Online conference system, method for controlling online conference system, and program
The online conference system addresses the misinterpretation of participant emotions by adjusting and presenting emotional feedback, improving communication quality through tailored emotional presentation.
Patent Information
- Application Number
- JP2024507238
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-03-15
- Filing Date
- 2022-03-15
- Publication Date
- 2024-11-21
- Estimated Expiration
- 2042-03-15
AI Technical Summary
Existing online conference systems fail to consider the unique emotional states of individual participants, leading to misinterpretation of their reactions, such as smiling participants not being recognized as having negative emotions.
An online conference system that includes an acquisition unit to recognize participant emotions, an adjustment unit to modify presentation content based on these emotions, and a presentation unit to present the adjusted content to other participants, allowing for tailored emotional feedback.
The system effectively presents participant reactions in a manner that suits the emotional context, enhancing communication by avoiding the presentation of unwanted negative emotions and facilitating smoother interactions.
Smart Images

Figure 2023175704000001 
Figure 2023175704000002
Abstract
Description
[Technical field]
[0001] The present invention relates to an online conference system, a control method for an online conference system, and a program. [Background technology]
[0002] In online conferences, there is a demand for technology that can display the reactions of other participants to a speaker.
[0003] Patent Document 1 describes a system that analyzes an image captured by a camera of a participant to recognize the state of the participant, and changes the background color of the participant according to the recognized state. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Japanese Patent No. 6872066 Summary of the Invention [Problem to be solved by the invention]
[0005] The system described in Patent Document 1 has a problem in that it does not take into account the specific state of each participant. For example, even if a participant who is in a good mood or smiles a lot has negative feelings toward a speaker's remarks, the problem is that the participant is not recognized as having negative feelings toward the speaker's remarks.
[0006] One aspect of the present invention has been made in consideration of the above problems, and one example of a purpose of the present invention is to provide a technique for appropriately presenting the reactions of participants in an online conference. [Means for solving the problem]
[0007] An online conference system according to one aspect of the present invention includes an acquisition means for acquiring an emotion recognition result of a first participant participating in an online conference, an adjustment means for adjusting presentation content of the emotion recognition result, and a presentation means for presenting the adjusted presentation content to a second participant different from the first participant.
[0008] A control method for an online conference system according to one aspect of the present invention includes at least one processor acquiring an emotion recognition result of a first participant participating in an online conference, adjusting presentation content of the emotion recognition result, and presenting the adjusted presentation content to a second participant different from the first participant.
[0009] A program according to one aspect of the present invention causes a computer to function as an acquisition means for acquiring an emotion recognition result of a first participant participating in an online conference, an adjustment means for adjusting presentation content of the emotion recognition result, and a presentation means for presenting the adjusted presentation content to a second participant different from the first participant. Effect of the Invention
[0010] According to one aspect of the present invention, the reactions of participants in an online conference can be presented in an appropriate manner. [Brief description of the drawings]
[0011] [Figure 1] FIG. 1 is a block diagram showing a configuration of an online conference system according to a first exemplary embodiment of the present invention. [Diagram 2] FIG. 2 is a flow chart showing the flow of a control method for an online conference system according to the first exemplary embodiment of the present invention. [Diagram 3] FIG. 11 is a block diagram showing a configuration of an online conference system according to an exemplary embodiment 2 of the present invention. [Figure 4] 11 is an example of an image displayed for acquiring the first setting information and the second setting information in the second exemplary embodiment of the present invention. [Diagram 5]FIG. 11 is a diagram showing an example of an avatar that a generation unit places in a virtual space in exemplary embodiment 2 of the present invention. [Figure 6] FIG. 11 is a sequence diagram showing the flow of a control method for an online conference system according to an exemplary embodiment 2 of the present invention. [Figure 7] FIG. 11 is a sequence diagram showing the flow of processing of an online conference executed in an online conference system according to an exemplary embodiment 2 of the present invention. [Figure 8] FIG. 11 is a diagram showing an example of a conference video of an online conference generated by a generation unit in exemplary embodiment 2 of the present invention. [Figure 9] FIG. 11 is a sequence diagram showing the flow of a control method for an online conference system according to an exemplary embodiment 3 of the present invention. [Figure 10] FIG. 11 is a diagram showing an example of the content presented on a display by a display unit in the third exemplary embodiment of the present invention. [Figure 11] FIG. 2 is a block diagram showing an example of the hardware configuration of an online conference system, a server, a virtual reality device, and an emotion recognition device in each exemplary embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Example 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described later.
[0013] (Overview of Online Conference System 1) The online conference system 1 according to this exemplary embodiment is a system that enables users of a plurality of computers to communicate with each other by providing each of the computers with video, audio, and data. The functions of the online conference system 1 may be provided in any of the plurality of computers, or may be provided in another computer (e.g., a server) that can communicate with each of the plurality of computers.
[0014] Moreover, the online conference system 1 according to this exemplary embodiment is a system that acquires an emotion recognition result of a first participant participating in an online conference, and adjusts the presentation content of the emotion recognition result and presents it to a second participant different from the first participant.
[0015] The first participant and the second participant are not particularly limited and may be any participants who are participating in the conference. For example, the first participant is a participant who listens to a speech, and the second participant is a participant who makes a speech.
[0016] Adjusting the presentation content of the emotion recognition result means changing the emotion or the degree of emotion included in the emotion recognition result and including the changed emotion recognition result in the presentation content. In other words, the presentation content includes the emotion recognition result, and the emotion recognition result is adjustable. The presentation content is presented to at least the second participant who speaks, but may also be presented to the first participant in addition to the second participant.
[0017] As an example, the online conference system 1 includes in the presentation content an emotion recognition result in which the degree of some or all of the emotions included in the emotion recognition result has been adjusted, or includes in the presentation content an emotion recognition result in which the degree of some or all of the emotions included in the emotion recognition result has been adjusted so as not to be included in the presentation content.
[0018] As a more specific example, the online conference system 1 includes positive emotion recognition results included in the emotion recognition results in the presentation content, and does not include negative emotion recognition results included in the emotion recognition results in the presentation content. Examples of positive emotions include happiness, joy, relief, etc. Examples of negative emotions include sadness, anger, anxiety, etc.
[0019] With this configuration, when a first participant has negative feelings after hearing a remark made by a second participant, the online conference system 1 does not present the negative feelings to the second participant. Therefore, the online conference system 1 does not present to the second participant negative feelings that the first participant does not want the second participant to know, and therefore can facilitate communication between the first participant and the second participant.
[0020] As another example, the online conference system 1 includes in the presentation content an emotion recognition result that has been modified by suppressing or emphasizing an emotion included in the emotion recognition result.
[0021] With this configuration, for example, if the first participant is an emotional person, the online conference system 1 suppresses the emotion, and if the first participant is an emotional person, the online conference system 1 emphasizes the emotion and includes it in the presentation content. Therefore, the online conference system 1 can appropriately present the reactions of the participants in the online conference.
[0022] The emotion recognition result can be obtained by using a known technology. One example of the technology for obtaining the emotion recognition result is a technology for obtaining the emotion recognition result by analyzing the physiological index, facial expression, and voice of the user and visualizing positive and negative emotions. One example of the physiological index of the user is a pulse wave, an electroencephalogram, a heart rate, and sweating.
[0023] (Configuration of online conference system 1) The configuration of an online conference system 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of an online conference system 1 according to this exemplary embodiment.
[0024] 1, the online conference system 1 according to this exemplary embodiment includes an acquisition unit 11, an adjustment unit 12, and a presentation unit 13. The acquisition unit 11, the adjustment unit 12, and the presentation unit 13 are components that respectively realize an acquisition means, an adjustment means, and a presentation means in this exemplary embodiment.
[0025] The acquisition unit 11 acquires an emotion recognition result of a first participant who participates in an online conference. The acquisition unit 11 supplies the acquired emotion recognition result to the adjustment unit 12.
[0026] The adjustment unit 12 adjusts the presentation content of the emotion recognition result supplied from the acquisition unit 11. The adjustment unit 12 supplies the adjusted presentation content to the presentation unit 13. The process of adjusting the presentation content by the adjustment unit 12 is as described above.
[0027] The presentation unit 13 presents the presentation content adjusted by the adjustment unit 12 to a second participant different from the first participant.
[0028] As described above, the online conference system 1 according to this exemplary embodiment employs a configuration including an acquisition unit 11 that acquires an emotion recognition result of a first participant participating in an online conference, an adjustment unit 12 that adjusts the presentation content of the emotion recognition result, and a presentation unit 13 that presents the adjusted presentation content to a second participant different from the first participant.
[0029] Therefore, according to the online conference system 1 according to this exemplary embodiment, the presentation content including the adjusted emotion recognition result of the first participant is presented to the second participant, thereby achieving the effect of appropriately presenting the reactions of the participants in the online conference.
[0030] (Flow of control method S1 of online conference system) The flow of the control method S1 of the online conference system 1 according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the control method S1 of the online conference system 1 according to this exemplary embodiment.
[0031] (Step S11) In step S11, the acquisition unit 11 acquires an emotion recognition result of a first participant who participates in the online conference. The acquisition unit 11 supplies the acquired emotion recognition result to the adjustment unit 12.
[0032] (Step S12) In step S12, the adjustment unit 12 adjusts the presentation content of the emotion recognition result supplied from the acquisition unit 11 in step S11. The adjustment unit 12 supplies the adjusted presentation content to the presentation unit 13. The process of adjusting the presentation content by the adjustment unit 12 is as described above.
[0033] (Step S13) In step S13, the presentation unit 13 presents the presentation content adjusted by the adjustment unit 12 in step S12 to a second participant different from the first participant.
[0034] As described above, in the control method S1 of the online conference system according to this exemplary embodiment, in step S11, the acquisition unit 11 acquires the emotion recognition result of a first participant participating in the online conference, in step S12, the adjustment unit 12 adjusts the presentation content of the emotion recognition result supplied from the acquisition unit 11 in step S11, and in step S13, the presentation unit 13 presents the presentation content adjusted by the adjustment unit 12 in step S12 to a second participant different from the first participant.
[0035] Therefore, according to the control method S1 for an online conference system according to this exemplary embodiment, the same effects as those of the online conference system 1 described above can be achieved.
[0036] Exemplary embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and the description thereof will be omitted as appropriate.
[0037] (Configuration of online conference system 100) The configuration of the online conference system 100 according to this exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the online conference system 100 according to this exemplary embodiment.
[0038] As shown in Fig. 3, the online conference system 100 according to this exemplary embodiment includes a server 2, a virtual reality device 3A, an emotion recognition device 4A, a virtual reality device 3B, and an emotion recognition device 4B. In this exemplary embodiment, when there is no need to distinguish between the virtual reality device 3A and the virtual reality device 3B, they are simply referred to as the "virtual reality device 3." Similarly, when there is no need to distinguish between the emotion recognition device 4A and the emotion recognition device 4B, they are simply referred to as the "emotion recognition device 4."
[0039] In addition, in this exemplary embodiment, a case will be described in which a user of the virtual reality device 3A is listening to a speech by a user of the virtual reality device 3B, but the online conference system 100 is not limited to this configuration. In other words, in the online conference system 100, the virtual reality device 3 and emotion recognition device 4 used by the user listening to the speech are referred to as the virtual reality device 3A and the emotion recognition device 4A, and the virtual reality device 3 and emotion recognition device 4 used by the user making the speech are referred to as the virtual reality device 3B and the emotion recognition device 4B. Here, the user listening to the speech is also referred to as the first participant, and the user making the speech is also referred to as the second participant.
[0040] In addition, in the online conference system 100 shown in FIG. 3, there are two virtual reality devices 3 and two emotion recognition devices 4, but the numbers of virtual reality devices 3 and emotion recognition devices 4 are not limited.
[0041] 3, the server 2, the virtual reality device 3A, the emotion recognition device 4A, the virtual reality device 3B, and the emotion recognition device 4B are communicatively connected to each other via a network N. The specific configuration of the network N does not limit the present embodiment, but as an example, a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination of these networks can be used.
[0042] In the online conference system 100, a server 2 enables users of virtual reality device 3A and virtual reality device 3B to communicate with each other by providing video, audio, and data to the virtual reality device 3A and virtual reality device 3B.
[0043] Furthermore, in the online conference system 100, the emotion recognition device 4A recognizes the emotion of the user of the virtual reality device 3A and outputs the emotion recognition result to the server 2. Then, the server 2 adjusts the presentation content of the acquired emotion recognition result and outputs the adjusted presentation content to the virtual reality device 3B.
[0044] (Server 2 configuration) As shown in FIG. 3, the server 2 includes a communication unit 21, a control unit 22, and a storage unit 23.
[0045] The communication unit 21 is a communication module that communicates with other devices via the network N. As an example, the communication unit 21 outputs data supplied from the control unit 22 (described later) to other devices via the network N, and obtains data output from other devices via the network N and supplies the data to the control unit 22.
[0046] The storage unit 23 stores data referenced by the control unit 22. Examples of the data stored in the storage unit 23 include emotion recognition results, presentation contents, and first setting information and second setting information, which will be described later.
[0047] (Functions of control unit 22) The control unit 22 controls each component included in the server 2. As shown in Fig. 3, the control unit 22 also functions as an acquisition unit 11, an adjustment unit 12, a presentation unit 13, and a generation unit 223. In this exemplary embodiment, the acquisition unit 11, the adjustment unit 12, the presentation unit 13, and the generation unit 223 are components that respectively realize an acquisition means, an adjustment means, a presentation means, and a generation means.
[0048] The acquisition unit 11 acquires data supplied from the communication unit 21. Examples of the data acquired by the acquisition unit 11 include emotion recognition results (the emotion recognition results of the first participant and the second participant), and first setting information, second setting information, first attribute information, second attribute information, and intimacy information, which will be described later. The acquisition unit 11 stores the acquired data in the storage unit 23.
[0049] The adjustment unit 12 acquires the emotion recognition result stored in the storage unit 23, and adjusts the presentation content of the emotion recognition result. Moreover, the adjustment unit 12 adjusts the presentation content of the emotion recognition result based on the data acquired from the virtual reality device 3. Furthermore, the adjustment unit 12 may adjust the speech content of the second participant, the movement of the second participant, and the avatar of the participant. An example of the process in which the adjustment unit 12 adjusts the presentation content will be described later. The adjustment unit 12 stores the adjusted presentation content in the storage unit 23.
[0050] The presentation unit 13 presents the presentation content to the second participant by outputting the presentation content to the virtual reality device 3B via the communication unit 21. As one example, the presentation unit 13 acquires a conference video generated by the generation unit 223 described later, the conference video including the adjusted presentation content, from the storage unit 23, and outputs the conference video to the virtual reality device 3 via the communication unit 21. As another example, the presentation unit 13 outputs a conference video not including the presentation content to the virtual reality device 3A, and outputs a conference video including the presentation content to the virtual reality device 3B. An example of the conference video output by the presentation unit 13 will be described later.
[0051] The presentation unit 13 may be configured to present the emotion recognition result in real time. In other words, the presentation unit 13 may output the presentation content at least when the presentation content changes during the online conference. With this configuration, the presentation unit 13 can present the emotion of the first participant to the second participant in real time.
[0052] The generation unit 223 generates a conference video of an online conference to be displayed on the virtual reality device 3. As an example, the generation unit 223 acquires presentation content stored in the storage unit 23, and generates a conference video including the presentation content. The generation unit 223 stores the generated conference video in the storage unit 23.
[0053] The generating unit 223 may generate a virtual space for holding an online conference. As an example, the generating unit 223 may generate a virtual space in which at least one of the avatars of the first participant and the second participant is arranged. The avatar used by the generating unit 223 may be acquired from the virtual reality device 3, or may be configured to associate a user ID for identifying a user from other users with an avatar and store them in advance in the storage unit 23, and acquire the user ID from the virtual reality device 3.
[0054] Furthermore, when the generation unit 223 generates a virtual space in which an avatar is placed, the generation unit 223 acquires the presentation content stored in the storage unit 23, and places an avatar in the virtual space according to the presentation content. The avatar that the generation unit 223 places in the virtual space according to the presentation content will be described later.
[0055] (Configuration of virtual reality device 3A) As shown in FIG. 3, the virtual reality device 3A includes a communication unit 31A, a control unit 32A, an input unit 34A, an imaging unit 35A, and a display 36A.
[0056] The communication unit 31A is a communication module that communicates with other devices via the network N. As an example, the communication unit 31A outputs data supplied from a control unit 32A (described later) to other devices via the network N, and obtains data output from other devices via the network N and supplies the data to the control unit 32A.
[0057] The input unit 34A is an interface that accepts input from a user. The input unit 34A supplies input information indicating the accepted input from the user to the control unit 32A. Examples of the input accepted by the input unit 34A include first setting information input by the first participant, first attribute information indicating the attribute of the first participant, and intimacy information indicating the intimacy between the first participant and the second participant.
[0058] Here, the first setting information is information that the first participant sets as to how the server 2 adjusts the content of the emotion recognition result to be presented. In other words, the first setting information is information that indicates how the first participant wants the emotion of the first participant to be adjusted and presented to the second participant. An example of the first setting information is to emphasize (or suppress) positive emotions (or negative emotions). An example of a process for receiving input of the first setting information will be described later.
[0059] Examples of the attributes of the first participant include gender, age, place of origin, place of residence, occupation, and clothing.
[0060] The imaging unit 35A is a camera capable of capturing moving images. The imaging unit 35A supplies the captured moving images to the control unit 32A. An example of a moving image captured by the imaging unit 35A is a moving image including a user's face as a subject.
[0061] The display 36A is a device for displaying images. A The display 36A acquires image data supplied from the control unit 32A and displays an image indicated by the image data. An example of an image displayed by the display 36A is an image of an online conference.
[0062] (Function of control unit 32A) The control unit 32A controls each of the components included in the virtual reality device 3 A. In addition, the control unit 32A also functions as an acquisition unit 321A and a display unit 322A, as shown in FIG.
[0063] The acquisition unit 321A acquires data supplied from the communication unit 31A, the input unit 34A, and the imaging unit 35A. The acquisition unit 321A outputs the acquired data to the server 2 via the communication unit 31A, or supplies the data to a display unit 322A described later.
[0064] The display unit 322A acquires an image to be displayed on the display 36A, and supplies image data indicating the image to the display 36A. Hereinafter, the display unit 322A supplying image data to the display 36A and the display 36A displaying an image indicated by the image data will also be expressed as the display unit 322A displaying an image on the display 36A.
[0065] (Configuration of emotion recognition device 4A) As shown in FIG. 3, the emotion recognition device 4A includes a communication unit 41A, a control unit 42A, and a sensor unit 44A.
[0066] The communication unit 41A is a communication module that communicates with other devices via the network N. As an example, the communication unit 41A outputs data supplied from a control unit 42A (described later) to other devices via the network N, and obtains data output from other devices via the network N and supplies the data to the control unit 42A.
[0067] The control unit 42A controls each component included in the emotion recognition device 4 A. In addition, the control unit 42A also functions as an emotion recognition unit 421A, as shown in FIG.
[0068] The emotion recognition unit 421A recognizes the emotion of the first participant based on sensor information supplied from the sensor unit 44A described later. The process of the emotion recognition unit 421A recognizing the emotion of the first participant is as described above. The emotion recognition unit 421A generates an emotion recognition result indicating the recognized emotion, and outputs the generated emotion recognition result to the server 2 via the communication unit 41A. Examples of the emotion recognition result include an emotion recognition result indicating that the first participant had a positive emotion and an emotion recognition result indicating that the first participant had a negative emotion.
[0069] Furthermore, the emotion recognition unit 421A may include the degree of the recognized emotion in the emotion recognition result. As an example, the emotion recognition unit 421A may indicate the degree of the emotion on a scale of 1 to 10 and include the degree in the emotion recognition result.
[0070] Furthermore, the emotion recognition unit 421A may output the sensor information supplied from the sensor unit 44A as an emotion recognition result to the server 2 via the communication unit 41A.
[0071] The sensor unit 44A is a sensor that detects at least one of a physiological index, a facial expression, and a voice of the first participant. The sensor unit 44A supplies sensor information indicating at least one of the detected physiological index, the facial expression, and the voice to the control unit 42A.
[0072] (Configuration of virtual reality device 3B and emotion recognition device 4B) The virtual reality device 3B is a virtual reality device used by the second participant. Instead of the first setting information acquired and output by the above-mentioned virtual reality device 3A, the virtual reality device 3B acquires and outputs second setting information input by the second participant, which is information that sets how the server 2 adjusts the content of the emotion recognition result to be presented. In other words, the second setting information is information that indicates how the second participant wants the emotion of the first participant to be adjusted and presented. The other configurations of the virtual reality device 3B are the same as those of the above-mentioned virtual reality device 3A, so the description will be omitted.
[0073] Moreover, the virtual reality device 3B acquires and outputs second attribute information indicating the attributes of the second participant and the degree of intimacy with the first participant, instead of the first attribute information indicating the attributes of the first participant and the degree of intimacy with the second participant acquired and output by the virtual reality device 3A described above. Examples of the attributes are as described above.
[0074] The emotion recognition device 4B is an emotion recognition device that recognizes the emotion of the second participant. The configuration of the emotion recognition device 4B is the same as that of the emotion recognition device 4A described above, and therefore a description thereof will be omitted.
[0075] (An example of a process for acquiring first setting information and second setting information) An example of a process in which the virtual reality device 3 acquires the first setting information and the second setting information will be described with reference to Fig. 4. Fig. 4 is an example of an image displayed to acquire the first setting information and the second setting information in this exemplary embodiment.
[0076] When the virtual reality device 3 receives an input from the first or second participant indicating that an online conference is to be started, the display unit 322 displays an image for acquiring the first setting information and the second setting information on the display 36. As an example, the display unit 322 displays the image shown in FIG.
[0077] As shown in Figure 4, the image for acquiring the first setting information and the second setting information includes items related to the first setting information (items included in "settings other than when speaking" in Figure 4) and items related to the second setting information (items included in "settings when speaking" in Figure 4).
[0078] As an example, if the user sets the item “Positive emotions” in “Settings other than when speaking” to “ON” and further sets it to “+20%”, the acquisition unit 321 acquires first setting information indicating that positive emotions are to be emphasized by 20%.
[0079] As another example, when the user sets the setting of the item "negative emotion" in the "settings when speaking" to "OFF", the acquisition unit 321 acquires second setting information indicating that negative emotions are not included.
[0080] Here, in the case where the speaker and the listeners are fixed, such as in a lecture, an image for acquiring either the first setting information or the second setting information may be displayed.
[0081] In addition, the image for acquiring the first setting information and the second setting information may include an item for selecting whether or not to make the conference an online conference including video images captured by the imaging unit 35, as shown in FIG. 4, and an item for selecting whether or not to make the conference an online conference in which an avatar is placed in a virtual space.
[0082] (Example 1 of a process in which the adjustment unit 12 adjusts the presentation content of the emotion recognition result) As one example, the adjustment unit 12 may adjust the presentation content based on the first setting information input by the first participant.
[0083] As an example, when the first setting information indicates that the positive emotion recognition result of the first participant is to be included in the presentation content and the negative emotion recognition result of the first participant is not to be included in the presentation content, the adjustment unit 12 may adjust the presentation content based on the first setting information.
[0084] In this configuration, when the emotion recognition result of the first participant is a negative emotion recognition result, the adjustment unit 12 adjusts the negative emotion recognition The results are not included in the presentation.
[0085] On the other hand, if the emotion recognition result of the first participant is a positive emotion recognition result, the adjustment unit 12 adjusts the positive emotion recognition Include the results in your presentation.
[0086] As another example, when the first setting information indicates that emotions are to be suppressed or emphasized, the adjustment unit 12 may adjust the presentation content based on the first setting information.
[0087] In this configuration, when the first setting information indicates that positive emotions are to be emphasized by 20% and the emotion recognition result of the first participant indicates that the first participant felt a positive emotion, the adjustment unit 12 includes in the presentation content the emotion recognition result adjusted by emphasizing the positive emotion indicated by the emotion recognition result by 20%.
[0088] On the other hand, when the first setting information indicates that negative emotions are to be suppressed by 30% and the emotion recognition result of the first participant indicates that the first participant felt a negative emotion, the adjustment unit 12 includes in the presentation content the emotion recognition result adjusted by suppressing the negative emotion indicated by the emotion recognition result by 30%.
[0089] In this way, by the adjustment unit 12 adjusting the presentation content based on the first setting information, the first participant can adjust the emotions of the first participant to be presented to the second participant, so that the reaction of the first participant participating in the online conference can be presented to the second participant in an appropriate manner.
[0090] For example, with this configuration, when a first participant who is feeling depressed joins an online conference, the adjustment unit 12 can adjust the content of the presentation to inform the second participant that the first participant is not feeling depressed because of the comments made by the second participant.
[0091] (Example 2 of a process in which the adjustment unit 12 adjusts the presentation content of the emotion recognition result) As another example, the adjustment unit 12 may adjust the presentation content based on second setting information input by the second participant.
[0092] As an example, when the second setting information indicates that a positive emotion recognition result is included in the presentation content and that a negative emotion recognition result is not included in the presentation content, and the emotion recognition result of the first participant is a negative emotion recognition result, the adjustment unit 12 recognition On the other hand, if the emotion recognition result of the first participant is a positive emotion recognition result, the adjustment unit 12 may include the positive emotion in the content to be presented. recognition The results may be included in the presentation.
[0093] As another example, when the second setting information indicates that emotions are to be suppressed or emphasized, the adjustment unit 12 may adjust the presentation content based on the second setting information.
[0094] As an example, if the second setting information indicates that positive emotions are to be emphasized by 20%, and the emotion recognition result of the first participant indicates that the first participant felt a positive emotion, the adjustment unit 12 may include in the presentation content the emotion recognition result adjusted to emphasize the positive emotion indicated by the emotion recognition result by 20%.
[0095] On the other hand, if the second setting information indicates that negative emotions are to be suppressed by 30% and the emotion recognition result of the first participant indicates that the first participant felt a negative emotion, the adjustment unit 12 may include in the presentation content the emotion recognition result adjusted by suppressing the negative emotion indicated by the emotion recognition result by 30%.
[0096] In this way, by the adjustment unit 12 adjusting the presentation content based on the second setting information, the second participant can adjust the emotions of the first participant to be presented to the second participant, so that the reaction of the first participant participating in the online conference can be presented to the second participant in an appropriate manner.
[0097] For example, with this configuration, if the second participant does not want to know that the first participant has felt negative emotions due to the second participant's comments, the adjustment unit 12 can adjust the presentation content to present to the second participant that the first participant does not have negative emotions, even if the first participant has negative emotions.
[0098] (Example 3 of a process in which the adjustment unit 12 adjusts the presentation content of the emotion recognition result) As yet another example, the adjustment unit 12 may adjust the presentation content based on attributes of one or both of the first participant and the second participant.
[0099] As an example, if the first attribute information indicates that the first participant is in his / her 50s, that his / her occupation is teacher, and that the emotion recognition result of the first participant is a positive emotion recognition result, the adjustment unit 12 may emphasize the positive emotion recognition result and include it in the presentation content. In other words, if the attribute indicated by the first attribute information is an attribute that is estimated to indicate that the first participant is not a very frank person, the adjustment unit 12 may emphasize the positive emotion recognition result of the first participant and include it in the presentation content.
[0100] In this case, the adjustment unit 12 may further change the speech content of the second participant to be polite. In other words, if the attribute indicated by the first attribute information is an attribute that indicates that the first participant is not a very frank person, the adjustment unit 12 may adjust the behavior of the second participant to be polite. Here, the adjustment unit 12 may adjust the behavior of the second participant to be polite in addition to the speech content.
[0101] As another example, when the second attribute information indicates that the second participant is in his / her 20s and that his / her occupation is related to entertainment, the adjustment unit 12 may change the speech content of the second participant to be polite. In other words, when the attribute indicated by the second attribute information is an attribute that indicates that the second participant is a frank person, the adjustment unit 12 may adjust the behavior of the second participant to be polite.
[0102] As yet another example, when a second participant participates in an online conference as an avatar and the first attribute information indicates that the first participant is dressed formally, the adjustment unit 12 may adjust the clothes of the avatar of the second participant so that the second participant is also dressed formally. In other words, when the attributes indicated by the first attribute information are attributes that indicate that the online conference is a serious conference, the adjustment unit 12 may adjust the clothes of the avatar of the second participant so that the clothes are suitable for a serious conference.
[0103] In this way, the adjustment unit 12 can facilitate communication between participants in an online conference by adjusting the presentation content based on the attributes of one or both of the first and second participants.
[0104] (Example 4 of a process in which the adjustment unit 12 adjusts the presentation content of the emotion recognition result) As yet another example, the adjustment unit 12 may adjust the presented content based on the degree of intimacy between the first participant and the second participant.
[0105] As an example, when the intimacy information indicating the intimacy with the second participant output from the virtual reality device 3A indicates that the first participant has met the second participant for the first time and the emotion recognition result of the first participant is a positive emotion recognition result, the adjustment unit 12 may emphasize the emotion recognition result and include it in the presentation content. In other words, when the intimacy between the first participant and the second participant is low, the adjustment unit 12 , No. One participant's positive emotion recognition results may be highlighted and included in the presentation.
[0106] Furthermore, when the intimacy level between the first participant and the second participant is low and at least one of the first participant and the second participant participates in the online conference as an avatar, the adjustment unit 12 may adjust the behavior of the participant participating as an avatar to be polite.
[0107] The adjustment unit 12 may further decrease the degree of emphasis on the emotion recognition result as the degree of intimacy between the first participant and the second participant increases.
[0108] Here, the adjustment unit 12 may estimate the degree of intimacy.
[0109] As an example, the adjustment unit 12 may estimate the intimacy level based on the number of online conferences held by the first participant and the second participant. For example, when the number of online conferences held by the first participant and the second participant exceeds a predetermined number (for example, 5 times), the adjustment unit 12 may be configured to increase the intimacy level.
[0110] As another example, the adjustment unit 12 may estimate the intimacy level based on the content of the statement made by the first participant. For example, if the content of the statement made by the first participant is frank, the adjustment unit 12 may be configured to increase the intimacy level.
[0111] In this way, the adjustment unit 12 can facilitate smooth communication between the participants of the online conference by adjusting the presented content based on the degree of intimacy between the first participant and the second participant.
[0112] (Example 5 of the process in which the adjustment unit 12 adjusts the presentation content of the emotion recognition result) As yet another example, the adjustment unit 12 may adjust the presentation content during an online conference based on at least one of sensor information detected during the online conference, an emotion recognition result of the first participant, and an emotion recognition result of the second participant.
[0113] As an example, when there are multiple first participants, the adjustment unit 12 may be configured to adjust the content presented during the online conference based on the emotion recognition results of each of the multiple first participants.
[0114] In this configuration, for example, if the emotion recognition result indicates that a predetermined number or more of the first participants (e.g., 80% or more of the total) among the multiple first participants felt positive emotions, the adjustment unit 12 may adjust the emotion recognition results of all the first participants to emotion recognition results indicating that they felt positive emotions.
[0115] As another example, the adjustment unit 12 may be configured to adjust the content presented during the online conference based on the emotion recognition result of the first participant.
[0116] In this configuration, for example, when the emotion recognition result of the first participant indicates that the first participant has had a positive emotion for a predetermined period of time (for example, five minutes) or more, the adjustment unit 12 may adjust the emotion recognition result of the first participant to an emotion recognition result that emphasizes the positive emotion.
[0117] As yet another example, the adjustment unit 12 may be configured to adjust the presentation content during the online conference based on the sensor information detected during the online conference and the emotion recognition result of the second participant.
[0118] For example, when the content of the statement indicated by the sensor information is a statement in which the second participant apologizes, the adjustment unit 12 may adjust the emotion recognition result of the second participant to an emotion with a stronger apologetic feeling.
[0119] As another example of this configuration, when the speaking speed of the second participant indicated by the sensor information is fast, the adjustment unit 12 , No. The speech speed of the second participant may be adjusted to be slower. Also, when the second participant participates in the online conference as an avatar, the adjustment unit 12 may adjust the movement of the second participant to be larger.
[0120] In this way, the adjustment unit 12 can facilitate communication between participants in an online conference by adjusting the content presented during the online conference based on at least one of the sensor information detected during the online conference, the emotion recognition result of the first participant, and the emotion recognition result of the second participant.
[0121] (Example of avatars depending on the content presented) An example of an avatar that the generation unit 223 places in the virtual space according to the presentation content will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of an avatar that the generation unit 223 places in the virtual space in this exemplary embodiment.
[0122] The generation unit 223 may control the facial expression of the avatar depending on the presented content.
[0123] As an example, if the emotion recognition result of the participant included in the presented content indicates that the emotion of the participant is neutral, the generation unit 223 places an avatar with a normal facial expression in the virtual space, as shown for participant PA4 in Figure 5.
[0124] As another example, if the emotion recognition result of the participant included in the presented content indicates that the emotion of the participant is positive, the generation unit 223 places in the virtual space an avatar with eyes wide open, corners of the mouth turned up, and a happy expression, as shown in participant PA5 in Figure 5.
[0125] As yet another example, if the emotion recognition result of the participant included in the presented content indicates that the emotion of the participant is positive and further emphasizes the positive emotion, the generation unit 223 places in the virtual space an avatar with narrowed eyes, turned up corners of the mouth, and a happier expression, as shown in participant PA6 in Figure 5.
[0126] As yet another example, if the emotion recognition result of a participant included in the presented content indicates that the emotion of the participant is negative, the generation unit 223 places an avatar with eyes closed and a sad expression in the virtual space, as shown by participant PA7 in Figure 5.
[0127] Furthermore, the generation unit 223 may control the movement of the avatar in accordance with the presented content. As an example, the generation unit 223 may control the movement of ears attached to the avatar, as shown in FIG.
[0128] As an example of this case, when the emotion recognition result of the participant included in the presented content indicates that the emotion of the participant is in a neutral state, the generation unit 223 places a participant PA4 with drooping ears in the virtual space.
[0129] As another example, if the emotion recognition result of the participant included in the presented content indicates that the emotion of the participant is positive, the generation unit 223 places participant PA5 or participant PA6, who has moved his / her entire ear, in the virtual space.
[0130] As yet another example, when the emotion recognition result of the participant included in the presented content indicates that the emotion of the participant is negative, the generation unit 223 places a participant PA6 with drooping ears in the virtual space.
[0131] Furthermore, the generating unit 223 may control the voice of the avatar according to the presented content. As an example, when the emotion recognition result of the participant included in the presented content indicates that the emotion of the participant is positive, the generating unit 223 generates a conference video including the voice of the avatar such as "happy," "fun," or "interesting."
[0132] Furthermore, the generation unit 223 may place, in addition to the avatar, an icon that changes depending on the emotion recognition result of the participant included in the presented content in the virtual space. An example of the generation unit 223 placing an icon in the virtual space will be described later.
[0133] In this way, the generation unit 223 controls the facial expression, movement, or voice of the avatar of the first participant placed in the virtual space according to the presentation content. With this configuration, the generation unit 223 can appropriately present the reaction of the participants of the online conference.
[0134] (Flow of control method S2 of online conference system 100) The flow of the control method S2 of the online conference system 100 according to this exemplary embodiment will be described with reference to Fig. 6. Fig. 6 is a sequence diagram showing the flow of the control method S2 of the online conference system 100 according to this exemplary embodiment.
[0135] (Step S21) In step S21, the acquisition unit 321A of the virtual reality device 3A acquires the first setting information via the input unit 34A.
[0136] (Step S22) In step S22, the acquisition unit 321A outputs the first setting information acquired in step S21 to the server 2 via the communication unit 31A.
[0137] (Step S23) In step S23, the acquisition unit 321B of the virtual reality device 3B acquires the second setting information via the input unit 34B.
[0138] (Step S24) In step S24, the acquisition unit 321B outputs the second setting information acquired in step S23 to the server 2 via the communication unit 31B.
[0139] The order of steps S21 and S22 and steps S23 and S24 is not limited.
[0140] (Step S25) In step S25, the acquisition unit 11 of the server 2 acquires the first setting information output from the virtual reality device 3A and the second setting information output from the virtual reality device 3B via the communication unit 21. The acquisition unit 11 stores the acquired first setting information and second setting information in the memory unit 23.
[0141] (Step S30) In step S30, the online conference is held in the online conference system 100.
[0142] (An example of online meeting processing) The flow of step S30, which is the processing of the online conference executed in the online conference system 100, will be described with reference to Fig. 7. Fig. 7 is a sequence diagram showing the flow of the processing of the online conference executed in the online conference system 100 according to this exemplary embodiment.
[0143] (Step S31) In step S31, the generation unit 223 generates a conference video of an online conference. As an example, when a moving image is acquired from the virtual reality device 3, the generation unit 223 generates a conference video including the moving image. As another example, when an avatar is used, the generation unit 223 generates a conference video including the avatar. The generation unit 223 stores the generated conference video in the storage unit 23.
[0144] (Step S32) In step S32, the presentation unit 13 acquires the conference video stored in the memory unit 23, and outputs the conference video to the virtual reality device 3 via the communication unit 21.
[0145] (Step S33) In step S33, the acquisition unit 321A of the virtual reality device 3A acquires the conference video output in step S32 via the communication unit 31A. The acquisition unit 321A supplies the acquired conference video to the display unit 322A.
[0146] (Step S34) In step S34, the display unit 322A displays the conference video supplied from the acquisition unit 321A in step S33 on the display 36A.
[0147] (Step S35) In step S35, the acquisition unit 321B of the virtual reality device 3B acquires the conference video output in step S32 via the communication unit 31B. The acquisition unit 321B supplies the acquired conference video to the display unit 322B.
[0148] (Step S36) In step S36, the display unit 322B displays the conference video supplied from the acquisition unit 321B in step S35 on the display 36B.
[0149] The order of steps S33 and S34 and steps S35 and S36 is not limited.
[0150] (Step S37) In step S37, the emotion recognition unit 421A of the emotion recognition device 4A acquires sensor information from the sensor unit 44A.
[0151] (Step S38) In step S38, the emotion recognition section 421A generates an emotion recognition result based on the sensor information acquired in step S37.
[0152] (Step S39) In step S39, the emotion recognition section 421A outputs the emotion recognition result generated in step S38 to the server 2 via the communication section 41A.
[0153] Here, the processing of steps S37 to S39 may be configured to be executed as a trigger when the emotion recognition unit 421A acquires a signal from the virtual reality device 3A indicating that step S34 has been executed, or may be configured to be executed at all times while step S30 is being executed, regardless of the processing of step S34.
[0154] (Step S40) In step S40, the emotion recognition unit 421B of the emotion recognition device 4B acquires sensor information from the sensor unit 44B. Here, the sensor information acquired by the emotion recognition unit 421B in step S40 includes at least the voice of the utterance of the second participant.
[0155] (Step S41) In step S41, the emotion recognition unit 421B outputs the sensor information acquired in step S40 to the server 2 via the communication unit 41B.
[0156] Here, the processes of steps S40 and S41 may be configured to be executed at all times during execution of step S30, regardless of the process of step S36. Also, in step S40, the emotion recognition unit 421B may generate an emotion recognition result based on the acquired sensor information. In this case, in step S41, the emotion recognition unit 421B may output the sensor information and the generated emotion recognition result to the server 2.
[0157] (Step S42) In step S42, the acquisition unit 11 of the server 2 acquires the emotion recognition result output in step S39 and the sensor information output in step S41 via the communication unit 21. The acquisition unit 11 stores the acquired emotion recognition result and sensor information in the storage unit 23.
[0158] (Step S43) The adjustment unit 12 acquires the first setting information, the second setting information, the emotion recognition result, and the sensor information stored in the storage unit 23, and adjusts the presentation content. The process of adjusting the presentation content by the adjustment unit 12 is as described above.
[0159] (Step S44) In step S44, the generation unit 223 generates a conference video of the online conference. As described above, the generation unit 223 may generate a virtual space for holding the online conference.
[0160] (Step S45) In step S45, the control unit 22 of the server 2 determines whether or not to end the conference. As one example, the control unit 22 determines to end the conference when a predetermined time has elapsed since the start of the conference. As another example, the control unit 22 determines to end the conference when an instruction to end the conference is received from the virtual reality device 3.
[0161] If it is determined in step S45 that the conference is to be ended, the online conference process shown in FIG. 7 ends.
[0162] On the other hand, if it is determined in step S45 that the conference has not ended, processing returns to step S32, and the presentation unit 13 acquires the conference video stored in the memory unit 23 (i.e., the updated conference video) and outputs the conference video to the virtual reality device 3 via the communication unit 21.
[0163] Here, as described above, the conference video stored in the storage unit 23 may be a conference video including the presentation content. Also, when the conference video stored in the storage unit 23 is a conference video not including the presentation content, the presentation unit 13 outputs the conference video and the presentation content to at least the virtual reality device 3B in step S32, which is executed again.
[0164] (Example of video footage from an online meeting) An example of a conference video of an online conference generated by the generation unit 223 will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of a conference video of an online conference generated by the generation unit 223 in this exemplary embodiment.
[0165] As an example, the generating unit 223 generates a conference video MP1 including a first participant PA1 and a second participant PA2 as shown in Fig. 8. The first participant PA1 and the second participant PA2 may each include a moving image including the first participant or the second participant as a subject acquired from the virtual reality device 3 in the conference video MP1 as is, or may be an avatar representing the first participant or the second participant, respectively.
[0166] 8, the generation unit 223 may include other first participants PA3 in the conference video MP1. The other first participants PA3 may be moving images acquired from the virtual reality device 3 or avatars, as in the above-described configuration.
[0167] Furthermore, as shown in FIG. 8, the generating unit 223 may include, in the conference video MP1, an icon ICN1 corresponding to the emotion recognition result of the first participant PA1, which is included in the presentation content.
[0168] Furthermore, when the emotion recognition result of the first participant PA1 included in the conference video MP1 changes, the generation unit 223 generates the conference video MP2 based on the emotion recognition result after the change, as shown in FIG.
[0169] As an example, when the emotion recognition result of the first participant PA1 changes to a positive emotion, the generation unit 223 changes the facial expression of the first participant PA1 to an expression indicating that the first participant PA1 has become positive. Furthermore, the generation unit 223 also changes the icon ICN1 corresponding to the emotion recognition result of the first participant PA1 to an icon ICN2 indicating that the first participant PA1 has become positive.
[0170] As another example, the generation unit 223 may be configured to change the background color according to the emotion recognition result of the first participant PA1. For example, when the first participant PA1 has a positive emotion, the generation unit 223 may change the background color of the first participant PA1 to green, and when the first participant PA1 has a negative emotion, the generation unit 223 may change the background color of the first participant PA1 to red.
[0171] As described above, in the online conference system 100 according to this exemplary embodiment, the acquisition unit 11 of the server 2 acquires the emotion recognition result of the first participant participating in the online conference from the emotion recognition device 4A, the adjustment unit 12 adjusts the presentation content of the emotion recognition result acquired by the acquisition unit 11, and the presentation unit 13 presents the adjusted presentation content to the virtual reality device 3B used by the second participant.
[0172] Therefore, according to the online conference system 100 according to this exemplary embodiment, the presentation content including the adjusted emotion recognition result of the first participant can be presented to the second participant, thereby achieving the effect of suitably presenting the reactions of the participants in the online conference.
[0173] Exemplary embodiment 3 A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the above exemplary embodiment are denoted by the same reference numerals, and the description thereof will not be repeated.
[0174] (Overview of the online conference system 100A) In the online conference system 100A according to this exemplary embodiment, presentation contents from a predetermined period or a predetermined time in the past are presented. More specifically, the presentation unit 13 of the server 2 acquires presentation contents from a predetermined period or a predetermined time in the past stored in the storage unit 23, and outputs them to the virtual reality device 3. Other configurations are the same as those of the online conference system 100 described above, and therefore will not be described.
[0175] (Online conference system 100 A(Flow of control method S3) The flow of the control method S3 of the online conference system 100A according to this exemplary embodiment will be described with reference to Fig. 9. Fig. 9 is a sequence diagram showing the flow of the control method S3 of the online conference system 100A according to this exemplary embodiment. Fig. 9 shows an example in which the server 2 outputs presentation content from a predetermined period or time in the past to the virtual reality device 3B, but the device to which the server 2 outputs the presentation content is not limited to the virtual reality device 3B.
[0176] (Step S51) In step S51, the acquisition unit 321B of the virtual reality device 3B acquires presentation information indicating a predetermined period or a predetermined point in time in the past via the input unit 34B.
[0177] (Step S52) In step S52, the acquisition unit 321B outputs the presentation information acquired in step S51 to the server 2 via the communication unit 31B.
[0178] (Step S53) In step S53, the acquisition unit 11 of the server 2 acquires the presentation information output from the virtual reality device 3B in step S52 via the communication unit 21. The acquisition unit 11 stores the acquired presentation information in the memory unit .
[0179] (Step S54) In step S54, the presentation unit 13 acquires the presentation information stored in the memory unit 23, and acquires presentation content corresponding to the presentation information from the memory unit 23. Then, the presentation unit 13 outputs the acquired presentation content to the virtual reality device 3B via the communication unit 21.
[0180] (Step S55) In step S55, the acquisition unit 321B of the virtual reality device 3B acquires the presentation content output from the server in step S54 via the communication unit 31B. The acquisition unit 321B supplies the acquired presentation content to the display unit 322B.
[0181] (Step S56) The display unit 322B acquires the presentation content supplied from the acquisition unit 321B in step S55. Then, the display unit 322B displays the acquired presentation content on the display 36B.
[0182] (Example of what is displayed) An example of the content presented by the display unit 322B on the display 36B will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of the content presented by the display unit 322B on the display 36B in this exemplary embodiment.
[0183] The display unit 322B may display a conference video of the online conference corresponding to the acquired presentation content, as shown in FIG. ,figure As shown in FIG. 10, as described in the above-described embodiment, the display unit 322B may display a conference video including a first participant PA1, a second participant PA2, and an icon ICN1 corresponding to the emotion recognition result of the first participant PA1.
[0184] As one example of this configuration, the presentation unit 13 of the server 2 may acquire a conference video including presentation content corresponding to the presentation information from the storage unit 23, and output the acquired conference video to the virtual reality device 3B. As another example of this configuration, the virtual reality device 3B may store previously acquired conference video in a storage unit (not shown), and acquire a conference video corresponding to the acquired presentation content from the storage unit.
[0185] In addition, when the acquired presentation content is the presentation content for a specified period, the display unit 322B may display the conference video including a seek bar SB indicating which point in the specified period is being played back, as shown in FIG. 10.
[0186] In this case, the display unit 322B may include a flag UF, which indicates the point in time at which the first participant spoke, above the seek bar SB, as shown in FIG.
[0187] As described above, in the online conference system 100A according to the present exemplary embodiment, the presentation unit 13 of the server 2 outputs presentation contents from a predetermined period or time in the past to the virtual reality device 3. Therefore, according to the online conference system 100A according to the present exemplary embodiment, it is possible to present later how the second participant responded to a statement made by the first participant.
[0188] [Software implementation example] Some or all of the functions of the online conference system 1, the server 2, the virtual reality device 3, and the emotion recognition device 4 may be realized by hardware such as an integrated circuit (IC chip), or may be realized by software.
[0189] In the latter case, the online conference system 1, the server 2, the virtual reality device 3, and the emotion recognition device 4 are realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter, referred to as computer C) is shown in FIG. 11. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the online conference system 1, the server 2, the virtual reality device 3, and the emotion recognition device 4. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the online conference system 1, the server 2, the virtual reality device 3, and the emotion recognition device 4.
[0190] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination of these. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these.
[0191] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may further include a communication interface for transmitting and receiving data to and from other devices. The computer C may further include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.
[0192] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. Furthermore, the program P can be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.
[0193] [Additional Note 1] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiment are also included in the technical scope of the present invention.
[0194] [Additional Note 2] Some or all of the above-described embodiments can be described as follows. However, the present invention is not limited to the following described aspects.
[0195] (Appendix 1) An online conference system comprising: an acquisition means for acquiring an emotion recognition result of a first participant participating in an online conference; an adjustment means for adjusting a presentation content of the emotion recognition result; and a presentation means for presenting the adjusted presentation content to a second participant different from the first participant. (Appendix 2) The online conference system according to claim 1, wherein the adjustment means includes positive emotion recognition results contained in the emotion recognition results in the presented content, and does not include negative emotion recognition results in the presented content.
[0196] (Appendix 3) 3. The online conference system according to claim 1, wherein the adjustment means includes in the presentation content the emotion recognition result that has been modified by suppressing or emphasizing the emotion included in the emotion recognition result.
[0197] (Appendix 4) The online conference system according to any one of claims 1 to 3, wherein the adjustment means adjusts the presented content based on attributes of one or both of the first participant and the second participant.
[0198] (Appendix 5) The online conference system according to any one of claims 1 to 4, wherein the adjustment means adjusts the presented content based on a degree of intimacy between the first participant and the second participant.
[0199] (Appendix 6) The online conference system of any one of appendices 1 to 5, wherein the acquisition means acquires an emotion recognition result of the second participant, and the pre-adjustment means adjusts the presentation content during the online conference based on at least one of sensor information detected during the online conference, the emotion recognition result of the first participant, and the emotion recognition result of the second participant.
[0200] (Appendix 7) 7. The online conference system according to claim 1, wherein the adjustment means adjusts the presented content based on first setting information input by the first participant.
[0201] (Appendix 8) 8. The online conference system according to claim 1, wherein the adjustment means adjusts the presented content based on second setting information input by the second participant.
[0202] (Appendix 9) 9. The online conference system according to claim 1, wherein the presentation means presents the emotion recognition result in real time.
[0203] (Appendix 10) 10. The online conference system according to claim 1, wherein the presentation means presents the presentation content from a predetermined period or a predetermined point in time in the past.
[0204] (Appendix 11) The online conference system according to any one of claims 1 to 10, further comprising a generating means for generating a virtual space for holding the online conference and controlling the facial expression, movement, or voice of an avatar of the first participant placed in the virtual space in accordance with the presented content.
[0205] (Appendix 12) A method for controlling an online conference system, comprising: at least one processor acquiring an emotion recognition result of a first participant participating in an online conference; adjusting presentation content of the emotion recognition result; and presenting the adjusted presentation content to a second participant different from the first participant.
[0206] (Appendix 13) A program that causes a computer to function as an acquisition means for acquiring an emotion recognition result of a first participant participating in an online conference, an adjustment means for adjusting presentation content of the emotion recognition result, and a presentation means for presenting the adjusted presentation content to a second participant different from the first participant.
[0207] [Additional Note 3] A part or all of the above-described embodiments can be further expressed as follows.
[0208] An online conference system comprising at least one processor, the processor executing an acquisition process for acquiring an emotion recognition result of a first participant participating in an online conference, an adjustment process for adjusting presentation content of the emotion recognition result, and a presentation process for presenting the adjusted presentation content to a second participant different from the first participant.
[0209] The online conference system may further include a memory, acquisition Processing and adjustment Processing and presentation The program for causing the processor to execute the above-mentioned processes may be stored in the storage medium. The program may be recorded in a non-transitory computer-readable tangible recording medium. [Explanation of symbols]
[0210] 1, 100, 100A Online Conference System 11 Acquisition Department 12 Adjustment part 13 Presentation part 2 Server 223 Generation part 3. Virtual reality devices 4 Emotion recognition device
Claims
1. An acquisition means for acquiring an emotion recognition result of a first participant participating in an online conference; an adjustment means for adjusting the presentation content of the emotion recognition result; a presentation means for presenting the adjusted presentation content to a second participant different from the first participant; An online conference system comprising:
2. The online conference system according to claim 1 , wherein the adjustment means includes a positive emotion recognition result contained in the emotion recognition result in the presented content, and does not include a negative emotion recognition result in the presented content.
3. The online conference system according to claim 1 , wherein the adjustment means includes in the presentation content the emotion recognition result that has been modified by suppressing or emphasizing an emotion included in the emotion recognition result.
4. The online conference system according to claim 1 , wherein the adjustment means adjusts the presented content based on attributes of one or both of the first participant and the second participant.
5. The online conference system according to claim 1 , wherein the adjustment unit adjusts the presented content based on a degree of intimacy between the first participant and the second participant.
6. The acquisition means acquires an emotion recognition result of the second participant, 6. The online conference system according to claim 1, wherein the pre-adjustment means adjusts the presentation content during the online conference based on at least one of sensor information detected during the online conference, an emotion recognition result of the first participant, and an emotion recognition result of the second participant.
7. The online conference system according to claim 1 , wherein the adjustment unit adjusts the presentation content based on first setting information input by the first participant.
8. The online conference system according to claim 1 , wherein the adjustment unit adjusts the presentation content based on second setting information input by the second participant.
9. At least one processor Obtaining an emotion recognition result of a first participant participating in an online conference; adjusting presentation content of the emotion recognition result; and presenting the adjusted presentation content to a second participant different from the first participant; A method for controlling an online conference system, comprising:
10. Computer, An acquisition means for acquiring an emotion recognition result of a first participant participating in an online conference; an adjustment means for adjusting the presentation content of the emotion recognition result; a presentation means for presenting the adjusted presentation content to a second participant different from the first participant; A program that functions as a
Citation Information
Patent Citations
System, method and program for implementing computer-mediated communication
JP6872066B1
