Communication support device, communication support method, and communication support processing program

Abstract

To provide a communication support device that can appropriately provide information to facilitate smooth communication with communication partners online. [Solution] The user terminal 20A, as a communication support device, comprises a first indicator value acquisition unit 274, a second indicator value acquisition unit 275, an identification unit 273, and a second notification unit 276. The first indicator value acquisition unit 274 acquires a first indicator value indicating the emotions of the first user. The second indicator value acquisition unit 275 acquires a second indicator value indicating the emotions of the second user. The identification unit 273 identifies a third user who has a predetermined relationship with the first user. The second notification unit 276 outputs notification information to the third user when the difference between the first indicator value and the second indicator value is greater than or equal to a predetermined threshold.

Description

Technical Field

[0001] The present invention relates to a communication support device, a communication support method, and a communication support processing program.

Background Art

[0002] With the recent spread of remote work, opportunities for online communication, such as online meetings, have increased through communication networks. When communicating online, it becomes more difficult for participants to read each other's feelings compared to having a conversation in real space. As a result, there is a possibility that the speech of a predetermined participant may be a harassment act against any other participant.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In view of this, there is a technology that monitors the speech content during an online meeting and notifies the speaker when detecting speech content that affects harassment (for example, Patent Document 1). By performing such a notification, the speaker can be made to recognize earlier that they have made a speech that affects harassment, and deterioration of the relationship between participants can be prevented.

[0005] However, whether a predetermined speech content affects harassment may vary depending on the relationship between users. In the above-described technology, processing considering the relationship between users is not performed, and there is a problem that appropriate notification regarding harassment may not be possible.

[0006] The present invention aims to provide a communication support device, a communication support method, and a communication support processing program that can appropriately provide information for smooth communication with a communication partner online. [Means for solving the problem]

[0007] A communication support device according to one or more embodiments for achieving the above objectives includes: a first index value acquisition unit that acquires a first index value indicating the emotions of a first user; a second index value acquisition unit that acquires a second index value indicating the emotions of a second user; an identification unit that identifies a third user having a predetermined relationship with the first user; and a notification unit that outputs notification information to the third user when the difference between the first index value and the second index value is greater than or equal to a predetermined threshold.

[0008] Furthermore, in one embodiment of one or more of the embodiments, the communication support method involves the communication support device acquiring a first indicator value indicating the emotions of a first user, acquiring a second indicator value indicating the emotions of a second user, identifying a third user who has a predetermined relationship with the first user, and outputting notification information to the third user when the difference between the first indicator value and the second indicator value is greater than or equal to a predetermined threshold.

[0009] Furthermore, a communication support program according to one or more embodiments provides a communication support device with a function to acquire a first index value indicating the emotions of a first user, a second index value indicating the emotions of a second user, a function to identify a third user having a predetermined relationship with the first user, and a function to output notification information to the third user when the difference between the first index value and the second index value is greater than or equal to a predetermined threshold. [Effects of the Invention]

[0010] According to a communication support device, communication support method, and communication support processing program according to one or more embodiments, it is possible to appropriately provide information for smooth communication with a communication partner online. [Brief explanation of the drawing]

[0011] [Figure 1] This is an overall diagram showing the configuration of a support system using a user terminal according to the first embodiment of the present invention. [Figure 2] This is a flowchart showing the process executed by a user terminal according to the first embodiment of the present invention. [Figure 3] This is an overall diagram showing the configuration of a support system using a user terminal according to a second embodiment of the present invention. [Figure 4] This is a flowchart showing the process executed by a user terminal according to a second embodiment of the present invention. [Figure 5] This is an explanatory diagram showing the difference between the average value of the first index value and the average value of the second index value acquired by the user terminal according to the second embodiment of the present invention. [Modes for carrying out the invention]

[0012] The following describes a support system constructed using a user terminal as a communication support device according to an embodiment of the present invention, with reference to the drawings. This support system is a system for multiple users to communicate smoothly in a virtual space constructed online via a communication network.

[0013] 《First Embodiment》 <Configuration of the support system according to the first embodiment> Figure 1 is a block diagram showing the configuration of the support system 1 according to this embodiment. The support system 1 is configured with user terminals 10A, 10B, 10C, and 10D connected to each other via a wide-area communication network NW. A virtual office, which is a virtual space, is constructed online via the wide-area communication network NW, and user terminals 10A, 10B, 10C, and 10D are pre-registered as terminals for users who will use this virtual office.

[0014] User terminal 10A is installed in the first physical space where user UA is located. User terminal 10B is a terminal used by user UB and is installed in the second physical space where user UB is located. User terminal 10C is a terminal used by user UC and is installed in the third physical space where user UC is located. User terminal 10D is a terminal used by user UD and is installed in the fourth physical space where user UD is located. The first physical space, the second physical space, the third physical space, and the fourth physical space are all different physical spaces.

[0015] In this embodiment, we will describe a case where four user terminals are connected to a wide-area communication network NW, but the number is not limited to this; two, three, or five or more user terminals may be connected, as long as there are two or more.

[0016] The user terminal 10A is composed of an information processing terminal such as a smart device, personal computer, or HMD (Head Mounted Display), and includes an input / output unit 11, an imaging unit 12, a sound collection unit 13, a speaker unit 14, a communication unit 15, a first storage unit 16, and a first control unit 17.

[0017] The input / output unit 11 is configured, for example, as a touch panel, and inputs operation information from the user UA (hereinafter referred to as "local terminal user") who is the user of the local user terminal 10A, and displays display information sent from the first control unit 17, as will be described later.

[0018] The imaging unit 12 captures the image of the user of the self-terminal UA. The sound collection unit 13 collects the voice of the user of the self-terminal UA. The speaker unit 14 outputs the voice information sent from the first control unit 17 as described below. The communication unit 15 performs information communication with the user terminals 10B, 10C, and 10D via the wide area communication network NW. In the present embodiment, the input / output unit 11, the imaging unit 12, the sound collection unit 13, and the speaker unit 14 are configured to be built in the self-terminal 10A, but these components may be externally connected devices.

[0019] The first storage unit 16 is composed of, for example, a RAM (Random Access Memory) or a ROM (Read Only Memory). The first storage unit 16 has a relationship information storage unit 161 and a threshold storage unit 162.

[0020] The relationship information storage unit 161 stores the intimacy that indicates the depth of the relationship between the user of the self-terminal UA and each of the other terminal users UB, UC, and UD who are the second users, which is calculated by the intimacy calculation unit 172 described below.

[0021] Specifically, in the relationship information storage unit 161, an intimacy of "3" is stored for the other terminal user UB (for example, the "boss in the same department" as seen from the user of the self-terminal UA) who has a deep relationship with the user of the self-terminal UA, and an intimacy of "2" is stored for the other terminal user UC (for example, the "colleague in the same department" as seen from the user of the self-terminal UA) who has a relatively deep relationship with the user of the self-terminal UA, and an intimacy of "1" is stored for the other terminal user UD (for example, the "colleague in another department" as seen from the user of the self-terminal UA) who has a weak relationship with the user of the self-terminal UA.

[0022] The threshold storage unit 162 stores a first threshold value regarding the first index value indicating the emotion of the user of the self-terminal UA, which is set for each intimacy between the user of the self-terminal UA and a user (hereinafter referred to as the "other terminal user") who is the user of another user terminal.

[0023] In this embodiment, the first index value is expressed with a normal state set to 0, and ranges from 1 to 5 for "unpleasantness," indicating the degree of negative emotions, and from -1 to -5 for "pleasantness," indicating the degree of positive emotions. The higher the value of this first index, the higher the degree of negative emotions, i.e., unpleasant emotions, and the lower the value, the higher the degree of positive emotions such as joy, satisfaction, and pleasantness of the user agent (UA) of the terminal. In this embodiment, only two classifications, pleasant and unpleasant, are used, but it is also possible to classify emotions into multiple categories and calculate the first index value for each classified emotion. For example, emotions such as joy, surprise, sadness, and anger can be quantified individually to perform a more detailed emotion analysis. For example, using Plutchik's Wheel of Emotions, emotions can be classified into eight basic emotions (joy, trust, fear, surprise, sadness, disgust, anger, and anticipation) and applied emotions formed by combinations of two adjacent emotions, and then applied to one of the patterns on the Wheel of Emotions.

[0024] The first threshold is set to a higher value the higher the intimacy with the user UA of the device. Specifically, a first threshold of "4" is set for intimacy level "3", a first threshold of "3" is set for intimacy level "2", and a first threshold of "2" is set for intimacy level "1".

[0025] The first control unit 17 is, for example, a CPU (Central Processing Unit) of a general-purpose microcomputer, and by installing and executing a program for communication support applications, it constitutes one or more information processing units as described below. Alternatively, the first control unit 17 may be a processor such as a GPU (Graphics Processing Unit), FPGA (Field-Programmable Gate Array), or quantum processor (quantum computer control chip). The first control unit 17 may load a program from the first storage unit 16 into memory and execute the program.

[0026] The first control unit 17 includes a communication information processing unit 171, a closeness calculation unit 172, a relationship information acquisition unit 173, a threshold acquisition unit 174, a first index value acquisition unit 175, and a first notification unit 176. The program, although not shown in the figures, is a computer program that implements the processing of the information processing method according to this disclosure. As a result, the first control unit 17 realizes the functions of the communication information processing unit 171, the threshold acquisition unit 174, the first index value acquisition unit 175, and the first notification unit 176. Some or all of the communication information processing unit 171, the threshold acquisition unit 174, the first index value acquisition unit 175, and the first notification unit 176 may be realized by hardware other than the first control unit 17, for example, by a general-purpose or dedicated circuit implemented in a semiconductor device.

[0027] The communication information processing unit 171 processes communication information to be sent and received with other user terminals 20B, 20C, and 20D.

[0028] The affinity calculation unit 172 calculates the affinity between the local terminal user UA and the other terminal users UB, UC, and UD, and stores this information in the relationship information storage unit 161. The affinity calculation unit 172 calculates an affinity value higher the deeper the relationship between the other terminal users UB, UC, and UD and the local terminal user UA.

[0029] The affinity calculation unit 172 may calculate the affinity for each other terminal user through manual operation by the local terminal user UA, or it may automatically calculate the affinity based on the communication history information between the local terminal user UA and the other terminal users.

[0030] For example, if the intimacy calculation unit 172 detects that user terminal 10A and user terminal 10B have mutually acknowledged that user UA and user UB of the other terminal have a favorable relationship, it may calculate the intimacy level between user UA and user UB of the other terminal as "3 (deep)".

[0031] The intimacy calculation unit 172 may also calculate the intimacy level between user UA and user UB on another terminal based on the number of times message information has been exchanged between user terminal 10A and user terminal 10B, the number of times one-on-one voice calls have been made, the number of times voice calls have been made as members of the same group, etc.

[0032] The relationship information acquisition unit 173 acquires the level of intimacy between the local terminal user UA and other terminal users from the relationship information storage unit 161. For example, this other terminal user refers to a user who is currently in a state of interaction with the local terminal user UA. The state of interaction may include the state of communication during activities such as sending and receiving messages with other terminal users (sending and receiving text messages, images, videos, voice messages, etc.), voice calls (real-time voice communication), video calls (real-time video and audio communication), file sharing (sharing documents, images, videos, audio files, etc.), and collaborative work (joint editing of online documents, use of project management tools, etc.). As a result, the relationship information acquisition unit 173 acquires the level of intimacy with other terminal users with whom it is interacting through these communication activities from the relationship information storage unit 161.

[0033] The threshold acquisition unit 174 acquires a first threshold corresponding to the intimacy level acquired by the relationship information acquisition unit 173 from the threshold storage unit 162.

[0034] The first indicator value acquisition unit 175 acquires a first indicator value that indicates the emotions of the user UA of the terminal. This first indicator value is calculated based on biometric information such as the heart rate, pulse, respiration, sweat, facial expression, and brain waves of the user UA of the terminal, which are measured by a wearable device worn by the user UA of the terminal.

[0035] For example, the first index value acquisition unit 175 analyzes the captured video of the user UA of the terminal to acquire a first index value indicating the emotion of the user UA. Specifically, the first index value acquisition unit 175 acquires the first index value using a trained model for facial expression recognition, which is obtained by pattern matching or machine learning (e.g., deep learning) of image information including the face of the user UA of the terminal. In this process, the first index value acquisition unit 175 uses the trained model for facial expression recognition to extract feature points for the contour of the user's face or each part of the face in the image, and by tracking the movement of the feature points, recognizes the facial expression of the user UA of the terminal and acquires a first index value indicating the emotion of the user UA of the terminal. Based on the captured image, the first index value acquisition unit 175 analyzes which of a number of pre-set emotion types the user UA's emotion belongs to, based on the user UA's facial expression (corners of the mouth, etc.). Alternatively, the first index value acquisition unit 175 may simply acquire a first index value classified as either "pleasant" or "unpleasant".

[0036] The first notification unit 176 transmits notification information to other user terminals in communication when the first index value acquired by the first index value acquisition unit 175 exceeds the first threshold acquired by the threshold acquisition unit 174.

[0037] User terminals 10B, 10C, and 10D have the same configuration as user terminal 10A.

[0038] <Operation of the support system according to the first embodiment> The following describes the processes executed by the user terminal 10A used by user UA, the first user in this embodiment, while the support system 1 is running.

[0039] Figure 2 is a flowchart showing the processes executed by user terminals 10A, 10C, and 10D. Here, as an example, we will explain the processes executed by user terminal 10A when user UA on its own terminal and user UB on another terminal communicate using the communication between user terminal 10A and user terminal 10B.

[0040] First, on user terminal 10A, the local user UA activates the communication support application by operating the input / output unit 11 and establishes communication with user terminal 10B via the communication unit 15 (YES in S1). Once communication between user terminal 10A and user terminal 10B is established, the relationship information acquisition unit 173 acquires the intimacy level between the local user UA and the other terminal user UB from the relationship information storage unit 161 (S2).

[0041] The threshold acquisition unit 174 acquires a first threshold corresponding to the intimacy level acquired by the relationship information acquisition unit 173 from the threshold storage unit 162 (S3). Specifically, it acquires a first threshold of "4" corresponding to the intimacy level of "3" of the other terminal user UB as seen from the perspective of the local terminal user UA.

[0042] Furthermore, once communication is established between user terminal 10A and user terminal 10B, the imaging unit 12 begins to capture images of its own terminal user UA, and the sound acquisition unit 13 begins to acquire the voice of its own terminal user UA. The communication information processing unit 171 acquires the image information captured by the imaging unit 12 and the voice information acquired by the sound acquisition unit 13, and transmits them to user terminal 10B via the communication unit 15.

[0043] Furthermore, the communication information processing unit 171 acquires imaging information and audio information from another terminal user UB transmitted from user terminal 10B via the communication unit 15, and outputs the acquired imaging information from the input / output unit 11 and the audio information from the speaker unit 14 (S4). In this way, imaging information and audio information are transmitted and received between user terminal 10A and user terminal 10B, and output at the receiving terminal, enabling the local terminal user UA and the other terminal user UB to communicate within the virtual office. The output referred to here is exemplified by the display of imaging information on the input / output unit 11 and the output of audio from the speaker unit 14, but may also include physical output by a printer or the like performed by the other terminal user UB, data output via a storage medium or the like, and output to other devices via data communication.

[0044] When communication between the local terminal user UA and the other terminal user UB begins, the first indicator value acquisition unit 175 acquires a first indicator value indicating the local terminal user UA's emotions at predetermined time intervals (S5). For example, the predetermined time interval is set to a range of 1 to 10 seconds.

[0045] When the first notification unit 176 detects that the first index value acquired by the first index value acquisition unit 175 has exceeded the first threshold acquired by the threshold acquisition unit 174 (YES in S6), it transmits notification information to the user terminal 10B that is communicating (S6). This notification information is information that the user UA of the terminal is experiencing negative (unpleasant) emotions. For example, if the first index value acquired by the first index value acquisition unit 175 is a value of "5" indicating "unpleasantness," this value exceeds the first threshold of "4," so the first notification unit 176 notifies the user UB of the other terminal. The notification method may be an indirect expression such as "User UA seems to have something on their mind" in a text message, or it may be a method that only notifies that the user is experiencing unpleasantness using tactile stimulation from a wearable device.

[0046] User terminal 10B outputs notification information transmitted from user terminal 10A from its input / output unit. In this way, by performing notification processing to other terminal users when the user UA of its own terminal experiences negative emotions, other terminal users UB can recognize the unpleasant emotions of the user UA of its own terminal and adjust their communication style, thereby preventing the deterioration of relationships between users.

[0047] In step S5, if the first index value acquired by the first index value acquisition unit 175 does not exceed the first threshold acquired by the threshold acquisition unit 174 ("NO" in S5), the process returns to step S4.

[0048] In step S5 described above, the first notification unit 176 may measure the number of times or duration that the first indicator value exceeds the first threshold after communication between its own terminal user UA and the other terminal user UB has started, and send notification information to the user terminal 10B when the measured value exceeds a predetermined value. By performing this processing, it is possible to prevent excessive notification processing to the other terminal user UB.

[0049] Furthermore, while the above-described embodiment explained the use of the support system 1 when a local terminal user UA and another terminal user UB communicate one-to-one, the support system 1 may also be used when a local terminal user UA conducts online meetings, etc., with multiple other terminal users UB, UC, and UD.

[0050] In this case, the processing described in steps S1 to S6 above may be performed between user terminal 10A and user terminal 10B, between user terminal 10A and user terminal 10C, and between user terminal 10A and user terminal 10D, respectively, or between user terminal 10A and one user terminal selected from among the multiple user terminals 10B, 10C, and 10D.

[0051] When performing the above-described process between user terminal 10A and one selected user terminal, the user terminal used by the user with the closest relationship to the current terminal user UA is selected as the target of the process.

[0052] According to the first embodiment described above, the user terminal as a communication support device includes: a first index value acquisition unit that acquires a first index value indicating the emotions of the user of the terminal using the device; a threshold acquisition unit that acquires a threshold related to the first index value, which is set based on the depth of the relationship between the user of the terminal and other terminal users with whom the user of the terminal is in a state of interaction; and a notification unit that outputs notification information to other terminal users when the first index value acquired by the first index value acquisition unit exceeds the threshold acquired by the threshold acquisition unit.

[0053] This allows the system to notify other users when a user on their own device experiences negative emotions, enabling them to recognize the user's unpleasant feelings and adjust their communication accordingly, thus preventing a deterioration of relationships between users.

[0054] The threshold further includes a closeness calculation unit that calculates the closeness between the first user and the second user, and the threshold may be set based on the closeness calculated by the closeness calculation unit. This makes it possible to prevent notification processing to other terminal users when the user has slightly negative feelings during a conversation with another terminal user with whom they have a close relationship, thereby appropriately protecting the privacy of the user of the terminal and preventing excessive notification processing to other terminal users.

[0055] Furthermore, the notification unit may output notification information when the number of times or the duration of the first indicator value exceeding a threshold exceeds a predetermined value. This makes it possible to more accurately prevent excessive notification processing to other terminal users.

[0056] 《Second Embodiment》 <Configuration of the support system according to the second embodiment> Figure 3 is a block diagram showing the configuration of the support system 2 according to this embodiment. Similar to the support system 1 described in the first embodiment, the support system 2 is configured such that user terminals 20A, 20B, 20C, and 20D, which are pre-registered in the virtual office, are connected to each other via a wide-area communication network NW.

[0057] The user terminal 20A includes an input / output unit 21, a communication unit 25, a sound pickup unit 23, a speaker unit 24, a second storage unit 26, and a second control unit 27. A detailed explanation of the parts of these functional units that have the same functions as those described in the first embodiment will be omitted.

[0058] The second storage unit 26 has a relationship information storage unit 261. Similar to the relationship information storage unit 161 described in the first embodiment, the relationship information storage unit 261 stores information indicating the relationship between the local terminal user UA and other terminal users for each combination.

[0059] The second control unit 27 includes a communication information processing unit 271, a closeness calculation unit 272, a specification unit 273, a first index value acquisition unit 274, a second index value acquisition unit 275, and a second notification unit 276.

[0060] The communication information processing unit 271 processes communication information to be sent and received with other user terminals 20B, 20C, and 20D. The intimacy calculation unit 272 calculates the intimacy between the local terminal user UA and the other terminal users UB, UC, and UD, similar to the intimacy calculation unit 172 described in the first embodiment, and stores it in the relationship information storage unit 261. The identification unit 273 identifies other terminal users whose relationship with the local terminal user UA is above a predetermined value, based on the information stored in the relationship information storage unit 261.

[0061] The first indicator value acquisition unit 274 acquires a first indicator value indicating the emotions of its own terminal user UA and sends it to the second notification unit 276, and also transmits it to other user terminals 20B, 20C, and 20D via the communication unit 25. The second indicator value acquisition unit 275 acquires second indicator values ​​indicating the emotions of other terminal users UB, UC, and UD, respectively, which are transmitted from the other user terminals 20B, 20C, and 20D.

[0062] The second notification unit 276, based on the average of the first indicator value acquired by the first indicator value acquisition unit 274 and the second indicator value acquired by the second indicator value acquisition unit 275, determines that there is a discrepancy in sentiment between the user UA of its own terminal and other terminal users UB, UC, and UD, and then outputs notification information to the user identified by the identification unit 273.

[0063] <Operation of the support system according to the second embodiment> The following describes the processes executed by the user terminal 20A used by user UA, the first user in this embodiment, while the support system 2 is running.

[0064] Figure 4 is a flowchart showing the processes executed by user terminals 20A, 20C, and 20D. As an example, this section describes the processes executed by user terminal 20A when users UA, UB, UC, and UD conduct an online meeting using communication between user terminals 20A, 20B, 20C, and 20D.

[0065] First, on user terminal 20A, user UA operates the input / output unit 21 to launch a communication support application and establishes communication with user terminals 10B, 10C, and 10D via the communication unit 25 (YES in S11).

[0066] Once communication is established between user terminals 20A, 20B, 20C, and 20D, the imaging unit 22 begins to capture images of its own terminal user UA, and the sound acquisition unit 23 begins to acquire the voice of its own terminal user UA. The communication information processing unit 271 acquires the imaging information captured by the imaging unit 12 and the voice information acquired by the sound acquisition unit 13, and transmits them to user terminals 10B, 10C, and 10D via the communication unit 25.

[0067] Furthermore, the communication information processing unit 271 acquires imaging information and audio information from other terminal users UB, UC, and UD transmitted from user terminals 20B, 20C, and 20D via the communication unit 25, and outputs the acquired imaging information from the input / output unit 11 and the audio information from the speaker unit 14 (S12). In this way, imaging information and audio information are transmitted and received between user terminal 20A and user terminals 20B, 20C, and 20D, and output at the receiving terminal, enabling user UA and other terminal users UB, UC, and UD to communicate within the virtual office.

[0068] When communication between the local terminal user UA and other terminal users UB, UC, and UD begins, the first indicator value acquisition unit 274 acquires a first indicator value P1 indicating the mood of the local terminal user UA at predetermined time intervals (S13). The first indicator value acquisition unit 274 sends the acquired first indicator value P1 to the second notification unit 276 and also transmits it to other user terminals 20B, 20C, and 20D via the communication unit 25.

[0069] Furthermore, the second indicator value acquisition unit 275 acquires second indicator values ​​from other user terminals 20B, 20C, and 20D, which represent the emotions of the other terminal users UB, UC, and UD (S14). The second indicator value acquisition unit 275 calculates the average value P2 of the second indicator values ​​acquired from the other terminal users UB, UC, and UD.

[0070] The second notification unit 276 determines whether or not there is a discrepancy in sentiment between the local terminal user UA and other terminal users UB, UC, and UD, based on the first index value P1 acquired by the first index value acquisition unit 274 and the average value P2 of the second index value calculated by the second index value acquisition unit 275 (S15).

[0071] Specifically, the second notification unit 276 determines that there is a discrepancy in emotions between the local terminal user UA and other terminal users UB, UC, and UD when the conditions are met: "The first index value P1 acquired by the first index value acquisition unit 274 is a value indicating 'unpleasant,' the average value P2 of the second index values ​​acquired by the second index value acquisition unit 275 is a value indicating 'pleasant,' and the difference between the first index value P1 and the average value P2 of the second index values ​​is greater than or equal to a predetermined second threshold."

[0072] For example, as shown in Figure 5, if the first index value P1 is a value of "4" indicating "unpleasantness," and the average value of the second index P2 is a value of "-3" indicating "pleasantness," then the difference between them is "7." Here, if the second threshold is set to "5," the second notification unit 276 determines that the difference "7" is greater than or equal to the second threshold "5," and that there is a discrepancy in emotions between the user UA of the local terminal and the users UB, UC, and UD of other terminals.

[0073] Furthermore, the second information unit 276 may measure the number of times or duration that the above conditions are met since the start of the online meeting, and may determine that there is a discrepancy in emotions between the user UA of its own terminal and the users UB, UC, and UD of other terminals when the measured value exceeds a predetermined value.

[0074] When the second notification unit 276 determines that there is a discrepancy in feelings between the local terminal user UA and other terminal users UB, UC, and UD, it requests the identification unit 273 to identify other terminal users whose relationship with the local terminal user UA is above a predetermined value, for example, "3" or higher.

[0075] The identification unit 273 identifies other terminal users UB whose relationship with the local terminal user UA is "3" or higher based on the information stored in the relationship information storage unit 261 (S16), and sends the identification information of the identified other terminal user UB to the second notification unit 276. The second notification unit 276 transmits notification information to the user terminal 10B of the other terminal user UB identified by the identification unit 273 (S17). This notification information is information that the local terminal user UA has different feelings from other users UB, UC, and UD.

[0076] Here, the identification unit 273 may identify one other terminal user with the highest relationship value among the other terminal users, send the identification information of the identified other terminal user to the second notification unit 276, and the second notification unit 276 may send notification information to the user terminal of the other terminal user corresponding to this identification information.

[0077] User terminal 10B outputs the notification information acquired from user terminal 10A from its input / output unit.

[0078] Here, "a discrepancy in emotions between the user A of the local terminal and the users B, C, and D of other terminals" refers to situations such as when only the user A of the local terminal has a different opinion from several other terminal users, or when only the user A of the local terminal is unable to understand the content of the meeting and is feeling anxious. In such cases, informing other terminal users who have a close relationship with the user A of the local terminal about the user A's unpleasant feelings allows those users to provide appropriate support to the user A of the local terminal. This can reduce the pressure to conform and ensure that the content of the meeting is reliably communicated to everyone.

[0079] Furthermore, the second notification unit 276 may change the content of the notification information depending on the relationship between the user UA and the other terminal user to whom the notification information is output. For example, if the other terminal user UB to whom the notification is output is a close colleague of the user UA, the notification information "They are in trouble, please help them" may be output. If the other terminal user UB is a superior to the user UA in terms of work, such as a boss, the notification information "Please follow up" may be output.

[0080] Furthermore, in the above-described embodiment, notification information may be output to the user terminal 10A if the user UA's emotions differ from those of multiple other terminal users. This allows the user UA to recognize that they are the only one whose emotions differ from those of other users and to take action to resolve the issue.

[0081] In the embodiment described above, the case in step S14 in which the second index value acquisition unit 275 calculates the average value P2 of the second index values ​​acquired from other terminal users UB, UC, and UD was explained, but instead of the average value, the median value may be calculated. In this case, in step S15, the second notification unit 276 determines whether the difference between the first index value P1 and the median value of the second index values ​​is greater than or equal to the second threshold.

[0082] According to the second embodiment described above, the user terminal as a communication support device includes a first indicator value acquisition unit that acquires a first indicator value indicating the emotions of the first user, the user of the terminal; a second indicator value acquisition unit that acquires a second indicator value indicating the emotions of the second user, the user of another terminal; a relationship information acquisition unit that identifies the user of another terminal whose relationship with the first user is greater than or equal to a predetermined value; and a notification unit that outputs notification information to the identified user of another terminal when the difference between the first indicator value and the second indicator value is greater than or equal to a predetermined threshold.

[0083] In this way, when a user's emotions differ from those of multiple other users, this information can be shared only with other users who have a close relationship with the original user. These users can then recognize the difference and provide support to the original user within the online meeting. This reduces the risk of individual users' opinions being suppressed and creates an environment where everyone participating in the online meeting can freely express their opinions.

[0084] Furthermore, the above-described process enables rapid detection of changes in the emotional state of the user of the device, and by notifying appropriate other users identified based on their relationship with the user of the device, it becomes possible to detect and resolve problems that occur during online meetings at an early stage.

[0085] Other users who receive notification information have a close and trusting relationship with the user on the same device, enabling them to provide prompt and sincere support. Furthermore, these users understand the progress of projects related to online meetings and the individual users' work situations, allowing them to provide specific and effective support.

[0086] Furthermore, information regarding the emotions of the user of one's own device is personal information, and disclosing it to all other users of other devices could constitute a violation of privacy. However, by disclosing it only to other users of other devices who have a close relationship with the user of one's own device, it is possible to share necessary information while protecting privacy.

[0087] Furthermore, while informing all other users of a device about the emotions of the user of the device could potentially create excessive pressure for conformity on either the user or other users, informing only other users who have a close relationship with the user of the device allows for necessary support to be provided to the user while preventing unnecessary pressure from arising.

[0088] Furthermore, the system includes a closeness calculation unit that calculates the level of closeness between the first user and the second user, and the threshold may be set based on the closeness level calculated by the closeness calculation unit. This makes it possible to avoid notification processing even if there is a slight emotional divergence between the user and other terminal users with whom they have a close relationship, thereby appropriately protecting the privacy of the user on their own terminal and preventing excessive notification processing to other terminal users.

[0089] Furthermore, the notification unit may output notification information to the third user when the number of times or the duration of the difference between the first indicator value and the second indicator value exceeds a predetermined threshold exceeds a predetermined value. This makes it possible to more accurately prevent excessive notification processing to other terminal users.

[0090] In the embodiments described above, the communication support device is configured as a user terminal such as a personal computer or smart device, and the case described is one in which multiple users in different physical spaces communicate using support systems 1 and 2 in a virtual office built online. However, the invention is not limited to this, and support systems 1 and 2 may also be used when the communication support device is configured as AR (Augmented Reality) glasses, and multiple users wear AR glasses and communicate while recognizing each other's virtual images.

[0091] Furthermore, by programming the functional configuration of the user terminal 10A or 20A described above and incorporating it into a computer, it is possible to construct a communication support program that enables the computer to function as a communication support device. [Explanation of symbols]

[0092] 1, 2 Support Systems 10A, 10B, 10C, 10D, 20A, 20B, 20C, 20D User Terminals 11, 21 Input / output section 12, 22 Imaging Unit 13, 23 Sound collection section 14, 24 Speaker section 15, 25 Communications Department 16 1st memory section 17. First Control Unit 26 2nd memory section 27 Second Control Unit 161 Relationship Information Storage Unit 162 Threshold memory unit 171, 271 Communication Information Processing Unit 172, 272 Intimacy calculation part 173 Relationship Information Acquisition Unit 174 Threshold acquisition unit 175, 274 First indicator value acquisition unit 176 First News Department 261 Relationship Information Storage Unit 273 Specific part 275 Second Indicator Value Acquisition Unit 276 Second News Department

Claims

1. A first indicator value acquisition unit that acquires a first indicator value that indicates the emotions of the first user, A second indicator value acquisition unit that acquires a second indicator value that indicates the emotions of a second user, A identifying unit that identifies a third user having a predetermined relationship with the first user, A communication support device comprising: a notification unit that outputs notification information to the third user when the difference between the first indicator value and the second indicator value is greater than or equal to a predetermined threshold;

2. The system further includes a closeness calculation unit that calculates the closeness between the first user and the second user, The communication support device according to claim 1, wherein the threshold is set based on the intimacy level calculated by the intimacy level calculation unit.

3. The communication support device according to claim 1, wherein the notification unit outputs notification information to the third user when the number of times or duration for which the difference between the first indicator value and the second indicator value exceeds a predetermined threshold value exceeds a predetermined value.

4. Communication support device, Obtain the first metric value that indicates the emotion of the first user, We obtain a second indicator value that shows the emotions of the second user, Identify a third user who has a predetermined relationship with the first user, A communication support method that causes notification information to be output to the third user when the difference between the first indicator value and the second indicator value is greater than or equal to a predetermined threshold.

5. In a communication support device, A function to obtain a first indicator value that shows the first user's emotions, A function to obtain a second indicator value that shows the emotions of the second user, A function to identify a third user who has a predetermined relationship with the first user, A communication support program for implementing a function that outputs notification information to the third user when the difference between the first indicator value and the second indicator value is greater than or equal to a predetermined threshold.

Images