Communication management device and communication management method

Abstract

In the present invention, a management unit manages connection between user terminals in an online session in which a plurality of users participate using user terminals held by the respective users. A change unit changes the length of the remaining time of the online session on the basis of the activity of communication between the plurality of users in the online session.

Description

Communication management device and communication management method

[0001] The present disclosure relates to a communication management device and a communication management method.

[0002] Chatting using online communication tools is an important activity that can help build relationships between users, reduce stress, and in business situations, it can be used as an icebreaker before getting into the main topic.However, users may feel awkward or bored when the conversation dies out due to lack of topics, or when the chat continues for a long time even though there is not much to talk about.

[0003] For example, the communication device disclosed in Patent Document 1 below has a short call mode in which the remaining time is set in advance, in addition to a normal call mode. A user can select either the normal call mode or the short call mode before starting a call. If the short call mode is selected, the user can extend the remaining time of the call as needed. This technology eliminates the need to set a call time longer than necessary for the topic, and communication ends before the topic runs out and the conversation dies out, thereby avoiding awkwardness and boredom.

[0004] JP 2016-184979 A

[0005] However, with the above-mentioned conventional technology, if the topic runs out and the conversation comes to a halt before the call time in short call mode expires, it is not possible to avoid the awkwardness and boredom that continues until the remaining time has elapsed.

[0006] The present disclosure aims to appropriately set the remaining time of an online session using an online communication tool.

[0007] The communication management device of the present disclosure includes a management unit that manages connections between user terminals in an online session in which multiple users participate using their own user terminals, and a modification unit that changes the length of the remaining time of the online session based on the level of communication activity between the multiple users in the online session.

[0008] The communication management method of the present disclosure involves using a computer to obtain the level of communication activity between multiple users in an online session in which the users participate using their own user terminals, and changing the remaining time of the online session based on the level of activity.

[0009] According to the present disclosure, the remaining time of an online session using an online communication tool is appropriately set.

[0010] 1 is a block diagram showing the configuration of the communication system 1. FIG. 2 is a block diagram showing the configuration of a user terminal 20-1. FIG. 3 is a block diagram showing the configuration of a communication management device 10. FIG. 4 is a diagram showing a schematic diagram of an increase in the remaining time length TR based on activity level D. FIG. 5 is a diagram showing a schematic diagram of an increase or decrease in the amount of change TH based on the remaining time length TR. FIG. 6 is a diagram showing a schematic diagram of an increase or decrease in the amount of change TH based on the remaining time length TR. FIG. 7 is a diagram showing a schematic diagram of a change in the remaining time length TR based on the relative value of activity level D. FIG. 8 is a diagram showing a schematic diagram of a change in the remaining time length TR based on the relative value of activity level D. FIG. 9 is a schematic diagram showing a display screen G1 during a web meeting. FIG. 10 is a flowchart showing the operation of the processing device 103 of the communication management device 10.

[0011] [Embodiment] [System Configuration] Fig. 1 is a block diagram showing the configuration of a communication system 1. The communication system 1 according to the embodiment includes a communication management device 10 and multiple user terminals 20 (20-1 to 20-3). The communication management device 10 and the user terminals 20-1 to 20-3 are connected to each other via a network N. The network N may be a WAN (Wide Area Network) or a LAN (Local Area Network).

[0012] The user terminals 20-1 to 20-3 are information processing terminals. More specifically, the user terminals 20 may be, for example, personal computers, tablets, or smartphones. The user terminals 20 are held by the users U (U-1 to U-3), respectively. In this embodiment, the communication system 1 includes three user terminals 20-1 to 20-3, but the number of user terminals 20 may be two, four, or more.

[0013] The communication management device 10 manages communication between users U-1 to U-3 using user terminals 20-1 to 20-3. More specifically, the communication management device 10 realizes online sessions using the user terminals 20-1 to 20-3. The online session may be, for example, a web meeting in which images (typically facial images of users U-1 to U-3) and audio are exchanged between the user terminals 20-1 to 20-3. The online session may also be a web seminar in which a large number of listeners listen to a presentation by one or a small number of presenters. The online session may also be a voice call using a voice call application or the like. However, the online session is not limited to these, and may be any communication realized by the network N, the user terminals 20-1 to 20-3, and the communication management device 10. In this embodiment, the online session is assumed to be a web meeting.

[0014] [User Terminals 20-1 to 20-3] Figure 2 is a block diagram showing the configuration of user terminal 20-1. Since the configurations of user terminals 20-1 to 20-3 are substantially the same, the following description will use user terminal 20-1 as an example. User terminal 20-1 is carried by user U-1. User terminal 20-1 includes a display 201, an input device 202, a microphone 203, a speaker 204, a camera 205, a communication device 206, a storage device 207, a processing device 208, and a bus 220 that interconnects these devices.

[0015] The display 201 is a device that displays information to the outside (for example, various display panels such as a liquid crystal display panel or an organic EL display panel). The input device 202 is a device that accepts input from the outside (for example, a keyboard, a mouse, a microphone, a switch, a button, or a sensor). The display 201 and the input device 202 may be integrated into one device (for example, a touch panel).

[0016] The microphone 203 picks up surrounding sounds and generates audio data. The audio picked up by the microphone 203 is, for example, the speech of user U-1 when he converses with other users U-2 to U-3 in a web meeting. The speaker 204 plays back audio data and outputs audio corresponding to the audio data. The audio data played back by the speaker 204 indicates, for example, the speech of users U-2 to U-3 when user U-1 converses with other users U-2 to U-3 in a web meeting. The microphone 203 and speaker 204 may not be included in the user terminal 20-1 and may be separate from the user terminal 20-1. In this case, the user terminal 20-1 is connected to the microphone 203 and speaker 204 by an interface (not shown).

[0017] The camera 205 generates an image of the space around the user terminal 20-1. The image captured by the camera 205 is, for example, an image of the user U-1, which is used when the user U-1 holds a web meeting with other users U-2 to U-3. The camera 205 has an imaging optical system and an imaging element. The imaging optical system is an optical system including at least one imaging lens. The imaging optical system may have various optical elements such as a prism, or may have a zoom lens or a focus lens. The imaging element is, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor.

[0018] The communication device 206 has an interface that can be connected to the network N, and communicates with other devices connected to the network N using wireless or wired communication.

[0019] The storage device 207 is a recording medium readable by the processing device 208. The storage device 207 includes, for example, a nonvolatile memory and a volatile memory. The nonvolatile memory is, for example, a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), and an EEPROM (Electrically Erasable Programmable Read Only Memory). The volatile memory is, for example, a RAM (Random Access Memory). The storage device 207 stores a program PG2. The program PG2 is a program for operating the user terminal 20-1.

[0020] The processing unit 208 includes one or more central processing units (CPUs). The one or more CPUs are examples of one or more processors. Each of the processors and CPUs is an example of a computer.

[0021] The processing device 208 reads the program PG2 from the storage device 207. By executing the program PG2, the processing device 208 functions as an input management unit 211, an output management unit 212, and a communication management unit 213. At least one of the input management unit 211, the output management unit 212, and the communication management unit 213 may be configured by a circuit such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array).

[0022] The input management unit 211 manages information input to the user terminal 20-1. Information is input to the user terminal 20-1 using the input device 202, microphone 203, and camera 205, for example.

[0023] The output management unit 212 manages the output of information from the user terminal 20-1. The output of information from the user terminal 20-1 is performed using the display 201 and speaker 204, for example.

[0024] The communication management unit 213 manages the communication between the user terminal 20-1 and the communication management device 10. The communication between the user terminal 20-1 and the communication management device 10 is performed using the communication device 206.

[0025] In this embodiment, a description will be given of the processing related to an online session (web meeting) by the input management unit 211, the output management unit 212, and the communication management unit 213. Typical processing related to a web meeting includes, for example, [1] booking a web meeting, and [2] participating in a web meeting.

[0026] [1] Web Meeting Reservation The output management unit 212 displays an icon corresponding to an application for web meetings (hereinafter referred to as the "web meeting app") on the display 201 (for example, a home screen, etc.). When the user U-1 presses the icon, the web meeting app starts. The output management unit 212 displays the user interface of the web meeting app on the display 201.

[0027] When user U-1 selects to reserve a web meeting in the user interface, the input management unit 211 accepts the selection. The communication management unit 213 sends a web meeting reservation request to the communication management device 10. The communication management unit 213 receives image data for displaying the web meeting reservation screen from the communication management device 10. The output management unit 212 displays the web meeting reservation screen on the display 201. The web meeting reservation screen has fields for inputting settings such as the start date and time of the web meeting, the meeting ID, and whether the camera 205 is on or off. The input management unit 211 accepts input from user U-1 to the web meeting reservation screen. The communication management unit 213 sends the information entered on the web meeting reservation screen to the communication management device 10.

[0028] When input into the reservation screen is complete, the communication management unit 213 receives a URL (Uniform Resource Locator) for participating in the reserved web meeting from the communication management device 10. The output management unit 212 displays the URL on the display 201. User U-1 then uses a communication tool such as email to send the URL to the user terminals 20-2 to 20-3 of other users U-2 to U-3 who are scheduled to participate in the web meeting. In this case, user U-1 becomes the host of the web meeting.

[0029] Note that online sessions may be "held immediately" rather than "reserved." In this case, the process is generally the same as above.

[0030] [2] Participating in a web meeting Based on the operation of user U-1, the output management unit 212 displays an email containing a URL for joining the web meeting on the display 201. When user U-1 clicks the URL, the web meeting application is launched. The communication management unit 213 sends a request to join the web meeting to the communication management device 10.

[0031] When the communication management device 10 permits user U-1 to participate in the web meeting, the input management unit 211 starts collecting the user U-1's speech using the microphone 203 and capturing an image of user U-1 using the camera 205. Depending on the settings of user U-1, at least one of the microphone 203 and the camera 205 may be turned off. The communication management unit 213 sends audio data indicating the user U-1's speech and image data of the captured image of user U-1 to the communication management device 10.

[0032] The communication management unit 213 also receives audio data indicating the speech of the other users U-2 to U-3 participating in the web meeting and image data of the other users U-2 to U-3 from the communication management device 10. The output management unit 212 outputs the speech of the other users U-2 to U-3 from the speaker 204. The output management unit 212 also displays images of the other users U-2 to U-3 on the display 201.

[0033] 3 is a block diagram showing the configuration of the communication management device 10. The communication management device 10 includes a communication device 101, a storage device 102, a processing device 103, and a bus 120 that interconnects these devices.

[0034] The communication device 101 has an interface that can be connected to a network N, and communicates with other devices connected to the network N using wireless or wired communication.

[0035] The storage device 102 is a recording medium readable by the processing device 103. The storage device 102 includes, for example, a nonvolatile memory and a volatile memory. The nonvolatile memory is, for example, a ROM, an EPROM, and an EEPROM. The volatile memory is, for example, a RAM. The storage device 102 stores a program PG1. The program PG1 is a program for operating the communication management device 10.

[0036] The processing device 103 includes one or more CPUs. The one or more CPUs are an example of one or more processors. Each of the processors and CPUs is an example of a computer. The processing device 103 executes the program PG1 to function as a session management unit 111, an estimation unit 112, a change unit 113, and a presentation unit 114. At least some of the functions of the session management unit 111, the estimation unit 112, the change unit 113, and the presentation unit 114 may be configured by circuits such as a DSP, an ASIC, a PLD, and an FPGA.

[0037] The session management unit 111 manages the connections between the user terminals 20-1 to 20-3 in an online session in which multiple users U-1 to U-3 participate using their own user terminals 20-1 to 20-3. The session management unit 111 is an example of a management unit. For example, when a request to participate in a web meeting is sent from the user terminal 20-2, the session management unit 111 checks with the user terminal 20 (user terminal 20-1) of the web meeting host (e.g., user U-1) whether the user can participate in the web meeting. If the web meeting host allows participation in the web meeting, the session management unit 111 notifies the user terminal 20-2 of permission to participate in the web meeting and begins receiving audio data and image data from the user terminal 20-2. The session management unit 111 also transmits audio data and image data received from the other user terminals 20-1 and 20-3 to the user terminal 20-2. The connection between the user terminals 20-1 to 20-3 in the online session corresponds to the transmission and reception of audio data and image data between the user terminals 20-1 to 20-3 via the session management unit 111, for example.

[0038] In this embodiment, the end timing of the web meeting is managed by the communication management device 10. When the end timing of the web meeting arrives, the session management unit 111 stops receiving audio data and image data from the user terminals 20-1 to 20-3. The session management unit 111 also stops transmitting audio data and image data received from other user terminals 20 to the user terminals 20-1 to 20-3. In other words, when the end timing of the web meeting arrives, the session management unit 111 disconnects the connection between the user terminals 20-1 to 20-3.

[0039] As will be described later, the end timing of the web meeting is dynamically changed by the change unit 113. The end timing of the web meeting is set to a default value, such as "one hour after the start of the web meeting." The default value may be set by the host of the web meeting.

[0040] The estimation unit 112 estimates the activity level D of communication between multiple users U-1 to U-3 in an online session (a web meeting in this embodiment). In this embodiment, the estimation unit 112 estimates the activity level D using voice data and image data transmitted from each of the user terminals 20-1 to 20-3. Specifically, from the voice data, for example, the amount of speech per unit time of each of the users U-1 to U-3, the volume and intonation of their voices when speaking, and the length of the back-and-forth pauses in the conversation can be analyzed. For example, from the image data, for example, the facial expressions, eye movements, and facial orientation of each of the users U-1 to U-3 can be analyzed.

[0041] For example, if the line of sight and facial direction of user U-1 in the image data tend to move away from the camera 205 (display 201), it is estimated that the level of concentration on the web meeting is low. Such an estimation result is a factor in estimating that the communication activity level D is low. Furthermore, for example, if user U-1's voice has a strong intonation, it is estimated that user U-1 is emotional, that is, the conversation between users U-1 to U-3 is lively. Such an estimation result is a factor in estimating that the communication activity level D is high.

[0042] The communication activity level D may be not only the audio data and image data described above, but also information obtained from other devices connected to the user terminals 20-1 to 20-3. For example, if user U-1 wears a wearable device (such as a smartwatch) and pairs it with the user terminal 20-1 using short-range wireless communication such as Bluetooth (registered trademark), biometric information such as heart rate can be acquired. For example, if user U-1's heart rate is high, it is estimated that user U-1 is emotionally excited, i.e., users U-1 to U-3 are having lively conversations. Such an estimation result is a factor in estimating that the communication activity level D is high.

[0043] Furthermore, for example, if the user terminal 20-1 is a personal computer and is paired with another information processing device (e.g., a smartphone) using short-range wireless communication, the user terminal 20-1 may be able to acquire the operation time for the other information processing device. For example, if user U-1 is operating another information processing device during a web meeting, it is estimated that user U-1 is less interested in the web meeting. Such an estimation result is a factor in estimating that the communication activity level D is low.

[0044] The estimation unit 112 may, for example, quantify the above analysis items and use a threshold value to estimate a classification such as "low" or "high" for the communication activity level D in the current web meeting. Preferably, the classification is set to, for example, two or more. In other words, it is preferable to be able to distinguish at least a state where the activity level D is low from other states (such as high or normal activity level D).

[0045] The estimation unit 112 may also estimate the activity level D of communication in the current web meeting using a learning model that has learned the relationship between the above analysis items and the activity level D. In this case, the activity level D may be expressed as a numerical value such as "activity level 30%." In the following description, the activity level D is expressed as a numerical value, and the larger the numerical value, the more lively the communication.

[0046] Generally, estimating activity level D requires audio data and image data spanning a predetermined period. The estimation unit 112 estimates activity level D using audio data and image data (hereinafter simply referred to as "data") for a predetermined period preceding the timing of estimating activity level D (hereinafter referred to as "estimated time TS"). The predetermined period may be rephrased as the length of data used to estimate activity level D. The predetermined period may be set arbitrarily, for example, to "N minutes (N is any natural number) preceding estimated time TS." The predetermined period may be determined, for example, based on the planned duration of the web meeting (the length of time from the start time of the web meeting to the initial end time of the web meeting). For example, the longer the planned duration, the longer the predetermined period may be set.

[0047] The change unit 113 changes the remaining time TR of the online session based on the communication activity level D between the multiple users U-1 to U-3 in the online session. Hereinafter, the "remaining time length" will be referred to as the "remaining time length TR." Specifically, the change unit 113 acquires the activity level D estimated by the estimation unit 112. Then, the change unit 113 increases the remaining time TR when the communication activity level D in the online session is high. Furthermore, the change unit 113 decreases the remaining time TR when the communication activity level D in the online session is low. In other words, the change unit 113 increases the remaining time TR when the activity level D exceeds a first threshold S1. Furthermore, the change unit 113 decreases the remaining time TR when the activity level D is below a second threshold S2 that is smaller than the first threshold S1. The remaining time TR may be increased or decreased based on the communication activity level D, or only one of them may be performed. In the following description, it is assumed that the remaining time length TR is both increased and decreased based on the activity level D.

[0048] 4A is a diagram schematically illustrating the increase in remaining time TR based on activity level D. The start time of the web meeting is T0, and the initial end time is TX. The initial end time TX corresponds to the "initial value of the end timing of the web meeting" described above. When the estimated time TS arrives, the estimation unit 112 estimates the activity level D at the estimated time TS using data of a predetermined period preceding the estimated time TS. The remaining time TR of the web meeting at the estimated time TS is defined as remaining time TR1.

[0049] The change unit 113 changes the remaining time TR1 based on the activity level D at the estimated time TS. In FIG. 4A , the activity level D exceeds the first threshold S1. The change unit 113 adds a change amount TH1 to the remaining time TR1, and sets the remaining time of the web meeting to the changed remaining time TR1+TH1. The "change amount" may be rephrased as "changed time length." As a result, the end time of the web meeting becomes the changed end time TN1, which is the change amount TH1 after the initial end time TX. In other words, the remaining time TR of the web meeting increases compared to before the change.

[0050] FIG. 4B is a diagram schematically illustrating the decrease in remaining time TR based on activity level D. As in FIG. 4A, the start time of the web meeting is T0 and the initial end time is TX. For ease of illustration, the ratio of time length to distance on the diagram may not match between FIG. 4B and FIG. 4A. When the estimated time TS arrives, the estimation unit 112 described above estimates the activity level D at the estimated time TS using data of a predetermined period preceding the estimated time TS. The remaining time TR of the web meeting at the estimated time TS is the remaining time TR1.

[0051] The change unit 113 changes the remaining time TR1 based on the activity level D at the estimated time TS. In FIG. 4B , the activity level D is assumed to be lower than the second threshold S2. The change unit 113 subtracts the change amount TH2 from the remaining time TR1 to set the remaining time of the web meeting as the changed remaining time TR1-TH2. As a result, the end time of the web meeting becomes the changed end time TN2, which is the change amount TH2 before the initial end time TX. In other words, the remaining time TR of the web meeting is reduced compared to before the change.

[0052] If the activity level D at the estimated time TS is not greater than the first threshold S1 and not less than the second threshold S2, the change unit 113 does not change the remaining time TR of the web meeting. In this case, the web meeting ends at the initial end time TX.

[0053] The change unit 113 may also determine the change amount TH for the remaining time TR based on the length of the remaining time TR at the estimated time TS. The estimated time TS is an example of a predetermined time point. In this case, the change amount TH is longer when the activity level D is a first value and the remaining time TR is a second length of time that is longer than the first length of time than when the activity level D is a first value and the remaining time TR is a first length of time.

[0054] 5A and 5B are diagrams illustrating an increase or decrease in the change amount TH based on the remaining time length TR. In FIGS. 5A and 5B, the activity level D at the estimated time TS3 and the activity level D at the estimated time TS4 are both activity levels D3. The activity level D3 is a value that exceeds the first threshold value S1. Therefore, in FIGS. 5A and 5B, the remaining time length TR is increased. The activity level D3 is an example of a first value.

[0055] In FIG. 5A , the remaining time at estimated time TS3 is TR3. Remaining time TR3 is an example of a second time. The change unit 113 adds a change amount TH3 to the remaining time TR3, setting the remaining time of the web meeting to the changed remaining time TR3+TH3. As a result, the end time of the web meeting becomes a changed end time TN3, which is the change amount TH3 after the initial end time TX.

[0056] On the other hand, in FIG. 5B, the remaining time at estimated time TS4 is TR4 (<TR3). The remaining time TR4 is an example of a first time length. In this case, the change unit 113 adds a change amount TH4 to the remaining time TR4, and sets the remaining time of the web meeting to the changed remaining time TR4 + TH4. The change amount TH4 is smaller than the change amount TH3 (TH4 < TH3). As a result, the end time of the web meeting becomes the changed end time TN4, which is the change amount TH4 after the initial end time TX. The changed end time TN4 is earlier than the changed end time TN3.

[0057] By doing so, the change amount TH for the remaining time length TR can be set according to the remaining time length TR, improving convenience. For example, if the change amount TH is "+1 minute" when the remaining time length TR is 1 hour, it is likely that users U-1 to U-3 will hardly feel the effectiveness of the extension. In contrast, if the change amount TH is "+1 minute" when the remaining time length TR is 5 minutes, it is likely that users U-1 to U-3 will feel that the change amount TH is appropriate.

[0058] The estimated time TS may be set, for example, as a predetermined percentage (e.g., 70%) of the planned duration of the web meeting (the duration from the start time T0 to the initial end time TX). The number of estimated times TS is not limited to one, and may be multiple. In this case, the estimated time TS may be set at predetermined intervals (e.g., every five minutes) after the start of the web meeting. When multiple estimated times TS are set, in other words, when the remaining time TR is changed multiple times by the change unit 113, the "initial end time TX" in the above description is replaced with the "most recent changed end time TN" for the second or subsequent change.

[0059] Furthermore, the change unit 113 may change the remaining time length TR based on the relative values ​​of the multiple activity levels D. Specifically, for example, the estimation unit 112 estimates an initial activity level D0 immediately after the start of the web meeting and an activity level D5 or D6 at a point in time when a predetermined period has elapsed since the start of the online session. The initial activity level D0 is an example of a first activity level, and the activity level D5 or D6 is an example of a second activity level. The change unit 113 changes the remaining time length based on the relative value of the activity level D5 with respect to the initial activity level D0. The relative value may be, for example, a magnitude relationship.

[0060] 6A and 6B are diagrams illustrating a change in the remaining time length TR based on the relative value of the activity level D. In Fig. 6A, the estimation unit 112 estimates the activity level D (initial activity level D0) immediately after the start time T0 of the web meeting. As described above, estimating the activity level D requires audio data and image data over a predetermined period of time. Therefore, the initial activity level D0 is calculated after a predetermined period of time has elapsed since the start time T0 of the web meeting.

[0061] The estimation unit 112 estimates activity level D5 at estimated time TS5, a predetermined time period after the start time T0. The remaining time at estimated time TS5 is TR5. In the example of FIG. 6A, activity level D5 is greater than initial activity level D0, i.e., the communication between users U-1 to U-3 in the web meeting is more active than immediately after the start. In this case, the modification unit 113 adds a modification amount TH5 to the remaining time TR5, setting the remaining time of the web meeting to the modified remaining time TR5 + TH5. As a result, the end time of the web meeting becomes the modified end time TN5, which is the change amount TH5 after the initial end time TX. In other words, the remaining time TR of the web meeting increases.

[0062] In FIG. 6B , as in FIG. 6A , the estimation unit 112 estimates the initial activity level D0 immediately after the start time T0 of the web meeting. The estimation unit 112 also estimates the activity level D6 at the estimated time TS6, a predetermined time period after the start time T0. The estimated time TS6 may be equal to TS5. The remaining time at the estimated time TS6 is TR6. In the example of FIG. 6B , the activity level D6 is smaller than the initial activity level D0, i.e., the communication between users U-1 to U-3 in the web meeting is less active than immediately after the start. In this case, the modification unit 113 subtracts the change amount TH6 from the remaining time TR6 to set the remaining time of the web meeting as the modified remaining time TR6-TH6. As a result, the end time of the web meeting becomes the modified end time TN6, which is the change amount TH6 before the initial end time TX. In other words, the remaining time TR of the web meeting is reduced.

[0063] In this way, by determining whether or not to change the remaining time length TR based on the relative values ​​of multiple activity levels D estimated at different times during a single web meeting, rather than the absolute value of the activity level D, a more appropriate remaining time length TR can be set regardless of the personalities of users U-1 to U-3. For example, if users U-1 to U-3 have proactive personalities, the activity levels D may be estimated to be higher throughout the entire web meeting. In this case, it is not possible to determine from the individual activity levels D alone whether users U-1 to U-3 actually want to extend the remaining time length TR. By calculating the relative values ​​of multiple activity levels D during a single web meeting, the level of communication activity between users U-1 to U-3 can be more accurately grasped, allowing a more appropriate remaining time length TR to be set.

[0064] The change unit 113 does not necessarily need to use the initial activity level D0 when changing the remaining time length TR based on the relative values ​​of the multiple activity levels D. For example, when the estimation unit 112 estimates the activity level D at predetermined time intervals (e.g., every 5 minutes), the remaining time length TR may be changed based on the relative value between the currently estimated activity level Dn and the previously estimated activity level Dn-1.

[0065] In the above description, the change amount TH may be determined based on the relative value between the initial activity level D0 and the activity level D5 or D6. For example, in FIG. 6A , consider a case where the initial activity level D0 is 10 and the activity level D5 is 20 (first pattern) and a case where the activity level D5 is 50 (second pattern). In this case, the change amount TH5 in the second pattern may be larger than the change amount TH5 in the first pattern.

[0066] The presentation unit 114 causes the user terminals 20-1 to 20-3 held by at least one of the multiple users U-1 to U-3 to present the remaining time TR. More specifically, the presentation unit 114 transmits remaining time information for outputting the remaining time to at least one of the user terminals 20-1 to 20-3. In the following description, the remaining time information is assumed to be information for displaying a character string indicating the remaining time on the display 201 of the user terminal 20-1 used by user U-1, who is the host of the web meeting. In this case, the remaining time information is transmitted to the user terminal 20-1 along with image data captured of users U-2 to U-3 (web meeting participants other than user U-1 using user terminal 20-1).

[0067] 7A is a schematic diagram showing the display screen G1 during a web meeting. Fig. 7A shows the display on the display 201 of the user terminal 20-1. The communication management unit 213 of the user terminal 20-1 receives image data and remaining time information for users U-2 to U-3 from the communication management device 10. The communication management unit 213 also receives audio data indicating the speech of users U-2 to U-3, but the following explanation will only cover the display of images.

[0068] The output management unit 212 of the user terminal 20-1 displays images of users U-2 to U-3 on the display 201 based on the image data of users U-2 to U-3. The output management unit 212 also displays an image of user U-1 on the display 201. The image of user U-1 may be displayed smaller than the images of users U-2 to U-3. The image of user U-1 may be an image captured by the camera 205 that is displayed directly on the display 201. Alternatively, the image of user U-1 captured by the camera 205 may be transmitted to the communication management device 10, integrated with the images of other users U-2 and U-3, and then transmitted to the user terminal 20-1.

[0069] In addition to images of users U-1 to U-3, the display screen G1 displays a character string DT indicating the remaining time TR. By looking at the character string DT, user U-1 can understand the remaining time of the web meeting. Note that, for example, when the remaining time TR is changed by the change unit 113, the display mode of the character string DT may be temporarily changed. For example, when the remaining time is changed, the character string DT may blink for a few seconds. This makes it easier for user U-1 to notice the change in the remaining time. Furthermore, when the remaining time TR is changed by the change unit 113, the change amount TH may be displayed as a character string.

[0070] The character string DT may be displayed not only from the start of the web meeting but also when the remaining time of the web meeting becomes short. That is, the presenting unit 114 may present the remaining time TR when the remaining time becomes equal to or less than a third time length. The third time length may be set to, for example, a fixed value (e.g., 5 minutes, but shorter than the planned duration of the web meeting) or may be set to a predetermined percentage of the planned duration (e.g., 5% of the planned duration).

[0071] After the presentation of the remaining time length TR has started, the remaining time length TR may be changed so that the changed remaining time length TR is greater than the third time length. In this case, it is preferable to continue presenting the remaining time length TR in order to prevent the user U-1 or the like from feeling uncomfortable.

[0072] By presenting the remaining time length TR of the web meeting after it becomes equal to or less than the third time length, it is possible to prevent the user U-1 from being more conscious of the remaining time length than necessary, and to allow communication between users in the web meeting to be carried out naturally.

[0073] The display screen G1 may also display an end button BT1 for ending the web meeting at the discretion of the user U-1, and an exit button BT2 for allowing only the user U-1 to exit the web meeting. This allows the user U-1 to end or exit the web meeting without relying on the management of the session management unit 111.

[0074] 7B is a schematic diagram showing a display screen G2 during a web meeting. FIG. 7B shows the display on the display 201 of the user terminal 20-3 used by user U-3. Images of users U-1 to U-3 are displayed on display screen G2, but the character string DT indicating the remaining time is not displayed. Therefore, user U-3 cannot grasp the remaining time TR of the web meeting. In other words, user U-3 can communicate with other users U-1 and U-3 in a natural way without being aware of the remaining time TR of the web meeting.

[0075] Also, the display screen G2 may display an exit button BT2 that allows only the user U-3 to exit the web meeting. This allows the user U-3 to exit the web meeting without relying on the management of the session management unit 111.

[0076] Although FIG. 7B illustrates an example in which the character string DT is not displayed on the display screen G2, it goes without saying that the character string DT may also be displayed on the displays 201 of the user terminals 20-2 and 20-3.

[0077] 8 is a flowchart showing the operation of the processing device 103 of the communication management device 10. The processing device 103 waits until a request to start a web meeting (denoted as "meeting" in the figure) is received (step S100: NO). When a request to start a web meeting is received (step S100: YES), the processing device 103 functions as the session management unit 111 and starts the web meeting. The initial value of the remaining time length of the web meeting is set to the time length from the start time T0 to the initial end time TX.

[0078] The processing device 103 functions as the session management unit 111 and receives image data of the user U-1 and audio data indicating the user U-1's speech from the user terminal 20-1 (step S102). The processing device 103 similarly receives image data and audio data from the other user terminals 20-2 to 20-3. The processing device 103 also functions as the session management unit 111 and transmits the image data and audio data received in step S102 to the other user terminals 20-2 to 20-3 (step S104). The processing device 103 also transmits the image data and audio data received from the user terminals 20-2 to 20-3 to the user terminal 20-1. The processing device 103 may also function as the presentation unit 114 and transmit remaining time length information to at least one of the user terminals 20-1 to 20-3 along with the image data and audio data.

[0079] The processing device 103 returns to step S102 and repeats the subsequent steps until the estimated time TS of the communication activity level D is reached (step S106: NO). When the estimated time TS of the communication activity level D is reached (step S106: YES), the processing device 103 functions as the estimation unit 112 and estimates the communication activity level D at the estimated time TS based on the image data and voice data (step S108).

[0080] The processing device 103 determines whether the activity level D estimated in step S108 exceeds a first threshold S1 (step S110). If the activity level D exceeds the first threshold S1 (step S110: YES), the processing device 103 functions as the change unit 113 and increases the remaining time TR of the web meeting (step S112). On the other hand, if the activity level D estimated in step S108 does not exceed the first threshold S1 (step S110: NO), the processing device 103 determines whether the activity level D estimated in step S108 is below a second threshold S2 (step S114). If the activity level D is below the second threshold S2 (step S114: YES), the processing device 103 functions as the change unit 113 and decreases the remaining time TR of the web meeting (step S116). If the activity level D estimated in step S108 is not lower than the second threshold value S2 (step S114: NO), the processing device 103 proceeds to step S118.

[0081] The processing device 103 returns to step S102 and repeats the subsequent steps until the remaining time of the web meeting reaches 0 (step S118: NO). When the remaining time reaches 0 (step S118: YES), the processing device 103 functions as the session management unit 111 and ends the web meeting (step S120). In other words, it disconnects the user terminals 20-1 to 20-3 from the web meeting.

[0082] [Summary of the embodiment] As described above, the communication management device 10 according to the embodiment changes the remaining time TR of the online session based on the activity level D of the communication between users U-1 to U-3 in the online session. This allows the duration of the online session to be set appropriately, improving the efficiency of communication using the network N.

[0083] Furthermore, the communication management device 10 increases the remaining time TR when the activity level D exceeds the first threshold value S1, and decreases the remaining time TR when the activity level D falls below the second threshold value S2. This makes it possible to avoid, for example, awkward moments when there is nothing to talk about, or disconnection when the end time arrives even though the conversation is getting lively.

[0084] Furthermore, the communication management device 10 determines the amount of change TH for the remaining time TR based on the remaining time TR at the estimated time TS of the activity level D. Therefore, when the remaining time TR is increased or decreased, the amount of change TH is set to an effective amount that does not cause discomfort.

[0085] Furthermore, the communication management device 10 changes the remaining time length TR based on the relative values ​​of the activity levels D at multiple points in time. Because the decision as to whether to change the remaining time length TR is based on the relative values ​​of multiple activity levels D during one online session, rather than the absolute value of the activity levels D, it is possible to set a more appropriate remaining time length TR regardless of the personalities of users U-1 to U-3.

[0086] Furthermore, the communication management device 10 causes at least one of the user terminals 20-1 to 20-3 to display the remaining time length TR. This allows at least one of the users U-1 to U-3 to grasp the remaining time length TR, improving convenience. Furthermore, the communication management device 10 causes at least one of the user terminals 20-1 to 20-3 to display the remaining time length TR when the remaining time length TR becomes equal to or less than a third time length. This allows users U-1 to U-3 to be more aware of the remaining time length TR, improving the efficiency of the online session, compared to when the remaining time length TR is displayed from the start of the online session.

[0087] [Modifications] Modifications of the above-described embodiment are shown below. Two or more of the following modifications may be arbitrarily selected and combined as long as they are not mutually contradictory.

[0088] [First Modification] In the above-described embodiment, image data and audio data are transmitted and received between the user terminals 20-1 to 20-3 during an online session (web meeting), and the communication management device 10 estimates the activity level D based on this information. On the other hand, there are cases in which only audio data is transmitted and received during an online session, or in which image data and audio data are transmitted from the presenter's user terminal 20 but not from the listener's user terminal 20, as in a web seminar. In such cases, the communication management device 10 may instruct the user terminal 20 to transmit both image data and audio data not for the online session but for the purpose of estimating the activity level D. Alternatively, the communication management device 10 (estimation unit 112) may estimate the activity level D using only the image data and audio data that can be obtained.

[0089] According to the first modification, it is possible to estimate the activity level D even in an online session that uses only either image data or audio data.

[0090] [Second Modification] In the above-described embodiment, the communication management device 10 estimated the activity level D. However, this is not limiting, and for example, the user terminals 20-1 to 20-3 may estimate the activity level D of each of the users U-1 to U-3. In this case, the function of the estimation unit 112 described above is realized by the processing device 208 of the user terminals 20-1 to 20-3. Furthermore, the change unit 113 of the communication management device 10 acquires the activity level D of each of the users U-1 to U-3 from the user terminals 20-1 to 20-3 and determines whether or not to change the remaining time length.

[0091] According to the second modified example, the processing load on the communication management device 10 is reduced. Furthermore, according to the second modified example, when the online session uses only image data or audio data, as in the first modified example, there is no need to transmit information that is not used in the online session to the communication management device 10. This reduces the communication load on the communication system 1. Furthermore, the psychological burden on users U-1 to U-3 caused by transmitting data that is not directly used in the online session is reduced.

[0092] [Third Modification] In the above-described embodiment, the communication management device 10 manages the remaining time of the web meeting itself. However, the communication management device 10 may also manage the remaining time of the icebreaker before getting to the main topic, the chat after the discussion of the main topic is finished, or the chat during a break during the discussion of the main topic. When the communication management device 10 manages the remaining time of the chat after the discussion of the main topic is finished, the web meeting itself can be ended, as in the above-described embodiment. Furthermore, when the communication management device 10 manages the remaining time of the icebreaker before getting to the main topic or the chat during a break during the discussion of the main topic, it outputs a notification (such as an alert) to each user terminal 20 when the remaining time reaches zero.

[0093] According to the third modification, online sessions such as web meetings can be conducted more effectively.

[0094] [Other] (1) In the above-described embodiment, ROM and RAM are given as examples of storage devices 102 and 207, but storage devices 102 and 207 may also be flexible disks, magneto-optical disks (e.g., compact disks, digital versatile disks, Blu-ray (registered trademark) disks), smart cards, flash memory devices (e.g., cards, sticks, key drives), CD-ROMs (Compact Disc-ROMs), registers, removable disks, hard disks, floppy (registered trademark) disks, magnetic strips, databases, servers, or other suitable storage media.

[0095] (2) In the above-described embodiments, the described information, signals, etc. may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0096] (3) In the above-described embodiment, input and output information may be stored in a specific location (for example, a memory) or may be managed using a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be transmitted to another device.

[0097] (4) In the above-described embodiment, the determination may be made based on a value represented by one bit (0 or 1), a Boolean value (true or false), or a comparison of numerical values ​​(e.g., a comparison with a predetermined value).

[0098] (5) The order of the process procedures, sequences, flowcharts, etc. illustrated in the above-described embodiments may be rearranged unless it is consistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0099] (6) Each function illustrated in Figures 2 and 3 is realized by any combination of hardware and / or software. Furthermore, the method for realizing each functional block is not particularly limited. That is, each functional block may be realized using a single device that is physically or logically coupled, or may be realized using two or more physically or logically separated devices that are connected directly or indirectly (e.g., via wire, wirelessly, etc.) and these multiple devices. A functional block may be realized by combining software with the single device or the multiple devices.

[0100] (7) The programs exemplified in the above-described embodiments should be broadly construed to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., regardless of whether they are called software, firmware, middleware, microcode, hardware description language, or by other names.

[0101] Software, instructions, information, etc. may also be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then these wired and / or wireless technologies are included within the definition of transmission media.

[0102] (8) In each of the foregoing embodiments, the terms "system" and "network" are used interchangeably.

[0103] (9) The information, parameters, etc. described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0104] (10) In the above-described embodiments, the portable device may be a mobile station (MS). Those skilled in the art may also refer to a mobile station as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other appropriate term. In this disclosure, the terms "mobile station," "user terminal," "user equipment (UE)," "terminal," etc. may be used interchangeably.

[0105] (11) In the above-described embodiments, the terms "connected," "coupled," or any variation thereof refers to any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, "connected" may be read as "access." As used in this disclosure, two elements may be considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0106] (12) In the above embodiments, the phrase "based on" does not mean "based only on," unless otherwise specified. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0107] (13) As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (e.g., searching in a table, database, or other data structure), ascertaining something as "determining" or "determining," and the like. Furthermore, "judgment" and "decision" may include regarding receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, and accessing (e.g., accessing data in memory) as having been "judgment" or "decision." Furthermore, "judgment" and "decision" may include regarding resolving, selecting, choosing, establishing, comparing, etc. as having been "judgment" or "decision." In other words, "judgment" and "decision" may include regarding some action as having been "judgment" or "decision." Furthermore, "judgment" may be interpreted as "assuming," "expecting," "considering," etc.

[0108] (14) In the above embodiments, when "include," "including," and variations thereof are used, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, the term "or," as used in this disclosure, is not intended to be an exclusive or.

[0109] (15) In this disclosure, when articles are added by translation, such as a, an, and the in English, the disclosure may include the nouns following these articles being plural.

[0110] (16) In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combined" may also be interpreted in the same way as "different."

[0111] (17) The aspects / embodiments described in this disclosure may be used alone, in combination, or switched depending on the implementation. Notification of predetermined information (e.g., notification that "X is true") is not limited to being explicit, but may be implicit (e.g., not notifying the predetermined information).

[0112] 1...communication system, 10...communication management device, 20 (20-1 to 20-3)...user terminal, 101...communication device, 102...storage device, 103...processing device, 111...session management unit, 112...estimation unit, 113...change unit, 114...presentation unit, 208...processing device, 211...input management unit, 212...output management unit, 213...communication management unit, N...network, U (U-1 to U-3)...user.

Claims

1. A communication management device comprising: a management unit that manages connections between user terminals in an online session in which multiple users participate using their own user terminals; and a change unit that changes the length of the remaining time of the online session based on the level of communication activity between the multiple users in the online session.

2. The communication management device according to claim 1, wherein the change unit increases the length of the remaining time when the activity level exceeds a first threshold value.

3. The communication management device according to claim 2, wherein the change unit decreases the length of the remaining time when the activity level falls below a second threshold that is smaller than the first threshold.

4. The communication management device of claim 1, wherein the activity level is the activity level at a predetermined point in time of the online session, and the change unit determines the amount of change to the remaining time based on the length of the remaining time at the predetermined point in time.

5. The communication management device described in claim 4, wherein the amount of change is longer when the activity level is the first value and the remaining time is a second length of time that is longer than the first length of time than when the activity level is a first value and the remaining time is a first length of time.

6. The communication management device of claim 1, wherein the activity level includes a first activity level immediately after the online session is started and a second activity level at a point in time when a predetermined period of time has elapsed since the online session was started, and the change unit changes the length of the remaining time based on the relative value of the second activity level to the first activity level.

7. The communication management device according to claim 1, further comprising a presentation unit that causes a user terminal held by at least one of the plurality of users to present the length of the remaining time.

8. The communication management device according to claim 7, wherein the presentation unit presents the remaining time when the remaining time is equal to or less than a third time length.

9. A communication management method, comprising: acquiring, by a computer, the activity level of communication between a plurality of users in an online session in which the users participate using their own user terminals; and changing the length of the remaining time of the online session based on the activity level.

Images