Audio conference device, audio conference system, and audio conference method

Abstract

To enable a manager to easily know the states of a plurality of voice conferences.SOLUTION: An audio conference device 1 according to an embodiment of the present invention is an audio conference device that provides a plurality of audio conferences via a network, and performs, with a plurality of user terminals used by a plurality of users participating in each of the plurality of audio conferences: A voice conference unit 121 that transmits and receives voices uttered in the voice conference, a voice analysis unit 122 that analyzes voices uttered in each of the plurality of voice conferences, and a display control unit 124 that displays an analysis result of the voices uttered in the voice conference by the voice analysis unit in association with each of the plurality of voice conferences on an administrator terminal used by an administrator who manages the plurality of voice conferences.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an audio conference device, an audio conference system, and an audio conference method for conducting an audio conference via a network. [Background technology]

[0002] In recent years, systems for holding audio conferences over a network among multiple information terminals have become increasingly popular. Patent Document 1 discloses a system that identifies a speaker who is speaking during an audio conference and visually indicates the identified speaker. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-299051 Summary of the Invention [Problem to be solved by the invention]

[0004] In schools or companies, there are cases where a manager such as a teacher or a boss manages multiple audio conferences. However, in the system disclosed in Patent Document 1, the participants of one audio conference are only shown the speaker in that audio conference, so the manager cannot easily know the status of multiple audio conferences.

[0005] The present invention has been made in view of these points, and has as its object to enable an administrator to easily know the status of a plurality of audio conferences. [Means for solving the problem]

[0006] A first aspect of the audio conference device of the present invention is an audio conference device that provides multiple audio conferences via a network, and includes an audio conference unit that transmits and receives audio uttered in the multiple audio conferences between multiple user terminals used by multiple users participating in each of the multiple audio conferences, an audio analysis unit that analyzes the audio uttered in each of the multiple audio conferences, and a display control unit that associates the analysis results of the audio uttered in each of the multiple audio conferences by the audio analysis unit on an administrator terminal used by an administrator who manages the multiple audio conferences.

[0007] The display control unit may cause the manager terminal to display the analysis results in association with each of the plurality of audio conferences while the plurality of audio conferences are continuing.

[0008] The audio conference unit may accept intervention by the manager terminal into any of the plurality of audio conferences using at least one of audio, text, and images.

[0009] The audio conference device may further include a suggestion unit that suggests to the manager terminal to intervene in any one of the plurality of audio conferences based on the analysis result.

[0010] The display control unit may cause a display mode of the audio conference in which the proposal unit proposes intervention to differ from a display mode of the other audio conferences.

[0011] The audio conference unit may automatically intervene in any of the plurality of audio conferences using at least one of audio, text, and images based on the analysis results.

[0012] The display control unit may display the analysis results on each of the plurality of user terminals used by the plurality of users participating in the audio conference while the audio conference is continuing.

[0013] The display control unit may display, between multiple captured images of the multiple users while the audio conference is ongoing, a symbol indicating the degree of exchange of speech between the multiple users in the audio conference.

[0014] The audio conference unit may accept input of a specified action from each of the multiple user terminals used by the multiple users participating in the audio conference while the audio conference is ongoing, and the display control unit may display on the administrator terminal, in addition to the results of the audio analysis, information indicating the action input in each of the multiple audio conferences.

[0015] The voice analysis unit may analyze the voice uttered in each of the plurality of voice conferences by comparing it with previously acquired voice patterns or model voice patterns.

[0016] The voice analysis unit may analyze the voice uttered in each of the plurality of voice conferences by comparing it with the voice pattern that differs for each purpose of the voice conference.

[0017] The voice analysis unit may analyze the voice uttered in each of the plurality of voice conferences by comparing it with the voice pattern that differs for each period of the voice conference.

[0018] The voice analysis unit may divide the plurality of users into a plurality of groups based on the analysis results, and the voice conference unit may start the voice conference in which the plurality of users included in each of the plurality of groups participate.

[0019] The voice analysis unit may divide the users into a plurality of groups based on attributes of each of the users in addition to the analysis result.

[0020] The voice analysis unit may divide the users into a plurality of groups based on the analysis result so that the amount of speech or speech tendencies of the users belonging to one group are similar.

[0021] A second aspect of the audio conference system of the present invention includes an audio conference device that provides multiple audio conferences via a network, and an administrator terminal used by an administrator that manages the multiple audio conferences. The audio conference device has an audio conference unit that transmits and receives audio uttered in the multiple audio conferences between multiple user terminals used by multiple users participating in each of the multiple audio conferences, an audio analysis unit that analyzes the audio uttered in each of the multiple audio conferences, and a display control unit that displays on the administrator terminal the analysis results of the audio uttered in the audio conferences by the audio analysis unit, associated with each of the multiple audio conferences. The administrator terminal has a display unit that displays the analysis results.

[0022] A third aspect of the audio conferencing method of the present invention includes the steps of transmitting and receiving audio uttered in multiple audio conferences via a network between multiple user terminals used by multiple users participating in each of the multiple audio conferences, executed by a computer; analyzing the audio uttered in each of the multiple audio conferences; and displaying the analysis results of the audio uttered in the audio conferences in the analysis step, in association with each of the multiple audio conferences, on an administrator terminal used by an administrator who manages the multiple audio conferences. [Effects of the Invention]

[0023] According to the present invention, an administrator can easily know the status of a plurality of audio conferences. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a schematic diagram of an audio conference system according to an embodiment; [Figure 2]1 is a block diagram of an audio conference system according to an embodiment. [Figure 3] FIG. 10 is a schematic diagram of a user terminal displaying an exemplary audio conference listing screen. [Figure 4] FIG. 2 is a schematic diagram of a user terminal displaying an exemplary audio conference screen. [Figure 5] FIG. 10 is a schematic diagram of an administrator terminal displaying an exemplary audio conference list screen. [Figure 6] FIG. 10 is a schematic diagram for explaining a method for displaying an action on an analysis result. [Figure 7] FIG. 10 is a schematic diagram for explaining a method for dividing a plurality of users into a plurality of groups based on the analysis results. [Figure 8] FIG. 1 is a flowchart illustrating an audio conference method executed by an audio conference device. DETAILED DESCRIPTION OF THE INVENTION

[0025] [Outline of Audio Conference System S] 1 is a schematic diagram of an audio conference system S according to this embodiment. The audio conference system S includes an audio conference device 1, a plurality of user terminals 2, and a manager terminal 3. The number of user terminals 2 and manager terminals 3 included in the audio conference system S is not limited. The audio conference system S may also include other devices such as servers and terminals.

[0026] The audio conference device 1 is a computer that provides multiple audio conferences via a network. An audio conference is the exchange of audio between multiple user terminals 2. An audio conference may also be the exchange of at least one of images (which may be still images or moving images) and text in addition to audio between multiple user terminals 2. The audio conference device 1 also analyzes the audio uttered in the audio conference and displays the analysis results on the user terminals 2 and the administrator terminal 3.

[0027] The audio conference device 1 is configured, for example, by a single computer or a cloud, which is a collection of computer resources. The audio conference device 1 is connected to a user terminal 2 and an administrator terminal 3 via a network such as a local area network or the Internet, either wired or wirelessly.

[0028] The user terminal 2 is an information terminal used by users (students, employees, etc.) who participate in an audio conference provided by the audio conference system S. The user terminal 2 is, for example, a personal computer, a smartphone, a tablet terminal, etc. The user terminal 2 may also be a wearable terminal worn by the user to input and output audio and images. The user terminal 2 accepts audio input from the user and transmits it to the audio conference device 1, and also receives and outputs audio input at other user terminals 2 from the audio conference device 1. In this embodiment, an example in which one user uses one user terminal 2 will be described, but multiple users may also use one user terminal 2.

[0029] The administrator terminal 3 is an information terminal used by an administrator (a teacher, a superior, or the like who is in a position to manage users) who manages one or more audio conferences provided by the audio conference system S. The administrator terminal 3 is, for example, a personal computer, a smartphone, a tablet terminal, or the like. The administrator terminal 3 may also be a wearable terminal worn by the administrator to input and output audio and images. The administrator terminal 3 displays the results of audio analysis by the audio conference device 1 and accepts intervention in the audio conference by the administrator.

[0030] In the audio conference system S according to this embodiment, the audio conference device 1 analyzes the audio uttered in each of the multiple audio conferences, associates the results of the audio analysis with each of the multiple audio conferences, and displays the results of the audio analysis on the manager terminal 3. This allows the audio conference system S to give the manager an overview of the status of the multiple audio conferences, making it possible for the manager to easily know the status of the multiple audio conferences.

[0031] [Configuration of Audio Conference System S] FIG. 2 is a block diagram of an audio conference system S according to this embodiment. In FIG. 2, arrows indicate the main data flows, and there may be other data flows not shown in FIG. 2. In FIG. 2, each block indicates a functional configuration rather than a hardware (device) configuration. Therefore, the blocks shown in FIG. 2 may be implemented in a single device, or may be implemented separately in multiple devices. Data may be exchanged between blocks via any means, such as a data bus, a network, or a portable storage medium.

[0032] The audio conference device 1 has a storage unit 11 and a control unit 12. The control unit 12 has an audio conference unit 121, a voice analysis unit 122, a proposal unit 123, and a display control unit 124. The storage unit 11 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, etc. The storage unit 11 stores in advance programs to be executed by the control unit 12. The storage unit 11 may be provided outside the audio conference device 1, in which case data may be exchanged between the storage unit 11 and the control unit 12 via a network.

[0033] The control unit 12 is a processor such as a CPU (Central Processing Unit), and functions as an audio conference unit 121, an audio analysis unit 122, a suggestion unit 123, and a display control unit 124 by executing programs stored in the storage unit 11. At least some of the functions of the control unit 12 may be performed by an electric circuit. Also, at least some of the functions of the control unit 12 may be performed by a program executed via a network.

[0034] The audio conference unit 121 executes multiple audio conferences by transmitting and receiving audio between multiple user terminals 2. The audio conference unit 121 transmits and receives audio generated in the audio conference (i.e., audio input to the audio input / output unit 22) between multiple user terminals 2 used by multiple users participating in the audio conference. The audio conference unit 121 also transmits and receives captured images of users participating in the audio conference (i.e., captured images captured by the imaging unit 23) between multiple user terminals 2 used by multiple users participating in the audio conference. The audio conference unit 121 also transmits and receives input contents (input characters, actions, etc.) of users participating in the audio conference between multiple user terminals 2 used by multiple users participating in the audio conference.

[0035] As a result, the audio conference unit 121 can hold an audio conference by sharing audio, captured images, and input contents among multiple user terminals 2. The audio conference unit 121 is not limited to the specific method shown here in order to hold an audio conference, and can use any known method.

[0036] Furthermore, the audio conference unit 121 may accept intervention using text or voice from the manager terminal 3, or may automatically intervene using text or voice, in an audio conference in which intervention has been proposed by the proposal unit 123. Intervention in an audio conference may involve, for example, outputting at least one of text, voice, or image input by the manager at the manager terminal 3 to the user terminal 2 of each user participating in the audio conference, or outputting at least one of text, voice, or image generated by the audio conference unit 121 to the user terminal 2 of each user participating in the audio conference.

[0037] For example, when an operation to select an intervention button 314 corresponding to one of the audio conferences is performed on the operation unit 34 of the administrator terminal 3, the audio conference unit 121 starts transmitting and receiving at least one of audio and text between the administrator terminal 3 and multiple user terminals 2 participating in the audio conference, thereby enabling the administrator terminal 3 to intervene in the audio conference. The audio conference unit 121 may transmit and receive at least one of audio and text only to some of the multiple user terminals 2 selected on the administrator terminal 3. Furthermore, with regard to the intervention, the audio conference unit 121 may transmit and receive an image (which may be a still image or a video) specified on the user terminal 2 and an image specified on the administrator terminal 3 in addition to or instead of at least one of audio and text.

[0038] The voice analysis unit 122 analyzes the voices uttered in each of the multiple voice conferences. For example, the voice analysis unit 122 calculates the amount of speech of each of the multiple users in association with each of the multiple voice conferences, and calculates the degree of interaction (speaker transition) between the multiple users.

[0039] The voice analysis unit 122 first determines which user has spoken (uttered) at each time interval (for example, every 10 to 100 milliseconds) during the voice conference based on the voice acquired by the voice conference unit 121. When the voice of one user is input to one user terminal 2, the voice analysis unit 122 determines which user has spoken based on the user terminal 2 from which the voice was acquired. When the voices of multiple users are input to one user terminal 2, the voice analysis unit 122 executes a known speaker separation process to determine which user has spoken in the voice acquired from one user terminal 2.

[0040] The voice analysis unit 122 identifies a continuous period from when a user starts speaking to when he or she finishes speaking as a speech period, and stores this in the storage unit 11. The voice analysis unit 122 also calculates the speech volume of each user for each time period, and stores this in the storage unit 11. For example, the voice analysis unit 122 calculates the speech volume for each time period by dividing the length of the user's speech in a certain time window (for example, 5 seconds) by the length of the time window.

[0041] The voice analysis unit 122 repeatedly calculates the amount of speech for each user per time period while shifting the time window by a predetermined time (for example, 1 second) from the start time of the voice conference to the current time (or the end time in the case of analysis after the end of the voice conference).The voice analysis unit 122 then detects a transition of speakers when a certain speech period is switched to another speech period.The voice analysis unit 122 tallyes the occurrence time of the transition detected in the discussion to be analyzed, the user at the transition source, and the user at the transition destination, and stores them in association with each other in the storage unit 11.

[0042] Furthermore, the voice analysis unit 122 may analyze the voices uttered in each of the multiple voice conferences by comparing them with previously acquired voice patterns or model voice patterns. In this case, the storage unit 11 stores previously acquired voice patterns or model voice patterns in advance. The model voice pattern is time-series data of the speech volume of the model person, created by, for example, acquiring in advance the voice pattern of a voice conference by the model person.

[0043] Previously acquired voice patterns or model voice patterns are defined for each purpose of the voice conference (use of the voice conference, type of users of the voice conference), for example, education, sales, interview, etc. The voice analysis unit 122 accepts in advance the selection of the purpose of the voice conference from the user terminal 2 or the administrator terminal 3, and compares the amount of speech of each user calculated from the voice of the voice conference with the voice patterns that differ for each purpose.

[0044] These voice patterns may also be defined for each period of the voice conference, for example. The voice analysis unit 122 receives in advance the scheduled time of the voice conference (for example, one hour) from the user terminal 2 or the administrator terminal 3, and compares the speech volume of each user calculated from the voice of the voice conference with different voice patterns for each period of the scheduled time (first half, middle, second half, etc.).

[0045] For example, in an audio conference for educational purposes where all participants are students (typically used for student debates), it is desirable for the volume of speech from each participant to be as uniform as possible throughout the scheduled time, and so the model audio pattern to be compared would be one such example. Also, even in an audio conference for educational purposes, if one participant plays a leadership role (typically used for an English conversation class at an English conversation school, etc., consisting of one instructor and one or more students), it is desirable for the student participants to speak more than the instructor, and if there are multiple students, it is desirable for the volume of speech from each student to be as uniform as possible, and so the model audio pattern to be compared would be one such example. Furthermore, in an audio conference for sales purposes (typically used for sales meetings with customers), it is desirable for the company's sales representative to speak more in the first half to middle of the scheduled time in order to explain their company's products or services, but it is desirable for the customer participants to speak more in the middle to latter half of the scheduled time in order to elicit their reactions, and so the model audio pattern to be compared would be one such example.

[0046] The voice analysis unit 122 calculates the degree of difference between the user's speech volume and the time series data of the speech volume indicated by the voice pattern (for example, the cumulative value of the difference between the speech volume of the voice pattern and the user's speech volume) as a comparison result and stores it in the memory unit 11.

[0047] The suggestion unit 123 suggests to the manager terminal 3 that the manager intervene in one of the multiple audio conferences based on the analysis result of the audio by the audio analysis unit 122. Intervention in an audio conference means that the manager participates in the audio conference using at least one of text and audio on the manager terminal 3. For example, when the analysis result of one audio conference by the audio analysis unit 122 satisfies a predetermined condition, the suggestion unit 123 identifies the audio conference as a target for intervention and notifies the manager terminal 3 of the identified audio conference as a target for intervention via the display control unit 124. For example, when there is a user in the audio conference whose speech volume is below a predetermined value, or when the total speech volume of multiple users participating in the audio conference is below a predetermined value, the suggestion unit 123 suggests the audio conference as a target for intervention.

[0048] The display control unit 124 displays the status of the audio conference and the analysis results on the user terminal 2 and the manager terminal 3 based on the analysis by the audio analysis unit 122. The detailed display contents by the display control unit 124 will be described later with reference to FIGS.

[0049] The user terminal 2 has a display unit 21, an audio input / output unit 22, an imaging unit 23, and an operation unit 24. The administrator terminal 3 has a display unit 31, an audio input / output unit 32, an imaging unit 33, and an operation unit 34. The display units 21, 31 include a liquid crystal display or the like capable of displaying information. The audio input / output units 22, 32 include a microphone or the like for inputting audio and a speaker or the like for outputting audio. When audio from multiple users is input to one user terminal 2, the audio input / output unit 22 may be configured with multiple microphones or a microphone array or the like depending on the speaker separation processing performed by the audio analysis unit 122. The imaging units 23, 33 include a camera or the like for outputting captured images of the user or administrator. The operation units 24, 34 include buttons, switches, touch panels, or the like that can accept human operation.

[0050] The audio conference device 1, the user terminal 2, and the administrator terminal 3 according to this embodiment are not limited to the specific configuration shown in Fig. 2. Each of the audio conference device 1, the user terminal 2, and the administrator terminal 3 is not limited to a single device, and may be configured by two or more physically separated devices connected by wire or wirelessly.

[0051] [How to hold an audio conference] (Display screen of user terminal 2) First, the following describes a process in which the audio conference device 1 causes the user terminal 2 to display information about an audio conference. In the audio conference device 1, the display control unit 124 causes the user terminal 2 to display an audio conference list screen including information about multiple audio conferences that the user can participate in.

[0052] The following explanation is for a case where a plurality of users participating in an audio conference held for a specific purpose are divided into a plurality of rooms (virtual rooms) for the audio conference. That is, each room (virtual room) is provided in an audio conference held for a specific purpose (an audio conference at a higher level), and a plurality of audio conferences at a lower level are held in each room (virtual room).

[0053] 3 is a schematic diagram of a user terminal 2 displaying an exemplary audio conference list screen. The user terminal 2 displays the audio conference list screen on the display unit 21 in response to control by the audio conference device 1. The audio conference list screen displays audio conference information 211 and an entry button 212 associated with each of a plurality of audio conferences (rooms). The audio conference information 211 indicates the number of users participating in the audio conference. The audio conference information 211 may also indicate the names of the users participating in the audio conference, the purpose of the audio conference, etc.

[0054] When an operation to select one of the entry buttons 212 is performed on the operation unit 24 of the user terminal 2, the audio conference unit 121 allows the user of the user terminal 2 to participate in the audio conference (room) corresponding to the selected entry button 212, and starts the audio conference. In addition, the audio conference unit 121 may automatically allow the user of the user terminal 2 to participate in a pre-assigned audio conference (room).

[0055] The display control unit 124 causes the user terminal 2 to display an audio conference screen including information about the audio conference in which the user is participating. Fig. 4 is a schematic diagram of the user terminal 2 displaying an exemplary audio conference screen. The user terminal 2 displays the audio conference screen on the display unit 21 in accordance with the control of the audio conference device 1.

[0056] The audio conference screen displays a user image 213, text information 214, an input field 215, an action field 216, and an analysis result 217 related to the audio conference (room) in which the user of the user terminal 2 is participating. The user image 213 is a captured image of the user captured by the imaging unit 23 of each of the multiple user terminals 2. If the captured image cannot be obtained from the user terminal 2 or if the user does not wish to make the captured image public, a predetermined image or text (such as the user's name) may be displayed in place of the user image 213.

[0057] The text information 214 represents a message inputted on the operation unit 24 of each of the plurality of user terminals 2. The text information 214 may also represent a message generated by the audio conference device 1 (for example, an automatic intervention message, which will be described later).

[0058] The input field 215 is an area where a user can input comments (e.g., impressions or annotations) during an audio conference. Comments input into the input field 215 are stored in the storage unit 11 of the audio conference device 1 in association with the time of input. The action field 216 is an area where a user can input actions during an audio conference. The action field 216 includes a plurality of buttons (icons) corresponding to a plurality of actions, such as like, applause, and laugh. Actions input into the action field 216 are stored in the storage unit 11 of the audio conference device 1 in association with the time of input.

[0059] The analysis result 217 represents the analysis result of the voice in the voice conference by the voice conference device 1 while the voice conference is continuing. The display control unit 124 displays, as the analysis result 217, an arrow symbol indicating the number of transitions that occurred in the voice conference from the start time to the current time for each combination of two users (i.e., the degree of interaction between multiple users) based on the analysis result by the voice analysis unit 122 while the voice conference is continuing. The arrow symbol of the analysis result 217 is displayed thicker the more the number of transitions, and thinner the fewer the number of transitions. This allows the voice conference system S to visually notify users of the degree of interaction between users in the voice conference while the voice conference is continuing.

[0060] Furthermore, while the audio conference is ongoing, the display control unit 124 changes the display mode of the user image 213 according to the volume of speech of each user based on the analysis results by the voice analysis unit 122. The display control unit 124 totals the volume of speech for each user from the start time of the audio conference to the current time. When the volume of speech of one user (or the ratio of speech compared to other users) is equal to or less than a predetermined value, the display control unit 124 makes the color, size, shape, border, etc. of the user image 213 of that user different from the user images 213 of the other users. This allows the audio conference system S to encourage users who speak less to speak.

[0061] The display control unit 124 may also switch whether to display the analysis result 217 on each of the multiple user terminals 2. The display control unit 124 may switch whether to display the analysis result 217 depending on, for example, the purpose of the audio conference. The display control unit 124 may also acquire information indicating the user's proficiency level or personality stored in advance in the storage unit 11, and display the analysis result 217 only on the user terminal 2 of a user whose acquired proficiency level or personality meets a predetermined condition (for example, being an advanced user or having a personality that is good at discussion). The display control unit 124 may also display the analysis result 217 only on the user terminal 2 of a user designated by an administrator. The display control unit 124 may also acquire the user's gaze direction using a known gaze detection method, and display the analysis result 217 only on the user terminal 2 of a user whose acquired gaze direction meets a predetermined condition (for example, not gazing at the analysis result 217 for a predetermined period of time or more).

[0062] Next, a description will be given of a process in which the audio conference device 1 causes the administrator terminal 3 to display information about an audio conference. In the audio conference device 1, the display control unit 124 causes the administrator terminal 3 to display an audio conference list screen including information about multiple audio conferences managed by the administrator.

[0063] (Display screen of administrator terminal 3) The following explanation also applies to a case where a plurality of users participating in an audio conference held for a specific purpose are divided into a plurality of rooms (virtual rooms) for the audio conference. That is, each room (virtual room) is provided in an audio conference held for a specific purpose (an audio conference at a higher level), and a plurality of audio conferences at a lower level are held in each room (virtual room).

[0064] In addition, when an administrator manages multiple audio conferences in higher hierarchical levels, even if multiple audio conferences are being held in lower hierarchical levels, an audio conference list screen including information about the multiple audio conferences in higher hierarchical levels may be displayed on the administrator terminal 3. In this case, as information about the audio conferences in higher hierarchical levels, information about audio conferences being held in the audio conferences in the higher hierarchical levels may all be displayed without distinguishing between lower hierarchical levels, or information about audio conferences in specific lower hierarchical levels among the audio conferences in the higher hierarchical levels may be selectively displayed, so that information about audio conferences in specific lower hierarchical levels within one audio conference in the higher hierarchical level and information about audio conferences in lower hierarchical levels within another audio conference in the higher hierarchical level may be displayed.

[0065] 5(a) and 5(b) are schematic diagrams of the administrator terminal 3 displaying an exemplary audio conference list screen. The administrator terminal 3 displays the audio conference list screen on the display unit 31 in accordance with control by the audio conference device 1. The audio conference list screen displays audio conference information 311 and analysis results 312 in association with each of multiple lower-level audio conferences (rooms) included in an upper-level audio conference managed by the administrator. The audio conference information 311 displays the names of users participating in the audio conference. The audio conference information 311 may also display the number of users participating in the audio conference, the purpose of the audio conference, etc. The analysis results 312 display the results of analysis of the audio conference by the audio analysis unit 122. The audio conference list screen also includes proposal information 313, an intervention button 314, comparison information 315, and a regroup button 316, which will be described later.

[0066] The display control unit 124 associates the analysis results 312 of the voices uttered in the voice conferences by the voice analysis unit 122 with each of the multiple voice conferences (rooms) and causes the manager terminal 3 to display the analysis results 312. It is desirable that the display control unit 124 associates the analysis results with each of the multiple voice conferences and displays the analysis results with each of the multiple voice conferences on the manager terminal 3 while the multiple voice conferences are ongoing. This allows the voice conference system S to notify the manager of the status of the multiple voice conferences currently being held, making it easy for the manager to determine whether or not to intervene in the multiple voice conferences.

[0067] For example, based on the analysis result by the audio analysis unit 122, the display control unit 124 displays an arrow symbol indicating the number of transitions that have occurred in an audio conference from the start time to the current time for each combination of two users participating in the audio conference (i.e., the degree of interaction between multiple users) as the analysis result 312. The arrow symbol of the analysis result 312 is displayed thicker the more the number of transitions, and thinner the fewer the number of transitions. This allows the audio conference system S to visually notify the administrator of the degree of interaction between users in the audio conference while the audio conference is ongoing.

[0068] For example, the display control unit 124 displays the analysis result 312 as a line graph showing the change over time in the speech volume of each of multiple users participating in one audio conference (room) based on the analysis result by the audio analysis unit 122. The display control unit 124 displays the speech volume per time indicated by the analysis result for each user as a line graph, with speech volume on the vertical axis and time on the horizontal axis. At this time, the display control unit 124 adds up the speech volumes of the users at each time point, i.e., displays the value obtained by sequentially summing the speech volumes of the users on the vertical axis. This allows the audio conference system S to visually notify the administrator of the speech volume of the entire audio conference, in addition to the speech volume of each user participating in the audio conference, while the audio conference is ongoing.

[0069] The display control unit 124 displays, for example, a bar graph showing the total speech volume of each of the multiple users participating in one audio conference based on the analysis result by the audio analysis unit 122 as the analysis result 312. This allows the audio conference system S to visually notify the administrator of the total speech volume of each user while the audio conference is ongoing.

[0070] When an operation to select an arrow symbol, a line graph, or a bar graph included in the analysis result 312 is performed on the operation unit 34 of the administrator terminal 3, the display control unit 124 may enlarge and display the selected arrow symbol, line graph, or bar graph in detail. The display control unit 124 may display, as the analysis result 312, not only arrow symbols, line graphs, and bar graphs, but also other analysis results obtained by analyzing the audio, in association with each of the multiple audio conferences (rooms).

[0071] The display control unit 124 displays information indicating the audio conference (room) for which the proposing unit 123 is proposing intervention on the manager terminal 3. As shown in FIG. 5(a), the display control unit 124 displays proposal information 313 indicating the audio conference for which the proposing unit 123 is proposing intervention on the audio conference list screen displayed on the manager terminal 3. The display control unit 124 displays the proposal information 313, for example, including characters that can identify the audio conference (room) for which the proposing unit 123 is proposing intervention and characters that explain why intervention is necessary.

[0072] Furthermore, the display control unit 124 differentiates the display mode of the audio conference information 311 corresponding to the audio conference in which the proposal unit 123 proposes intervention from the display mode of the audio conference information 311 corresponding to the other audio conferences among the multiple audio conferences included in the audio conference list screen displayed on the administrator terminal 3. In the example of Fig. 5(a), the display control unit 124 changes the border of the audio conference information 311 as the display mode, but the display control unit 124 may also change the color, size, shape, etc. of the audio conference information 311.

[0073] 5(b), the display control unit 124 may display comparison information 315 showing the comparison result calculated by the voice analysis unit 122. The display control unit 124 may display, for example, proposal information 313 including text that can identify the voice conference that was compared with the voice pattern and text that explains the comparison result. The display control unit 124 may also display the comparison result of the voice with the voice pattern not only on the administrator terminal 3 but also on the user terminal 2. This allows the voice conference system S to present to the administrator the voice analysis result based on the voice pattern acquired in advance, making it easier for the administrator or user to interpret the analysis result.

[0074] The display control unit 124 may display, on the audio conference list screen displayed on the administrator terminal 3, actions and comments input by users while the audio conference is ongoing, in addition to the analysis results by the audio analysis unit 122. As described above, an action is any one of a plurality of actions, such as like, applause, and laughter, input by a user on the user terminal 2 during the audio conference, and is stored in the storage unit 11 in association with the time at which it was input. A comment is an impression, annotation, or the like input by a user on the user terminal 2 during the audio conference, and is stored in the storage unit 11 in association with the time at which it was input.

[0075] 6 is a schematic diagram for explaining a method for displaying actions and comments on the analysis results. The display control unit 124 displays characters indicating the contents of the actions and comments at positions corresponding to the time when the actions and comments were input on a line graph showing the time change of the speech volume in the analysis results 312 included in the audio conference list screen, for example. The display control unit 124 may also display the actions and comments not only on the administrator terminal 3 but also on the user terminal 2. This allows the audio conference system S to present the analysis results together with the actions and comments of users input during the audio conference, making it easier for the administrator or users to interpret the analysis results.

[0076] Furthermore, while the audio conference is in progress, the display control unit 124 may display information indicating an action input at one user terminal 2 on the other user terminals 2. This allows a user to notify other users of their opinions, such as agreement, without interrupting the ongoing conversation.

[0077] (User division processing) 5(a) and 5(b) on the operation unit 34 of the administrator terminal 3, the voice analysis unit 122 may divide multiple users into multiple groups for holding multiple voice conferences based on the analysis results of the multiple voice conferences in the voice conference device 1. That is, the voice analysis unit 122 may propose multiple groups that are preferable for holding the next and subsequent voice conferences based on the analysis results.

[0078] 7 is a schematic diagram illustrating a method for dividing multiple users into multiple groups based on the analysis results. The voice analysis unit 122 analyzes the voices of two audio conferences corresponding to groups G1 and G2 into which multiple users are divided, and stores the analysis results in the memory unit 11. The voice analysis unit 122 divides the multiple users into new groups G1' and G2' based on the analysis results stored in the memory unit 11.

[0079] Note that the explanation here refers to regrouping between rooms (virtual rooms) (between lower-level audio conferences) by forming groups G1 and G2 into new groups G1' and G2', but the regrouping may also include dividing an audio conference in a certain room (virtual room) into smaller lower-level audio conferences.

[0080] For example, based on the analysis results, the voice analysis unit 122 divides multiple users into multiple groups so that the volume of speech or speech tendencies of multiple users belonging to one group are similar. In this case, the voice analysis unit 122 clusters the volume of speech of the multiple users using a known method, and creates each cluster into a group.

[0081] Furthermore, the voice analysis unit 122 may group together users who have a tendency to speak, such as, for example, often interrupting others in the middle of their speech during an audio conference, or tending to speak more in the first, middle, or last half of an audio conference. Conversely, the voice analysis unit 122 may divide multiple users into multiple groups so that the amount of speech or speech tendencies of multiple users belonging to one group are far apart.

[0082] Furthermore, the voice analysis unit 122 may divide the users into multiple groups based on the attributes of each of the users in addition to the analysis results. The user attributes may be, for example, the user's school grades or the results of a personality test previously administered to the user. In this case, the voice analysis unit 122 divides the users into multiple groups so that the attributes of the users belonging to one group are similar or dissimilar.

[0083] The audio conference unit 121 automatically starts an audio conference in which multiple users included in each of the multiple groups generated by the audio analysis unit 122 participate. The display control unit 124 may notify the user of the audio conference to participate in by displaying information indicating the audio conference corresponding to the group in which the user is included on the audio conference list screen displayed on the user terminal 2 of FIG.

[0084] In this way, the audio conference system S can efficiently train and evaluate users by suggesting groups for audio conferences between similar or dissimilar users.

[0085] (Automatic intervention based on voice analysis results) The audio conference unit 121 may automatically intervene in an audio conference (room) for which the suggestion unit 123 has proposed intervention, using at least one of voice and text. In this case, the audio conference unit 121 determines the content of intervention based on the analysis result by the audio analysis unit 122. For example, when there is a user in the audio conference whose speech volume is below a predetermined value, the audio conference unit 121 determines that the content of intervention is to encourage the user to speak. Furthermore, for example, when the total speech volume of multiple users participating in the audio conference is below a predetermined value, the audio conference unit 121 determines that the content of intervention is to encourage all users to speak.

[0086] The audio conference unit 121 then generates an automated voice (mechanical voice) indicating the content of the intervention and outputs it to the multiple user terminals 2 participating in the audio conference, or generates text indicating the content of the intervention and displays it on the multiple user terminals 2 participating in the audio conference. This allows the audio conference system S to automatically intervene in an audio conference that requires intervention based on the results of the audio analysis, thereby facilitating the audio conference while reducing the burden on the administrator. The audio conference unit 121 may automatically intervene only in some user terminals 2 selected based on predetermined conditions from among the multiple user terminals 2. In this case, the audio conference unit 121 selects, for example, the user terminals 2 of users whose speech volume is below a predetermined value as targets for intervention.

[0087] As mentioned above, the audio conference unit 121 may, in addition to or instead of at least one of audio and text, send and receive images specified by the user terminal 2 (the images may be still images or moving images) and images specified by the administrator terminal 3 in relation to intervention.

[0088] [Audio conference flow] 8 is a flowchart of the audio conference method executed by the audio conference device 1. In the audio conference device 1, the display control unit 124 causes the user terminal 2 to display an audio conference list screen including information about multiple audio conferences that the user can participate in.

[0089] The audio conference unit 121 accepts participation of any of the users in the audio conference in response to an operation on the operation unit 24 of the user terminal 2 (S11). The audio conference unit 121 starts the audio conference by starting to exchange audio with the multiple user terminals 2 of the multiple users participating in the audio conference (S12).

[0090] The voice analysis unit 122 analyzes the voices uttered in each of the multiple voice conferences (S13). For example, the voice analysis unit 122 calculates the amount of speech of each of the multiple users in association with each of the multiple voice conferences, and calculates the degree of interaction (speaker transition) between the multiple users.

[0091] The display control unit 124 displays the analysis results by the voice analysis unit 122 on the user terminal 2 (S14). For example, while the voice conference is continuing, the display control unit 124 displays arrow symbols indicating the degree of interaction between multiple users on the user terminal 2 based on the analysis results by the voice analysis unit 122. Furthermore, for example, while the voice conference is continuing, the display control unit 124 changes the display mode of the user image in accordance with the amount of speech by the user based on the analysis results by the voice analysis unit 122.

[0092] The display control unit 124 associates each of the plurality of audio conferences with the analysis result 312 of the audio uttered in the audio conference by the audio analysis unit 122 and causes the manager terminal 3 to display the result (S15). For example, while the audio conference is ongoing, the display control unit 124 causes the manager terminal 3 to display an arrow symbol indicating the degree of interaction between the plurality of users based on the analysis result by the audio analysis unit 122.

[0093] Furthermore, for example, while the audio conference is continuing, the display control unit 124 causes the administrator terminal 3 to display a line graph indicating the change over time in the speech volume of each of the multiple users participating in one audio conference based on the analysis result by the speech analysis unit 122. Furthermore, for example, while the audio conference is continuing, the display control unit 124 causes the administrator terminal 3 to display a bar graph indicating the total value of the speech volume of each of the multiple users participating in one audio conference based on the analysis result by the speech analysis unit 122.

[0094] [Effects of this embodiment] According to the audio conference system S of this embodiment, the audio conference device 1 analyzes the audio uttered in each of the multiple audio conferences, associates the results of the audio analysis with each of the multiple audio conferences, and displays the results of the audio analysis on the manager terminal 3. In this way, the audio conference system S allows the manager to have an overview of the status of the multiple audio conferences, and enables the manager to easily know the status of the multiple audio conferences.

[0095] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by functionally or physically distributing or integrating any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combination also have the effects of the original embodiments.

[0096] The processor of the audio conference device 1 is responsible for each step (process) included in the audio conference method shown in Fig. 8. That is, the processor of the audio conference device 1 reads a program for executing the audio conference method shown in Fig. 8 from the storage unit 11, and executes the program to control each unit of the audio conference device 1, thereby executing the audio conference method shown in Fig. 8. Some of the steps included in the audio conference method shown in Fig. 8 may be omitted, the order of the steps may be changed, or multiple steps may be performed in parallel. [Explanation of symbols]

[0097] S Audio Conference System 1 Audio conference equipment 12 Control Unit 121 Audio Conference Section 122 Voice Analysis Unit 123 Proposal Department 124 Display control unit 2. User terminal 3. Administrator terminal

Claims

1. An audio conference device for providing multiple audio conferences via a network, an audio conference unit for transmitting and receiving audio generated in the audio conference between a plurality of user terminals used by a plurality of users participating in each of the plurality of audio conferences; a voice analysis unit that analyzes first time series data of speech volume of each of the plurality of users in the voices uttered in each of the plurality of voice conferences by comparing the first time series data of speech volume in a previously acquired voice pattern or a model voice pattern; a display control unit that displays the analysis result by the voice analysis unit on an information terminal; An audio conference device comprising:

2. the analysis result is information indicating the degree of difference between the message volume indicated by the first time series data and the message volume indicated by the second time series data; The audio conference device of claim 1 .

3. the audio conference unit accepts intervention into any of the plurality of audio conferences using at least one of audio, text, and image from an administrator terminal used by an administrator who manages the plurality of audio conferences; 3. The audio conference device according to claim 1 or 2.

4. a suggestion unit that suggests to the manager terminal that the manager should intervene in any one of the plurality of audio conferences based on the analysis result; The audio conference device of claim 3.

5. the voice analysis unit analyzes the voice uttered in each of the plurality of voice conferences by comparing it with the voice pattern that differs for each purpose of the voice conference. The audio conference device according to any one of claims 1 to 4.

6. the voice analysis unit analyzes the voice uttered in each of the plurality of voice conferences by comparing it with the voice pattern that differs for each period of the voice conference; The audio conference device according to any one of claims 1 to 4.

7. the voice analysis unit divides the users into a plurality of groups based on the analysis result; the audio conference unit starts the audio conference in which the plurality of users included in each of the plurality of groups participate. The audio conference device according to any one of claims 1 to 6.

8. the voice analysis unit divides the plurality of users into a plurality of groups based on the analysis result as well as the attributes of each of the plurality of users. The audio conference device of claim 7.

9. the voice analysis unit divides the users into a plurality of groups based on the analysis result so that the amount of speech or speech tendencies of the users belonging to one group are similar to each other; The audio conference device of claim 7.

10. an audio conference device for providing a plurality of audio conferences via a network; a plurality of user terminals used by a plurality of users participating in each of the plurality of audio conferences; Including, The audio conference device an audio conference unit for transmitting and receiving audio generated in the audio conference between the plurality of user terminals; a voice analysis unit that analyzes first time series data of speech volume of each of the plurality of users in the voices uttered in each of the plurality of voice conferences by comparing the first time series data of speech volume in a previously acquired voice pattern or a model voice pattern; a display control unit that displays the analysis result by the voice analysis unit on an information terminal; An audio conference system comprising:

11. The computer executes transmitting and receiving voices generated in the audio conference between a plurality of user terminals used by a plurality of users participating in each of the plurality of audio conferences via a network; analyzing first time series data of speech volume of each of the plurality of users in the speech uttered in each of the plurality of audio conferences by comparing it with second time series data of speech volume in a previously acquired speech pattern or a model speech pattern; a step of displaying the analysis result in the analyzing step on an information terminal; 1. An audio conferencing method comprising:

Images