Meeting support system, meeting support method, and meeting support program

Abstract

To provide an environment for understanding the contents of meetings held prior to the present time. According to one aspect of the present disclosure, a conference support system includes at least one processor that records conference data including audio from an online conference, acquires a point in time going back in the online conference from a user's terminal, generates content corresponding to the conference data for a time range from the point in time, and plays the content on the user's terminal at a playback speed faster than the original playback speed of the conference data while the online conference is in progress.

Description

[Technical Field]

[0001] One aspect of the present disclosure relates to a conference support system, a conference support method, and a conference support program. [Background technology]

[0002] There are known mechanisms for helping participants understand the contents of online conferences via a network. For example, Patent Document 1 describes a network conference system that records conference information while the conference is in progress, and when a participant joins the conference midway, creates a summary of the conference information up to that point and provides the summary to the participant individually. Patent Document 2 describes a video conference system that rewinds and plays back video or audio of a speech made by a participant in an electronic conference if that speech is missed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-339033 [Patent Document 2] Japanese Patent Application Laid-Open No. 2008-236553 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a need for a mechanism that allows participants in an online meeting to understand the content of the meeting prior to the current time. [Means for solving the problem]

[0005] A conference support system according to an aspect of the present disclosure includes at least one processor that records conference data including audio of an online conference, acquires a point in time going back in the online conference from a user's terminal, generates content corresponding to the conference data for a time range after the point in time, and plays the content on the user's terminal at a playback speed faster than the original playback speed of the conference data while the online conference is in progress.

[0006] In this aspect, content corresponding to conference data from a point in time prior to the online conference is generated. The content is then played back at high speed on the user's device so as to catch up with the ongoing online conference. This provides an environment for understanding the content of the conference prior to the current time. [Effects of the Invention]

[0007] According to one aspect of the present disclosure, it is possible to provide an environment for understanding the content of a meeting held before the current time. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of application of a conference support system according to an embodiment. [Figure 2] FIG. 1 is a diagram illustrating an example of a hardware configuration related to a conference support system according to an embodiment. [Figure 3] FIG. 1 is a diagram illustrating an example of a functional configuration related to a conference support system according to an embodiment. [Figure 4] FIG. 10 is a diagram illustrating an example of a conference screen. [Figure 5] 5A and 5B are diagrams showing examples of playback screens. Example (a) of Fig. 5 is an example of a playback screen showing one frame constituting the first half of the content. Example (b) of Fig. 5 is an example of a playback screen showing one frame constituting the second half of the content. [Figure 6] FIG. 2 is a sequence diagram illustrating an operation of the conference support system according to the embodiment. [Figure 7]FIG. 10 is a diagram illustrating an example of a functional configuration related to a conference support system according to another embodiment. [Figure 8] 8A and 8B are diagrams illustrating another example of a conference screen. Example (a) of Fig. 8 is an example of a conference screen that displays the status and progress. Example (b) of Fig. 8 is another example of a conference screen that displays the status and progress. [Figure 9] FIG. 10 is a sequence diagram illustrating an operation of a conference support system according to another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are designated by the same reference numerals, and redundant description will be omitted.

[0010] [System Overview] The conference support system according to the embodiment is a computer system that supports users in online conferences. An online conference is a conference held via multiple user terminals connected to a network, and is also called a web conference or a network conference. A user is a person who uses the conference support system. A user terminal is a computer used by one or more users. "Supporting a user" is achieved by providing the user with content representing the progress of an online conference prior to the current time. Content refers to data that allows a person to recognize some information at least through hearing. Content may be moving images (video) including audio, or audio only. "Providing" refers to the process of transmitting information to a user terminal via a network.

[0011] The conference support system receives from a user terminal a request specifying a point in time to go back in the online conference. The point in time to go back in the online conference is the point at which content playback starts (hereinafter referred to as the "content start point"). The conference support system generates content data, which is electronic data representing the content, based on the content start point and electronic data recording the online conference, and transmits the content data to the user terminal. The user terminal receives and processes the content data, and performs catch-up playback of the content at high speed. Chase-up playback refers to a function that plays back audio being recorded or video being recorded with a delay.

[0012] The "contents (progress) of the conference (online conference) before the current time point" includes the progress of the conference in a first range from the content start time to the time when the content start time is specified (in other words, the time when catch-up playback is instructed). The real-time conference continues even while catch-up playback of the content corresponding to the first range is being executed. The "contents (progress) of the conference (online conference) before the current time point" may further include the progress of the conference in a second range from the time when the content start time is specified (the time when catch-up playback is instructed) to the current time point. The progress of the conference in the second range is the content of the conference that continues to progress during catch-up playback.

[0013] FIG. 1 is a diagram illustrating an example of an application of a conference support system 1. In this embodiment, the conference support system 1 includes a server 10. The server 10 is a computer (conference support server) that transmits content to at least one user terminal 20. The server 10 is connected to multiple user terminals 20 via a communication network N. Although five user terminals 20 are shown in FIG. 1, the number of user terminals 20 is not limited. The configuration of the communication network N is not limited. For example, the communication network N may be configured to include the Internet or an intranet. As illustrated in FIG. 1, the type of user terminal 20 is not limited. For example, the user terminal 20 may be a portable terminal such as a high-function mobile phone (smartphone), a tablet terminal, a wearable terminal (e.g., a head-mounted display (HMD), smart glasses, etc.), a laptop personal computer, or a mobile phone. Alternatively, the user terminal 20 may be a stationary terminal such as a desktop personal computer.

[0014] The content in the present disclosure is a moving image that combines live-action images and audio. Live-action images refer to images that show the real world and are obtained by an imaging device such as a camera. The conference support system 1 may be used for various purposes. For example, the conference support system 1 may be used for video conferences, online seminars, etc. In other words, the conference support system 1 may be used as a communication means for sharing moving images among multiple users. Alternatively, the conference support system 1 may be used for telephone conferences where only audio is shared.

[0015] [System Configuration] 2 is a diagram showing an example of a hardware configuration related to the conference support system 1. As an example, the server 10 includes a processor 101, a main memory unit 102, an auxiliary memory unit 103, and a communication unit 104 as hardware components.

[0016] The processor 101 is a computing device that executes an operating system and application programs. Examples of processors include a central processing unit (CPU) and a graphics processing unit (GPU), but the type of processor 101 is not limited to these.

[0017] The main memory unit 102 is a device that stores programs for operating the server 10, calculation results output from the processor 101, etc. The main memory unit 102 is configured by, for example, at least one of a ROM (Read Only Memory) and a RAM (Random Access Memory).

[0018] The auxiliary storage unit 103 is generally a device capable of storing a larger amount of data than the main storage unit 102. The auxiliary storage unit 103 is configured by a non-volatile storage medium such as a hard disk or flash memory. The auxiliary storage unit 103 stores a server program P1 for causing at least one computer to function as the server 10, and various data. In this embodiment, the conference support program is implemented as the server program P1.

[0019] The communication unit 104 is a device that executes data communication with other computers via the communication network N. The communication unit 104 is configured by, for example, a network card or a wireless communication module.

[0020] Each functional element of server 10 is realized by loading server program P1 onto processor 101 or main memory unit 102 and executing that program. Server program P1 includes code for realizing each functional element of server 10. Processor 101 operates communication unit 104 in accordance with server program P1, and executes reading and writing of data from and to main memory unit 102 or auxiliary memory unit 103. Each functional element of server 10 is realized by such processing.

[0021] The server 10 may be configured by one or more computers. When multiple computers are used, these computers are connected to each other via a communication network N to logically configure one server 10.

[0022] As an example, the user terminal 20 includes, as hardware components, a processor 201, a main memory unit 202, an auxiliary memory unit 203, a communication unit 204, an input interface 205, an output interface 206, and an imaging unit 207.

[0023] The processor 201 is a computing device that executes an operating system and application programs. The processor 201 may be, for example, a CPU or a GPU, but the type of the processor 201 is not limited to these.

[0024] The main storage unit 202 is a device that stores programs for causing the user terminal 20 to function, calculation results output from the processor 201, etc. The main storage unit 202 is configured with at least one of a ROM and a RAM, for example.

[0025] The auxiliary storage unit 203 is generally a device capable of storing a larger amount of data than the main storage unit 202. The auxiliary storage unit 203 is configured by a non-volatile storage medium such as a hard disk or flash memory. The auxiliary storage unit 203 stores the client program P2 for causing the computer to function as the user terminal 20 and various data.

[0026] The communication unit 204 is a device that executes data communication with other computers via the communication network N. The communication unit 204 is configured by, for example, a network card or a wireless communication module.

[0027] The input interface 205 is a device that accepts data based on a user's operation or action. For example, the input interface 205 is configured with at least one of a keyboard, operation buttons, a pointing device, a microphone, a sensor, and a camera. The keyboard and operation buttons may be displayed on a touch panel. Since the type of the input interface 205 is not limited, the data to be input is also not limited. For example, the input interface 205 may accept data entered or selected by a keyboard, operation buttons, or a pointing device. Alternatively, the input interface 205 may accept voice data entered by a microphone. Alternatively, the input interface 205 may accept, as motion data, data indicating the user's non-verbal behavior (e.g., gaze, gestures, facial expressions, etc.) detected by a motion capture function using a sensor or camera.

[0028] The output interface 206 is a device that outputs data processed by the user terminal 20. For example, the output interface 206 is configured with at least one of a monitor, a touch panel, an HMD, and a speaker. The display device, such as a monitor, a touch panel, or an HMD, displays the processed data on a screen. The speaker outputs audio represented by the processed audio data.

[0029] The imaging unit 207 is a device that captures images of the real world, and more specifically, is a camera. The imaging unit 207 may capture moving images (video) or still images (photographs). When capturing moving images, the imaging unit 207 processes the video signal based on a given frame rate to obtain a series of frame images arranged in time series as a moving image. The imaging unit 207 can also function as the input interface 205.

[0030] Each functional element of the user terminal 20 is realized by loading a client program P2 onto the processor 201 or the main memory unit 202 and executing that program. The client program P2 includes code for realizing each functional element of the user terminal 20. The processor 201 operates the communication unit 204, the input interface 205, the output interface 206, or the imaging unit 207 in accordance with the client program P2, and reads and writes data from and to the main memory unit 202 or the auxiliary memory unit 203. Each functional element of the user terminal 20 is realized by this processing.

[0031] At least one of the server program P1 and the client program P2 may be provided by being fixedly recorded on a tangible recording medium such as a CD-ROM, a DVD-ROM, or a semiconductor memory. Alternatively, at least one of these programs may be provided as a data signal superimposed on a carrier wave via the communications network N. These programs may be provided separately or together.

[0032] FIG. 3 is a diagram showing an example of a functional configuration related to the conference support system 1. The server 10 includes functional elements including a conference control unit 11, a recording unit 12, a request receiving unit 13, a content generation unit 14, and an output unit 15. The conference control unit 11 is a functional element that controls the display of an online conference on a user terminal 20. The recording unit 12 is a functional element that records conference data including the audio of the online conference. The request receiving unit 13 is a functional element that receives a content generation request including a content start time from the user terminal 20. The content generation unit 14 is a functional element that generates content data based on the content start time and conference data. The content data has a time span from the content start time to catching up with the real-time conference. The content data is, for example, one or more pieces of data in a streaming format. The output unit 15 is a functional element that transmits the content data to the user terminal 20.

[0033] The user terminal 20 includes, as functional elements, a conference display unit 21, a request transmission unit 22, and a content playback unit 23. The conference display unit 21 is a functional element that cooperates with the conference control unit 11 of the server 10 to display an online conference. The request transmission unit 22 is a functional element that transmits a content generation request to the server 10. The content playback unit 23 is a functional element that plays back content data received from the server 10.

[0034] The conference database 30 is a non-transitory storage medium or storage device that stores conference data, which is electronic data of an online conference. The conference data in this disclosure is video images including audio from the online conference. The conference data may further include user identification information that identifies the user who is the speaker of the audio.

[0035] [System Operation] FIG. 4 is a diagram showing an example of a conference screen 300. The conference screen 300 is a screen that displays an ongoing online conference in real time. The conference screen 300 is displayed on the user terminal 20 of each user participating in the online conference. For example, the conference screen 300 is displayed on the user terminal 20 of each of four users (user A, user B, user C, and user D). The conference screen 300 includes, for example, display areas 301 to 304, name display fields 301A to 304A, a time point input field 305, and a catch-up playback button 306.

[0036] Display areas 301-304 are screen areas that display moving images of users. The moving images of users are moving images captured by user terminal 20. The number of display areas 301-304 corresponds to the number of users. For example, four display areas 301-304 display moving images of four users, respectively. As the number of users increases or decreases, the display areas also increase or decrease. Display areas 301-304 may display one frame image constituting a moving image, or may display one still image. Display areas 301-304 may be highlighted when a displayed user is speaking.

[0037] Name display fields 301A-304A are screen areas that display the names of users participating in an online conference. A user's name may be set by receiving a user input when joining the online conference. The user's name may also be recorded in conference database 30 as user identification information. Name display fields 301A-304A correspond one-to-one to display areas 301-304, respectively. For example, a video of user A is displayed in display area 301, and the name of user A is displayed in name display field 301A.

[0038] The time point input field 305 is a screen element for receiving a user input regarding the content start time. The time point input field 305 receives an input operation or a selection operation of the content start time, for example, 5 minutes before. The catch-up play button 306 is a screen element for performing catch-up play from the content start time input in the time point input field 305. The aspects of the time point input field 305 and the catch-up play button 306 are not limited to this, and for example, the content start time may be set as a fixed value and the catch-up play button 306 may be displayed alone.

[0039] The display of the conference screen 300 is controlled by cooperation between the conference control unit 11 of the server 10 and the conference display unit 21 of the user terminal 20. For example, the conference display unit 21 captures moving images of users and transmits the moving images and user identification information to the server 10. The conference control unit 11 generates the conference screen 300 based on the moving images and user identification information received from multiple user terminals 20, and transmits the conference screen 300 to the user terminals 20 of each user. The conference display unit 21 processes the received conference screen 300 and displays it on a display device.

[0040] FIG. 5 is a diagram illustrating an example of a playback screen 400. The playback screen 400 displays the progress of a past online conference. More specifically, the playback screen 400 displays the progress of a past online conference, recorded from the start of the content until the progress is caught up with the real-time progress. The playback screen 400 is displayed on the user terminal 20, triggered, for example, by pressing the catch-up playback button 306 on the conference screen 300. A user may miss the contents of a conference due to various reasons, such as being away from their desk or poor communication over the communication network N, or may simply want to listen to the contents of the conference again. In such cases, the user can check the contents of the conference by catch-up playback of the content. For example, user D, who temporarily leaves the conference, performs catch-up playback from the point where he left after returning to his desk. In this case, the first half of the content shows scenes in which user D is absent, and the second half of the content shows scenes in which user D, having returned to his desk, is playing back the content. The following description assumes that the playback screen 400 is displayed on the user terminal 20 of user D, who temporarily leaves the conference.

[0041] 5A shows a playback screen 400A, which is one frame constituting the first half of the content, as an example of playback screen 400. Playback screen 400A is a screen for understanding the contents of the conference as time passes. Playback screen 400A includes display areas 401-404, name display fields 401A-404A, a playback speed field 405, an operation interface 406, a playback time field 407, and a progress bar 408.

[0042] Display areas 401-404 and name display fields 401A-404A correspond to display areas 301-304 and name display fields 301A-304A, respectively, of conference screen 300. Display area 401 is highlighted with a double frame, and user D is not displayed in display area 404. In other words, playback screen 400A indicates that user A is speaking and user D is away from his desk.

[0043] The playback speed field 405 is a screen element that displays the playback speed of the content. The playback speed of the content is a playback speed that is faster than the original playback speed of the conference data. The original playback speed means that the playback speed of the conference data has not been changed. The playback speed of the content is, for example, n times faster (n>1.0) than the original playback speed. In one example, the playback speed of the content is 2.0 times faster. The playback speed field 405 may accept user input regarding a change in the playback speed of the content.

[0044] The operation interface 406 is a user interface for performing various operations related to the playback of content, such as switching between playback and pause, cueing, and so on, from the user.

[0045] The playback time field 407 is a screen element that displays the elapsed time from the start of playback of the content. The progress bar 408 is a screen element that displays the progress rate within the time span of the content. In other words, the playback time field 407 and the progress bar 408 indicate the playback position of the content.

[0046] Example (b) of FIG. 5 shows, as an example of playback screen 400, playback screen 400B, which is one frame constituting the latter half of the content. Playback screen 400B is a screen for understanding the content of a conference that continues to progress during catch-up playback. On playback screen 400B, the playback position indicated by playback time field 407 and progress bar 408 is later than on playback screen 400A. That is, playback screen 400B indicates that more time has passed since playback screen 400A. A video of user D is displayed in display area 404. This indicates that user D has returned to his seat. Playback screen 400B shows an online conference in which user D is playing back content. That is, playback screen 400B shows an online conference being held by users A, B, and C, while user D, who is playing back the content, is not participating in the conference.

[0047] The operation of the conference support system 1 will be described with reference to Fig. 6, and a conference support method according to this embodiment will be described. Fig. 6 is a sequence diagram showing the operation of the conference support system 1 as a processing flow S1. In the following, it is assumed that four users (user A, user B, user C, and user D) are participants in an online conference. The conference control unit 11 of the server 10 and the conference display unit 21 of the user terminal 20 work together to display a conference screen 300 (see Fig. 4) on each of the user terminals 20 of the four users.

[0048] In step S11, the recording unit 12 of the server 10 records the video including the audio of the online conference as conference data in the conference database 30. The recording unit 12 continues to record the conference data as the online conference progresses.

[0049] The conference data may further include user identification information. The server 10 receives video images captured at the same time from the user terminals 20 of the respective users. Therefore, the recording unit 12 can identify the correspondence between the audio at a given time and the user identification information. The recording unit 12 associates this correspondence with the conference data in chronological order and records it in the conference database 30.

[0050] In step S12 and onward, the description will be given assuming that the user terminal 20 is the terminal of the user (user D in the example of FIG. 5) who is attempting to perform catch-up playback. In step S12, the conference display unit 21 of the user terminal 20 accepts a user input regarding the content start time. For example, the conference display unit 21 accepts a user input regarding the content start time via the time input field 305 of the conference screen 300. In one example, the conference display unit 21 accepts a user input specifying that the content start time is five minutes ago.

[0051] In step S13, the request sending unit 22 of the user terminal 20 sends a content generation request including the content start time (a time going back in time in the online conference) to the server 10. For example, the request sending unit 22 acquires the content start time entered in the time input field 305, triggered by pressing the chase play button 306. The request sending unit 22 generates a content generation request including the content start time, and sends the content generation request to the server 10. The request receiving unit 13 of the server 10 receives the content generation request and acquires the content start time.

[0052] In step S14, the content generator 14 of the server 10 reads conference data for a time range from the start of the content from the conference database 30, and generates content data corresponding to the conference data. In one example, the content generator 14 generates content data corresponding to conference data from five minutes ago. The method and data structure of the content data are not limited. For example, the content generator 14 may generate content data by associating the voice speaker with user identification information. The content generator 14 continues generating content data until the playback of the content on the user terminal 20 catches up with the real-time online conference. Therefore, the end point of the time range may vary depending on, for example, the playback speed of the content or the length of the playback time of the content.

[0053] In step S15, the output unit 15 of the server 10 transmits the content data to the user terminal 20. In the user terminal 20, the content playback unit 23 receives the content data.

[0054] In step S16, the content playback unit 23 plays back the content at a playback speed faster than the original playback speed of the conference data while the online conference is in progress. The content playback unit 23 processes the content data received from the server 10 and displays the content on the display device. If content rendering has not been performed on the server 10 side, the content playback unit 23 displays the content by performing rendering based on the content data. If the content data represents the content itself, the content playback unit 23 displays the content as is. The user terminal 20 outputs audio from the speaker in accordance with the display of the content. In this way, the content playback unit 23 displays a playback screen 400 (see example (a) and example (b) in Figure 5) on the user terminal 20.

[0055] The playback speed of the content may be any speed faster than the original playback speed of the conference data. For example, the playback speed of the content may be 2.0 times faster. The content playback unit 23 plays back the content at a high speed while the online conference is in progress. The playback speed of the content may be determined by the content generation unit 14 or the content playback unit 23. When the playback of the content catches up with the real-time online conference, the content playback unit 23 ends the playback of the content. Then, the conference display unit 21 displays the conference screen 300 again on the user terminal 20. In this way, the display on the user terminal 20 switches from the playback screen 400 to the conference screen 300.

[0056] In relation to step S14, the end point of the time range may be determined. For example, the end point of the time range may be the acquisition time when the content start point is acquired. The acquisition time may be the time when the server 10 receives a content generation request, or the time of a user operation such as pressing the chase playback button 306. In this case, content data from the content start point to the time indicating the end point of the time range is generated and transmitted to the user terminal 20. Then, in step S16, the content playback unit 23 may play the content, and the conference display unit 21 may display the online conference. In other words, content playback and real-time online conference display may be executed in parallel. When content playback reaches the end point of the time range, the content playback unit 23 ends content playback. Meanwhile, the conference display unit 21 continues displaying the online conference.

[0057] FIG. 7 is a diagram showing an example of a functional configuration related to the conference support system 1A. The conference support system 1A differs from the conference support system 1 in that the server 10 further includes a status determination unit 16 as a functional element, and the user terminal 20 further includes a sharing unit 24 as a functional element. The status determination unit 16 is a functional element that determines the user's status and the progress of content playback. The status refers to the user's participation state in the conference. The progress refers to the progress of content playback. The sharing unit 24 is a functional element that cooperates with the status determination unit 16 of the server 10 to share the user's status and the progress of content playback.

[0058] FIG. 8 is a diagram showing another example of the conference screen 300. Example (a) of FIG. 8 shows a conference screen 300A that displays a status and progress. Example (a) of FIG. 8 assumes that user D is currently playing content. The conference screen 300A includes a time display field 304B and a status message 307. The time display field 304B is a screen element that displays the time remaining until the content playback ends. The time display field 304B may be displayed within a display area that displays the video of the user currently playing the content. For example, the time display field 304B may be displayed within the display area 304 that displays the video of user D. The time display field 304B displays the remaining time, such as "0 minutes 30 seconds remaining." The status message 307 is a screen element that displays the status that a user is currently playing content. The status message 307 displays information indicating which user is currently playing the content, such as "User D is currently playing catch-up content." The form of the time display field 304B and the status message 307 is not limited to this, and for example, the time display field 304B and the status message 307 may be displayed integrally.

[0059] Example (b) of FIG. 8 shows a conference screen 300B that displays a status and progress. In example (b) of FIG. 8, it is assumed that user D is currently playing content. The conference screen 300B includes an indicator 304C and a status message 308. Indicator 304C is a screen element that displays the progress rate of the content playback. Indicator 304C may be displayed in a display area that displays a video of the user currently playing the content. For example, indicator 304C may be displayed in the display area 304 that displays the video of user D. Indicator 304C displays the progress rate in terms of the time span of the content. For example, indicator 304C displays the progress rate using a progress bar or a percentage. Status message 308 is a screen element that displays the status of the voice speaker along with the progress of the content playback. Status message 308 includes an embedded portion 309 that displays user identification information. Status message 308 displays information such as, for example, "User D is currently playing back the speech of 'speaker'." Here, "speaker" corresponds to embedding unit 309. User identification information can be displayed in embedding unit 309 along with the progress of content playback. In one example, user identification information for user A is displayed in embedding unit 309, such as "User D is currently playing back the speech of 'User A'." The aspects of indicator 304C and status message 308 are not limited to this, and, for example, indicator 304C and status message 308 may be displayed integrally.

[0060] The time display field 304B, the indicator 304C, and the status messages 307 and 308 described above may not be displayed, may be displayed independently, or may be displayed in any combination.

[0061] The operation of the conference support system 1A will be described with reference to FIG. 9. FIG. 9 is a sequence diagram showing the operation of the conference support system 1A as a processing flow S2. In the following, it is assumed that four users (user A, user B, user C, and user D) are participants in an online conference. The conference control unit 11 of the server 10 and the conference display unit 21 of the user terminal 20 cooperate to display a conference screen 300 (see FIG. 4) on each of the user terminals 20 of the four users. In addition, the user terminal 20 of the user who plays back the content will be referred to as a first user terminal, and the user terminal 20 of a user other than the user will be referred to as a second user terminal.

[0062] Steps S21 to S26 are similar to steps S11 to S16 of the processing flow S1, respectively, and therefore will not be described here.

[0063] In step S27, the sharing unit 24 of the first user terminal notifies the server 10 of the playback speed of the content. For example, the sharing unit 24 acquires the playback speed of the content displayed in the playback speed field 405, triggered by playback of the content. The sharing unit 24 notifies the server 10 of the playback speed. By receiving the notification of the playback speed from the first user terminal, the status determination unit 16 may determine that the user of the first user terminal is currently playing back content. The processing of step S27 may be further executed, triggered, for example, by a change in the playback speed of the content, a cue to the content, or the like.

[0064] In step S28, the status determination unit 16 of the server 10 calculates the progress status based on the playback speed and elapsed time of the content. In one example, the status determination unit 16 multiplies the playback speed of the content by the elapsed time to calculate the playback position in the length of the playback time of the content as the progress status. The elapsed time may be obtained from, for example, the first user terminal, or may be measured using the time when the notification of the playback speed is received in the processing of step S27 as the start time. The status determination unit 16 may calculate the time until the playback of the content ends as the progress status. Alternatively, the status determination unit 16 may calculate the progress rate of the playback of the content as the progress status.

[0065] In step S29, the conference control unit 11 performs conference display control for the second user terminal. For example, the conference control unit 11 transmits the status and progress to the second user terminal. In the second user terminal, the conference display unit 21 acquires the progress and status.

[0066] In step S30, the conference display unit 21 displays the progress and status. For example, the conference display unit 21 of the user terminal 20 of users A, B, and C displays a conference screen 300A (see example (a) in FIG. 8) on the display device. The time display field 304B and the status message 307 on the conference screen 300A allow users A, B, and C to share the status of user D and the progress of the content playback.

[0067] In relation to step S27, if the playback speed of the content has been determined on the server 10 side, the sharing unit 24 does not need to notify the playback speed.

[0068] [effect] As described above, a conference support system according to one aspect of the present disclosure includes at least one processor that records conference data including audio from an online conference, acquires a point in time going back in the online conference from a user's terminal, generates content corresponding to the conference data for a time range from the point in time, and plays the content on the user's terminal at a playback speed faster than the original playback speed of the conference data while the online conference is in progress.

[0069] A conference support method according to an aspect of the present disclosure is executed by a conference support system including at least one processor, and includes the steps of recording conference data including audio of an online conference, acquiring a time point going back in the online conference from a user terminal, generating content corresponding to the conference data for a time range after the time point, and playing the content on the user terminal at a playback speed faster than an original playback speed of the conference data while the online conference is in progress.

[0070] A conference support program according to one aspect of the present disclosure causes a computer to perform the steps of recording conference data including audio from an online conference, obtaining a point in time going back in the online conference from a user's terminal, generating content corresponding to the conference data for a time range after the point in time, and playing the content on the user's terminal at a playback speed faster than the original playback speed of the conference data while the online conference is in progress.

[0071] In this aspect, content corresponding to conference data from a point in time prior to the online conference is generated. The content is then played back at high speed on the user's device so as to catch up with the ongoing online conference. This provides an environment for understanding the content of the conference prior to the current time.

[0072] The above-mentioned Patent Document 1 describes a network conferencing system that records conference information while the conference is in progress, and when a late joiner is detected, creates a summary of the conference information up to that point and provides the summary to the late joiner individually. However, the technology in Patent Document 1 is not a technology that catches up and plays back conference content that was missed while participating in the conference. Furthermore, the technology in Patent Document 1 may result in missing conference content due to the creation of a summary.

[0073] The above-mentioned Patent Document 2 describes a video conference system that rewinds and plays back the video or audio of a speech made by an electronic conference participant if that speech is missed. However, the technology described in Patent Document 2 does not replay the audio or video at high speed when rewinding. Therefore, participants cannot quickly grasp the connection between the contents of the conference.

[0074] In contrast, in the above-described aspects of the present disclosure, content corresponding to conference data from a time range after a point in time during an online conference is played back at high speed. Therefore, it is possible to catch up and play back conference content that a user missed while participating in the conference without missing anything. Furthermore, by playing back the content at high speed, a user can quickly grasp the connection between the previous and next conference content.

[0075] In another aspect of the conference support system, at least one processor may cause a terminal of a user other than the user to display a status indicating that the user is playing content. In this case, the status of the user playing content is shared among users participating in the online conference. Since other users can understand the status, the online conference can proceed smoothly.

[0076] In another aspect of the conference support system, at least one processor may calculate a progress status based on the playback speed and elapsed time of the content, and display the progress status on the terminal of another user. In this case, the progress status of the user playing the content is shared among users participating in the online conference. Since other users can understand the progress status, the online conference can proceed smoothly.

[0077] In another aspect of the conference support system, at least one processor may acquire user identification information identifying the user who is speaking the audio, associate the audio with the user identification information to generate content, and display the user identification information along with a status on a terminal of another user in accordance with the progress of the online conference. In this case, the users participating in the online conference share information about who is speaking as the user playing the content is listening to. This allows the other users to understand the details of the progress of the conference.

[0078] In another aspect of the conference support system, at least one processor may calculate the time remaining until the playback of the content is completed as the progress status. In this case, the time remaining until the playback of the content is completed is shared with other users. This allows the other users to accurately grasp the progress status.

[0079] In another aspect of the conference support system, at least one processor may calculate a progress rate of the content playback as the progress status. In this case, the progress rate of the content playback is shared with other users. This allows the other users to intuitively understand the progress status.

[0080] In another aspect of the conference support system, the end point of the time range may be the time when the time point is acquired. At least one processor may play back the content and display the online conference on the user's terminal. In this case, the content from the previous time point to the time when the time point is acquired is played back at high speed, and the online conference from the time when the time point is acquired is displayed in real time. This reduces the time required to play the content.

[0081] [Variations] The present disclosure has been described in detail above based on the embodiments. However, the present disclosure is not limited to the above embodiments. Various modifications of the present disclosure are possible without departing from the spirit and scope of the present disclosure.

[0082] The content generation unit 14 may generate content data in text data format by performing voice recognition on the conference data. For example, the content generation unit 14 may generate content data in which at least the user's speech is converted into text. The content generation unit 14 may generate content data consisting of only text data format, or may generate content data consisting of a combination of text data format and audio or video. The content generation unit 14 may generate content data that identifies the speaker of each audio by associating user identification information with text data. The content playback unit 23 may display the content data in text data format on a display device. In this case, an environment can be provided in which the contents of the conference can be quickly understood.

[0083] The content playback unit 23 may execute skip playback, which skips a part of the content. Skip playback may be executed, for example, triggered by a change in the playback position on the progress bar 408 or a cue operation on the operation interface 406. Skip playback can reduce the time required to play the content.

[0084] Content may be labeled with one or more labels. For example, the content generation unit 14 may detect the volume or number of speakers in the conference data over time and determine whether the detected value is equal to or greater than a predetermined threshold. The content generation unit 14 may generate content data labeled with a label such as "the conference is active" for a time period when the detected value is equal to or greater than the threshold. Other examples of labeling include "the conference is quiet," "a specific user is speaking," and "the speaker is changing." The content playback unit 23 may perform skip playback using the label as a cue position. Cueing may be triggered by a user operation or may be performed automatically without receiving a user operation. In one example, the content playback unit 23 may perform automatic cue skip playback so as to play only the content within the time range indicated by the label. By enabling playback of parts of a conference where activity was active, user convenience is improved.

[0085] In the above embodiment, the online conference is described as a conference format in which video images are shared, but it may also be a conference format using only audio. Also, in the above embodiment, it is described that the status determination unit 16 calculates the progress status, but the progress status may be shared from the first user terminal to the second user terminal. As another example, the second user terminal may send an interruption request to interrupt the playback of content to the first user terminal. The conference display unit 21 of the first user terminal that receives the interruption request may display the conference screen 300.

[0086] In the above embodiments, the conference support system 1, 1A is configured using the server 10. However, the conference support system may also be applied to an online conference between user terminals 20 without using the server 10. In this case, each functional element of the server 10 may be implemented in one of the user terminals, or may be implemented separately in multiple user terminals. In this regard, the conference support program may be implemented as a client program. The conference support system may be configured using a server or may be configured without using a server. That is, the conference support system may be a client-server system, or may be a client-client system using P2P (Peer to Peer) or E2E (End to End) encryption. The client-client system improves the confidentiality of online conferences. In one example, the conference support system can prevent online conference audio and other data from being leaked to third parties by E2E encrypting the online conference between the user terminals 20.

[0087] In this disclosure, the expression "at least one processor executes a first process, executes a second process, ... executes an nth process" or a corresponding expression is a concept that includes cases where the entity executing the n processes from the first process to the nth process (i.e., the processor) changes midway through. In other words, this expression is a concept that includes both cases where all n processes are executed by the same processor and cases where the processor changes among the n processes according to an arbitrary policy.

[0088] The processing procedure of the method executed by at least one processor is not limited to the examples in the above embodiment. For example, some of the steps (processing) described above may be omitted, or the steps may be executed in a different order. Furthermore, any two or more of the steps described above may be combined, or some of the steps may be modified or deleted. Alternatively, other steps may be executed in addition to the steps described above.

[0089] Any part or all of the functional units described in this specification may be realized by a program. The program mentioned in this specification may be distributed by being non-temporarily recorded on a computer-readable recording medium, or may be distributed via a communication line (including wireless communication) such as the Internet, or may be distributed in a state where it is installed on any terminal.

[0090] Based on the above description, a person skilled in the art may conceive additional effects or various modifications of the present disclosure, but the aspects of the present disclosure are not limited to the individual embodiments described above. Various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and spirit of the present disclosure, which is derived from the content defined in the claims and their equivalents.

[0091] For example, a configuration described herein as one device (or component; the same applies hereinafter) (this includes a configuration depicted as one device in the drawings) may be realized by multiple devices. Alternatively, a configuration described herein as multiple devices (this includes a configuration depicted as multiple devices in the drawings) may be realized by one device. Alternatively, some or all of the means or functions included in a certain device (e.g., a server) may be included in another device (e.g., a user terminal).

[0092] Not all of the features described in this specification are essential requirements. For example, features described in this specification but not claimed may be considered optional additional features.

[0093] The applicant is only aware of the known technology described in the "Prior Art Documents" section of this specification. Please note that this disclosure does not necessarily aim to solve the problems in that known technology. The problem to be solved by this disclosure should be identified in consideration of the entire specification. For example, if this specification states that a specific configuration achieves a certain effect, it can also be said that the problem corresponding to that effect is solved. However, the description of that effect does not necessarily intend that such a specific configuration be an essential requirement. [Explanation of symbols]

[0094] 1,1A...conference support system, 10...server, 11...conference control unit, 12...recording unit, 13...request receiving unit, 14...content generation unit, 15...output unit, 16...status determination unit, 20...user terminal, 21...conference display unit, 22...request sending unit, 23...content playback unit, 24...sharing unit, 30...conference database, 300,300A,300B...conference screen, 304B...time display column, 304C...indicator, 307,308...status message, 309...embedding unit, 400,400A,400B...playback screen, P1...server program, P2...client program.

Claims

1. at least one processor; the at least one processor: Recording meeting data including audio from online meetings, Performing speech recognition on the conference data to convert it into text; Obtaining a time going back in time to the online conference from a user's terminal; outputting text that is labeled with one or more labels indicating whether the online conference is active or not, corresponding to the conference data for a time range after the time point and based on whether the volume of the conference data or the number of speakers is equal to or greater than a predetermined threshold; displaying the text on the user's terminal while the online conference is in progress; Meeting support system.

2. A system comprising at least one processor, the at least one processor: Recording meeting data including audio from online meetings, Performing speech recognition on the conference data to convert it into text; acquiring user identification information for identifying a user who is the speaker of the voice, and associating the user identification information with the conference data that has been converted into text; Obtaining a time going back in time to the online conference from a user's terminal; outputting text corresponding to the conference data for a time range after the time point; displaying the text on the user's terminal while the online conference is in progress; displaying, on a terminal of a user other than the user, a status indicating that the user is currently displaying the text, and the user identification information along with a progress status of displaying the text; Meeting support system.

3. A system comprising at least one processor, the at least one processor: Recording meeting data including audio from online meetings, Performing speech recognition on the conference data to convert it into text; Obtaining a time going back in time to the online conference from a user's terminal; outputting text corresponding to the conference data for a time range after the time point; displaying the text on the user's terminal while the online conference is in progress; calculating a remaining time until the display of the text ends, and displaying the remaining time on a terminal of a user other than the user; Meeting support system.

4. 4. The conference support system according to claim 1, wherein the at least one processor causes a terminal of a user other than the user to display a status indicating that the user is currently displaying the text.

5. 4. The conference support system according to claim 1, wherein the at least one processor causes the text to be displayed on the user's terminal and the online conference to be displayed on the user's terminal.

6. The conference support system according to claim 2 or 3, wherein the at least one processor outputs the text with one or more labels.

7. the at least one processor: generating content corresponding to the conference data for a time range after the time point; During the online conference, the content is played back on the user's terminal at a playback speed faster than the original playback speed of the conference data. The conference support system according to any one of claims 1 to 3.

8. A conference support method executed by a conference support system having at least one processor, comprising: Recording conference data including audio of the online conference; performing speech recognition on the conference data to convert it into text; acquiring a time point going back in time to the online conference from a user's terminal; outputting text corresponding to the conference data for a time range after the time point, the text being labeled with one or more labels indicating whether the online conference is active or not, based on whether the volume of the conference data or the number of speakers is equal to or greater than a predetermined threshold; displaying the text on the user's terminal while the online conference is in progress; A meeting support method including:

9. A conference support method executed by a conference support system having at least one processor, comprising: Recording conference data including audio of the online conference; performing speech recognition on the conference data to convert it into text; acquiring user identification information for identifying a user who is the speaker of the voice, and associating the user identification information with the conference data that has been converted into text; acquiring a time point going back in time to the online conference from a user's terminal; outputting text corresponding to the conference data for a time range after the time point; displaying the text on the user's terminal while the online conference is in progress; a step of displaying the user identification information on a terminal of a user other than the user along with a status indicating that the user is currently displaying the text and a progress status of the display of the text; A meeting support method including:

10. A conference support method executed by a conference support system having at least one processor, comprising: Recording conference data including audio of the online conference; performing speech recognition on the conference data to convert it into text; acquiring a time point going back in time to the online conference from a user's terminal; outputting text corresponding to the conference data for a time range after the time point; displaying the text on the user's terminal while the online conference is in progress; calculating a remaining time until the display of the text ends, and displaying the remaining time on a terminal of a user other than the user; A meeting support method including:

11. Recording conference data including audio of the online conference; performing speech recognition on the conference data to convert it into text; acquiring a time point going back in time to the online conference from a user's terminal; outputting text corresponding to the conference data for a time range after the time point, the text being labeled with one or more labels indicating whether the online conference is active or not, based on whether the volume of the conference data or the number of speakers is equal to or greater than a predetermined threshold; displaying the text on the user's terminal while the online conference is in progress; A meeting support program that causes a computer to execute the above.

12. A step of recording conference data including audio of an online conference; performing speech recognition on the conference data to convert it into text; acquiring user identification information for identifying a user who is the speaker of the voice, and associating the user identification information with the conference data that has been converted into text; acquiring a time point going back in time to the online conference from a user's terminal; outputting text corresponding to the conference data for a time range after the time point; displaying the text on the user's terminal while the online conference is in progress; a step of displaying the user identification information on a terminal of a user other than the user along with a status indicating that the user is currently displaying the text and a progress status of the display of the text; A meeting support program that causes a computer to execute the above.

13. A step of recording conference data including audio of an online conference; performing speech recognition on the conference data to convert it into text; acquiring a time point going back in time to the online conference from a user's terminal; outputting text corresponding to the conference data for a time range after the time point; displaying the text on the user's terminal while the online conference is in progress; calculating a remaining time until the display of the text ends, and displaying the remaining time on a terminal of a user other than the user; A meeting support program that causes a computer to execute the above.

Images