Smart Query Buffering Mechanism

Abstract

Methods, apparatus, and computer program products for controlling a web-based presentation are described. In one embodiment of the present invention, the presentation's previously presented and future content is analyzed. When a user question is received from a user terminal communicatively coupled to the web-based presentation, the user question is buffered so that the system can determine, based on the analyzed content, whether content answering the user question has been or will be presented in the presentation. In response to the determination, the system performs an action. In an embodiment, the system either: 1) if the content has been or will be presented in the presentation, the system provides a notification in a user interface regarding the content to be answered and maintains the user question in a buffered state; or 2) if the content has not been presented and will not be presented, the system releases the user question from the buffered state so that the user question can be forwarded to the presenter of the presentation.

Description

[Technical Field]

[0001] The present disclosure relates generally to user interfaces and, more particularly, to a user interface for conducting a live web conferencing session. [Background technology]

[0002] When presenting in a live web conferencing session, presenters are often interrupted by other participants. Often, other participants will ask questions that have been or will be addressed by the presenter in other parts of the presentation. The question may have been addressed in a previous slide when the other participants were not paying attention or did not understand the slide's importance due to a lack of context. Or, due to organizational choices about the presentation made by the presenter, the question will be answered in upcoming material.

[0003] These unnecessary questions are redundant, time-inefficient, and distracting to other participants from the presentation.

[0004] Further improvements to the user interface are needed to address this issue. Summary of the Invention

[0005] According to the present disclosure, methods, apparatus, and computer program products for controlling a web-based presentation are described. In one embodiment of the present invention, the presentation's previously presented and future content is analyzed. When a user question is received from a user terminal communicatively coupled to the web-based presentation, the user question is buffered so that the system can determine, based on the analyzed content, whether content answering the user question has been or will be presented in the presentation. In response to the determination, the system performs an action. In an embodiment, the system either: 1) if the content has been or will be presented in the presentation, the system provides a notification in a user interface regarding the content to be answered and maintains the user question in a buffered state; or 2) if the content has not been presented and will not be presented, the system releases the user question from the buffered state so that the user question can be forwarded to the presenter of the presentation.

[0006] An apparatus embodiment includes a processor and computer memory holding computer program instructions for executing the above-described method to control a web-based presentation.

[0007] In a computer program product embodiment, the computer program product includes a computer program product stored in a non-transitory computer readable medium for use with a data processing system and carrying computer program instructions executed by the data processing system for controlling a web-based presentation to perform the aforementioned method.

[0008] In another embodiment of the present invention, a method for controlling a web-based presentation includes analyzing content of the web-based presentation and receiving a user question from a user terminal communicatively coupled to the web-based presentation. The user question is buffered while determining whether content answering the user question will be presented in the presentation. A user interface of the web-based presentation is modified based on the analyzed content, including whether the content answering the user question will be presented in the presentation.

[0009] The foregoing has outlined some of the more pertinent features of the disclosed subject matter. These features should be construed as merely illustrative. Many other beneficial results can be attained by applying the disclosed subject matter in a different manner or by modifying the invention, as will be explained.

[0010] For a more complete understanding of the present invention, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which: [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 depicts an exemplary block diagram of a distributed data processing environment in which exemplary aspects of the illustrative embodiments may be implemented. [Figure 2] 1 is an exemplary block diagram of a data processing system in which exemplary aspects of the illustrative embodiments may be implemented; [Figure 3] 1 is a flow diagram of one embodiment of the present invention for conducting a web conference. [Figure 4] 1 is a flow diagram for one embodiment of the present invention for preparing a system to conduct a web conference. [Figure 5] FIG. 1 is an architecture diagram of one embodiment of the present invention. [Figure 6] 4 is a flow diagram for another embodiment of the present invention for conducting a web conference. [Figure 7] FIG. 10 is a diagram of a user interface for a user according to one embodiment of the present invention. [Figure 8] 1 is a flow diagram for one embodiment of the present invention for collecting information related to questions and answers during a web conference. DETAILED DESCRIPTION OF THE INVENTION

[0012] At a high level, the present invention provides a method, system, and computer program product for conducting a web conference. When a participant indicates that they would like to ask a question, the question is buffered before being forwarded to the presenter. While the question is being held, the system searches the presentation to see if the question is answered. If the question is found in the presentation, the answer is presented to the participant. If the answer is not found in the presentation, the question is forwarded to the presenter.

[0013] Embodiments will now be described more fully herein with reference to the accompanying exemplary drawings. However, the present invention may be embodied in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. The embodiments are provided so that this disclosure conveys the principles of the invention, and those skilled in the art will understand that variations on these examples are within the scope of the invention. In this description, details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the presented embodiments.

[0014] 1-2, exemplary diagrams of data processing environments are provided in which exemplary embodiments of the present disclosure may be implemented. It should be understood that FIGS. 1-2 are exemplary only and are not intended to assert or imply any limitation with regard to the environments in which aspects or embodiments of the disclosed subject matter may be implemented. Many modifications to the depicted environments may be made without departing from the scope of the present invention.

[0015] Referring now to the drawings, Figure 1 shows a visual representation of an exemplary distributed data processing system in which aspects of the exemplary embodiments may be implemented. Distributed data processing system 100 may include a network of computers in which aspects of the exemplary embodiments may be implemented. Distributed data processing system 100 includes at least one network 102, which is the medium used to provide communications links between the various devices and computers connected to each other within distributed data processing system 100. Network 102 may include connections such as wires, wireless communications links, fiber optic cables, etc.

[0016] In the illustrated example, server 104 and server 106 are connected to network 102 along with network storage unit 108. Additionally, clients 110, 112, and 114 are also connected to network 102. Clients 110, 112, and 114 may be, for example, smartphones, tablet computers, personal computers, network computers, etc. In the illustrated example, server 104 provides data such as boot files, operating system images, and applications to clients 110, 112, and 114. Clients 110, 112, and 114 are clients to server 104 in the illustrated example. Distributed data processing system 100 may include additional servers, clients, and other devices not shown. One or more of the server computers may be mainframe computers connected to network 102. The mainframe computer may be, for example, an IBM® System z® mainframe running the IBM® z / OS® operating system. A mainframe storage unit and workstations (not shown) are connected to the mainframe. A workstation can be either a personal computer connected directly to the mainframe, communicating via a bus, or a console terminal connected directly to the mainframe via a display port.

[0017] In the depicted example, distributed data processing system 100 is the Internet, where network 102 represents a worldwide collection of networks and gateways that use the Transmission Control Protocol / Internet Protocol (TCP / IP) suite of protocols to communicate with one another. At the heart of the Internet is a backbone of high-speed data communication lines between major nodes or host computers, consisting of thousands of commercial, government, educational, and other computer systems that route data and messages. Of course, distributed data processing system 100 may also be implemented to include a number of different types of networks, such as, for example, an intranet, a local area network (LAN), a wide area network (WAN), etc. As stated above, FIG. 1 is intended as an example, and not as an architectural limitation for various embodiments of the disclosed subject matter, and thus the specific elements illustrated in FIG. 1 should not be considered limitations on the environments in which illustrative embodiments of the invention may be implemented.

[0018] 2, a block diagram of an exemplary data processing system 200 is shown in which aspects of the exemplary embodiments may be implemented. Data processing system 200 is an example of a computer, such as client 114 of FIG. 1, in which computer usable code or instructions implementing the processes for the exemplary embodiments of the present disclosure may be located.

[0019] Referring now to Figure 2, a block diagram of a data processing system in which illustrative embodiments may be implemented is shown. Data processing system 200 is an example of a computer, such as server 104 or client 110 in Figure 1, in which computer-usable program code or instructions implementing the processes for the illustrative embodiments may be located. In this illustrative example, data processing system 200 includes a communications fabric 202 that facilitates communications between a processor unit 204, a memory 206, persistent storage 208, a communications unit 210, an input / output (I / O) unit 212, and a display 214.

[0020] Processor unit 204 serves to execute instructions for software that may be loaded into memory 206. Processor unit 204 may be a set of one or more processors or may be a multi-processor core, depending on the particular implementation. Additionally, processor unit 204 may be implemented using one or more heterogeneous processor systems in which a main processor resides on a single chip with secondary processors. As another illustrative example, processor unit 204 may be a symmetric multi-processor (SMP) system that includes multiple processors of the same type.

[0021] Memory 206 and persistent storage 208 are examples of storage devices. A storage device is any piece of hardware that can store information on a temporary and / or permanent basis. In these examples, memory 206 may be, for example, a random access memory or any other suitable volatile or non-volatile storage device. Persistent storage 208 may take various forms depending on the particular implementation. For example, persistent storage 208 may include one or more components or devices. For example, persistent storage 208 may be a hard drive, flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination of the above. The media used by persistent storage 208 may also be removable. For example, a removable hard drive may be used for persistent storage 208.

[0022] In such an example, communications unit 210 facilitates communication with other data processing systems or devices. In such an example, communications unit 210 is a network interface card. Communications unit 210 may facilitate communication through the use of physical and / or wireless communications links.

[0023] Input / output unit 212 allows for the input and output of data to and from other devices that may be connected to data processing system 200. For example, input / output unit 212 may provide a connection for user input through a keyboard and mouse. Additionally, input / output unit 212 may send output to a printer. Additionally, input / output unit may provide a connection for a microphone for audio input from a user and a speaker for providing audio output from the computer. Display 214 provides a mechanism for displaying information to a user.

[0024] Instructions for the operating system and applications or programs are located on persistent storage 208. These instructions may be loaded into memory 206 for execution by processor unit 204. The processes of the various embodiments may be performed by processor unit 204 using computer-implemented instructions, which may be located in a memory, such as memory 206. These instructions are referred to as program code, computer-usable program code, or computer-readable program code, which may be read and executed by a processor in processor unit 204. The program code in the various embodiments may be embodied on different physical or tangible computer-readable media, such as memory 206 or persistent storage 208.

[0025] The program code 216 is located in a functional form on a selectively removable computer-readable medium 218 and may be loaded onto or transferred to the data processing system 200 for execution by the processor unit 204. The program code 216 and the computer-readable medium 218 form a computer program product 220 in these examples. As an example, the computer-readable medium 218 may be in a tangible form, such as an optical or magnetic disk, that is inserted into or placed into a drive or other device that is part of the persistent storage 208 for transfer onto a storage device, such as a hard drive that is part of the persistent storage 208. In a tangible form, the computer-readable medium 218 may also take the form of persistent storage, such as a hard drive, thumb drive, flash memory, or the like, that is connected to the data processing system 200. The tangible form of the computer-readable medium 218 is also referred to as a computer-recordable storage medium. In some cases, the computer-recordable medium 218 may not be removable.

[0026] Alternatively, program code 216 may be transferred to data processing system 200 from computer-readable medium 218 through a communications link to communications unit 210 and / or a connection to input / output unit 212. The communications link and / or connection may be physical or wireless in illustrative examples. The computer-readable medium may also take the form of intangible media, such as a communications link or wireless transmission containing program code. The various components illustrated for data processing system 200 are not meant to provide architectural limitations to the manner in which various illustrative embodiments may be implemented. Various illustrative embodiments may be implemented in data processing systems including components in addition to or instead of those illustrated for data processing system 200. Other components illustrated in FIG. 12 may differ from the illustrated illustrative example. By way of example, storage devices in data processing system 200 are any hardware apparatus capable of storing data. Memory 206, persistent storage 208, and computer-readable medium 218 are examples of tangible forms of storage devices.

[0027] As another example, a bus system may be used to implement communications fabric 202 and may be comprised of one or more buses, such as a system bus or an input / output bus. Of course, the bus system may be implemented using any suitable type of architecture that provides for a transfer of data between different components or devices attached to the bus system. Further, a communications unit may include one or more devices used for transmitting and receiving data, such as a modem or a network adapter. Further, a memory may be, for example, memory 206 or a cache such as found in interfaces and memory controller hubs that may be present in communications fabric 202.

[0028] Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java™, Smalltalk®, C++, C#, Objective-C, and traditional procedural programming languages ​​such as Python or C. The program code may run entirely on the user's computer, as a standalone software package, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet Service Provider).

[0029] Those skilled in the art will appreciate that the hardware in Figures 1-2 may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash memory, equivalent non-volatile memory, optical disk drives, etc., may be used in addition to or in place of the hardware depicted in Figures 1-2. Additionally, the processes of the illustrative embodiments may be applied to multiprocessor data processing systems other than the SMP systems described above without departing from the scope of the disclosed subject matter.

[0030] The techniques described herein may operate with a standard client-server paradigm, such as that shown in FIG. 1, in which a client machine communicates with an Internet-accessible, web-based portal running on a set of one or more machines. End users operate Internet-enabled devices (e.g., desktop computers, notebook computers, Internet-enabled mobile devices, etc.) that can access and interact with the portal. Typically, each client or server machine is a data processing system, such as that shown in FIG. 2, including hardware and software; these entities communicate with each other over a network, such as the Internet, an intranet, an extranet, a private network, or any other communications medium or link. A data processing system typically includes one or more processors, an operating system, one or more applications, and one or more utilities.

[0031] Embodiments will now be described more fully herein with reference to the accompanying exemplary drawings. However, the present invention may be embodied in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. The embodiments are provided so that this disclosure conveys the principles of the invention, and those skilled in the art will understand that variations on these examples are within the scope of the invention. In this description, details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the presented embodiments.

[0032] In an embodiment of the present invention, when a participant indicates that they would like to ask a question, the system records the participant's spoken question before allowing the system to forward the question to the presenter or before allowing the participant microphone to unmute for the web conference. This allows the system to hold or buffer the question while operations occur in the background. A natural language processing (NLP) module renders the question into a format that can be used to perform a lookup to determine whether an answer to the question is already in the presentation. A search module uses the NLP output to understand the question spoken by the user and generate an answer if one exists in the presentation. In an embodiment, if the answer to the question has already been answered in a previous slide, the answer is presented to the user by referring the participant to the previous slide or in a new pop-up window. In an embodiment of the present invention, if the answer will be answered in a future slide, the system informs the user that the question will be addressed in a future slide. Alternatively, a pop-up window may present the user with the answer in a future slide. In another embodiment, the user's question is entered in written form rather than spoken form. The interface used to enter questions, for example a chat-type user interface, can also be used to present answers.

[0033] In some embodiments, the system analyzes the user's question to determine whether it is appropriate for the presentation, e.g., whether it is within the scope of the presentation, to avoid questions that would distract from the presentation. If it is not deemed appropriate, the system prevents the question from interrupting the presentation. A notification may be sent to the participant who asks an out-of-scope question. In some embodiments, the out-of-scope questions are collected and later forwarded to the presenter, allowing the presenter to answer the out-of-scope questions using a different means, such as email.

[0034] If the system determines that the question has not yet been answered in the presentation materials, that the question is appropriate for the presentation, and that there is no upcoming content that will address the question, the system unmutes the participant. The presenter will hear the question, possibly with some delay, but the question will eventually be relayed to the presenter. The presenter may design the presentation so that questions can be forwarded to the questioner periodically at predetermined times or gaps in the presentation. In an embodiment of the present invention, a module accepts a set of break points in the presentation from the presenter. While the presentation is being presented, the system determines when the next break point in the set of break points occurs in the presentation. The system waits until the next break point before releasing the user question from the buffer state.

[0035] In an alternative embodiment, the system determines whether the presenter is speaking and waits for a gap in the presentation to relay the question. In yet another embodiment, a natural break in the presentation, such as a slide change, may be the event that triggers the relay of the question to the presenter. While the participant is waiting, in some embodiments, a notification may be sent to the participant that their question is in queue.

[0036] In embodiments of the present invention, presentation materials may be augmented with additional question and answer materials. For example, a presenter may have given a particular presentation or a similar presentation on a previous date, so questions and answers from previous sessions are stored in a frequently asked questions (FAQ) database. If the presenter is a subject matter expert, there may be other prepared materials that can augment the presentation materials. In some embodiments, a chatbot is used to present answers to participants who ask questions in a chat interface.

[0037] In an embodiment of the present invention, the system groups questions by intent, i.e., groups questions from multiple participants, to prevent the same question from being relayed or asked. In an embodiment of the present invention, grouping is performed for questions that are not answered in the presentation or supporting material related to the presentation. The group of questions with the highest frequency may be moved to the top of the queue for the presenter to address first. NLP analysis may be performed on groups of similar questions to determine which of the groups of questions is most complete or representative. The most complete or representative of the group of questions, or a reduced subset of the group of questions, is forwarded to the presenter.

[0038] One embodiment of the present invention is shown in Figure 3. In step 301, a conference call system is set up in accordance with an embodiment of the present invention. In an embodiment, the required amount of the presentation is downloaded to the participants. Voice and chat functionality is enabled using the query buffering function described herein. That is, rather than immediately forwarding questions to the presenter, they are buffered so that additional processing can be performed without interrupting the presentation. In this step, the query buffering mechanism begins listening to the voice conversation and processing chat about the presentation. Particularly relevant to the present invention, participant input is considered for questions to the presenter. In a preferred embodiment, these tasks are performed by a smart listener module.

[0039] Step 303 involves a natural language processing (NLP) module. In the previous step, the presentation is converted into a usable format, e.g., written into a "document," so that as the presentation progresses, the user context can be summarized to understand user questions in context, creating a user context for the question. In an embodiment, the "context document" is a usable format that can be queried later in the process. The context document may include time references for when slides are presented in the presentation, as well as time references for NLP rendering of audio from the presenter and other participants. The context document is used to determine the current context for the user question so that the most relevant content in the presentation material can be selected. In a preferred embodiment, the presentation material is ingested by a machine learning component, and a usable format is created. In an embodiment of the present invention, a component from the IBM Watson™ product family (IBM Watson TM Products within the product family are used. As will be described, presentation materials include written and audiovisual materials, as well as audio and chat streams captured by the system. This information is also optionally stored in a frequently asked questions (FAQ) database or other suitable knowledge store, database, or other suitable format in step 305.

[0040] In step 307, a user asks a question during the presentation, either via voice or a chat channel. In step 309, the question is temporarily delayed and muted (if necessary) to allow for additional processing without interrupting the presentation. In step 311, an NLP module derives the meaning of the question in a usable format. In step 313, the system determines whether the question has been answered in the presentation. In this step, the system considers the set of questions that have been answered in the presentation, either before the question was asked or in slides that will be presented later in the presentation so far. In embodiments of the present invention, cognitive discovery functionality is used against the conversation during the presentation, e.g., stored in a context document, and the presentation document to detect whether the question has been or will be answered. The context document created in step 303 is referenced by the cognitive discovery functionality to determine the current context for the user question so that the most relevant content in the presentation material can be selected.

[0041] In another embodiment, questions and answers are indexed in a FAQ database and the system uses a search engine to find answers.

[0042] If the question is answered, the system retains the question from the user participant for the duration of the presentation and displays the answer in the interface for that particular user in step 315. Also in this step, if the question will be answered in upcoming materials, the system retains the question from the user and informs the user that the question will be answered in upcoming materials.

[0043] An embodiment of the present invention determines whether a question is out of scope / inappropriate for a presentation. For example, a question about college football may be considered out of scope for a presentation on machine learning. Determining whether a question is out of scope may be performed by semantic analysis, comparing the words used in the question to a set of keywords in the presentation. If the question is out of scope, the system retains the question from the user for the duration of the presentation and notifies the presenter of the question after the presentation. The system also notifies the participants in a notification that the question is out of scope for the presentation. The notification may inform the participants that the question is being forwarded to the presenter for later response. The presenter may want to follow up with the user after the presentation using another channel or review whether the system correctly classified the question as out of scope or inappropriate. In future versions of the presentation, the presenter may want to include the question in the presentation or in a set of FAQs.

[0044] If the question is not answered in the presentation or supporting materials, after buffering the system allows the question to pass through to the presenter.

[0045] In an embodiment of the present invention, questions not answered in the presentation or supporting materials are grouped by intent to prevent duplication. Once duplicates are removed from each group, the system allows the remaining questions, i.e., the reduced group of questions, to be passed to the presenter through the web conferencing user interface. The system may select the most complete or representative question from the group of questions with the same intent so that the reduced group of questions can become a single question.

[0046] The questions and answers are recorded by the system. In an embodiment of the present invention, both answered and unanswered questions are fed into a learning mechanism to improve system quality. System quality can be improved by adding new questions and answers to the set of FAQs and understanding the various ways in which users ask questions about the presentation subject. Furthermore, the recorded questions and answers provide the presenter with information to know the audience topics of interest.

[0047] In the above embodiment, there is an assumption that the answer is not on the current slide because that is not the usual case. However, in an alternative embodiment, there may be times when it is appropriate to have the participant pose a question, and the question should be allowed. In this embodiment, a cognitive discovery function determines that the answer is on the current slide. In an embodiment, a cognitive discovery function in the interface may point the user to a specific location in the current slide that answers the question. The user may feel that the slide does not address the question. With additional user input, such as a negative response to "Does this answer your question, yes or no?", the system presents the question to the presenter.

[0048] While the above-described embodiment assumes an audiovisual presentation in which participants communicate with the presenter through voice questions, in another embodiment of the invention, participants communicate with the presenter using a written interface into which questions are typed. While written questions are not distracting to the set of participants, questions viewed by the presenter can still be distracting to the presenter, and a distracted presenter will result in a degraded presentation.

[0049] In an embodiment of the present invention, audio from a presenter on the audio channel of a presentation is captured, analyzed by an NLP application, and converted into a format that can be used by a learning system. Thus, presentation materials, e.g., a slide show, which may already contain both text and image information, are supplemented with actual words spoken by the presenter. These words may also provide answers to participants' questions. Spoken words from other participants, which are allowed to become part of the presentation after a buffering period, may also be useful for determining the context of the question. In an embodiment of the present invention, such words also become part of the context document.

[0050] FIG. 4 shows a flow diagram for one embodiment of the present invention for preparing a system to conduct a web conference. In step 401, a presentation is populated by machine learning so that the presentation content can be searched by the system during the presentation for answers to questions asked by web conference participants. In an embodiment of the present invention, answers can be tiered in a hierarchy. 1) A simple answer to the question is presented: Q: What is the largest waterfall in North America? A: Niagara Falls; 2) A slide on which the answer is provided; and 3) An excerpt from an FAQ related to the understood question. The hierarchy of answers can be linked by a hyperlink or GUI control in the first panel. Machine learning prepares materials from the presentation so that a Q&A interface can be presented to participants in future web conferences. Although not shown, information can be added to an FAQ database.

[0051] In step 403, the system determines whether there is a previous version of the web conference presentation. If so, in step 405, the previous presentation and any associated questions and answers, an FAQ database, and other material marked as relevant to the previous presentation are retrieved. Note that such material may have already been retrieved and classified by machine learning, and thus this step may be to associate the already retrieved material with the current presentation. Step 407 indicates that all of the retrieved material related to the previous presentation may be stored in a single FAQ database rather than multiple information types. In another embodiment where related material is stored in multiple forms, it may still be useful to store the presentation information in the FAQ database. However, this step is not performed in some embodiments of the present invention.

[0052] There may be other material on the topic prepared by the presenter. There may be related material that the presenter is aware of but that has been prepared by others. This may be the first time the presenter has prepared a presentation on the topic. Thus, in step 409, the system searches for other material designed by the presenter that is relevant to the current presentation. In a preferred embodiment, to give the presenter a measure of control, only material designed by the presenter is used. In an alternative embodiment, a general search of available knowledge may be performed and ingested by the system. In step 411, other relevant material is ingested. In embodiments where the relevant material is compiled into an FAQ database, in step 413, pertinent, related ingested material is added to the FAQ database.

[0053] Once the presentation, previous versions of the presentation, and related materials have been captured and converted into a usable format, the process ends at step 415. Those skilled in the art will note that in different embodiments, information may be converted into a number of different formats.

[0054] 5 is an architectural diagram of one embodiment of the present invention. A set of participants 501 and questioner participants 503 are connected to a presenter 507 via a network 505. The web conference is run by a web service 508.

[0055] In some embodiments, the web service is divided into multiple modules. For example, a natural language module (NLP) 509 is used to understand questions posed by participants and convert the questions into a format usable by other modules in the web service. The NLP module 509 may perform other NLP tasks for other modules in the web service as needed. A graphical user interface module 511 is used to present the presenter's presentation, i.e., change slides when appropriate, present a chat interface, and even present the question and answer interface of the present invention within the chat interface or a separate chat interface.

[0056] A search module 513 is used to search the presentation and designated related materials for answers to one or more currently posed questions. In some embodiments, the search module only searches the actual presentation for answers. However, in preferred embodiments, the system has already indexed or captured material from the presentation, previous related presentations, or designated "related" materials into a database or other usable format.

[0057] Many aspects of the system can be found in a smart listener module 515 and an actions module 517 in embodiments of the present invention. The listener module 515 identifies, categorizes, and groups responses from participants so that the system or presenter can respond appropriately. Audience participants 501, 503 submit responses using a voice or text (chat) interface. The listener module may categorize responses, i.e., whether they are questions, feedback, votes, comments, or complaints. In embodiments of the present invention, categorized responses use response types with more detailed information (e.g., question, complaint, high priority) and weights to help prioritize required actions. In embodiments of the present invention, only questions or complaints are forwarded to the presenter, as they must respond to these categorization inputs. This step determines how relevant a response or question is to the presentation. If a question or response is relevant, a higher weight is assigned to the answer; i.e., a higher weight is more likely to create an interruption in the presentation. If the response or question is not relevant, the system discards the response or adds it to a list of questions that can be addressed later by the presenter. Preferably, the listener module 515 utilizes semantic content analysis, either natively or through the NLP module 509. Semantic content analysis is used during online conference sessions to understand questions from participants. For example, semantic content analysis performs an analysis of terms and words in the initial question / response for terms relevant to searching the content document created from the ingestion process. Semantic content analysis may also be used to analyze the presenter's speech to prepare a context document used to place participant questions in the appropriate context.

[0058] In embodiments of the present invention, web conference participants are identified by login or IP-based information. In some embodiments, speakers (i.e., presenters, questioners, other participants) may be identified based on Mel-Frequency Cepstral Coefficient (MFCC)-based speaker identification. Machine learning, such as long-short-term memory (LSTM) or other artificial recurrent neural network (RNN) architectures, is used for command or question buffering by a smart listener module.

[0059] In an embodiment of the present invention, the following operations are performed by the listener module 515: Information from the participant question is passed through a sentence parser, resulting in parsed tokens; semantically-adaptive searching methods are used to reduce the search space in a database, e.g., an FAQ database, and find relevant answers from the presentation or accompanying database via NLP; MFCC application or other methods can be used for speaker identification, which is useful for establishing the context of the question.

[0060] In embodiments, the listener module 515 uses statistical clustering and classification methods to extract audio and visual contextual conversation information features about participant questions, index the feature space, and classify related and other information into semantic categories. As an example of statistical clustering, some embodiments use K-means clustering to cluster related data. As described elsewhere, supervised or unsupervised learning is used to label data. As is well known, a weighted variation of Euclidean distance may be used for phrase comparison to determine whether a current question is similar to other questions posed by participants. In embodiments, Euclidean distance compares all components of a question's feature vector with equally weighted similar components of other questions. The listener module 515 then creates a reduced group of questions by excluding duplicate questions within a given group of questions. In some embodiments, question grouping is used for questions not answered in the presentation or supporting materials. Questions answered in the presentation or supporting materials will be addressed with appropriate notice to users.

[0061] The listener module 515 counts and groups similar questions to identify the severity of the event / question. For example, in an event where 50% of the remote audience is asking a question because they cannot see the presentation screen, the event / question needs to be immediately surfaced to the presenter so that the web conference can be corrected or restarted.

[0062] The listener module 515 passes the results of the semantic analysis and clustering to the action module 517 .

[0063] The action module 517 is responsible for deciding whether to interrupt the presenter and alert the presenter to a participant question or a high-severity event affecting multiple participants that requires immediate attention. Examples of presentation actions requiring immediate attention include video or audio issues that can be fixed by increasing the presenter's volume, advancing a slide, or maximizing the view of an item in the presentation. The action module 517 also makes the decision not to interrupt the presenter and instead present the answer to the participants.

[0064] Actions taken by the action module 517 may be configurable from manual mode to automatic mode. Manual mode may be used to allow the system to configure the listener module 515 to collect specific data and score that specific data based on the mode selected by the user. For example, in automatic mode, the listener module 515 may listen for online audience navigation in a presentation channel and determine that the behavior of many attendees indicates that a change is needed. For example, when many attendees try to turn up the audio or zoom in on their screens, this indicates to the listener module that something is unclear in the presentation. The clarity in the presentation may be due to audio, slide, text, or slow connection issues. If the majority of participants exhibit the same behavior, this may cause the action module 517 to interrupt the presenter to correct the situation. The action module may suggest recommended actions. In an embodiment of the present invention, the action module provides a pop-up notification to inform the presenter or hosting administrator to modify the presentation, for example, increase the volume, or in severe cases, reconnect participants to resolve the issue the attendees are experiencing.

[0065] In manual mode, the presenter has greater control over the presentation. The presenter may choose to manually check the settings needed for speech intelligibility. In manual mode, the presenter may choose how the action module 517 handles questions and feedback actions. That is, the presenter may specify when questions are posed during the presentation, for example, only after a slide or at a specified point in the presentation. By limiting questions to specified points in the presentation, the selection has the effect of consolidating participant questions and avoiding duplicate questions.

[0066] In an embodiment of the present invention, the listener module will then pass the appropriate information to the action module.

[0067] The FAQ database 519 represents a database containing data from past presentations or related materials designated by the presenter as relevant to the current presentation. In some cases, the presenter is a subject matter expert (SME) on the current topic. There may be a presenter website where a forum is presented where SMEs or other forum members answer follower questions, or the presenter may be an expert on a social media website where follower questions are posed and answered. Additionally, the presenter may have conducted significant research on the topic. All of these may provide additional materials that may provide answers to attendee questions. Depending on the perceived reliability of the relevant information, the presenter may designate information from such sources as relevant to the presentation. Furthermore, the presenter may designate each piece of information as belonging to a certain level in the hierarchy of information displayed in the interface when attendees drill down into the information. Information in the FAQ database 519 may be compiled and annotated by the presenter according to the information's perceived accuracy.

[0068] Some embodiments of the present invention use a machine learning module 521. In embodiments, an action module may use historical data to learn about the relationship between recommended actions and the meeting setup specified by the presenter. Features used as input features include whether the web conference presentation includes remote participants, and if so, how many or what percentage, the topic or title of the presentation, the number of conference audience members, the number of conference lines for the audience, and the channel used to connect to the conference.

[0069] The machine learning module 521 is also used in some embodiments of the present invention to ingest information from the presentation, previous related presentations, and other related materials. In some embodiments, the machine learning module 521 is used to provide answers to participant questions. As is well known, reconfigurable weights are applied to a set of data inputs to determine a machine learning output. As an example of a suitable machine learning module used in some embodiments for image analysis, python + workflow for OpenCV for Regional Convolutional Neural Network (R-CNN) contextual analysis allows a set of recommended actions to be implemented by the action module. As is well known to those skilled in the art, OpenCV is an image segmentation algorithm. In embodiments of the present invention, the system ingests information in the presentation. The presentation likely includes images, and the system needs to extract information from the images.

[0070] For text and audio information, other well-known algorithms can be used to extract information. For example, text transcription can be performed using natural language processing (NLP), and audio information can be analyzed using well-known speech recognition algorithms.

[0071] Figure 6 shows a flow diagram for another embodiment of the present invention for conducting a web conference. Element 601 shows chat that may take place during a presentation. Due to space limitations, the presentation itself is not shown, but would also be displayed on the screen. Those skilled in the art will understand that the chat may be a spoken chat channel between participants, a chatbot, and the presenter (host).

[0072] As displayed on the screen, the chat includes a first question from User 1: "What time is the transfer?" followed by a response from the presenter (host): "10 AM." This illustrates a case where the chatbot cannot find the answer in the presentation materials. The chat also includes a second question from User 2: "Who is our contact?" In this case, the system may respond with "Bob" because the answer is within the presentation or supporting materials. It may be a question asked by another user in a previous presentation. The chat also includes a response from User 2: "Thank you." The system may recognize this as a polite response that is not a question that needs to be answered. Some chatbots are programmed to respond to such polite responses. However, in a preferred embodiment, the system is programmed to not interrupt the presentation in response to some types of user input that are not questions directed to the presenter, such as polite responses, feedback, votes, comments, complaints, etc.

[0073] In this embodiment, chatbot 603 interacts with the user using a natural language API. In step 605, the conversation is recorded according to the understood words. In some embodiments, indexed documents 607 are also generated for use by search engines. The indexed documents may include the presenter's audio, time stamps from when slides were changed, and other items about the presentation to improve search results. A machine learning system 609 utilizes cognitive discovery capabilities to read the recorded or indexed conversation and create annotations to supplement the indexed documents 607. In some embodiments of the present invention, indexed documents 607 also include presentation materials and supporting materials.

[0074] Step 611 represents a portion of the chat: the question, "Who is our contact?" As shown in step 613, the chatbot's natural language API detects that this phrase is a question that should receive a response. In step 615, the system introduces a delay to buffer the question while the query is processed. In step 617, machine learning 609 determines whether the question is answered. For this question, machine learning 609 determines that the question was answered in the presentation or supporting materials. Thus, in step 619, machine learning 609 does not allow the question to be relayed to the presenter and responds, "Your question was answered previously. Our meeting contact is Bob." Otherwise, if the question was not answered, for example the previous question, "What time is the transfer?", in step 621 the system stops delaying the question and allows the question to pass through the web conferencing channel to the host presenter, so that the host can respond, "10:00 AM."

[0075] 7 is a diagram of a user interface for a user according to one embodiment of the present invention. In a first panel 701, the user types the question "What is enhanced machine learning?" into the NLP interface. In this case, the system responds that the answer can be found within the presentation materials and that the answer to your question can be found in slide 4. A thumbnail of slide 4 may be presented in the panel.

[0076] In the second panel 703, the user types the question, "When should I use machine learning?" In this case, the chatbot was unable to find the answer in the presentation materials, but was able to find several FAQs that appear relevant. The system responds by referring you to the following FAQs that may answer your question: 1) Using Supervised ML, 2) Using Unsupervised ML, and 3) Using Neural Networks. The FAQ listing may include the entire FAQ text or be a summary with a hyperlink to the full text, depending on the available display space in the panel.

[0077] In the third panel 705, the user types the question, "What clustering algorithm will be used?" In this example, the system cannot find a suitable answer either in the presentation or in the supporting materials. Instead, the system responds, "Your question is in the presenter's queue," indicating that the question has been forwarded to the presenter.

[0078] FIG. 8 shows a flow diagram for one embodiment of the present invention for collecting information related to questions and answers during a web conference. Collecting this information improves the system based on feedback learning. This information is used to continually train the system on relevance scores for new questions. Embodiments enable an active learning process to improve presentation materials. For example, after a presentation, a presenter may review questions asked by participants and manually mark those that are highly relevant. Machine learning designates some questions as worthy of the presenter's attention based on content similarity to the presentation, while also marking other questions that are not relevant to the presentation.

[0079] In step 801, a question is received from a web conference participant. In step 803, the system determines whether the question is similar to a previous question already asked. The method used to determine similarity can be semantic analysis, as used above for queue questions. Alternatively, in this step, the determination of similar questions is made based on FAQs in an FAQ database or other supporting materials. If the question is similar to a question already asked, the system removes the question from the queue so that the presenter is not faced with the task of wading through repetitive questions. In some embodiments, before removing a question, the number of similar questions and the time at which similar questions were asked in the presentation may be recorded. By prioritizing the most frequently asked questions, the system and the quality of the presentation are improved.

[0080] If not, in step 805, the system adds the question to the presenter. In some embodiments, the time the question was asked is added to the question queue to help provide context to the presenter and / or the system. In step 807, a search is performed to determine if the answer is found in supplemental materials. The supplemental materials may include the presentation, previous versions of the presentation, an FAQ database, and other supplemental materials selected by the presenter as relevant to the presentation. The supplemental materials may also include questions and answers from the presentation just given. The participant's reaction to the proposed answer, for example, whether the participant accepts the answer or asks a new, related question, is used as an indication of whether the answer is considered correct. In step 809, the system adds candidate answers to the presenter's queue for the question.

[0081] In step 811, the presenter accepts one of the candidate answers to the question or provides an alternative answer. In step 813, the presenter may decide to add the question and answer to an FAQ or other supplemental material. Or, based on participant feedback, a significant number of participants had issues or questions with the way the material was presented, so in step 815 the presenter may decide to add the material to the presentation. In step 817, the current question is removed from the queue. Although not shown in the drawing, the questions and answers may be presented to the presenter sequentially, so that each individual issue or question may be presented individually, and the flow diagram then returns to step 811. After all of the questions have been addressed by the presenter, the process ends in step 819.

[0082] As before, in embodiments of the present invention, the incorporation of the presentation, previous related presentations, and related material shown is performed by a machine learning-based process. In such embodiments, cosine similarity is used to identify similar information about previous questions asked to determine relevance to the presentation, or previous presentation content is also utilized to determine relevance to the current presentation. In such embodiments, NLP is used for entity recognition, and cosine similarity is used for correlation of content information.

[0083] The present invention has many advantages over the prior art. By buffering questions and enabling contextual forwarding of questions to the presenter, presentations are conducted with minimal unnecessary interruptions. This improves the quality of the presentation for all participants. By determining the context of the question / comment, i.e., based on identifying the type of participant response, only the question and related questions are forwarded to the presenter. Questions relayed to the presenter are forwarded at the appropriate time in the presentation to optimize necessary interruptions, provided the presenter has not or will not answer a similar question during the web conference (in which case the system will respond to the participant). If the question has already been answered, the participant remains muted, and the system pops up a notification providing the answer. In some embodiments, the answer is provided via a chatbot on the participant's screen or via an audio interface only on the participant's line during the presentation. In this way, participant questions do not disturb the presenter and other participants during the conference session.

[0084] If a question is to be posed in the future, a similar mechanism will notify the user that this topic will be covered shortly. Additionally, the system will hide inappropriate or out-of-scope questions to prevent the meeting from being delayed or going off topic.

[0085] The order of steps in which the method is described is not intended to be construed as a limitation, and any number of the described method blocks may be combined in any order to implement the method or alternative methods. Additionally, individual blocks may be deleted from the method without departing from the scope of embodiments of the present invention.

[0086] Those skilled in the art will appreciate that the division of tasks among modules is exemplary, and that other embodiments of the present invention will divide tasks differently. Furthermore, in describing the exemplary processes above, the order in which the processes are described with reference to the flowcharts is not intended to be construed as limiting, and any number of the described method blocks may be combined in a different order to implement the method or alternative methods. Furthermore, individual blocks may be deleted from the method without departing from the scope of the subject matter described herein.

[0087] Although preferred operating environments and use cases have been described, the techniques herein may be used in any other operating environment in which it is desirable to deploy services.

[0088] As described, the foregoing functionality may be implemented in a stand-alone manner, e.g., as one or more software-based functions executed by one or more hardware processors, or may be available as managed services (including web services via SOAP / XML or RESTful interfaces). The specific hardware and software implementation details described herein are for illustration purposes only and are not intended to limit the scope of the described subject matter.

[0089] More generally, computing devices within the context of the disclosed subject matter are each data processing systems, including hardware and software, that communicate with one another over a network, such as the Internet, an intranet, an extranet, a private network, or any other communications medium or link. Applications on the data processing systems provide native support for Web and other well-known services and protocol services, including, but not limited to, support for HTTP, FTP, SMTP, SOAP, XML, WSDL, UDDI, and WSFL, among others. Information about SOAP, WSDL, UDDI, and WSFL is available from the World Wide Web Consortium (W3C), which develops and maintains these standards, and more information about HTTP, FTP, SMTP, and XML is available from the Internet Engineering Task Force (IETF).

[0090] In addition to cloud-based environments, the techniques described herein may be implemented within or with a variety of server-side architectures, including simple n-tier architectures, web portals, federation systems, etc.

[0091] Still more generally, the subject matter described herein may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment containing both hardware and software elements. In a preferred embodiment, modular functions are implemented in software, which software includes, but is not limited to, firmware, resident software, microcode, etc. Furthermore, the interfaces and functions may take the form of a computer program product accessible from a computer-usable or computer-readable medium that provides program code for use by or in connection with a computer or any instruction execution system. In this description, a computer-usable or computer-readable medium may be any apparatus that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The medium may be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device). Examples of computer-readable media include semiconductor or solid-state memory, magnetic tape, removable computer diskettes, random access memory (RAM), read-only memory (ROM), rigid magnetic disks, and optical disks. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read / write (CD-R / W), and DVD. A computer-readable medium is a tangible, non-transitory item.

[0092] A computer program product may be a product having program instructions (or program code) for implementing one or more of the described functions. Such instructions or code may be stored in a computer-readable storage medium within a data processing system after being downloaded over a network from a remote data processing system. Alternatively, such instructions or code may be stored in a computer-readable storage medium within a server data processing system and adapted to be downloaded over a network to a remote data processing system for use in the computer-readable storage medium within the remote system.

[0093] In a representative embodiment, the techniques are implemented in software executed by a dedicated computing platform, preferably one or more processors. The software may be maintained in one or more data stores or memories associated with the one or more processors, and the software may be implemented as one or more computer programs. Collectively, this dedicated hardware and software include the functionality described above.

[0094] In a preferred embodiment, the functionality provided herein is implemented as an adjunct or extension to an existing cloud compute deployment management solution.

[0095] While the above describes a particular order of operations performed by some embodiments of the present invention, it should be understood that such orders are exemplary, as alternative embodiments may perform operations in a different order, combine some operations, overlap some operations, etc. Reference herein to a given embodiment indicates that the described embodiment may include a particular feature, structure, or characteristic, but that not every embodiment necessarily includes the particular feature, structure, or characteristic.

[0096] In a preferred embodiment of the invention described herein, a method for controlling a web-based presentation is provided, the method including analyzing content of the web-based presentation, receiving a user question from a user terminal communicatively coupled to the web-based presentation, buffering the user question while determining, based on the analyzed content, whether content answering the user question will be presented in the presentation, the determination including whether the answering content will be presented in the presentation, and modifying a user interface of the web-based presentation in response to the determination. The method may further include, in response to determining that the answering content will be presented in the presentation, maintaining the user question in a buffered state and providing a notification to the user regarding the answering content in the user interface. The method may further include, in response to determining that the content will not be presented in the presentation, releasing the user question from the buffered state to enable the user interface of the web-based presentation to forward the user question to a presenter of the presentation. The method may further include analyzing presented content of the presentation, where the presented content includes audio information presented on an audio channel and visual information presented on a visual channel of the presentation, creating a context document using the analyzed presented content, and determining a current context for the user question using the context document.

[0097] Finally, while certain components of the system are described separately, those skilled in the art will appreciate that some of the functionality may be combined or shared among given instructions, program sequences, code portions, etc.

Claims

1. A method for controlling a web-based presentation implemented in a device having a processor, comprising: said processor analyzing the presented and future content of said presentation; receiving a user query from a user terminal communicatively coupled to the web-based presentation; buffering the user question while the processor determines, based on the analyzed content, whether content answering the user question has been or will be presented in the presentation; and The processor, in response to the determination, In response to determining that the answering content has been or will be presented in the presentation, maintaining the user question in a buffered state and providing a notification to the user regarding the answering content at a user interface; or and releasing the user query from the buffer state to allow the user query to be forwarded to a presenter of the presentation in response to determining that the content has not been and will not be presented. To implement one of the following: A method comprising:

2. The method of claim 1, further comprising, in response to the processor determining that the answer content has been presented, providing the answer content to the user in the notification within the user interface.

3. The method of claim 1, further comprising, in response to the processor determining that the answering content will be presented, including in the notification that the answering content will be presented in the presentation.

4. The method of claim 1 , wherein analyzing the upcoming content of the presentation includes analyzing visual information within the presentation and notes of the presentation.

5. The method of claim 1, further comprising the processor preventing the user question from being forwarded to a presenter.

6. The method of claim 1, further comprising the processor analyzing the presented content of the presentation, including audio information on an audio channel and visual information on a visual channel of the presentation.

7. A plurality of user queries are received, and parsing, receiving, and buffering are performed for each of the queries; the processor, after determining, for the question for which the determination is that the content has not been and will not be presented, before releasing the user question; grouping a set of questions received from a plurality of participants, the questions being grouped according to the intent of each question; For each group of questions, creating a reduced group of questions by eliminating overlapping questions; forwarding the reduced group of questions to the presenter; To carry out The method of claim 1 further comprising:

8. The method of claim 7, wherein the processor prioritizes each group of questions based on frequency; said processor forwarding the reduced group of questions to said presenter based on highest priority; The method of claim 1 further comprising:

9. 1. An apparatus comprising: a processor; a computer memory holding computer program instructions executed by said processor for controlling a web-based presentation, said computer program instructions comprising: program code operable to analyze the presented and future content of said presentation; program code operable to receive a user query from a user terminal communicatively coupled to the web-based presentation; program code operable to buffer the user question while determining, based on the analyzed content, whether content answering the user question has been or will be presented in the presentation; In response to the determination, In response to determining that the answering content has been or will be presented in the presentation, maintaining the user question in a buffered state and providing a notification to the user regarding the answering content at a user interface; or and releasing the user query from the buffer state to allow the user query to be forwarded to a presenter of the presentation in response to determining that the content has not been and will not be presented. and program code operable to implement one of the following: the computer memory including An apparatus comprising:

10. program code operable to, in response to determining that the response content has been submitted, provide the response content to the user in the notification within the user interface; The apparatus of claim 9 further comprising:

11. program code operable to accept a set of breakpoints within the presentation; program code operable to determine when the next breakpoint in the set of breakpoints occurs within the presentation; program code operable to wait until said next breakpoint before releasing said user query from said buffer state; The apparatus of claim 9 further comprising:

12. 12. The apparatus of claim 11, further comprising program code operable to provide a notification that the question is in a queue while the apparatus waits for the next breakpoint.

13. program code operable to determine whether there is previous presentation material or designated related material for the web-based presentation; program code operable to include the previous presentation material or the specified related material in the set of presentation materials; program code operable to analyze the set of presentation materials prior to executing the web-based presentation; program code operable to create a knowledge source from the presentation materials; program code operable to determine whether there is content in the knowledge source that answers a user question; program code operable to use the answering content in the knowledge source in the notification; The apparatus of claim 9 further comprising:

14. program code operable to filter a plurality of user questions to find a set of user questions if the determination is that the content has not been and will not be presented; program code operable to group respective questions from the set of questions received from a plurality of participants, the questions being grouped by intent of the respective questions to generate at least a first group of questions; program code operable to determine which of the first group of questions is the most complete or representative question; program code operable to forward a most complete or representative question of the first group of questions to the presenter; The apparatus of claim 9 further comprising:

15. A computer program product for causing a computer to carry out the method according to any one of claims 1 to 8.

16. 16. A computer-readable medium having recorded thereon the computer program of claim 15.

Images