Real-time Virtual Conference Transcript Summarization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Participants in virtual conferences often struggle to keep track of discussions due to distractions or lack of focus, leading to the need for real-time summaries to facilitate engagement and note-taking.
Innovation Solution
The implementation of real-time summarization systems that utilize automatic speech recognition (ASR) to generate transcripts, followed by machine-learning (ML) and large language model (LLM) processes to segment and summarize utterances into coherent summaries, which are then provided to participants during the meeting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If real-time summarization is implemented using ASR, ML, and LLM processes, then participant engagement and note-taking effectiveness are improved, but system complexity and processing time increase
Solution Approach 1:
The summarization process is divided into distinct stages: automatic speech recognition (ASR) converts audio to text, machine learning models perform content segmentation to identify key topics, and large language models generate concise summaries. This segmentation allows each component to specialize and process independently, managing overall system complexity while maintaining high productivity.
Solution Approach 2:
The patent introduces intermediate processing layers between raw conference audio and final summaries. ASR transcripts serve as an intermediary representation of spoken content, while ML-based segmentation creates structured intermediate representations that bridge unstructured audio and coherent summaries, reducing the computational burden on final summary generation.
2Productivity
If real-time summarization is implemented using ASR, ML, and LLM processes, then participant engagement and note-taking effectiveness are improved, but processing speed and computational resources are consumed
Solution Approach 1:
The system performs preliminary speech-to-text conversion and content segmentation during the conference proceeding, preparing structured intermediate representations in advance. This preliminary processing of ASR transcription and topic segmentation enables faster final summary generation by the LLM, reducing overall processing delays while maintaining high engagement quality.
Data Source
AI summary
One example method includes receiving, during a virtual conference from a client device, a request to generate a real-time summary of the virtual conference, a plurality of client devices participating in the virtual conference and exchanging audio and video streams; receiving, during the virtual conference, a plurality of utterances generated by automatic speech recognition (“ASR”) of the audio streams; generating a group of consecutive utterances from the plurality of utterances; determining a segment based on the group of consecutive utterances; generating, using a large language model (“LLM”), a segment summary based on the segment; and providing the segment summary to the client device.