Streaming Audio Keyword Detection With Lightweight Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for extracting keywords from audio content are inefficient for streaming multimedia and require heavy resource usage, making them unsuitable for on-device analysis.
Innovation Solution
Segment audio content into smaller segments, apply lightweight models for speech-to-text conversion, generate phrase candidates and sentence embeddings, calculate an average embedding value, and label candidates above a threshold as keywords for real-time extraction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the entirety of text data is used for keyword detection after audio broadcast, then keyword extraction accuracy is improved, but processing time and resource usage increase significantly
Solution Approach 1:
The patent segments audio content into multiple audio segments and processes each segment independently to generate text segments. This segmentation allows for incremental keyword detection rather than waiting for complete audio processing, thereby reducing overall processing time while maintaining keyword extraction accuracy through cumulative analysis of segment embeddings.
Solution Approach 2:
The patent performs preliminary processing by generating text segments and their corresponding embeddings before final keyword detection. By pre-computing sentence embeddings for each text segment and storing them for later comparison, the system prepares data in advance, enabling faster real-time keyword detection without sacrificing accuracy.
2Measurement precision
If heavy keyword detection models are used to ensure accurate keyword extraction, then keyword detection accuracy is improved, but device resource requirements and complexity increase
Solution Approach 1:
The patent divides the keyword detection task into smaller sub-tasks by processing audio in segments. Each segment undergoes lightweight speech-to-text conversion and embedding generation, followed by comparison with average segment embeddings. This segmentation enables accurate keyword detection through multiple small processing steps rather than requiring a single heavy model.
Solution Approach 2:
The patent introduces sentence embeddings as an intermediary representation between raw text and keyword detection. By converting text segments into embedding vectors and comparing these embeddings to identify keywords, the system achieves accurate keyword extraction using lightweight models rather than complex traditional NLP pipelines.
3Loss of information
If complete audio processing is performed before keyword detection, then comprehensive keyword coverage is improved, but real-time processing capability is lost
Solution Approach 1:
The patent segments audio content into multiple audio segments that are processed independently and in sequence. Each segment is converted to text, embedded, and analyzed for keywords immediately, enabling real-time keyword detection. The cumulative effect of processing all segments ensures comprehensive keyword coverage while maintaining real-time processing capability.
Solution Approach 2:
The patent maintains continuous keyword detection by processing audio segments as they become available rather than waiting for complete audio processing. Each text segment is immediately converted to embeddings and compared to identify keywords, ensuring uninterrupted real-time keyword extraction while maintaining comprehensive coverage through continuous accumulation of segment results.
Data Source
AI summary
Examples of the present disclosure describe improved systems and methods for detecting keywords in audio content. In one example implementation, audio content is segmented into one or more audio segments. One or more text segments is generated, each text segment corresponding to each of the audio segments. For each text segment, one or more phrase candidate values is generated using a textual analysis, and one or more sentence embedding values is generated using a sentence embedding analysis. Next, an average sentence embedding value is calculated using the one or more sentence embedding values. Each of the one or more phrase candidate values is compared to the average sentence embedding value. Each phrase candidate value having a comparison value above a threshold value is labeled as representing a keyword.