Hierarchical LSTM Speaker Identification via Cognitive Print Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for minimizing transcription word error rates in subtitling HTTP live streaming (HLS) while improving word recognition rates are inefficient, particularly when dealing with multiple speakers, as they require biometric identification which can be computationally infeasible and impractical for broad domains.
Innovation Solution
A computer-implemented method using a hierarchical Long Short-Term Memory (LSTM) model to identify speakers by matching spoken words to a cognitive print, which classifies traits such as tone, stress, pitch, sentiment, and accent, allowing for accurate speaker identification and annotation of subtitles without the need for biometric matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If biometric identification is used to identify speakers, then speaker identification accuracy is improved, but computational complexity and processing time increase significantly
Solution Approach 1:
The patent segments the speaker identification process into multiple independent trait classifications (tone, stress, pitch, sentiment, accent) rather than using a single complex biometric identification system. Each trait is classified separately by dedicated LSTM layers, reducing the computational complexity of each individual classification while maintaining overall identification accuracy through the combination of multiple simpler classifications.
Solution Approach 2:
The patent introduces cognitive prints as an intermediary representation that captures speaker characteristics through multiple traits rather than directly comparing raw biometric data. This intermediary layer (the hierarchical LSTM model processing multiple traits) mediates between the input audio signal and the final speaker identification, reducing the direct computational burden of biometric matching.
2Measurement precision
If biometric identification is used to identify speakers, then speaker identification accuracy is improved, but processing time increases making it impractical for live streaming
Solution Approach 1:
The patent divides the speaker identification task into parallel trait classification processes (tone, stress, pitch, sentiment, accent) that can be processed more efficiently than sequential biometric matching. This segmentation enables faster processing suitable for live streaming while maintaining accuracy through the combined analysis of multiple traits.
Solution Approach 2:
The patent performs preliminary classification of speaker traits using the hierarchical LSTM model before final speaker identification. By pre-processing and extracting multiple trait features in advance, the system reduces the time required for actual speaker identification during live streaming, making the process practical for real-time applications.
3Measurement precision
If multiple traits are classified using a hierarchical LSTM model, then speaker identification accuracy is improved, but model complexity increases
Solution Approach 1:
The hierarchical LSTM model segments the complex task of speaker identification into multiple specialized sub-tasks, with each layer dedicated to classifying a specific trait (tone, stress, pitch, sentiment, accent). This segmentation reduces the complexity of each individual layer compared to a single monolithic model, while the hierarchical structure combines these simpler classifications to achieve high overall accuracy.
Data Source
AI summary
Aspects of the present invention provide devices that subtitle streaming video with audio and identify a speaker in a streaming video with audio according to words spoken by the speaker matched to a cognitive print. The cognitive print includes traits classified according a hierarchical long short term model (LSTM). The hierarchical LSTM includes layers of LSTMs and each layer corresponds to the classification of one trait. A processor annotates a subtitle of the words spoken by the speaker, which decorates the subtitle with a label representative of the identified speaker, and streams the decorated subtitle with the streaming video with audio.