Speaker Diarization With Bounded Vector Memory for Long Audio
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices face challenges in efficiently determining the speech sections of multiple speakers from audio data, particularly when the length of the audio data increases, leading to exponential increases in computational resources and memory requirements for similarity calculations.
Innovation Solution
The electronic device employs a method to manage computational and memory resources by periodically deleting vectors used in speaker diarization, maintaining the total number of vectors below a preset threshold, thereby optimizing resource usage and maintaining efficient performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the electronic device processes long audio data to determine speech sections of multiple speakers, then the accuracy and completeness of speaker diarization is improved, but the computational resources and memory requirements increase exponentially
Solution Approach 1:
The patent divides the audio data into multiple frames and processes them sequentially, maintaining only a limited number of recent frame vectors in memory. This segmentation approach allows the system to handle long audio data while managing computational resources by processing data in manageable chunks rather than loading the entire audio stream at once.
Solution Approach 2:
The patent dynamically manages the vector storage by deleting older frame vectors when the number of vectors exceeds a preset threshold. This dynamic adjustment of memory content based on real-time conditions allows the system to maintain optimal performance for speaker diarization while adapting to varying audio lengths and computational constraints.
2Reliability
If the electronic device stores all frame vectors for speaker diarization, then the completeness of speech section determination is improved, but the memory requirements increase significantly
Solution Approach 1:
The patent implements a mechanism that discards older frame vectors when the vector queue reaches a preset length threshold. By strategically removing the oldest vectors and retaining the most recent ones, the system maintains sufficient information for accurate speaker diarization while significantly reducing memory consumption compared to storing all vectors indefinitely.
Solution Approach 2:
The patent extracts and stores only the necessary recent frame vectors required for current speaker diarization tasks, rather than retaining all historical vectors. This extraction approach allows the system to maintain the essential information needed for accurate speech section determination while minimizing memory usage by removing redundant historical data.
3Measurement precision
If the electronic device performs similarity calculations for all vectors, then the accuracy of speaker identification is improved, but the computational time and processing speed decrease
Solution Approach 1:
The patent segments the vector set into a manageable queue of recent frame vectors, processing only these relevant vectors for similarity calculations rather than comparing all historical vectors. This segmentation reduces the computational complexity of speaker identification while maintaining accuracy by focusing comparisons on the most pertinent recent audio frames.
Solution Approach 2:
The patent performs similarity calculations only on a partial set of recent vectors rather than all available vectors. This partial action approach maintains sufficient speaker identification accuracy by focusing on the most relevant recent audio data while significantly improving processing speed by avoiding unnecessary comparisons with historical vectors.
Data Source
AI summary
According to an embodiment, an electronic device, while obtaining audio data by using a microphone, obtains a plurality of frames by dividing the audio data. The electronic device obtains first vectors respectively corresponding to the plurality of frames. The electronic device determines a speaker corresponding to each of the first vectors by using groups in which the first vectors are respectively included, which are obtained by grouping the first vectors, and second vectors stored in the memory. The electronic device stores within the memory information, which is determined by using a speaker respectively corresponding to the first vectors, indicating a speaker of at least one time section of the audio data. The electronic device, based on a total number of the first vectors and the second vectors greater than a preset number, deletes at least one vector among the first vectors and the second vectors from the memory to adjust the total number to be lower than or equal to the preset number.