Audio Processing With Cached Prompt Features for Lower Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio processing methods using large language models suffer from significant delays due to the overhead of processing prompt text and repeated processing of the same text for different audio segments, leading to inefficient use of computing resources.
Innovation Solution
Pre-processing of prompt text to obtain an intermediate feature, caching this feature, and incrementally processing target audio to generate a processing result, reducing latency and resource waste by reusing cached features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If prompt text is processed for each audio segment using large language models, then audio processing accuracy is maintained, but processing latency increases significantly and computing resources are wasted
Solution Approach 1:
The patent extracts and caches intermediate features (KV cache) from prompt text in advance, before actual audio processing occurs. This preliminary extraction of text representations allows the system to reuse these features across multiple audio segments, eliminating redundant processing and significantly reducing latency while maintaining accuracy
Solution Approach 2:
The patent creates cached copies of intermediate features from prompt text and stores them for reuse. Instead of reprocessing the original text for each audio segment, the system copies and reuses the pre-computed intermediate representations, reducing computational overhead and processing time
2Reliability
If prompt text is reprocessed for different audio segments, then processing completeness is ensured, but computing resource efficiency decreases
Solution Approach 1:
The patent merges the processing of multiple audio segments by caching intermediate features that can be reused across different segments. Instead of separately processing prompt text for each segment, the system combines efforts by reusing cached representations, reducing redundant computation and improving resource efficiency while maintaining complete processing coverage
Solution Approach 2:
The patent recovers and reuses cached intermediate features from previous processing operations. Rather than discarding the computational work done on prompt text, the system recovers these intermediate representations and applies them to subsequent audio segments, eliminating waste and improving efficiency
Data Source
AI summary
Embodiments of the present disclosure provide a method of audio processing method, an electronic device, and a storage medium. The method includes: obtaining an intermediate feature of prompt text as a first intermediate feature, where the first intermediate feature is obtained by pre-processing the prompt text based on a language model; inputting the first intermediate feature and acquired target audio into the language model, to output, as a second intermediate feature, an intermediate feature corresponding to the target audio, where the target audio corresponds to the prompt text, and the first intermediate feature and the second intermediate feature are both cached as a key value (KV); and inputting the first intermediate feature and the second intermediate feature into the language model to generate a processing result corresponding to the target audio.