Audio Co-Processor Offload for Low-Latency, Low-Power Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio processing methods on computing devices face challenges in minimizing both latency and power consumption, particularly in scenarios involving concurrent render, capture, and loopback streams, as they often require CPU resources and cannot dynamically modify audio device graphs, leading to increased latency and power usage.
Innovation Solution
Offload the audio pump to an audio co-processor, utilizing a shared buffer and processing pipeline with audio processing objects (APOs) to handle multiple audio streams in parallel, allowing dynamic graph topology updates and reducing CPU load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If CPU-based audio pump is used to process multiple concurrent audio streams, then audio processing capability is provided, but CPU resources are monopolized and power consumption increases
Solution Approach 1:
The patent segments the audio processing workload by separating the audio pump functionality from the main CPU and relocating it to a dedicated audio co-processor. This allows the CPU to handle high-level audio graph management while the co-processor handles the periodic audio mixing and processing tasks, reducing CPU power consumption during audio playback and recording operations.
Solution Approach 2:
The patent introduces an audio co-processor as an intermediary component between the CPU and audio endpoints. The co-processor acts as a mediator that receives audio data from the CPU, performs the computationally intensive mixing and processing operations, and outputs to audio devices, thereby offloading the CPU from continuous audio processing tasks.
2Loss of time
If fast pump cadence is used to reduce audio latency, then audio latency decreases, but power consumption increases
Solution Approach 1:
The patent replaces the CPU-based audio pump mechanism with a dedicated audio co-processor that is optimized for audio processing tasks. The co-processor can operate at fast cadences required for low latency audio processing while consuming less power than the CPU would require to perform the same operations, due to its specialized architecture and lower operational overhead.
3Productivity
If audio pump runs periodically at fixed cadence, then audio mixing is performed, but CPU must remain active increasing power consumption
Solution Approach 1:
The patent extracts the audio pump functionality from the CPU and relocates it to a dedicated audio co-processor. This extraction allows the CPU to enter low-power states between audio processing intervals, while the co-processor independently handles the periodic audio mixing tasks, significantly reducing overall system power consumption during audio operations.
4Productivity
If series arrangement of CPU mixer and co-processor mixer is used, then audio processing is distributed, but latency increases
Solution Approach 1:
The patent inverts the traditional architecture by making the co-processor the primary audio processing unit and the CPU a secondary control unit. Instead of the CPU mixing audio and then passing to co-processor, the co-processor performs mixing directly from application buffers, eliminating the series arrangement and reducing latency while maintaining distributed processing benefits.
5Loss of energy
If audio device graphs are static on audio co-processor, then hardware optimization is achieved, but dynamic modification capability is lost
Solution Approach 1:
The patent introduces dynamic reconfiguration capability to the audio co-processor, allowing the audio device graph topology to be modified at runtime. The system can dynamically add, remove, or reconfigure audio processing nodes and connections based on current application requirements, while maintaining the power efficiency benefits of co-processor-based processing through optimized execution paths.
Data Source
AI summary
Techniques are described for offloading an audio pump from a central processing unit (CPU) of a computing device to an audio co-processor. A device graph for an audio endpoint can be modified to include a processing pipeline for an audio stream from an application. Audio data from the audio stream can be retrieved by the audio co-processor from a shared buffer accessible to the application and the audio co-processor. The processing pipeline for the audio stream can include a series of audio processing objects (APOs) linked by connection buffers. Audio data from the audio stream can be processed by executing an audio pump thread with the audio co-processor. Multiple audio streams can be executed in parallel by the audio pump thread, including render, capture, and loopback streams, with better glitch resilience, relatively low latency, and a better power profile.