Deep Learning Compression Pipeline with Direct C2C Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI models, particularly in latency-critical applications like speech, image, or video processing, face inefficiencies due to processor limitations, with CPU bottlenecks occurring when using GPU-based entropy coding for machine learning compression, leading to slow execution and inefficient data processing.
Innovation Solution
Implementing a system with two hardware accelerators connected via Chip-to-Chip (C2C) communication, where a first accelerator processes input data and transfers intermediate results directly to a second accelerator, bypassing the CPU, enabling parallelized lossy and lossless compression/decompression using different hardware architectures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If GPU-based entropy coding is used for machine learning compression, then compression capability is improved, but CPU bottlenecks occur leading to slow execution
Solution Approach 1:
The system segments the compression pipeline into two distinct hardware accelerators: one for lossy compression and one for lossless entropy coding. This segmentation allows each accelerator to be optimized for its specific function and eliminates the CPU bottleneck by removing sequential dependencies between compression stages.
Solution Approach 2:
A memory interface acts as an intermediary between the two hardware accelerators, enabling direct data transfer without CPU intervention. This intermediary component facilitates efficient communication while maintaining the independence of each accelerator, resolving the bottleneck caused by CPU-mediated data transfer.
2Productivity
If multiple hardware accelerators are used for parallelized compression, then processing speed is improved, but data transfer overhead increases
Solution Approach 1:
The system merges the data transfer path between hardware accelerators directly through a shared memory interface, eliminating the need for separate communication channels. This consolidation reduces data transfer overhead while maintaining parallel processing capabilities, as both accelerators access the same memory space efficiently.
3Device complexity
If CPU is used for data transfer between processing stages, then system simplicity is maintained, but CPU bottlenecks occur
Solution Approach 1:
The system extracts the data transfer function from the CPU and assigns it to a dedicated memory interface. This extraction removes the burden of data transfer from the CPU, allowing the processor to focus on computation while the memory interface handles data movement between accelerators, thereby eliminating CPU bottlenecks.
Data Source
AI summary
Deep learning data compression using multiple hardware accelerator architectures is provided herein. A system includes a computing device and first and second hardware accelerators coupled thereto. The first and second hardware accelerators may be of different types, such as a tensor streaming processor and a field programmable gate array. The first and second hardware accelerators may be directly connected to one another, such as by a chip-to-chip connection. The first and second accelerators may implement different stages of a data pipeline, such as lossless and lossy compression stages of a learned image compression.