SoC Compression Block for Spill Data Latency and Power Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial reality systems face challenges in efficiently managing large data sets, particularly during machine learning training, which often leads to data spills that require storage in off-die memory, resulting in increased latency and power consumption.
Innovation Solution
The implementation of a compression block within a System-on-a-Chip (SoC) that compresses spill data using an encoder and stores it in local memory, reducing the need for off-die Dynamic Random-Access Memory (DRAM) and minimizing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in off-die memory, then storage capacity is sufficient, but power consumption increases and latency increases
Solution Approach 1:
The patent segments memory into local on-die SRAM for frequently accessed data and off-die DRAM for bulk storage. The compression block segments data into compressed and uncompressed portions, storing only essential uncompressed data locally while keeping compressed data in off-die memory, thus reducing power consumption for local memory operations while maintaining sufficient total storage capacity.
Solution Approach 2:
The patent implements a nested memory hierarchy where compressed data resides in off-die DRAM and uncompressed data is nested within local on-die SRAM when needed. The compression block acts as an intermediary layer that manages this nesting, allowing the system to benefit from both the high capacity of off-die memory and the low power consumption of on-die memory for active data.
2Loss of time
If data is stored in local memory, then latency is reduced, but storage capacity is limited
Solution Approach 1:
The patent changes the compression parameter to achieve a balance between compressed data size and decompression overhead. By optimizing the compression ratio, the system fits more data into the limited local memory capacity while minimizing the time penalty of compression and decompression operations, thus addressing both latency and storage capacity constraints.
3Quantity of substance
If compression is applied to data, then storage efficiency improves, but processing complexity increases
Solution Approach 1:
The patent introduces a dedicated compression block as an intermediary component between the compute elements and memory subsystem. This specialized hardware module handles compression and decompression operations, isolating the processing complexity from the main compute elements and memory controllers. The compression block acts as a mediator that manages the complexity of data transformation while presenting a simple interface to the rest of the system.
4Speed
If distributed local memory is used, then bandwidth is high and latency is low, but energy consumption increases when sharing with distant compute elements
Solution Approach 1:
The patent applies local quality by keeping frequently accessed uncompressed data in local on-die SRAM close to compute elements, ensuring high bandwidth and low latency for hot data. Less frequently accessed compressed data is stored in off-die memory, reducing the energy cost of maintaining large amounts of data in high-speed local memory. The compression block manages this spatial distribution of data quality.
Data Source
AI summary
A system on a chip (SoC) includes a first subsystem, a second subsystem and a compression block connected to the first and second subsystems, wherein the compression block includes a decoder and an encoder. The compression block receives spill data generated by a compute element in one of the first and second subsystems, compresses the spill data using the encoder and stores the compressed spill data in a data block in local memory of one of the compute elements.