Processor Compression Instructions for Read-Only Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current memory compression techniques either incur performance overhead due to OS defragmentation or reduce effective memory bandwidth due to metadata access overhead, and they do not effectively address the energy-intensive data transfer between memory and processor in deep learning systems and other applications using read-only data.
Innovation Solution
A processor compression instruction that compresses multiple adjacent data blocks of read-only data into one block, storing it in memory and decompressing it in a prefetch buffer for improved memory bandwidth and reduced latency, using general compression algorithms like Bit-Plane Compression or Base-delta-immediate compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If memory compression techniques are used to increase effective memory capacity, then cache capacity is expanded, but metadata access overhead reduces effective memory bandwidth
Solution Approach 1:
The patent extracts the compression and decompression operations from the critical data path by implementing them in the memory controller rather than in the cache hierarchy. This separation allows the cache to maintain full bandwidth for data access while the memory controller handles compression independently, eliminating the metadata access overhead that would otherwise bottleneck memory bandwidth.
Solution Approach 2:
The memory controller acts as an intermediary between main memory and the cache system, performing compression and decompression operations. This intermediary approach allows the cache to see only uncompressed data with full bandwidth, while the memory controller manages the compression overhead, effectively decoupling the two concerns.
2Volume of stationary object
If OS defragmentation is used to compress memory data, then effective memory capacity increases, but performance overhead is incurred
Solution Approach 1:
The system performs compression automatically in the memory controller without requiring OS-level defragmentation or user intervention. The compression occurs transparently during memory operations, eliminating the performance overhead associated with traditional OS defragmentation while still achieving effective memory capacity expansion.
Solution Approach 2:
The patent implements compression as a preliminary action that occurs in the memory controller before data reaches the cache hierarchy. By pre-compressing data in the memory controller, the system avoids the need for subsequent defragmentation operations that would degrade performance, as compression is already completed before data enters the critical cache path.
3Productivity
If data is transferred between memory and processor in deep learning systems, then computation can be performed, but energy consumption increases significantly
Solution Approach 1:
The patent changes the density parameter of stored data by implementing compression in the memory controller. This parameter change allows more data to be stored in the same physical memory space, reducing the number of transfer operations needed and thereby lowering energy consumption while maintaining computation throughput.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A processor that includes compression instructions to compress multiple adjacent data blocks of uncompressed read-only data stored in memory into one compressed read-only data block and store the compressed read-only data block in multiple adjacent blocks in the memory is provided. During execution of an application to operate on the read-only data, one of the multiple adjacent blocks storing the compressed read-only block is read from memory, stored in a prefetch buffer and decompressed in the memory controller. In response to a subsequent request during execution of the application for an adjacent data block in the compressed read-only data block, the uncompressed adjacent block is read directly from the prefetch buffer.