Value Cache Compression for Datacenter Memory Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cache management systems are inefficient as they store duplicate values, leading to reduced cache hits, increased cache pollution, and higher memory bandwidth, which can elevate read and write latency in datacenter scenarios.
Innovation Solution
A cache management system that compresses multiple cache blocks with the same value into a single block, using a value cache to store frequent values and pointers to their locations, thereby maximizing storage space for distinct values and reducing cache pollution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If duplicate values are cached in the primary cache, then cache storage is simple and fast, but cache storage space is wasted and cache hit ratios are reduced
Solution Approach 1:
The patent extracts duplicate values from the primary cache and stores them in a separate value cache. The primary cache retains only unique values, while the value cache stores duplicate values with a frequency counter. This separation resolves the contradiction by allowing the primary cache to maintain high access speed with unique values only, while the value cache efficiently handles duplicates through compression and frequency-based storage.
2Productivity
If more cache memory is allocated to store all data, then cache hit ratios are maximized, but cache cost increases exponentially
Solution Approach 1:
The patent changes the storage parameter by introducing a frequency-based compression mechanism in the value cache. Instead of storing all data in the primary cache, it stores only unique values there and uses the value cache to store duplicate values with frequency counters. This parameter change allows the system to achieve high cache hit ratios by intelligently storing frequently accessed duplicate values, while keeping the overall cache memory size and cost manageable through compression.
3Quantity of substance
If duplicate values are stored in the cache, then all data can be cached, but cache pollution increases and memory bandwidth is consumed
Solution Approach 1:
The patent creates a compressed copy of duplicate values in the value cache, storing them with frequency counters instead of full duplicates. This copying mechanism allows the system to represent multiple cache blocks with the same value using a single stored value plus a frequency counter, thereby reducing cache pollution and memory bandwidth consumption while still enabling efficient retrieval of all cached data including duplicates.
Data Source
AI summary
The disclosure is directed to a cache management system (“system”) for storing data in a cache of a computer system in a compact form. The system identifies multiple instances of a candidate value stored in a primary cache and stores those instances as a single value in a value cache. The system stores multiple pointers to the candidate value in the value cache, e.g., as many as the number of occurrences of the candidate value in the primary cache, each of which corresponds to an address in a memory of the computer system storing an instance of the primary cache candidate value. By storing multiple instances of the candidate value as a single instance, the system reduces the cache pollution and the memory bandwidth and facilitates storing more number of distinct values in the primary cache of a given size.