Cache Size Estimation Using HLL Counters for Variable Blocks

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing cache size determination methods, such as Miss Ratio Curve (MRC), struggle to support variable-sized data blocks and Time to Live (TTL) in caching platforms, leading to resource consumption and irrelevant information generation.

Innovation Solution

A method and apparatus that utilize HyperLogLog (HLL) counters to associate requests with data block sizes, determining cache size based on unique hashed keys and their block sizes, allowing for variable-sized blocks and TTL support.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If MRC technique is used to determine cache size, then cache size prediction accuracy is improved, but resource consumption increases and support for variable-sized blocks and TTL is lost

Engineering Contradiction:

Improvecache size prediction accuracyVSAvoidresource consumption

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent replaces the complex mechanical tracking system of MRC (which requires tracking unique entries, generating histograms, and calculating stack distances) with a probabilistic counting approach using HyperLogLog algorithms. This substitution maintains measurement precision for cache size prediction while dramatically reducing resource consumption by using compact probabilistic data structures instead of detailed tracking mechanisms.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent changes the fundamental parameter from tracking detailed entry information (stack distances, unique counts) to using probabilistic cardinality estimation. By transforming the measurement approach from exact counting to probabilistic estimation, the system achieves comparable accuracy with significantly reduced computational and memory resources.

Inventive Principle:

Principle #35Parameter changes

2Measurement precision

If MRC technique is used to determine cache size, then cache size prediction is achieved, but support for variable-sized data blocks and TTL is lost

Engineering Contradiction:

Improvecache size predictionVSAvoidsupport for variable-sized blocks and TTL

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent segments the cache analysis by creating separate HyperLogLog counters for different data block size ranges (e.g., small, medium, large blocks). This segmentation enables the system to handle variable-sized blocks effectively by categorizing them into manageable groups while maintaining the ability to predict cache size accurately for each segment.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces dynamic adaptability by making the cache size determination process responsive to TTL constraints. The system dynamically adjusts the cache size prediction based on the minimum TTL value observed in the workload, allowing it to adapt to different data lifecycles and access patterns without requiring fixed assumptions about data persistence.

Inventive Principle:

Principle #15Dynamics

3Loss of information

If MRC is generated by tracking unique entries, then miss ratio curve is obtained, but computing and memory resources are consumed

Engineering Contradiction:

Improvemiss ratio informationVSAvoidcomputing and memory resources

Core Design Contradiction:

Loss of informationVSUse of energy by moving object

Solution Approach 1:

The patent employs HyperLogLog probabilistic counters that use compact memory representations (disposable data structures) to estimate unique entry counts. These probabilistic structures consume minimal memory and computing resources compared to exact tracking methods, providing sufficient information for cache size prediction without the high resource cost of maintaining detailed unique entry records.

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

4Measurement precision

If MRC technique is used, then cache size determination is achieved, but irrelevant information such as high-miss ratio regions is generated

Engineering Contradiction:

Improvecache size determinationVSAvoidirrelevant information generation

Core Design Contradiction:

Measurement precisionVSLoss of information

Solution Approach 1:

The patent extracts only the essential information needed for cache size determination by using HyperLogLog to estimate the number of unique keys directly. This approach eliminates the generation of irrelevant intermediate information such as detailed miss ratio curves and high-miss ratio regions, focusing computational effort solely on obtaining the cache size metric that matters for resource allocation.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS12499051B2Method of determining a cache size using an estimation of a number of requests for cache memory and a size of the requests

Publication Date: 2025.12.16 HUAWEI CLOUD COMPUTING TECHNOLOGIES CO LTD

AI summary

The present disclosure provides a method and system for determining a cache size and allocating cache. A batch of requests for storing data in cache is received and associated with a set of counters configured to obtain respective estimated counts of unique hashed keys of requests associated therewith. Requests may be associated with respective counters in accordance with respective size of each request and a respective data block size associated with each counter. Estimated count of unique hashed keys of requests may be obtained at each counter for a selected expiry time based on Time to Live indicators of requests. Cache size to be allocated is computed by, for counters in the set, summing over a product of respective estimated count of hashed keys of requests at a counter with the data block size associated with the counter.