Dynamically Stacked Roaring Bitmaps for Local Cache Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large cache sizes in web servers lead to network partition issues and inefficiencies, as they are often hosted on remote clusters, causing access problems and resource wastage due to the inability to support bitmap operations in local memory caches.
Innovation Solution
Implementing a dynamically compressed local cache that adjusts its elasticity based on cardinal values, using a local cache compression engine to apply compression algorithms like roaring bitmaps, allowing for reduced resource usage and synchronization with remote caches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If local cache stores large amounts of data to reduce network traffic, then network efficiency improves, but memory usage and processor cycles increase
Solution Approach 1:
The patent applies compression algorithms that dynamically adjust compression parameters based on data characteristics. By changing the parameter of data representation from uncompressed to compressed format, the system achieves both reduced memory usage and reduced network traffic, resolving the contradiction between these two parameters.
Solution Approach 2:
The patent uses composite data structures combining compressed data with bitmap indexes. This composite approach allows efficient storage of large datasets in compressed form while enabling fast retrieval operations through the bitmap component, thus reducing both memory usage and network traffic simultaneously.
2Quantity of substance
If local cache is compressed to reduce memory usage, then memory efficiency improves, but access speed and bitmap operation support deteriorate
Solution Approach 1:
The patent segments the cache into compressed data storage and separate bitmap index structures. This segmentation allows the compressed data to save memory while the bitmap indexes provide fast access paths, resolving the contradiction between memory efficiency and access speed.
Solution Approach 2:
The bitmap structures serve as intermediaries between the compressed data and retrieval operations. Instead of directly accessing compressed data which would be slow, the system uses bitmaps as intermediaries to quickly locate and retrieve specific data, maintaining fast access speeds while using compressed storage.
3Quantity of substance
If remote cache clusters are used to host large caches, then cache capacity increases, but network partition issues and access reliability worsen
Solution Approach 1:
The patent inverts the traditional approach by bringing the cache closer to the application (local cache) rather than hosting it remotely. By compressing the local cache to fit larger capacities locally, the system eliminates network dependency for cache access, thus improving reliability during network partitions while maintaining large cache capacity.
4Adaptability or versatility
If uncompressed local cache is used to support bitmap operations, then operation capability improves, but resource allocation efficiency worsens
Solution Approach 1:
The patent creates a multi-functional system where compressed data structures serve both storage efficiency and bitmap operation capabilities. The compressed format maintains the ability to perform bitmap operations while simultaneously providing compression benefits, making the system universally capable of both functions without requiring separate uncompressed storage.
Data Source
AI summary
A method for compressing data in a local cache of a web server is described. A local cache compression engine accesses values in the local cache and determines a cardinality of the values of the local cache. The local cache compression engine determines a compression rate of a compression algorithm based on the cardinality of the values of the local cache. The compression algorithm is applied to the cache based on the compression rate to generate a compressed local cache.