Sliding Window Data Caching with Multi-Granularity Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing caching techniques for sliding window data in computer systems are inefficient in managing cache buffer sizes, determining data to purge, and minimizing database calls, leading to high computing resource consumption and delays.
Innovation Solution
Caching data at multiple granularity levels, with one portion stored at a higher granularity for higher resolution and another portion at a lower granularity for memory efficiency, and refreshing the cache by obtaining new data from the storage unit when the sliding window moves, while optimizing cache updates based on usage patterns and system load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data is stored at higher granularity level to obtain higher resolution sliding window, then measurement precision is improved, but memory consumption increases
Solution Approach 1:
The patent divides the sliding window data into multiple segments, each stored at different granularity levels. Recent data is stored at higher granularity for detailed analysis, while older data is stored at lower granularity to reduce memory usage. This segmentation allows the system to maintain high measurement precision for current trends while consuming less memory overall.
Solution Approach 2:
Different portions of the sliding window are stored with different quality levels (granularity). The patent applies local quality by storing recent time periods at higher granularity levels where detailed information is most valuable, while storing historical periods at lower granularity levels where aggregate information suffices, thereby optimizing the balance between precision and memory consumption.
2Measurement precision
If a large cache is used to hold higher resolution sliding window data, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent implements dynamic cache management where the granularity level and buffer allocation are adjusted based on current system conditions, query patterns, and data age. The system dynamically determines which portions of the sliding window require high-resolution storage and which can use aggregated data, allowing the cache structure to adapt to changing requirements rather than maintaining a fixed large cache.
Solution Approach 2:
The system changes parameters such as granularity level, buffer size, and retention period based on operational context. By varying these parameters dynamically, the patent can maintain high measurement precision when needed while reducing cache complexity during periods when aggregated data suffices, thus avoiding the need for a consistently large complex cache.
3Reliability
If the cache is updated frequently to maintain current sliding window data, then reliability is improved, but productivity decreases due to increased database calls
Solution Approach 1:
The patent implements periodic cache updates at strategically determined intervals rather than continuous updates. The system updates the cache periodically based on the sliding window movement and query patterns, refreshing data at intervals that maintain sufficient reliability while significantly reducing the frequency of database calls compared to continuous synchronization.
Solution Approach 2:
The system uses feedback from query patterns and data usage to intelligently determine when cache updates are necessary. By monitoring access patterns and determining which portions of the sliding window are most frequently queried, the patent can prioritize updates for those specific segments, maintaining reliability for high-demand data while avoiding unnecessary updates for less-accessed portions, thus reducing overall database calls.
Data Source
AI summary
Disclosed are methods, systems, paradigms and structures for caching data associated with a sliding window in computer systems. A sliding window can include a time window that progresses with time, and the data can include time series data. As time progresses, the sliding window changes bringing in new data. The cache is updated with new data as and when the sliding window moves. The sliding window data is cached at various granularity levels. The method includes storing a first portion of the data at a first granularity level and a second portion at a second granularity level. The data is cached at various granularity levels in order to effectively use the cache considering at least cache updating criteria such as (i) number of times a storage unit is queried to retrieve the data for updating the cache, (ii) the day/date/time at which the storage unit is queried.