Cache Service Platform with Distributed Soft Locks for Stampede Prevention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large-scale cloud-based computer systems, concurrent access to cached data objects can lead to a 'stampede' scenario where multiple requests attempt to reload the same data from the database simultaneously, wasting resources and causing long wait times due to the lack of efficient cache management and invalidation strategies.
Innovation Solution
Implementing a cache service platform with distributed soft locks and asynchronous cache refresh mechanisms, along with extended TTL strategies and multi-level caching, to arbitrate among parallel application threads and minimize database access, ensuring only one thread reloads the data and allowing other threads to use cached data if it remains valid.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data objects are cached to minimize direct database accesses, then database access frequency is reduced and system performance is improved, but when cached data expires or is missing, multiple concurrent requests can cause a stampede scenario that wastes network and database resources
Solution Approach 1:
The patent introduces a lock object as an intermediary between concurrent requests and the cache loading mechanism. This lock object serializes access to the cache loading process, ensuring that only one thread can load data from the database at a time while other threads wait. This mediator prevents the stampede scenario by controlling the flow of concurrent requests, thereby reducing unnecessary database and network resource wastage while maintaining high database access efficiency through effective caching.
Solution Approach 2:
The patent implements preliminary action by acquiring a lock before attempting to load data into cache. Threads acquire the lock object before checking if data needs to be loaded, and the first thread to acquire the lock performs the cache loading operation. This preliminary locking mechanism prevents multiple threads from simultaneously initiating database accesses, thereby eliminating resource wastage while preserving the productivity benefits of caching.
2Reliability
If cache expiration periods are set to ensure data freshness, then data consistency is maintained, but cache hit ratio decreases and database access frequency increases
Solution Approach 1:
The patent applies dynamics by extending the cache lifetime beyond the standard expiration period for requests that occur during the grace period. When a thread acquires the lock and finds that the cached data is within the extended grace period and has not been invalidated, it can serve the request from cache even though the original expiration time has passed. This dynamic extension of cache validity based on invalidation status maintains data consistency (reliability) while increasing cache hit ratio (productivity) by preventing premature cache evictions.
Solution Approach 2:
The patent changes the parameter of cache validity by introducing an extended grace period mechanism. Instead of strictly enforcing the original expiration time, the system dynamically adjusts the effective cache lifetime based on whether the data has been invalidated. This parameter change allows the cache to remain effective longer for unchanged data, improving productivity while maintaining reliability through invalidation checks.
3Reliability
If cache invalidation strategies are implemented to maintain data accuracy, then data consistency is improved, but system complexity increases due to additional tracking and validation mechanisms
Solution Approach 1:
The patent merges the cache invalidation mechanism with the existing lock object and cache loading process. The lock object serves dual purposes: it serializes cache loading operations and simultaneously checks for data invalidation before serving cached data. This merging of functions reduces system complexity by eliminating separate invalidation tracking mechanisms while maintaining data accuracy through integrated validation logic in the lock acquisition and data serving process.
Data Source
AI summary
A request for retrieving a cached data object from a data object cache used to cached data objects retrieved from one or more primary data sources is received from a data object requester. Responsive to determining that the cached data object in the data object cache is expired, it is determined whether the cached data object in the data object cache is still within an extended time period. If the cached data object in the data object cache is still within an extended time period, it is determined whether the cached data object is free of a cache invalidity state change caused by a data change operation. If the cached data object is free of a cache invalidity state change, the cached data object is returned to the data object requester.