Multi-state midtier cache replacement for latency reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current middle-tier cache systems face inefficiencies when handling large data objects, as they typically use a binary validity system, leading to long latency for mobile devices that only need a small portion of the data, while server configurations require the entire data object, and separate caches for each application configuration result in data duplication and increased overhead.
Innovation Solution
Implementing a middle-tier cache with multiple validity states that allows incremental loading and updating of data objects based on application configurations, where data is subdivided into portions corresponding to different application needs, and cache partitions are dynamically resized and managed with independent cache replacement policies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a binary validity system is used in traditional cache, then the cache structure is simple, but mobile devices experience long latency when only needing a small portion of large data objects
Solution Approach 1:
The patent segments the binary validity state into multiple granular validity states (e.g., valid_for_mobile, valid_for_desktop, valid_for_server) that correspond to different application configurations. This allows the cache to track which portions of data objects are valid for which device types, enabling mobile devices to retrieve available portions without waiting for complete data object loading.
Solution Approach 2:
The patent adds a new dimension to the traditional binary validity system by introducing configuration-specific validity states. Instead of a single valid/invalid binary state, the system incorporates a third dimension (application configuration type) into the validity tracking, allowing simultaneous validity assessment across multiple device configurations.
2Adaptability or versatility
If separate caches are created for each application configuration, then each cache can be optimized for its specific needs, but data duplication and overhead increase
Solution Approach 1:
The patent merges multiple configuration-specific caches into a single unified cache that serves all application configurations. The unified cache uses configuration-specific validity states to track which data portions are valid for which configurations, eliminating data duplication while maintaining the ability to optimize for specific device types through selective data serving.
3Reliability
If the entire data object must be loaded before serving mobile devices, then data consistency is maintained, but latency increases significantly for mobile configurations
Solution Approach 1:
The patent applies partial action by serving mobile devices with portions of data objects that are valid for mobile configurations, even when the complete data object is not yet loaded. The configuration-specific validity states ensure that only appropriate portions are served to each device type, maintaining data consistency for served portions while reducing overall latency.
Solution Approach 2:
The system performs preliminary loading of data object portions that are likely to be needed by mobile devices first, tracking their validity states separately. This allows mobile-optimized portions to be prepared and served earlier, while other portions continue loading in the background for desktop or server configurations.
Data Source
AI summary
A server includes a data cache for storing data objects requested by mobile devices, desktop devices, and server devices, each of which may execute a different configuration of an application. When a cache miss occurs, the cache may begin loading portions of a requested data object from various data sources. The cache itself may be divided into multiple partitions, and each of the partitions may be assigned to a specific attribute, such as an application configuration. Portions of the data object may be loaded into corresponding cache partitions based on the attributes of each. Although part of a single cache, each of the partitions may be independently assigned different cache replacement policies. Performance metrics for each of the partitions may be monitored and used to update the cache replacement policy for each partition at runtime without interrupting response traffic.