Application Server Cache Pinning for Volatile Memory Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing application server systems face challenges in managing memory efficiently, leading to slower response times and increased timeouts due to inadequate storage space for applications, especially when experiencing sudden spikes in popularity or usage, as they rely on inefficient cache eviction policies that can evict frequently used applications, causing delays in loading requested applications.
Innovation Solution
Implement a method where an application server maintains a subset of applications in volatile memory, identifies exempt applications from eviction, selects an eviction target based on cache policies like least-recently-used or least-frequently-used, and loads the requested application while ensuring that pinned applications are not evicted, thereby optimizing memory usage and response times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the application server uses standard cache eviction policies to manage volatile memory, then memory storage capacity is optimized, but frequently used applications may be evicted causing slower response times and increased timeouts
Solution Approach 1:
The patent extracts frequently accessed applications from the normal cache eviction process by designating them as 'pinned' applications. These pinned applications are removed from the pool of applications subject to eviction, ensuring they remain in volatile memory regardless of memory pressure. This allows the system to maintain fast access to hot applications while still optimizing overall memory utilization through eviction of non-pinned applications.
Solution Approach 2:
The system performs preliminary identification and pinning of frequently accessed applications before memory eviction becomes necessary. By monitoring application access patterns and proactively designating pinned applications, the system prevents the performance degradation that would occur if these applications were evicted. This advance preparation ensures that when memory pressure occurs, the right applications are protected in advance.
2Speed
If the application server loads more applications into volatile memory to reduce loading time, then response speed improves, but available memory for new applications decreases
Solution Approach 1:
The patent applies different quality treatments to different applications in the cache. Pinned applications receive preferential treatment by being marked as exempt from eviction, while non-pinned applications follow standard eviction policies. This local differentiation allows the system to optimize for both speed (by keeping pinned apps in memory) and capacity (by evicting non-pinned apps when needed), rather than applying a uniform policy to all applications.
3Productivity
If the application server evicts less frequently used applications to make space for new applications, then memory utilization efficiency improves, but the system cannot handle sudden spikes in popularity of existing applications
Solution Approach 1:
The system implements self-service through dynamic pinning where applications that demonstrate high usage patterns automatically qualify for pinned status. The monitoring mechanism identifies frequently accessed applications and designates them for protection from eviction. This self-organizing behavior allows the system to adapt to changing popularity patterns automatically, ensuring that applications that become popular are protected while maintaining efficient memory utilization for less popular applications.
Data Source
AI summary
An application server maintains a first plurality of applications in non-volatile memory. The application server loads into volatile memory a subset of the first plurality of applications. The subset is a second plurality of applications. The application server receives a request to execute an application that is not currently loaded in volatile memory, and in response performs a set of operations: (1) determining that there is inadequate storage space in volatile memory to store the application; (2) identifying at least one application in the second plurality of applications that is designated as exempt from eviction from volatile memory; (3) selecting from among the second plurality of applications an eviction target, where the selection process excludes the designed applications; (4) evicting from volatile memory the eviction target; and (5) loading and executing the application in response to the received request, and returning a result responsive to the received request.