Off-Heap Caching for Java Web Service Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Java-based web service applications, particularly those involved in video advertising, face performance issues due to garbage collection overhead, which can cause stalls and missed decision windows in systems requiring rapid data access, as garbage collection can occur at any time and is triggered by insufficient heap memory.
Innovation Solution
Implementing a network service system with off-heap caching, where application servers and cache servers utilize both heap and off-heap memory, allowing data to be stored and accessed outside the virtual memory subject to automatic garbage collection, thereby reducing the need for garbage collection and minimizing system interruptions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If Java automatic garbage collection is used to manage heap memory, then memory management is automated and programmers are spared manual management burden, but application programs are put on hold during garbage collection, causing performance stalls and missed decision windows
Solution Approach 1:
The patent segments memory management into two distinct parts: heap memory for short-lived objects and off-heap memory for long-lived cached objects. This segmentation allows the application to continue executing without interruption while garbage collection occurs only on the heap portion, resolving the contradiction between automated memory management and application execution speed.
Solution Approach 2:
The patent extracts long-lived cached objects from the heap memory to off-heap memory. By removing these objects from the garbage-collected heap, the system eliminates the performance stalls caused by garbage collection of cached data, while maintaining automated memory management for the remaining heap objects.
2Reliability
If objects are stored in old generation space of heap memory, then long-lived objects are properly managed, but garbage collection overhead becomes very high, causing application programs to be put on hold
Solution Approach 1:
The patent extracts long-lived cached objects from the old generation space of heap memory and stores them in off-heap memory. This extraction eliminates the high garbage collection overhead associated with old generation objects, while the heap memory continues to properly manage short-lived objects through automatic garbage collection.
Solution Approach 2:
The patent introduces off-heap memory as an intermediary storage location for cached objects. This intermediary allows the system to maintain reliable object lifecycle management for long-lived objects without subjecting them to the time-consuming garbage collection process that affects heap memory.
3Speed
If heap memory is used for caching data in video advertising systems, then data can be quickly accessed, but garbage collection can occur at any time and trigger stalls during critical decision windows
Solution Approach 1:
The patent extracts cached advertising data from heap memory to off-heap memory. This extraction ensures that critical decision windows are never stalled by garbage collection, while data access speed is maintained through direct off-heap memory access with custom memory management.
Solution Approach 2:
The patent implements dynamic memory management for cached objects in off-heap memory, allowing the system to adaptively manage memory allocation and deallocation based on actual usage patterns. This dynamic approach ensures timely data access for video advertising decisions without the rigid constraints of automatic garbage collection cycles.
Data Source
AI summary
A method for providing data over a network using an application server having off-heap caching includes receiving at an application server coupled to a network a request for requested data, using an key index stored on the application server to locate where the requested data is stored in off-heap memory of the application server, retrieving the requested data from the off-heap memory of the application server, and resolving the request.