Dynamic Data Structure Memory Management for Sparse Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Dynamic data structures used to manage sparsely accessed data in digital computers tend to grow and become less memory-efficient, leading to performance degradation due to memory fragmentation, as they allocate and free memory without effectively recovering memory from sparsely populated portions.
Innovation Solution
A method is implemented where pages of memory are dynamically allocated and de-allocated within a dynamic data structure, with a probability-based decision to un-map de-allocated pages inversely proportional to a superlinear function of the number of allocated pages, allowing for memory recovery by redistributing free and mapped pages across the structure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a dynamic data structure is used to manage sparsely accessed data, then memory allocation flexibility is improved, but memory efficiency deteriorates as the structure grows and becomes sparsely populated
Solution Approach 1:
The patent applies the discarding and recovering principle by identifying sparsely populated portions of the dynamic data structure and actively recovering memory from these regions. When access patterns indicate sparsity, the system discards unused or rarely accessed data portions and recovers the underlying memory for reallocation, thereby maintaining memory efficiency while preserving the flexibility of dynamic data structures.
Solution Approach 2:
The patent implements dynamics by making the data structure's memory allocation adaptive to changing access patterns. The system continuously monitors access frequencies and dynamically adjusts memory allocation by consolidating sparsely populated regions and reallocating their memory, allowing the structure to transition between dense and sparse states efficiently without manual intervention.
2Adaptability or versatility
If memory is dynamically allocated and freed during program execution, then program adaptability is improved, but memory fragmentation increases leading to performance degradation
Solution Approach 1:
The patent recovers memory from sparsely populated regions created by dynamic allocation and freeing operations. By identifying and consolidating fragmented memory regions, the system recovers contiguous memory blocks that can be efficiently allocated, thereby reducing fragmentation-induced performance degradation while maintaining program adaptability.
Solution Approach 2:
The patent employs feedback mechanisms by monitoring access patterns to detected memory fragmentation and performance degradation. Based on this feedback, the system triggers memory consolidation operations that reallocate memory from sparsely used regions to frequently accessed areas, optimizing performance while preserving program adaptability.
3Quantity of substance
If the dynamic data structure grows to accommodate sparsely accessed data, then data coverage is improved, but processing time increases due to degraded performance
Solution Approach 1:
The patent recovers memory from sparsely populated portions of the growing data structure, consolidating data into denser regions. This reduces the effective size of the active data structure that needs to be processed, thereby maintaining comprehensive data coverage while decreasing processing time by operating on a more compact, optimized structure.
Data Source
AI summary
When a dynamic data structure is used for managing sparsely accessed data stored in memory of a digital computer, pages of the memory are dynamically allocated and de-allocated to respective portions of the dynamic data structure so that the pages become distributed over the portions of the dynamic data structure and the de-allocated pages include free pages that are mapped in the dynamic data structure and free pages that are not mapped in the dynamic data structure. To reduce memory fragmentation and recover memory, upon de-allocating a page of memory from a portion of the data structure, a determination is made whether or not to un-map the de-allocated page from the portion of the dynamic data structure so that un-mapping of the de-allocated page has a probability that is the inverse of a superlinear function of the number of allocated pages in the portion of the dynamic data structure.