Dynamic Memory Manager for Sorter Processes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems have a fixed memory allocation for processes, leading to inefficient utilization of available memory and increased resource consumption due to frequent data spilling to disk during sorting operations, as the maximum memory allocated to each process cannot be dynamically adjusted based on overall system memory usage.
Innovation Solution
A memory manager dynamically adjusts the memory allocation to processes based on current usage and overall system memory availability, periodically calculating a threshold limit to optimize memory utilization and minimize disk spilling by releasing excess memory and reusing freed buffers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the maximum memory allocated to each sorter process is fixed, then the system ensures stable memory usage and prevents memory starvation, but the available system memory is underutilized when there is unused memory and sorters cannot allocate more memory even when needed
Solution Approach 1:
The patent implements dynamic memory allocation where the memory manager continuously monitors system memory usage and adjusts the maximum memory threshold for sorter processes in real-time. This allows the system to adapt memory allocation to current system conditions, enabling sorters to utilize more memory when system resources are available while preventing memory exhaustion when resources are constrained, thereby resolving the contradiction between adaptability and stability.
2Productivity
If the maximum memory threshold for sorter processes is increased, then sorters can perform more sorting operations in-memory without spilling to disk, but other processes may be starved of memory resources and overall system functioning is adversely impacted
Solution Approach 1:
The patent implements a feedback mechanism where the memory manager continuously monitors both the memory usage of sorter processes and the overall system memory availability. Based on this feedback, the memory manager dynamically adjusts the maximum memory threshold allocated to sorters. When system memory is abundant, sorters are allowed to use more memory to improve sorting efficiency. When system memory becomes constrained, the threshold is reduced to prevent other processes from being starved, thus maintaining system stability while optimizing sorting performance.
3Loss of time
If sorters are allocated fixed memory amounts, then memory management is simple and predictable, but frequent spilling to disk occurs when memory is exhausted, increasing resource consumption and execution time
Solution Approach 1:
The patent implements dynamic memory allocation where the memory manager continuously monitors system memory usage and adjusts the maximum memory threshold for sorter processes in real-time. This allows the system to adapt memory allocation to current system conditions, enabling sorters to utilize more memory when system resources are available while preventing memory exhaustion when resources are constrained, thereby resolving the contradiction between adaptability and stability.
Solution Approach 2:
The patent changes the memory allocation parameter from a fixed value to a dynamic value that adjusts based on system conditions. The memory manager modifies the maximum memory threshold parameter for sorter processes based on real-time system memory availability, allowing the system to optimize sorting performance by enabling in-memory sorting when possible while maintaining simple memory management through automated control.
Data Source
AI summary
Techniques for dynamically changing the amount of memory that is allocated to and used by processes (e.g., sorter processes) executed concurrently by a data processing system in a manner that is adaptive to the overall memory usage of the system. The ability to dynamically change the amount of memory that can be allocated to and used by individual processes increases and optimizes the overall memory utilization of the data processing system, which in turn results in the operations (e.g., sorting) performed by the processes to be performed in a more efficient manner. A memory manager is provided that is responsible for periodically collecting memory related information for the data processing system and for the processes (e.g., sorter processes) executed by the data processing system, and based on the collected information, for periodically and dynamically causing the memory used by the processes to be changed during the lifetime of the processes.