Log Flush Rate Regulation via Virtual Utilization Feedback

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing techniques face challenges in efficiently managing the flush rate of log entries in data storage systems, leading to resource shortages and inefficient resource utilization, with existing methods failing to provide smoothness and stability in flush processing rates and proper reclamation of resources.

Innovation Solution

The proposed solution involves determining a target utilization point for each resource and adjusting the flush rate based on virtual utilization deviations, using regulation actions such as varying the number of processing cores to optimize the flush rate, and implementing a flush working set switch mechanism to adapt to changing workloads.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If the flush rate is increased to improve resource reclamation, then resource shortages are reduced, but system stability and smoothness of flush processing deteriorate

Engineering Contradiction:

Improveresource reclamation rateVSAvoidflush processing stability

Core Design Contradiction:

ProductivityVSStability of the object's composition

Solution Approach 1:

The system dynamically adjusts the flush rate based on real-time virtual utilization metrics and workload conditions. The flush rate is not fixed but varies adaptively to match actual system needs, allowing the system to achieve both high resource reclamation and stable processing by responding to changing conditions rather than maintaining a constant rate

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system implements feedback control by continuously monitoring virtual utilization (VU) metrics and adjusting the flush rate accordingly. When VU exceeds thresholds, the system increases flushing activity; when VU is low, it reduces flushing. This closed-loop feedback mechanism ensures resource reclamation is optimized while maintaining processing stability through adaptive response to system state

Inventive Principle:

Principle #23Feedback

2Stability of the object's composition

If the flush rate is decreased to improve system stability, then resource shortages increase, but productivity of resource reclamation deteriorates

Engineering Contradiction:

Improveflush processing stabilityVSAvoidresource reclamation rate

Core Design Contradiction:

Stability of the object's compositionVSProductivity

Solution Approach 1:

The system uses dynamic adjustment of flush rate based on real-time conditions rather than maintaining a fixed low rate. The flush rate adapts to workload intensity and resource utilization patterns, ensuring that resource reclamation remains effective during high-demand periods while maintaining stability during low-demand periods

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes the flush rate parameter adaptively based on monitored virtual utilization metrics and workload characteristics. By adjusting this key parameter in response to system state, the system optimizes both stability and productivity, preventing resource shortages during critical periods while avoiding excessive flushing during stable periods

Inventive Principle:

Principle #35Parameter changes

3Productivity

If more processing cores are allocated to increase flush capacity, then resource reclamation improves, but device complexity and resource consumption increase

Engineering Contradiction:

Improveflush processing capacityVSAvoidprocessing core allocation

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system allocates processing resources partially rather than fully, using only the necessary number of processing cores based on actual flush workload. Instead of provisioning maximum capacity always, the system dynamically assigns cores only when needed, reducing overall system complexity and resource consumption while maintaining adequate flush capacity when required

Inventive Principle:

Principle #16Partial or excessive action

4Device complexity

If existing flush management techniques are used, then system simplicity is maintained, but resource utilization efficiency and reclamation rates deteriorate

Engineering Contradiction:

Improveflush management simplicityVSAvoidresource utilization efficiency

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The system implements self-service flush management where the system automatically monitors its own virtual utilization metrics and adjusts flushing behavior without external intervention. This self-regulating mechanism maintains simplicity by eliminating the need for complex external control systems while significantly improving resource utilization efficiency through adaptive self-optimization

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS20240134712A1Techniques for efficient flushing and providing optimal resource utilization

Publication Date: 2024.04.25 DELL PROD LP

AI summary

In at least embodiment, processing can include: receiving virtual utilizations (VUs) for resources consumed in connection with recording entries in a log for commands or operations, wherein each resource has a corresponding VU which denotes a number of allocated units of the resource associated with entries of the log which have not been flushed from the log; determining a first resource having a maximum VU with respect to VUs of consumed resources; determining whether the maximum VU is within an acceptable VU range, wherein the acceptable VU range has bounds determined in accordance with a maximum deviation threshold associated with the first resource; and responsive to determining the maximum VU is not within the acceptable VU range of the first resource, performing one of a plurality of regulation actions that varies an amount of a system resource to accordingly vary a flush rate of entries from the log.