Dynamic Compression Job Allocation in Hardware Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Central processing unit (CPU) bandwidth is significantly consumed by compression and decompression processes in storage systems, leading to performance bottlenecks in scenarios like backup, restore, and replication, which can be improved using a hardware accelerator but requires dynamic allocation of compression jobs across multiple levels to achieve optimal performance.
Innovation Solution
A method and system that dynamically allocates compression jobs between multiple compression levels within a hardware accelerator, such as a QuickAssist Technology compatible accelerator or GPU, based on performance metrics like compression ratio, resource consumption, and speed, to achieve a guaranteed average performance of 3× compression ratio over time, while monitoring CPU workload and allocating tasks accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If compression jobs are handled by CPU, then system flexibility and control are maintained, but CPU bandwidth is significantly consumed leading to performance bottlenecks
Solution Approach 1:
The patent extracts the compression function from the CPU by introducing a dedicated hardware accelerator. The compression engine is implemented as a separate physical component that can be attached to the storage device, offloading compression tasks from the CPU and reducing its bandwidth consumption while maintaining system control through the controller that manages both the storage device and compression engine.
Solution Approach 2:
The patent introduces a controller as an intermediary component that manages the interaction between the storage device and the compression engine. The controller receives compression requests from the storage device, coordinates with the compression engine to perform compression operations, and manages data flow between components, thereby maintaining system flexibility while enabling hardware-accelerated compression.
2Productivity
If hardware accelerator is used for compression, then CPU bandwidth is reduced, but dynamic allocation of compression jobs across multiple levels is required to achieve optimal performance
Solution Approach 1:
The patent implements dynamic compression level selection where the compression engine can operate at multiple compression levels (first level, second level, third level) and the controller dynamically selects the appropriate compression level based on current system conditions, data characteristics, and performance requirements. This allows the system to adapt to varying workload demands while utilizing hardware acceleration.
Solution Approach 2:
The patent changes the compression parameter (compression level) dynamically based on workload conditions. The controller monitors system state and adjusts the compression level parameter selected for the hardware accelerator, enabling the system to optimize between compression ratio and speed by selecting from multiple predefined compression levels rather than using a fixed compression setting.
3Productivity
If multiple compression levels are implemented in hardware accelerator, then compression performance is optimized, but monitoring and allocating jobs across levels increases system complexity
Solution Approach 1:
The patent implements a feedback mechanism where the controller monitors the performance of the compression engine and uses this information to make informed decisions about job allocation and compression level selection. The controller receives status information from the compression engine and adjusts its scheduling and parameter selection based on observed performance, enabling automated optimization without manual intervention.
Solution Approach 2:
The compression engine is designed to autonomously manage its own operation at the hardware level, executing compression tasks based on instructions from the controller without requiring complex external management. The engine independently handles the actual compression processing, allowing the controller to focus on high-level scheduling and parameter selection rather than micro-managing each compression operation.
Data Source
AI summary
One embodiment provides a computer implemented method of dynamically allocating compression jobs including monitoring compression performance at a plurality of compression levels within a hardware accelerator; comparing compression performance between the plurality of compression levels; and dynamically allocating compression jobs between the plurality of compression levels to achieve a guaranteed average performance.