Hardware Compression Bridge Between Backup Server and Storage

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current data compression methods in storage systems consume significant CPU bandwidth, particularly during backup, restore, and replication processes, where CPU resources are bottlenecked, and existing hardware accelerators face challenges in balancing compression rates and resource utilization between LZ and GZ algorithms.

Innovation Solution

A method utilizing a hardware accelerator, such as a QuickAssist Technology (QAT) compatible accelerator or GPU, to communicate instructions for compressing data in LZ format, decompressing it, and then recompressing into GZ format, optimizing CPU resource usage and network bandwidth by aggregating data buffers and calculating checksums, while enabling compatibility with both LZ and GZ formats.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data compression is performed using traditional CPU-based methods, then compression functionality is available, but CPU bandwidth is significantly consumed and system performance is bottlenecked

Engineering Contradiction:

Improvecompression throughputVSAvoidCPU bandwidth consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

A hardware accelerator is introduced as an intermediary device between the CPU and storage system to perform compression and decompression operations. The accelerator receives compression requests from the CPU, performs the actual compression using dedicated hardware circuits, and returns compressed data to the CPU, thereby offloading the computationally intensive compression tasks from the CPU and reducing CPU bandwidth consumption while maintaining or improving compression throughput

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the software-based compression mechanism (CPU executing compression algorithms) with a hardware-based compression mechanism (dedicated compression accelerator). This substitution transitions the compression function from a software process that consumes CPU cycles to a hardware process that uses specialized circuits, thereby significantly reducing CPU bandwidth consumption while enabling higher compression throughput through parallel processing capabilities

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Use of energy by moving object

If hardware accelerator is used for compression, then CPU resource consumption is reduced, but compatibility between different compression formats (LZ and GZ) must be maintained

Engineering Contradiction:

ImproveCPU resource consumptionVSAvoidcompression format compatibility

Core Design Contradiction:

Use of energy by moving objectVSAdaptability or versatility

Solution Approach 1:

The hardware accelerator is designed with multi-functionality to support multiple compression formats including both LZ and GZ formats. The accelerator contains configurable compression engines that can be programmed or switched to operate with different compression algorithms, allowing it to handle various compression formats while maintaining a single hardware device. This universality enables the system to reduce CPU resource consumption while preserving compatibility across different compression formats through a single versatile hardware component

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Loss of energy

If compression rate is increased to reduce network bandwidth, then network efficiency improves, but CPU bandwidth consumption increases

Engineering Contradiction:

Improvenetwork bandwidthVSAvoidCPU bandwidth

Core Design Contradiction:

Loss of energyVSUse of energy by moving object

Solution Approach 1:

The hardware accelerator serves as an intermediary that enables high compression rates without proportionally increasing CPU bandwidth consumption. By offloading the compression computations to dedicated hardware circuits, the system can apply more aggressive compression algorithms (higher compression rates) that would be too computationally intensive for the CPU, thereby reducing network bandwidth requirements while keeping CPU bandwidth consumption within acceptable limits

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system changes the parameter of where compression computations are performed - moving from software-based CPU execution to hardware-based accelerator execution. This parameter change enables the system to achieve higher compression rates because the hardware accelerator can process compression operations in parallel with greater efficiency, reducing the effective computational overhead and allowing more aggressive compression settings without proportionally increasing CPU bandwidth consumption

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS10840943B1System and method of data compression between backup server and storage

Publication Date: 2020.11.17 EMC IP HLDG CO LLC

AI summary

One embodiment provides a computer implemented method of data compression using a hardware accelerator. The method includes communicating, using a hardware accelerator, to a client device a set of instructions that compressed data be sent in LZ compressed format; receiving LZ compressed data at the hardware accelerator in LZ compressed format; decompressing the LZ compressed data using the hardware accelerator to create decompressed data; recompressing the decompressed data into GZ format, using the hardware accelerator, to create GZ compressed data; and sending the GZ compressed data to a storage device.