Inline Data Compression Using Hybrid Hardware-Software Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face inefficiencies in resource utilization and performance due to either software compression, which burdens computing resources, or hardware compression, which consumes physical resources and is costly, and both methods are typically used alone, leading to over-commitment of storage resources and high disk traffic.
Innovation Solution
Implementing a hybrid approach that combines inline software compression and inline hardware compression, where software compression is used for less busy times and hardware compression during peak activity, allowing for efficient data handling without storing uncompressed data, thereby optimizing resource usage and reducing disk traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If software compression is used, then compression can be implemented without specialized hardware, but computing resources are burdened and may displace resources needed to service host IO requests
Solution Approach 1:
The patent segments the compression function into two separate implementations: software compression and hardware compression. Each handles different portions of data traffic based on system conditions, allowing the system to leverage the cost-effectiveness of software while minimizing its impact on host IO performance by offloading to hardware when needed.
Solution Approach 2:
The system dynamically changes the compression mode parameter based on workload conditions. During periods of heavy host IO activity, the system switches to hardware compression to preserve computing resources for host services. During lighter periods, it uses software compression to utilize available computing capacity, thus optimizing the balance between implementation cost and performance.
2Productivity
If hardware compression is used, then compression is faster and does not displace computing resources, but physical resources such as space, power, and cooling are consumed and it is expensive
Solution Approach 1:
The patent implements a dynamic compression architecture where the system adapts its compression approach based on real-time conditions. The hardware compression capability is selectively activated during periods of heavy write activity when speed is critical, while software compression handles lighter workloads, thus optimizing the use of physical resources while maintaining high compression speed when needed.
3Reliability
If background compression is used, then compression can be performed on already-stored data, but storage resources are over-committed and disk traffic increases significantly
Solution Approach 1:
The patent applies compression preliminarily - that is, compression is performed at the time data is initially written to storage, rather than in the background after data is already stored. This inline compression approach ensures that only compressed data occupies storage space from the outset, eliminating the need for over-commitment of storage resources and avoiding the generation of additional disk traffic that would result from reading and rewriting data in background compression.
Data Source
AI summary
A technique for managing data storage applies both inline software compression and inline hardware compression in a data storage system, using both types of compression together. The data storage system applies inline software compression for compressing a first set of newly arriving data and applies inline hardware compression for compressing a second set of newly arriving data. Both sets of data are directed to a data object, and the data storage system compresses both sets of data without first storing uncompressed versions thereof in the data object.