Selective Data Compression for Non-Compressible Stream Bypass
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Compressing data in network storage systems consumes excessive processor cycles, particularly when data is not compressible, leading to performance degradation and negatively impacting high availability, speed, and reliability.
Innovation Solution
Implementing a method to selectively compress data using the Lempel-Ziv-Oberhumer (LZO) lossless data compression algorithm, where data streams with a success rate below a threshold are bypassed, thereby avoiding unnecessary processing cycles and optimizing performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data compression is applied to all incoming data streams, then storage efficiency is improved, but processor cycle consumption increases and performance deteriorates
Solution Approach 1:
The patent changes the parameter of compression application from universal to selective based on data characteristics. It introduces a success rate threshold parameter to determine whether compression should be applied, transforming the compression process from a fixed operation to a conditional one that adapts to data compressibility, thereby resolving the contradiction between storage efficiency and processor performance
Solution Approach 2:
The system performs self-evaluation by testing data samples to determine compressibility before full compression. The data streams essentially serve themselves by providing sample data that the system uses to make compression decisions, avoiding unnecessary processing of non-compressible data while maintaining efficient compression of suitable data
2Quantity of substance
If compression is applied to non-compressible data, then storage space is reduced, but excessive processor cycles are consumed
Solution Approach 1:
Instead of applying full compression to all data, the patent applies partial action by first testing a sample portion of the data stream. If the sample fails to meet the success rate threshold, the system stops further compression processing for that stream. This partial testing approach prevents excessive processor cycle consumption on non-compressible data while still achieving storage space reduction for compressible data
3Productivity
If selective compression is implemented with success rate threshold, then processor performance is optimized, but system complexity increases
Solution Approach 1:
The patent segments the compression process into distinct stages: sample extraction, compression testing, success rate calculation, and decision-making. This segmentation allows the system to manage complexity by breaking down the selective compression logic into manageable modular components, each handling a specific aspect of the decision process
Data Source
AI summary
Methods, non-transitory computer readable media, and computing devices that assists with selectively compressing data includes identifying data stream characteristics in a received data stream from a client. A data processing operation to perform on the received data stream is determined based on stored compression instructions data obtained using the identified one or more characteristics. The determined data processing operation is performed on the received data stream. The received data stream is performed upon performing the determined data processing operation.