Adaptive Data Compression in Shared Resource Pools
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As the demand for data storage in shared pools of configurable resources increases, existing systems face challenges in adapting to demand while minimizing storage costs, as network bandwidth and compute power decrease with increased load, necessitating efficient data compression and storage optimization.
Innovation Solution
A real-time analytical model is employed to select the appropriate data format and storage location, determining whether to compress data and choosing an appropriate compression technique, with compression potentially occurring at the client or within the shared pool, to minimize overall storage costs and optimize data footprint.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in a shared pool of resources without compression, then storage capacity is maximized, but network bandwidth and compute power decrease as load increases
Solution Approach 1:
The system dynamically changes the compression parameter of data based on storage pool load conditions. When the storage pool is under heavy load, the system applies compression to reduce the volume of incoming data, thereby preserving network bandwidth and compute power while still utilizing available storage capacity. This parameter adjustment resolves the contradiction by allowing the system to maintain both storage capacity utilization and operational productivity.
2Loss of energy
If data compression is applied to reduce storage footprint, then storage costs and network bandwidth usage decrease, but compression processing requires additional compute resources
Solution Approach 1:
The system employs self-service by automatically evaluating data characteristics and making intelligent compression decisions without requiring external intervention. The evaluation manager analyzes data properties and determines optimal compression strategies, allowing the system to balance compression benefits against compute resource costs autonomously. This self-service mechanism resolves the energy loss contradiction by optimizing compression application to achieve storage and bandwidth savings while minimizing unnecessary compression processing.
3Volume of stationary object
If compression is applied to all data, then storage footprint is minimized, but data access and retrieval operations become more complex
Solution Approach 1:
The system applies local quality by selectively compressing only specific data blocks or files based on their individual characteristics, access patterns, and priority levels rather than uniformly compressing all data. The evaluation manager determines which data portions benefit most from compression while keeping frequently accessed or critical data in uncompressed form. This selective approach minimizes storage footprint for appropriate data while maintaining simple access operations for data that requires frequent or complex retrieval.
Data Source
AI summary
Embodiments of the invention relate to evaluation and storage of data in a computer system configured with a shared pool of resources. A multi-level adaptive compression technique is employed to minimize the cost of data storage based upon the type of data being stored and their access pattern. The costs of data storage include capacity, bandwidth, and compute cycles. Data is transformed local to a client in communication with the shared pool, local to the shared pool, or as a combination with a partial transformation local to the client and a partial transformation local to the shared pool.