3D GRID Compression Layer for Load-Balanced Storage Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data compression in computer systems consumes significant CPU resources, making it challenging for efficient storage and processing, especially in GRID storage environments where random access operations are affected.
Innovation Solution
A three-dimensional GRID compression architecture is introduced, incorporating a new compression layer with additional compression nodes that operate between interface and data nodes, allowing for real-time compression and load balancing without impacting performance, using a data distribution scheme that divides volumes into sections for efficient processing and redistribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data compression is implemented in traditional CPU-based systems, then storage efficiency is improved, but CPU resource consumption increases significantly
Solution Approach 1:
The system segments the compression function from the storage function by introducing dedicated compression nodes that operate independently from data storage nodes. This segmentation allows compression operations to be performed by specialized nodes rather than consuming CPU resources on general-purpose processors, thereby improving storage efficiency while reducing CPU resource consumption.
Solution Approach 2:
Compression nodes act as intermediary components between data sources and storage systems. These intermediary nodes handle all compression and decompression operations, preventing CPU-intensive compression tasks from directly impacting the performance of storage and retrieval operations. The intermediary layer absorbs the computational burden while maintaining efficient data flow.
2Adaptability or versatility
If compression nodes are added to the GRID architecture, then data compression capability is improved, but system complexity increases
Solution Approach 1:
Compression nodes are designed as universal components that can handle multiple functions including compression, decompression, and data routing. By creating multi-functional nodes, the system achieves enhanced compression capability without proportionally increasing complexity, as each node serves multiple purposes within the GRID architecture.
Solution Approach 2:
The patent introduces compression nodes as a new dimension or layer in the GRID storage architecture. Rather than complicating existing nodes, the system adds a vertical layer of compression functionality that operates independently. This dimensional addition allows compression capability to be integrated without fundamentally restructuring the horizontal data flow of the original GRID system.
3Quantity of substance
If random access operations are performed in compressed storage, then storage density is improved, but access time increases
Solution Approach 1:
Data is pre-compressed before storage, and the system maintains indexes and metadata that allow rapid location of compressed data blocks. By performing compression in advance during data ingestion rather than during access operations, the system achieves high storage density while maintaining fast access times through pre-established data structures that enable quick retrieval of compressed blocks.
Data Source
AI summary
Various embodiments for data compression by a processor. Levels of data distribution are configured for data processing, including a first level of the data distribution incorporating a GRID network of data storage nodes, and a second level of the data distribution incorporating a GRID network of compressive nodes in communication with the GRID network of data storage nodes. Input/output (I/O) for an associated storage volume is load balanced between the data storage nodes, as data passes through the first level into the second level to be compressed or uncompressed.