Big Data Storage Redundancy via Asymmetric Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems for big data processing face challenges in efficiently managing storage capacity due to redundant data duplication, leading to increased space occupation and processing complexities.
Innovation Solution
A method involving a data storage system with multiple storage devices, where data is redundantly stored across devices, with non-compressed data stored in primary devices and compressed data stored in a third device, allowing for parallel access and decompression operations to ensure data availability and optimize storage space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is redundantly duplicated three times across storage devices, then data reliability and availability are improved, but storage capacity is wasted and space occupation increases
Solution Approach 1:
The patent changes the physical state of data by applying compression to one copy while keeping other copies uncompressed. This allows the system to maintain redundancy for reliability while reducing the total storage capacity required. The compressed copy serves as space-efficient backup that can be decompressed when needed.
Solution Approach 2:
Instead of treating all redundant copies equally (symmetric storage), the patent introduces asymmetry by compressing data in one storage device while leaving other devices with uncompressed data. This asymmetric approach maintains the reliability benefits of redundancy while optimizing storage capacity utilization.
2Quantity of substance
If data is stored in compressed format, then storage capacity is reduced, but access speed and processing efficiency may be degraded due to decompression requirements
Solution Approach 1:
The patent performs compression in advance during the data writing phase, so that when data needs to be accessed, the compressed version is already prepared and ready for decompression. This preliminary compression action separates the storage optimization from the access operation, allowing efficient retrieval without real-time compression overhead.
Solution Approach 2:
Instead of compressing all redundant copies (excessive action), the patent applies compression to only one copy (partial action). This partial compression approach achieves storage capacity reduction while maintaining uncompressed copies that can provide faster access when needed, balancing storage efficiency with access speed.
3Reliability
If multiple storage devices are used for redundant storage, then data reliability is improved, but system complexity increases
Solution Approach 1:
The patent assigns different qualities to different storage devices: one device stores compressed data while others store uncompressed data. This local quality differentiation allows the system to optimize for storage capacity in one location while maintaining fast access capabilities in others, reducing overall system complexity by having specialized roles for each device.
Data Source
AI summary
An operating method of a data storage system comprising a processor and multiple storage devices, the operating method comprising: a first storage operation of selecting a first storage device, a second storage device, and a third storage device among the multiple storage devices and transmitting and storing data generated by the processor in the first storage device and the second storage device, a second storage operation of transmitting, to the third storage device, the data stored in the second storage device and compressing and storing the data in the third storage device, a first access operation of accessing the data in the first storage device, by the processor, after the first storage operation is completed, and a second access operation of accessing the data in the second storage device after fail of the first access operation.