Androgynous Zone Storage for Opportunistic XOR Backup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional geographically diverse data storage systems face inefficiencies due to the need for all front-end storage devices to send chunks before creating a complete backup, leading to imbalances and increased storage space usage, as some devices process data faster than others.
Innovation Solution
Implementing partial backup chunks that can be created and stored before receiving all chunks, allowing for iterative compression and dynamic role assignment of storage devices, including androgynous zone storage components that can adapt between front-end and back-end roles without reconfiguration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional geographically diverse data storage systems wait for all front-end storage devices to send chunks before creating a complete backup, then data integrity is ensured, but storage space usage increases and processing efficiency decreases due to imbalances in data processing speeds
Solution Approach 1:
The system creates partial backup chunks containing data from a subset of front-end storage devices before all devices have sent their chunks. Instead of waiting for complete data sets, the system performs backup operations on available partial data, thereby improving productivity without significantly compromising data integrity through subsequent updates when additional chunks arrive.
Solution Approach 2:
The system performs preliminary backup operations by creating partial backup chunks as soon as data from some front-end devices is available, rather than waiting for all devices. This preliminary action reduces the overall backup creation time and improves storage efficiency by utilizing available data earlier in the process.
2Reliability
If the system waits for all chunks to be received before creating complete backups, then data completeness is maintained, but storage imbalances occur as some devices process data faster than others
Solution Approach 1:
The system creates partial backup chunks from available data subsets before all front-end devices have contributed their chunks. This approach maintains reasonable data completeness while preventing storage imbalances by allowing faster devices to contribute to backups without being blocked by slower devices, thereby improving operational ease and storage balance.
Solution Approach 2:
The system dynamically adjusts backup creation operations based on the availability of data chunks from front-end devices. Instead of following a rigid wait-for-all protocol, the system adapts its backup operations to the current state of data availability, creating partial backups when possible and updating them as more data becomes available, thus maintaining both reliability and storage balance.
3Device complexity
If storage devices have fixed roles as front-end or back-end, then system architecture is simplified, but adaptability decreases when devices need to be reconfigured
Solution Approach 1:
The system implements androgynous zone storage components that can function in multiple roles - acting as front-end devices when receiving user data and as back-end devices when storing backup chunks. This multi-functionality allows devices to be dynamically reassigned based on system needs without requiring complex reconfiguration, thereby maintaining simple architecture while improving adaptability.
Solution Approach 2:
The system enables dynamic role assignment where storage devices can transition between front-end and back-end roles based on operational requirements. This dynamic capability allows the system to adapt to changing conditions, such as device failure or load balancing needs, while maintaining a relatively simple underlying architecture through the use of standardized androgynous components.
Data Source
AI summary
Opportunistic combining of data chunks is disclosed. Data chunks stored in storage devices of different zones of a zone storage system can be convolved to conserve memory. The zone storage system can be a geographically diverse storage system. A convolved chunk can be stored at a zone that does not contribute a local data chunk to the data represented in the convolved chunk. A zone storage component can be androgynous, rather than being explicitly configured to act as a front/back end storage device. This androgyny can enable the zone storage system to store a complete chunk at a zone based on real time use. In an aspect, an androgynous zone storage component can take on, or transition between, a de facto front-end storage device character or de facto back-end storage device character in response to deployment of the androgynous zone storage component in the storage system.