Distributed Storage Data Re-protection via Local Copy Creation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed replicated data storage systems, data transfer over wide area networks is slower than within local area networks, leading to potential data loss and reduced system performance during replication, especially when detecting impending or actual failures in storage zones.
Innovation Solution
The system creates a remote copy of impacted data within a geographically separate storage zone before transferring it back to the primary location, ensuring data integrity and minimizing downtime by maintaining a healthy copy within the secondary zone.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is replicated over wide area networks, then data redundancy and system reliability are improved, but data transfer speed and system performance deteriorate
Solution Approach 1:
The system proactively detects storage zone failures before they cause data loss and initiates data replication in advance. When a failure is detected in a primary storage zone, the system immediately replicates the impacted data to a secondary storage zone over the wide area network, rather than waiting for data loss to occur. This preliminary action ensures data redundancy is established before transfer speed limitations become critical.
Solution Approach 2:
The system segments data replication into two phases: first replicating only the impacted data segments that are at risk, rather than replicating entire data sets. This selective segmentation reduces the volume of data transferred over the wide area network, thereby improving transfer speed while maintaining the necessary data redundancy for reliability.
2Reliability
If data replication is performed over wide area networks during failure detection, then data security is improved, but system performance and productivity deteriorate
Solution Approach 1:
The system performs partial replication by transferring only the specific impacted data that is at risk, rather than replicating all data. This partial action approach maintains data security for the critical portions while minimizing the performance impact on the overall system during the replication process.
Solution Approach 2:
When a storage zone failure is detected, the system rushes through the replication process by prioritizing and accelerating the transfer of impacted data over the wide area network. This rushing through approach ensures data security is maintained by completing the critical replication quickly, while minimizing the duration of performance degradation.
3Reliability
If data transfer over wide area networks is performed during failure recovery, then data integrity is improved, but transfer speed and operational efficiency worsen
Solution Approach 1:
The system performs preliminary detection of storage zone failures and initiates data replication before data integrity is compromised. By detecting failures early and acting preliminarily, the system ensures data integrity is maintained while reducing the total time required for the transfer process, as the replication begins immediately upon detection rather than after data loss occurs.
Solution Approach 2:
The system transfers only the partial set of impacted data that requires replication to maintain integrity, rather than transferring all data. This partial action reduces the overall transfer time while ensuring that the critical data integrity requirements are met for the affected portions.
Data Source
AI summary
Data re-protection in a distributed replicated data storage system is disclosed. The method may be implemented on a server or controller. A method includes storing first data in a first zone and storing a replica of the first data in a second zone. The zones are at different, separate locations. When an actual or impending failure with the first data in the first zone is detected, the system automatically initiates transitioning to a copy of impacted data at the first zone obtained from the second zone. The transitioning includes creating a remote copy of the impacted data at the second zone within a local area network before transferring the copy to the first zone over a wide area network. The methods allow the system to return to a fully protected state faster than if the impacted data was transferred from the second zone to the first zone without making a copy at the second zone.