Non-Interruptive Data Reconstruction via Traffic Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data reconstruction methods for cloud services are interruptive, causing service disruptions and inefficiencies, especially when upgrading or downgrading storage systems, as they require downtime or simultaneous processing of read and write operations.
Innovation Solution
A system for non-interruptive data reconstruction involving a source storage, target storage, a server with an agent module and path IO module, a traffic modeling unit, and a data moving service unit that collects performance parameters, records changed blocks, and schedules data copying and switching operations to minimize access disruptions, allowing continuous service during data migration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If offline method is used for data reconstruction, then implementation simplicity is improved, but service continuity deteriorates due to required shutdown and reboot
Solution Approach 1:
The system performs preliminary actions by creating snapshots of the source storage before the actual data migration. This allows the migration process to start in advance without interrupting the running service, as the snapshot is taken while the service is still accessible. The bitmap recording of changed blocks is also prepared in advance to track modifications during migration.
Solution Approach 2:
The patent uses snapshot copying to create a point-in-time copy of the source storage before migration. This snapshot can be copied to the target storage without requiring the source storage to be taken offline. The copying process uses bitmap records to efficiently track and replicate only the changed blocks, enabling continuous service during the copy operation.
2Reliability
If mirroring method is used for data reconstruction, then service continuity is improved, but storage compatibility and performance deteriorate due to vendor restrictions and role switching requirements
Solution Approach 1:
The system segments the data migration process into distinct phases: snapshot creation, bitmap recording of changes, and selective copying based on the bitmap. This segmentation allows independent operation of each phase without requiring the entire system to switch roles or depend on specific vendor mirroring capabilities, thereby improving compatibility.
Solution Approach 2:
The patent introduces a bitmap as an intermediary structure to track changed blocks between snapshots. This bitmap acts as a mediator that enables the system to identify and copy only the necessary blocks to the target storage, eliminating the need for complex role switching and vendor-specific mirroring protocols while maintaining service continuity.
3Productivity
If dual write and snapshot image copying method is used, then data reconstruction capability is improved, but service interruption occurs due to simultaneous read and write operations
Solution Approach 1:
The system takes a preliminary snapshot of the source storage before the dual write operation begins. This preliminary snapshot captures the state of data at the start, and the bitmap is used to track only the changes that occur during the dual write phase. This allows the service to remain accessible throughout the process without interruption.
Solution Approach 2:
The bitmap serves as a feedback mechanism that continuously tracks which blocks have changed between snapshots. This feedback information is used to selectively copy only the changed blocks to the target storage, enabling the system to manage read and write operations efficiently without causing service interruption, as the feedback loop allows for coordinated processing.
4Adaptability or versatility
If in-band data moving service unit is used, then data migration functionality is added, but system complexity and performance bottleneck increase
Solution Approach 1:
The patent extracts the data migration functionality from the main service path by using offline snapshot copying. Instead of requiring an in-band service unit that processes all data through a complex infrastructure, the system creates snapshots and copies them using a separate, simpler mechanism. This extraction reduces the complexity of the in-band infrastructure while maintaining migration capability.
Solution Approach 2:
The system uses snapshot copying as a simplified alternative to in-band data moving. By copying snapshots of the source storage to the target storage using a straightforward copying mechanism (rather than complex in-band infrastructure), the patent achieves data migration functionality without introducing significant complexity or creating performance bottlenecks in the main service path.
Data Source
AI summary
A system for achieving non-interruptive data reconstruction is disclosed. The system includes a source storage, a target storage, a server, a traffic modeling unit and at least one data moving service unit. With the help of traffic modeling unit, a period of time of low access can be estimated. Data reconstruction (copying or moving) can be carried on during the period of time. Thus, non-interruptive data reconstruction can be done.