Parallel Data Restore With Incremental Compression Under I/O Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data restore operations in computing systems are prone to latency issues due to increased load, memory capacity, reduced bandwidth, and resource faults, leading to blocking and slowdowns in data restoration processes.
Innovation Solution
A non-blocking object-level restore system employing parallel I/O operations and incremental data compression to read and write data objects in parallel, utilizing a data restore appliance with components like a volume mount service, parallel file read/write services, and a data compressor to manage data transfer and compression efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data restore operations are performed using sequential reads and writes, then resource usage is reduced, but the operation speed and productivity deteriorate due to latency issues
Solution Approach 1:
The patent divides the data restore operation into multiple parallel stages: reading data objects from backup storage into buffers, compressing data in parallel, and writing compressed data to target storage. This segmentation allows simultaneous execution of multiple operations that would otherwise be sequential, directly improving restore speed while managing complexity through structured parallelism
Solution Approach 2:
The patent introduces a new dimension of parallel processing by implementing multi-threaded I/O operations where multiple threads read different data objects simultaneously into different buffers, and multiple threads write compressed data simultaneously to different locations. This dimensional expansion from sequential to parallel operations resolves the productivity-complexity contradiction
2Productivity
If parallel I/O operations are implemented to improve restore speed, then productivity increases, but resource usage and device complexity increase
Solution Approach 1:
The patent performs data compression on buffered data objects before writing to target storage, rather than compressing during the write operation. This preliminary compression action allows the I/O subsystem to operate in parallel without the overhead of real-time compression during data transfer, improving productivity while managing resource consumption through staged processing
Solution Approach 2:
The patent introduces buffers as intermediary storage between the backup storage and target storage. Data objects are first read into buffers, then compressed, and finally written from buffers to target storage. These intermediary buffers decouple the read and write operations, enabling parallel execution while controlling resource usage through buffered rather than direct streaming
3Productivity
If data compression is applied during restore operations, then data transfer efficiency improves, but processing time and device complexity increase
Solution Approach 1:
The patent implements continuous parallel compression where multiple data objects are compressed simultaneously in an pipeline fashion. While one data object is being compressed, another is being read into a buffer, and a third is being written from its compressed form. This continuous overlapping of read-compress-write operations across multiple threads maintains high data transfer efficiency while minimizing idle processing time
Solution Approach 2:
The patent performs compression on data objects while they are in the buffer stage, before the write operation begins. This preliminary compression action separates the compression workload from the critical I/O path, allowing data transfer to proceed efficiently while compression occurs in parallel on buffered data, thus improving overall transfer efficiency without adding significant processing time to the critical path
Data Source
AI summary
As part of a data restore operation from a backup data store, a system reads data objects of the backup data store in parallel into a buffer, which includes reading chunks of a first data object into the buffer. The data objects are provided from the buffer to a data compressor that incrementally compresses the data objects as the data objects are received at the data compressor to produce compressed data objects. The compressed data objects are transferred in parallel to a target data store as part of the data restore operation.