Backup Object Parallel Storage for Network Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud storage systems face challenges in efficiently accessing and managing backup files across multiple client systems, leading to network delays and latency issues when writing and reading backup objects in network storage.
Innovation Solution
The system generates backup objects from metadata and data set records in a backup file, allowing concurrent transmission and storage across multiple containers in network storage, enabling parallel writing and reading to reduce network delays and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If backup objects are transmitted sequentially to network storage, then network storage reliability is maintained, but network delays and latency increase
Solution Approach 1:
The backup file is divided into multiple backup objects that can be transmitted concurrently to different containers in parallel, transforming a sequential transmission process into a parallel one, thereby reducing overall transmission time and network latency while maintaining data integrity through proper ordering information
Solution Approach 2:
Order information is embedded in the backup object information before transmission occurs. This preliminary action ensures that even though objects are transmitted in parallel, they can be correctly reassembled in the proper sequence at the destination, maintaining reliability without sacrificing speed
2Loss of time
If multiple backup objects are transmitted concurrently to network storage, then network delays and latency are reduced, but device complexity increases
Solution Approach 1:
Backup object information acts as an intermediary data structure that contains ordering information and metadata. This intermediary enables parallel transmission by providing the necessary coordination and tracking mechanisms, allowing the system to manage concurrent operations without significant complexity increases
Solution Approach 2:
The system creates and manages copies of backup objects in parallel across multiple containers. By using identical copy operations for each backup object and relying on the backup object information for ordering, the system reduces complexity through operational uniformity while achieving parallel transmission
3Productivity
If backup objects are stored in separate containers, then concurrent writing is enabled, but data access complexity increases
Solution Approach 1:
The backup data is segmented into multiple backup objects stored in separate containers, enabling concurrent writing operations. Each container can be written to independently and in parallel, significantly improving productivity and reducing write operations time
Solution Approach 2:
The backup object information contains ordering information that provides feedback about the correct sequence of backup objects. This feedback mechanism allows the system to track and reassemble objects in the proper order despite being stored in separate containers, managing access complexity through systematic tracking
Data Source
AI summary
Provided are a computer program product, system, and method for accessing records of a backup file in a network storage. A backup file comprises a sequential file of metadata records and data set records is processed. The metadata records include metadata on data of data sets in the data set records. Backup objects store the metadata and the data set records in the backup file. A determination is made of at least one container in the network storage to store the backup objects. Backup object information indicates for each of the backup objects, an order of the metadata or data set record, stored in the backup object, in the backup file, and a container of the at least one container in which the backup object is stored. Multiple of the backup objects are concurrently transmitted to the network storage to concurrently write to the at least one container.