Universal Connector for NAS Incremental Backup Batching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for managing backups of large datasets stored in Network Attached Storage (NAS) face inefficiencies in bandwidth management, resource optimization, and protocol compatibility, leading to suboptimal performance and increased costs.
Innovation Solution
The implementation of a universal connector that provides a protocol-agnostic, multi-threaded interface for managing backups. This connector optimizes backups by batching small files, managing incremental backups, reducing roundtrips for network bandwidth efficiency, and parallelizing backup processes based on resource availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If incremental backups are performed for each small file individually, then backup completeness is improved, but network bandwidth usage increases and backup time extends
Solution Approach 1:
The patent combines multiple small file backup operations into a single batched incremental backup operation. The system groups files that have changed since the last backup and processes them together in one network transaction, rather than sending individual backup requests for each file. This merging approach maintains complete backup coverage while significantly reducing network bandwidth consumption and roundtrip overhead.
2Measurement precision
If metadata requests are sent for each file to determine changes, then backup accuracy is improved, but the number of roundtrips increases and backup performance deteriorates
Solution Approach 1:
The system merges multiple individual file metadata requests into a single batched metadata retrieval operation. By grouping files and querying their metadata collectively, the system maintains accurate detection of changed files while reducing the number of network roundtrips from potentially thousands (one per file) to a manageable number of batched requests, thereby improving overall backup performance.
3Adaptability or versatility
If a universal connector with multi-threaded interface is implemented, then protocol compatibility and resource optimization are improved, but system complexity increases
Solution Approach 1:
The patent implements a universal connector that provides a multi-threaded interface capable of handling multiple backup protocols and operations simultaneously. This universal component can process different protocol types (NFS, SMB, etc.) and manage multiple backup streams in parallel, providing enhanced protocol compatibility and resource optimization. The complexity is managed by consolidating these capabilities into a single versatile connector rather than implementing separate handlers for each protocol and operation type.
Data Source
AI summary
A method for managing data includes obtaining, by a universal connector, a request for an incremental backup of a set of files, in response to the request: performing a metadata batching of the set of files to obtain a set of file batches, performing a request conversion to generate a set of converted metadata requests each corresponding to one of the set of file batches, obtaining a results array from a network attached storage (NAS) server, wherein the results array comprises metadata for each file in the set of files, performing a metadata comparison with an attributes table stored in the universal connector to determine a subset of files, and initiating an incremental backup of the subset of files by sending the set of converted batch requests to a network attached storage (NAS) server to obtain the subset of files and generating the incremental backup.


