Dynamic Scanning Thread Allocation for Backup Speed
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Companies face challenges in efficiently backing up large volumes of data stored in network-based storage environments without the ability to install data agents or filter drivers, particularly in third-party managed systems, which limits their control over scanning processes and resource allocation.
Innovation Solution
A data agent within the primary storage environment determines the optimal number of scanning threads to identify files for backup in a network-based storage environment, analyzing network performance and using historical data to dynamically adjust thread instances, enabling efficient backup operations even in remote or third-party managed storage settings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple scanning threads are used to scan large volumes of data, then backup productivity is improved, but device complexity and resource management difficulty increase
Solution Approach 1:
The patent divides the scanning process into multiple independent threads that can operate simultaneously on different portions of the storage system. Each thread is responsible for scanning specific volumes or file systems, enabling parallel processing that increases backup productivity while maintaining manageable complexity through modular thread design
Solution Approach 2:
The patent implements dynamic thread management where the number of scanning threads is determined based on real-time system conditions such as available resources, storage volume size, and network bandwidth. The thread configuration can be adjusted during operation to optimize performance, balancing productivity gains with complexity management through adaptive rather than fixed threading strategies
2Productivity
If data agents are installed on third-party storage systems to enable scanning, then backup capability is improved, but ease of operation deteriorates due to installation restrictions
Solution Approach 1:
The patent introduces a data agent as an intermediary component that can be deployed on third-party storage systems to enable backup scanning without requiring direct access to the underlying storage architecture. The data agent acts as a mediator between the backup system and the storage system, providing scanning capabilities while simplifying operation through standardized interfaces and reducing the need for complex system-level access or installation
3Reliability
If comprehensive scanning of all files is performed, then data protection completeness is improved, but loss of time increases due to scanning duration
Solution Approach 1:
The patent segments the scanning process into multiple parallel threads that simultaneously scan different portions of the storage system. This segmentation enables comprehensive scanning of all files for complete data protection while reducing total scanning time through parallel processing. Each thread can independently scan specific volumes, directories, or file types, and results are aggregated to maintain completeness
Solution Approach 2:
The patent implements continuous scanning operations where multiple threads operate simultaneously and continuously through the storage system. Rather than sequential scanning that would complete one portion before starting another, the system maintains continuous scanning action across multiple fronts, improving both completeness through comprehensive coverage and reducing time through overlapping scan operations
Data Source
AI summary
Embodiments disclosed herein address the need to more efficiently backup a network-based storage environment that may be remote from a primary storage environment. For example, embodiments herein can provide a more efficient backup of a storage managed by a third-party entity. To improve the backup process, embodiments herein may optimize the number of scanning threads that are used to identify files that are to be backed up by, for example, analyzing the characteristics of the network and/or the network storage system to determine a number of scanning threads that will enable faster scanning of the network storage system while at the same time not overburden or be slowed down by a network between the network-based storage environment and the primary storage environment.