Adaptive Throttling of Metadata Requests for Backup Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The primary storage system experiences CPU resource overburden and performance degradation due to indiscriminate metadata requests from the backup system during incremental backups, potentially leading to CPU overheating and malfunction.
Innovation Solution
Adaptive throttling of metadata requests is implemented using a token request system, where requests are managed based on a determined performance metric of the primary storage system, ensuring that metadata requests are sent proximate to when the backup system requires them and can ingest the data, thereby preventing CPU resource overutilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the backup system sends metadata requests for all objects during incremental backup, then the backup system can determine which objects have changed, but the primary storage system becomes overburdened with CPU intensive operations
Solution Approach 1:
The backup system performs a full directory traversal of the primary snapshot to pre-determine which objects need to be included in the incremental backup before sending metadata requests. This preliminary action allows the backup system to have a targeted list of objects, reducing the number of metadata requests sent to the primary storage system and thereby reducing its CPU burden while still ensuring accurate backup of all necessary objects.
Solution Approach 2:
Instead of sending metadata requests for all objects in the file system, the backup system sends requests only for the subset of objects that were determined to be included in the incremental backup during the directory traversal phase. This partial action approach reduces the total number of metadata requests, lowering the CPU intensive operations on the primary storage system while maintaining backup reliability for the necessary objects.
2Loss of information
If the primary storage system responds to all metadata requests, then the backup system can identify all changed objects, but the available CPU resources for other processes are significantly reduced
Solution Approach 1:
The backup system performs directory traversal of the primary snapshot beforehand to create a definitive list of objects that need to be backed up. This preliminary action ensures that the backup system has complete information about which objects are included in the incremental backup, so it only requests metadata for those specific objects rather than all objects, thus maintaining information completeness while reducing CPU consumption.
Solution Approach 2:
The backup system uses the primary snapshot as a reference copy to determine which objects need to be backed up. By working with this snapshot copy and its directory structure, the backup system can identify target objects without requiring metadata from the primary storage system for objects that are not included in the incremental backup, thereby reducing the number of metadata requests and CPU resource consumption on the primary system.
3Speed
If metadata requests are sent continuously during directory traversal, then the backup system can process objects in real-time, but the primary storage system may become overloaded and experience performance degradation
Solution Approach 1:
The backup system performs the full directory traversal of the primary snapshot before sending any metadata requests to the primary storage system. This preliminary action creates a complete roadmap of which objects need to be backed up, allowing the backup system to then send metadata requests in a controlled manner rather than continuously during traversal, preventing overload on the primary storage system while maintaining backup functionality.
Data Source
AI summary
An identification of a primary snapshot created for a primary storage system is received. A first request for a first metadata of a first file directory structure object associated with the primary snapshot is issued. A second request for data content of the first file directory structure object associated with the primary snapshot is determined to be sent to a recipient device based on a received response to the first request. A third request for a second metadata of a second file directory structure object associated with the primary snapshot is determined to be sent to the recipient device. Timing and ordering of issuance of a plurality of requests that at least include the second request and the third request to the recipient device are managed based on a determined performance metric of the recipient device and corresponding relative impact to the performance metric of the recipient device.