Grid Server Deduplication Rate Adjustment for Backup Windows
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deduplication-based data storage systems face availability issues due to reliance on a single compute server for deduplication, leading to increased end-to-end backup times as data volume grows, exceeding backup window limits and violating service level agreements.
Innovation Solution
A scalable server grid system dynamically adjusts deduplication and replication rates based on ingestion rates, allowing parallel processing of ingestion, deduplication, and replication, with failover models to maintain availability and reduce recovery point objective times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single compute server is used for deduplication processing, then device complexity is reduced, but reliability deteriorates and backup time increases as data volume grows
Solution Approach 1:
The system divides the deduplication processing function into multiple independent compute servers organized in a grid. Each server handles a portion of the deduplication workload independently, eliminating the single point of failure and allowing the system to scale with data volume while maintaining reliability and constant backup windows.
2Quantity of substance
If data volume increases and additional disk storage units are added, then storage capacity is improved, but backup time increases beyond service level agreement limits
Solution Approach 1:
The system segments the deduplication workload across multiple compute servers in a grid architecture. As storage capacity increases with additional disk units, the compute grid expands proportionally to maintain constant backup windows, ensuring service level agreements are met regardless of data volume growth.
Solution Approach 2:
The system dynamically scales the compute grid capacity to match storage capacity growth. The number of compute servers is adjusted based on the amount of data to be processed, maintaining optimal performance and constant backup windows as the system evolves over time.
3Loss of time
If deduplication processing capacity is increased to handle larger data volumes, then backup time is reduced, but device complexity and cost increase
Solution Approach 1:
Each compute server in the grid is designed to be multi-functional, handling both data ingestion and deduplication processing. This universal design allows flexible resource allocation and scaling without requiring specialized hardware, reducing overall system complexity while maintaining high performance.
Data Source
Figure 1
Figure 2a
Figure 2b
AI summary
A system, a method, and a computer program product for or managing grid server system resource(s) are disclosed. A processor of a grid server in a plurality of grid servers determines an ingestion rate of data received by the grid server for a backup. The ingestion rate corresponds to a speed of transferring of the received data into a memory of the grid server. The processor determines a deduplication and replication rate of the received data. Deduplication represents a deduplication of the received data by the grid server. Replication represents replication of the deduplicated received data to another server in the plurality of servers. The processor adjusts the deduplication and replication rate based on the ingestion rate.