Storage Workload Scheduling for Deduplication Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing workload scheduling methods in storage systems using deduplication do not effectively optimize deduplication ratios, leading to suboptimal performance in deduplication hit rates and read-cache behavior due to limited memory and temporal correlations between workload processes.
Innovation Solution
A method that determines dynamic correlations between workload processes within a prior time window, calculates deduplication ratios, and schedules multiple workload processes based on the highest deduplication ratio to optimize deduplication efficiency, utilizing an epoch graph structure to capture recent deduplication relationships and adjust scheduling accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If workload processes are scheduled without considering deduplication correlations, then scheduling simplicity is maintained, but deduplication ratios and hit rates deteriorate
Solution Approach 1:
The system performs preliminary analysis of deduplication correlations between workload processes before scheduling decisions are made. By pre-computing correlation metrics and storing them in a correlation matrix, the system prepares scheduling information in advance, enabling optimized scheduling without real-time computation overhead.
Solution Approach 2:
The patent replaces traditional scheduling mechanisms with an intelligence-driven approach. Instead of using simple round-robin or priority-based scheduling, the system employs machine learning models and correlation analysis to dynamically determine optimal scheduling decisions, substituting mechanical scheduling algorithms with cognitive computing approaches.
2Reliability
If workload processes are scheduled based on deduplication correlations, then deduplication hit rates improve, but processing time increases
Solution Approach 1:
The system pre-computes deduplication correlations and stores them in a correlation matrix for rapid retrieval during scheduling. By performing the computationally intensive correlation analysis beforehand rather than in real-time, the system achieves high deduplication hit rates without adding significant processing overhead to the scheduling decision itself.
Solution Approach 2:
The scheduling system dynamically adjusts its behavior based on workload characteristics and correlation strength. For workloads with strong deduplication correlations, the system applies optimized scheduling; for workloads with weak correlations or time-sensitive requirements, the system reverts to standard scheduling, thereby balancing performance optimization with processing efficiency.
3Reliability
If memory capacity is increased to store more deduplication data, then deduplication hit rates improve, but hardware cost increases
Solution Approach 1:
The system changes the parameter of data organization by implementing hierarchical storage structures and compression algorithms for deduplication data. Instead of storing all deduplication data uniformly in high-speed memory, the system uses a multi-level storage hierarchy that places frequently accessed deduplication data in fast memory while storing less frequently accessed data in slower, cheaper storage media.
Solution Approach 2:
The patent extracts only the most relevant deduplication data for immediate processing, separating frequently accessed deduplication entries from the full deduplication database. By extracting and prioritizing hot data in fast memory while leaving cold data in slower storage, the system achieves high deduplication hit rates without requiring proportionally large increases in expensive memory capacity.
Data Source
AI summary
A computer-implemented method enables workload scheduling in a storage system for optimized deduplication. The method includes determining dynamic correlations of deduplications between workload processes in a prior time window. Workload processes include one or more tasks with defined execution timing parameters. The method further includes determining deduplication ratios based on the correlations of the deduplications between the workload processes. The method further includes scheduling multiple workload processes based on a highest determined deduplication ratio of the determined deduplication ratios.