Dynamic Storage Scheduling via Constraint Satisfaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems are overprovisioned, leading to inefficient resource allocation and requiring developers to modify source code to adapt to changing system conditions, which is impractical and separates the duties of software developers and data center operators.
Innovation Solution
A method and system that dynamically schedules data storage by generating a scheduling problem based on node states and constraints, using metadata and machine learning to optimize resource allocation, ensuring constraints are met while minimizing power usage and ensuring data durability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If developers modify source code to adapt storage system behavior to changing conditions, then the system can respond to current states, but this comingles the duties of software developers and data center operators and requires constant redeployment
Solution Approach 1:
The system transitions from static, pre-defined storage behavior embedded in source code to dynamic, real-time scheduling decisions. The scheduling system evaluates current node states (online/offline, storage capacity, workload) and dynamically assigns storage tasks to optimal nodes without requiring code modifications. This enables the system to adapt to changing conditions while maintaining clear separation between developer and operator responsibilities.
Solution Approach 2:
A scheduling system acts as an intermediary layer between the storage tasks and the physical storage nodes. This mediator receives storage requests, evaluates current system state, and makes intelligent routing decisions based on predefined policies and real-time conditions. The scheduling system translates high-level storage requirements into specific node assignments without requiring changes to the underlying storage software or hardware.
2Reliability
If storage systems are overprovisioned to ensure capacity, then data durability and availability are maintained, but resource allocation efficiency decreases and costs increase
Solution Approach 1:
The scheduling system dynamically adjusts storage task allocation based on changing system parameters such as node online status, available storage capacity, and current workload. When nodes become available, the system automatically assigns pending storage tasks to utilize previously idle resources. This approach maintains data durability through proper replication while improving resource utilization by responding to real-time parameter changes without requiring overprovisioning.
Solution Approach 2:
The system continuously monitors the state of storage nodes and uses this feedback to make intelligent scheduling decisions. By tracking which nodes are online, their available capacity, and their current workload, the scheduling system can dynamically allocate storage tasks to optimize both reliability and resource efficiency. This feedback mechanism ensures data durability requirements are met while minimizing wasted capacity from overprovisioning.
3Ease of operation
If storage tasks are assigned without dynamic scheduling, then system operation is simple, but resource utilization is suboptimal and power consumption increases
Solution Approach 1:
The scheduling system implements a practical approach by focusing optimization on storage task allocation while leaving other system operations unchanged. It selectively applies intelligent scheduling to storage write and read operations, evaluating node states and making assignments that reduce power consumption by utilizing naturally idle nodes rather than activating additional infrastructure. This partial optimization approach delivers energy efficiency improvements without requiring complete system redesign.
Data Source
AI summary
A system and method is disclosed for scheduling and allocating data storage. An example method comprises generating a scheduling problem based at least on states of each of the plurality of storage nodes, a received plurality of storage tasks and received constraints, wherein the scheduling problem is a constraint satisfaction problem, selecting one or more approaches to solving the scheduling problem based on metadata associated with the storage tasks and constraints, solving the scheduling problem to generate a scheduling solution based on the one or more approaches, determining whether the given constraints are satisfied by the scheduling solution, executing, by the processor, the scheduling solution by assigning storage of data to each of the plurality of storage nodes when the constraints are satisfied by the scheduling solution and determining another scheduling solution based on the one or more approaches when the constraints are not satisfied by the scheduling solution.