Workload Scheduling Using Historical Node Health Profiles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing scheduling systems in computing environments face inefficiencies due to the lack of consideration for historical node health when selecting computing resources, leading to potential workload failures and unstable systems, as they primarily rely on instantaneous node states rather than historical performance metrics.
Innovation Solution
A proactive fault-tolerant scheduling mechanism that utilizes a historical profile of aggregated node health, calculated through a time series database and exponential weighted moving averages, to score nodes based on past performance, thereby selecting more reliable computing resources for workload distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If scheduling systems rely on instantaneous node states for resource selection, then the scheduling process is simple and fast, but the system reliability deteriorates due to potential workload failures on unhealthy nodes
Solution Approach 1:
The system performs preliminary actions by continuously collecting and storing node health metrics in a time-series database before scheduling decisions are made. Historical health data is pre-processed and maintained ready for quick retrieval during scheduling, allowing the system to account for node reliability trends without adding complexity to the actual scheduling decision process
Solution Approach 2:
An intermediary component is introduced between the scheduler and node health data - a time-series database that stores historical metrics and an exponential weighted moving average calculator. This intermediary layer processes raw health data into meaningful scores that the scheduler can use, isolating the complexity from the core scheduling logic while improving reliability
2Reliability
If scheduling systems use historical performance data and complex scoring mechanisms, then the system reliability improves, but the computational overhead and processing time increase
Solution Approach 1:
The system performs preliminary calculations by continuously computing exponential weighted moving averages of node health metrics in the background, before scheduling decisions are required. This pre-computation of reliability scores ensures that when a scheduling decision is needed, the system can quickly retrieve and use pre-calculated scores without performing complex real-time analysis
Solution Approach 2:
The system transforms raw node health metrics into a standardized scoring parameter using exponential weighted moving averages. This parameter transformation converts multiple varying health indicators into a single comparable score that simplifies the scheduling decision process while maintaining reliability information
3Reliability
If scheduling systems consider aggregated node health profiles, then workload scheduling failures are reduced, but the data processing and storage requirements increase
Solution Approach 1:
The system extracts only the essential health metrics from node data and stores them in a time-series database with automatic retention policies. By extracting and storing only relevant historical data points rather than complete node state information, the system maintains workload execution success rate while managing data volume efficiently
Solution Approach 2:
The system implements data retention policies that discard old historical data beyond a certain time horizon, as it becomes less relevant for predicting future node health. The exponential weighted moving average calculation naturally discounts older data points, giving more weight to recent measurements while automatically reducing the influence of historical data over time
Data Source
AI summary
Apparatuses, systems, and techniques for selecting computing resources based on software programs scoring past performance of computing resources. In at least one embodiment, a processor comprising circuitry may cause software programs to be performed using computing resources based on software programs to score past performance of the one or more computing resources. In at least one embodiment, a processor selects a computing system to perform a software workload based on attributes of computer systems that are recorded over periods of time. In at least one embodiment, a processor comprises circuits to use attribute values of computer systems that are computed over periods of time to select computer systems to perform software workloads.