Workload Prediction Model for Computing Cluster
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current capacity planning in z/OS computing clusters is inefficient due to reliance on manual expert judgment and offloading data to external storage, leading to inaccurate workload predictions and increased complexity, which can result in over- or under-commitment of resources, affecting performance and costing.
Innovation Solution
A method for performance predictions in a computing cluster that involves sampling historic workload data, applying time-series analysis to detect cyclic and repeating events, training a workload prediction model using additional data about acyclic events, and reassigning resources based on predicted future demand, all performed within the cluster using Apache Spark and machine learning algorithms like Gradient Boosted Trees.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual expert judgment and external storage are used for capacity planning, then resource allocation decisions can be made, but prediction accuracy decreases and system complexity increases
Solution Approach 1:
The patent extracts the data processing functionality from external storage systems and implements it directly within the z/OS computing cluster using Apache Spark. This eliminates the need to offload data to external storage while maintaining the ability to perform complex time-series analysis and machine learning workloads, thereby improving prediction accuracy without increasing external system dependencies
Solution Approach 2:
The computing cluster performs its own capacity planning and workload prediction using internally deployed Apache Spark and machine learning algorithms. This self-service approach eliminates reliance on external storage systems and manual expert judgment, enabling automated, accurate predictions while reducing system complexity through internalization of previously external functions
2Productivity
If manual expert judgment is used for capacity planning, then resource allocation can be performed, but productivity and response time decrease
Solution Approach 1:
The system implements automated capacity planning through self-service mechanisms where Apache Spark and machine learning models continuously analyze workload data and generate predictions without manual intervention. This automation dramatically improves productivity by eliminating manual expert judgment while reducing response time through real-time or near-real-time analysis capabilities
Solution Approach 2:
The patent implements preliminary action by continuously training machine learning models with historical workload data and performing predictive analysis before capacity planning decisions are needed. This advance preparation enables rapid response to changing workload demands without requiring manual analysis at the time of decision-making
Data Source
AI summary
Performance predictions in a computing cluster can be provided by sampling and storing historic workload request data of the computing cluster as time-stamped workload values, forecasting an expected total number of workload requests for a defined time interval in the future based on a time-series analysis of the time-stamped workload values, where the time-series analysis detects cyclic and repeating events in the time-stamped workload values. In response to a result of the time-series analysis, training a workload prediction model by using additional data about acyclic events in expected workload requests, where the training applies a statistical regression technique for predicting a future workload demand for the computing cluster, and in response to exceeding a predefined threshold value of the predicted future workload demand, reassigning resources of the computing cluster.