Workload Clustering via Machine Learning for Distributed Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed computing systems, managing a large number of workloads individually is inefficient, as existing techniques require manual selection and management of workloads, which can be time-consuming and labor-intensive, especially when dealing with similar workloads.
Innovation Solution
Implementing a machine-learning algorithm, such as the k-means clustering algorithm, to identify and manage workload clusters based on characteristics like processing usage, memory usage, and operating system type, allowing for efficient grouping and management of similar workloads as a group.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If manual selection and management of workloads is used, then workload management can be performed with simple techniques, but the time and effort required for managing multiple workloads increases significantly
Solution Approach 1:
The patent merges multiple individual workload management tasks into a single cluster-level operation. By grouping similar workloads into clusters based on shared characteristics (CPU usage, memory usage, I/O patterns, etc.), operators can perform management actions on entire clusters simultaneously rather than individually handling each workload, thereby dramatically reducing the time and effort required.
Solution Approach 2:
The patent creates abstract representations (clusters) that copy and generalize the common characteristics of multiple workloads. Instead of managing each workload's unique attributes individually, the system creates a clustered view that captures shared properties, allowing management actions to be applied based on these copied characteristics rather than handling each workload separately.
2Measurement precision
If individual workload management is performed, then each workload can be managed with specific precision, but the complexity of managing a large number of workloads increases
Solution Approach 1:
The patent segments the large set of workloads into smaller, manageable clusters based on their characteristics. This segmentation reduces the complexity of the management system by breaking down the monolithic task of managing all workloads individually into multiple smaller tasks of managing distinct clusters, each with similar properties.
Solution Approach 2:
The patent creates a universal management approach where a single management action can be applied to multiple workloads through their common cluster representation. The cluster serves as a multi-functional abstraction that enables operators to manage diverse workloads using unified procedures, reducing the need for separate management mechanisms for each individual workload.
3Productivity
If workload clustering using machine-learning algorithms is implemented, then workload management efficiency is improved, but the complexity of the management system increases
Solution Approach 1:
The patent replaces manual, mechanical workload management processes with automated machine-learning algorithms. Instead of operators manually analyzing and grouping workloads based on their characteristics, the system uses machine-learning algorithms to automatically identify patterns, similarities, and optimal cluster formations, thereby improving efficiency while the added computational complexity is handled automatically by the algorithm.
Solution Approach 2:
The machine-learning algorithm enables the workload management system to perform self-service by automatically analyzing workload characteristics, identifying clustering opportunities, and forming clusters without requiring manual intervention. The system serves itself by autonomously making management decisions based on the learned patterns from historical workload data.
Data Source
AI summary
Methods, systems, and computer readable mediums for optimizing a system configuration are disclosed. In some examples, a method includes determining whether a system configuration for executing a workload using a distributed computer system is optimizable and in response to determining that the system configuration is optimizable, modifying the system configuration such that at least one storage resource for storing workload data is located at a server node that is executing the workload in the distributed computer system.