Excess Capacity Grid Orchestration for Idle AI and ML Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enterprise customers face challenges in maximizing the utilization of server hardware resources, which often sit idle during non-business hours, leading to high energy expenditure and carbon footprint, while seeking to engage energy-intensive workloads like machine learning and AI, and managing excess capacity in virtualized computing environments.
Innovation Solution
Leveraging web assembly to form an excess capacity grid that securely executes workloads within existing virtual machines, utilizing underutilized compute and memory resources without specialized accelerators, and dynamically deploying AI and machine learning processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If server hardware resources are deployed to meet peak demand, then service reliability is improved, but resource utilization deteriorates (most resources sit idle during non-business hours)
Solution Approach 1:
The system dynamically allocates computational resources between production workloads and AI/ML training workloads based on real-time capacity availability. During business hours, resources are dedicated to production services; during non-business hours, excess capacity is automatically repurposed for AI/ML training, creating a dynamic resource sharing model that adapts to varying demand patterns
Solution Approach 2:
The same server hardware resources serve multiple functions: production workloads during business hours and AI/ML training during non-business hours. This multi-functionality allows the infrastructure to support both operational requirements and advanced computing initiatives without requiring separate dedicated hardware for each purpose
2Power
If energy-intensive AI and machine learning workloads are executed, then computational capability is improved, but energy consumption increases
Solution Approach 1:
AI/ML training workloads are executed periodically during non-business hours when production workload demand is low. This periodic execution pattern allows the system to leverage off-peak capacity for energy-intensive computations without impacting production services, effectively using time-based resource allocation to manage energy consumption
Solution Approach 2:
The system automatically identifies and utilizes excess capacity for AI/ML training without requiring separate dedicated resources. The infrastructure serves itself by repurposing unused computational resources, eliminating the need for additional energy-consuming hardware while still advancing AI/ML capabilities
3Productivity
If excess capacity is utilized for AI and machine learning processes, then resource utilization is improved, but service performance may deteriorate
Solution Approach 1:
The system implements dynamic resource allocation that continuously monitors production workload performance and AI/ML training progress. Resource allocation between production and training workloads is adjusted in real-time based on current system state, ensuring that service performance requirements are always met while maximizing resource utilization
Solution Approach 2:
The system schedules AI/ML training workloads to execute during predetermined non-business hours when production workload demand is expected to be minimal. This advance scheduling ensures that training activities do not compete with production workloads for resources, preserving service performance while enabling resource utilization
Data Source
AI summary
Aspects of providing an excess capacity grid for artificial intelligence, machine learning, and lower-priority processes are described. A grid orchestration client is installed on a virtual machine or a physical device that performs a production workload for an enterprise. The grid orchestration client communicates with a grid orchestration server as part of an excess capacity grid that performs grid workloads. A request to execute a grid workload is received. The grid orchestration client causes the grid workload to be executed.