Cloud ML Task Execution With Dynamic Resource Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning model (MLM) technologies face inefficiencies due to fixed-sized cluster constraints, leading to suboptimal resource utilization and latency issues when task complexity overwhelms or underutilizes subscribed cloud computing resources.
Innovation Solution
An application programming interface (API) facilitates cloud-based development and deployment of MLMs, utilizing a workflow engine to allocate flexible and on-demand hardware resources, enabling scalable and efficient MLM operations through containerization and priority queuing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If fixed-sized cluster constraints are used for cloud computing resources, then resource allocation is simplified and management is easier, but resource utilization becomes suboptimal and latency increases when task complexity varies
Solution Approach 1:
The patent implements dynamic resource allocation by allowing the cluster size to change based on task requirements. The system can scale the number of computing nodes up or down dynamically, transforming the static fixed-sized cluster into a dynamic structure that adapts to varying task complexities and workload demands, thereby optimizing resource utilization efficiency.
Solution Approach 2:
The system changes the parameter of cluster size from a fixed value to a variable that can be adjusted based on task requirements. By modifying this key parameter dynamically, the system resolves the contradiction between operational simplicity and productivity, allowing easy management through automated scaling while maintaining high resource utilization.
2Device complexity
If fixed-sized cluster constraints are used, then infrastructure cost is reduced and system complexity is lowered, but latency increases when task complexity overwhelms subscribed resources
Solution Approach 1:
The system introduces dynamic scaling capabilities that allow the cluster to adapt its size in real-time based on task complexity. When tasks overwhelm subscribed resources, the system automatically provisions additional computing nodes, reducing latency without permanently increasing system complexity through automated resource management.
Solution Approach 2:
The patent creates a universal resource pool that can handle varying task complexities by providing multi-functionality. The same infrastructure can serve both simple and complex tasks by dynamically allocating resources, eliminating the need for separate fixed clusters for different task types and reducing overall system complexity.
3Ease of operation
If fixed-sized cluster constraints are used, then subscription management is simplified, but resource utilization becomes suboptimal leading to idle time when task load is light
Solution Approach 1:
The system implements dynamic resource provisioning that scales the cluster size down when task load is light, eliminating idle computing resources. This maintains subscription management simplicity through automated policies while preventing energy waste from underutilized fixed-sized clusters.
Solution Approach 2:
The patent enables self-service resource management where the system automatically adjusts cluster size based on workload demands without requiring manual intervention. This maintains operational simplicity while optimizing resource utilization by provisioning or de-provisioning resources as needed, eliminating both over-provisioning and under-provisioning issues.
Data Source
AI summary
Disclosed are devices, systems, and techniques for provisioning of scalable machine learning operations on a cloud-based server. The techniques include receiving from a client device, via a cloud service API, authorization data and receiving from the client device, via the cloud service API, a selection of a task to be executed in association with a machine learning model. The techniques further include allocating, from a shared pool of cloud computing resources, one or more processors to execute the task, wherein the shared pool of cloud computing resources is being concurrently used for execution of a plurality of additional tasks received from one or more additional client devices. The techniques further include instantiating an execution container comprising one or more compute backends, receiving, using the authorization data, the user data into the execution container, and executing, using the one or more processors, the task in the execution container.