Multi-tenant Workload Service Queue for Dynamic Resource Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cloud computing models require users to obtain and pay for dedicated resource capacity for a minimum period, leading to inefficiencies and increased costs due to excess capacity usage, especially when handling concurrent tasks, and involve significant downtime during server startup and shutdown processes.
Innovation Solution
A workload service that allows users to pre-load code and supporting files into a queue, enabling asynchronous job processing on a shared resource basis, with dynamic resource allocation and usage-based pricing, minimizing downtime and optimizing resource utilization by processing jobs only when capacity is available.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If dedicated server capacity is provisioned for a minimum period, then service availability is improved, but resource utilization deteriorates due to excess capacity usage
Solution Approach 1:
The patent implements dynamic resource allocation where server capacity is not statically assigned but dynamically provisioned based on actual job queue conditions. The system monitors job queues and activates server capacity only when jobs are present, allowing the resource allocation to adapt dynamically between active and idle states rather than remaining fixed throughout a minimum commitment period.
Solution Approach 2:
The patent creates a shared resource pool that serves multiple customers concurrently. A single pool of server capacity is universally allocated to handle jobs from multiple customers based on demand, rather than dedicating separate capacity to each customer. This multi-functional approach allows the same physical resources to serve different customers at different times, maximizing utilization.
2Productivity
If sufficient capacity is obtained to handle concurrent tasks, then task processing capability is improved, but cost increases due to paying for unused capacity
Solution Approach 1:
The system implements self-service resource allocation where customers submit jobs to a shared queue and the system automatically allocates capacity based on queue depth and availability. Customers do not need to pre-provision capacity or manage dedicated servers; the system serves them automatically when resources are available, eliminating the need to pay for standby capacity that would otherwise remain unused.
Solution Approach 2:
The patent merges multiple customers' job queues into a single shared queue and consolidates server capacity into a shared pool. This combining allows the system to efficiently match available capacity with actual job demand across all customers, ensuring that processing capability is sufficient for concurrent tasks while avoiding payment for excess capacity that would sit idle.
3Ease of operation
If code is reinstalled and server restarted for each customer, then dedicated access is ensured, but downtime increases
Solution Approach 1:
The patent pre-loads and caches application code in the shared server environment before jobs are submitted. When jobs arrive, the code is already prepared and available in the shared pool, eliminating the need for time-consuming installation and restart operations. This preliminary preparation of the execution environment allows rapid job deployment without service interruption.
Solution Approach 2:
The system introduces a shared code cache and job queue as intermediary layers between customer submissions and server execution. Instead of directly installing code on dedicated servers for each customer, the system uses the shared queue and cached code as intermediaries to facilitate rapid job deployment. This intermediary mechanism decouples code deployment from server restart operations, eliminating downtime.
Data Source
AI summary
Tasks to be performed for a user can be outsourced to a workload service operable to process those types of tasks. Information for the tasks, such as code to be executed and data to be processed, can be stored to a queue for the workload service. The workload service can pull the task when an appropriate resource is available and provide the result to the user. The user can be charged only for the approximate time of the processing, without need to obtain the resource for an extended period of time resulting in unused capacity. The user can take advantage of queuing, data, and code storage services whereby tasks can contain pointers to code, data, and other information needed for a task. The workload service can pre-load the supporting code or information needed to provide an environment in which code of the type for the user task can be executed.