Task-Centric Scheduler for Cross-Framework Resource Negotiation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing parallel computing frameworks face challenges in executing programs written for different frameworks, particularly when porting software between platforms, as they often require significant efforts and are not compatible with non-native environments.
Innovation Solution
A distributed parallel computing system with a master node and worker nodes, utilizing a scheduler that acts as an intermediary to negotiate resource allocation with a resource manager, allowing foreign programs to execute by allocating computing resources as containers, and employing policy engines to optimize resource allocation based on pre-determined constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a parallel computing system executes foreign programs written for different frameworks, then the system's versatility and adaptability improve, but the complexity of resource allocation and framework compatibility increases
Solution Approach 1:
The patent introduces a scheduler as an intermediary component between foreign programs and the parallel computing system's resource manager. The scheduler translates resource requests from foreign program frameworks into the native framework's resource allocation protocols, enabling compatibility without direct integration. This mediator handles framework-specific protocols and interfaces, allowing the system to execute programs from multiple frameworks while maintaining a unified resource management architecture.
2Manufacturing precision
If a scheduler negotiates resource allocation through multiple iterations with a resource manager, then resource allocation precision improves, but the time required for resource allocation increases
Solution Approach 1:
The patent implements preliminary action by having the scheduler perform initial resource allocation based on program requirements before execution begins. The scheduler pre-negotiates resource allocation with the resource manager, establishing baseline resource reservations. During execution, the policy engine performs iterative negotiations to adjust allocations, but the preliminary allocation reduces the total time required compared to starting from scratch.
Solution Approach 2:
The patent employs feedback mechanisms where the policy engine continuously monitors resource usage and performance metrics during program execution. Based on this feedback, the engine iteratively adjusts resource allocations by negotiating with the resource manager, balancing precision requirements with time constraints through adaptive reallocation rather than exhaustive pre-negotiation.
3Productivity
If the policy engine balances multiple analyzer suggestions and policies to generate resource requests, then resource allocation optimization improves, but the computational overhead increases
Solution Approach 1:
The patent applies partial action by having the policy engine selectively process analyzer suggestions rather than exhaustively evaluating all possible combinations. The engine prioritizes critical resource allocation decisions based on program requirements and policy constraints, making sufficient but not necessarily optimal decisions in all cases. This approach achieves adequate resource utilization efficiency while limiting computational overhead to manageable levels.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for a task centric resource scheduling framework. A scheduler executing on a master node computer of a distributed parallel computing system allocates computing resources of the parallel computing system to a program according to one or more policies associated with the program. Each policy includes a set of pre-determined computing resource constraints. Allocation of the computing resources includes performing multiple iterations of negotiation between the scheduler and a resource manager of the parallel computing system. In each iteration, a policy engine of the scheduler submits requests to get more resources from, or requests to release already acquired resources to, the resource manager. The policy engine generates the requests by balancing suggestions provided by analyzer components of the policy engine and a corresponding policy. The policy engine can then determine an allocation plan on how to allocate resources.