In-Network Compute Resource Allocation via Dynamic Topology Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for allocating in-network compute-resources in parallel computing jobs are limited by precomputed trees, which may not accurately match the specific needs of each job, leading to denied requests due to insufficient resource availability and network mismatches.
Innovation Solution
A system that dynamically selects in-network compute-resources based on network topology, resource usage state, and job requirements without relying on pre-generated trees, allowing for ad-hoc allocation tailored to each application's needs and managing resource usage directly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If precomputed trees are used for resource allocation, then resource allocation speed is improved, but resource matching accuracy deteriorates
Solution Approach 1:
The system dynamically selects in-network compute-resources based on current network topology, resource usage state, and job requirements rather than using static precomputed trees. This dynamic approach allows the resource allocation to adapt to changing conditions and match specific job needs accurately while maintaining efficient allocation through real-time decision making.
2Device complexity
If pre-generated trees are used for resource allocation, then allocation complexity is reduced, but adaptability to specific job needs deteriorates
Solution Approach 1:
The resource allocation process is segmented into distinct selection criteria: network topology considerations, resource usage state assessment, and job requirement matching. This segmentation allows the system to evaluate multiple factors independently and combine them for optimal resource selection, achieving both manageable complexity and high adaptability to specific job needs.
Solution Approach 2:
The system employs dynamic selection of in-network compute-resources based on current network topology, resource usage state, and specific job requirements. This dynamic approach enables the allocation mechanism to adapt to varying job needs and changing network conditions without relying on fixed pre-generated trees, thereby maintaining both operational simplicity and high versatility.
3Manufacturing precision
If ad-hoc allocation is implemented, then resource matching accuracy is improved, but allocation complexity increases
Solution Approach 1:
The ad-hoc allocation process is divided into structured segments: evaluating network topology, assessing resource usage state, and matching job requirements. This segmentation transforms the complex ad-hoc allocation into a systematic process with clear evaluation criteria, improving resource matching accuracy while keeping the allocation logic organized and manageable.
Solution Approach 2:
The system performs preliminary assessments of network topology and resource usage state before making allocation decisions. By preparing this foundational information in advance, the system simplifies the subsequent resource matching process, achieving high accuracy without proportionally increasing overall allocation complexity.
Data Source
AI summary
In one embodiment, an in-network compute resource assignment system includes a network device to receive a request to select resources to perform a processing job, wherein the request includes at least one resource requirement of the processing job, and end point devices assigned to perform the processing job, a memory to store a state of in-network compute-resources indicating resource usage of the in-network compute-resources by other processing jobs, and a processor to manage the stored state, and responsively to receiving the request, selecting ones of the in-network compute-resources to perform the processing job based on: (a) a network topology of a network including the in-network compute-resources; (b) the state of the in-network compute-resources; and (c) the at least one resource requirement of the processing job.