NIC Bandwidth Prioritization for Hierarchical Collective Messages
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Collective algorithms in parallel or distributed computing are susceptible to bottlenecks due to delays in lower hierarchical levels, which impact overall performance and efficiency, especially in large-scale systems.
Innovation Solution
A network interface card (NIC) or data processing unit (DPU) allocates different bandwidths to packets based on their hierarchical levels, prioritizing delivery to lower levels to prevent blocking and ensure efficient communication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If hierarchical communication patterns are used in collective algorithms, then data dependencies are satisfied and correct computation is ensured, but bottlenecks occur at lower hierarchical levels that delay overall algorithm completion
Solution Approach 1:
The patent changes the bandwidth parameter dynamically based on hierarchical level. Lower hierarchical levels are allocated greater bandwidth while higher levels receive lesser bandwidth. This parameter change allows lower levels to complete their communication tasks faster, reducing the bottleneck effect while maintaining the hierarchical dependency structure needed for correct computation.
Solution Approach 2:
The patent applies different bandwidth allocation policies to different parts of the hierarchical structure. Specifically, lower hierarchical levels receive preferential bandwidth treatment compared to higher levels. This local differentiation in resource quality addresses the bottleneck at critical lower levels without affecting the overall hierarchical computation correctness.
2Device complexity
If equal bandwidth is allocated to all packets in hierarchical levels, then resource allocation is simple, but lower hierarchical levels experience delays that propagate through the hierarchy
Solution Approach 1:
The patent introduces differentiated bandwidth parameters based on packet hierarchical level. Instead of a single equal bandwidth parameter for all packets, the system uses multiple bandwidth parameters where lower-level packets receive higher bandwidth allocation. This resolves the time loss issue while adding manageable complexity to the allocation mechanism.
Solution Approach 2:
The patent performs preliminary bandwidth allocation based on packet hierarchical level before transmission begins. By pre-assigning greater bandwidth to lower-level packets, the system proactively prevents delays before they occur, rather than reacting to bottlenecks as they arise. This preliminary differentiation simplifies the overall control logic while effectively reducing transmission delays.
Data Source
AI summary
Embodiments herein relate to a NIC providing more bandwidth to deliver a packet that is part of a lower hierarchical level of a collective algorithm than a packet that is part of a higher hierarchical level of the collective algorithm, when both packets are ready for transmission. The NIC can allocate an appropriate amount of bandwidth to each packet that ensures the delivery of the packet associated with a respectively lower hierarchical level is prioritized over the packet associated with a respectively higher hierarchical level, which can resolve data dependencies and result in faster execution of the collective algorithm.