Bandwidth-proportioned Datacenter Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-scale datacenters face bandwidth bottlenecks due to mismatched bandwidths between storage and computation devices and network interfaces, leading to inefficient data transmission and processing.
Innovation Solution
Provisioning storage and computation nodes with network interface components that match or are proportioned to the bandwidth of their respective node components, ensuring that data can be communicated at the full bandwidth of the node components, thereby eliminating bottlenecks and enhancing performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If network switches are arranged in a traditional hierarchy with core switches and top of rack switches, then network structure is simplified and easier to manage, but bandwidth bottlenecks occur and datacenter efficiency decreases
Solution Approach 1:
The patent segments the network interface bandwidth into separate virtual interfaces, each dedicated to specific node types (storage nodes, computation nodes). This segmentation allows independent bandwidth allocation and prevents bottlenecks while maintaining the hierarchical switch structure for management simplicity.
Solution Approach 2:
The patent implements dynamic bandwidth allocation where virtual network interfaces can be provisioned with different bandwidth capacities based on the specific needs of storage and computation nodes. This dynamic provisioning allows the system to adapt bandwidth distribution without changing the physical network hierarchy.
2Speed
If storage devices are provisioned with disks having greater collective bandwidth than network interface bandwidth, then storage capacity and speed are improved, but network bandwidth scarcity causes congestion and inefficiency
Solution Approach 1:
The patent applies local quality by provisioning each storage node with network interface bandwidth that matches its specific disk bandwidth requirements. Instead of uniform network provisioning, each node receives tailored bandwidth allocation, ensuring that fast storage devices are not bottlenecked by insufficient network interface capacity.
Solution Approach 2:
The patent changes the bandwidth parameter of network interfaces through virtualization, allowing the same physical network interface to present different bandwidth capacities to different nodes based on their storage and computation requirements. This parameter adjustment resolves the mismatch between disk bandwidth and network interface bandwidth.
3Speed
If computation devices are provisioned with input/output bus having greater bandwidth than collective network interface bandwidth, then computation processing speed is improved, but network bandwidth limitation causes congestion and inefficiency
Solution Approach 1:
The patent provisions each computation node with network interface bandwidth that matches its specific I/O bus bandwidth requirements. This localized bandwidth matching ensures that high-speed computation devices can fully utilize their processing capability without being constrained by insufficient network interface bandwidth.
Solution Approach 2:
The patent modifies the effective bandwidth parameter of network interfaces through virtualization to match the I/O bus bandwidth of computation nodes. This parameter change allows the network infrastructure to support high-speed computation without requiring uniform over-provisioning of all network interfaces.
4Productivity
If Map-Reduce model is used to resolve bandwidth bottlenecks, then datacenter efficiency is improved, but program complexity increases and requires fragmenting programs into smaller routines
Solution Approach 1:
The patent introduces virtual network interfaces as an intermediary layer between the physical network infrastructure and the storage/computation nodes. This intermediary enables bandwidth proportioning without requiring changes to the Map-Reduce programming model, thus maintaining program simplicity while achieving efficiency improvements.
Data Source
AI summary
A system including at least one storage node and at least one computation node connected by a switch is described herein. Each storage node has one or more storage units and one or more network interface components, the collective bandwidths of the storage units and the network interface components being proportioned to one another to enable communication to and from other nodes at the collective bandwidth of the storage units. Each computation node has logic configured to make requests of storage nodes, an input/output bus, and one or more network interface components, the bandwidth of the bus and the collective bandwidths of the network interface components being proportioned to one another to enable communication to and from other nodes at the bandwidth of the input/output bus.