Hierarchical Token Bucket for Virtual I/O Bandwidth Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current server infrastructure struggles to efficiently manage varying workload demands, leading to resource over-commitment and increased costs due to the need for physical scaling, which fails to address demand spikes effectively, resulting in suboptimal performance and potential service failures.
Innovation Solution
Implementing a hierarchical token bucket mechanism to control traffic injection rates in a distributed system with virtualized I/O subsystems over a switch fabric network, allowing for dynamic bandwidth allocation and quality of service management across the network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If infrastructure is built to peak loads, then service availability is improved, but resource over-commitment and costs increase
Solution Approach 1:
The patent implements dynamic resource allocation through virtualization, allowing computing, storage, and network resources to be dynamically assigned and reassigned based on real-time demand. Virtual machine migration and resource pooling enable the system to adapt to workload fluctuations without requiring peak-capacity infrastructure for all users simultaneously, thus maintaining service availability while reducing overall resource commitment.
Solution Approach 2:
The virtualized infrastructure creates universal resource pools that can serve multiple functions and users simultaneously. A single physical server can host multiple virtual machines serving different applications and users, and resources can be dynamically allocated across different workloads based on demand, eliminating the need for dedicated peak-capacity infrastructure for each individual user or application.
2Quantity of substance
If infrastructure is built to average loads, then resource costs are reduced, but productivity decreases during demand spikes
Solution Approach 1:
The system dynamically scales resources during demand spikes by activating additional virtual machine instances and allocating more computing, storage, and network resources on-demand. Load balancing and automatic resource provisioning ensure that productivity is maintained during peak periods without requiring permanent peak-capacity infrastructure, thus avoiding the costs associated with always-building-for-peak.
Solution Approach 2:
The virtualized platform provides self-service capabilities where applications and users can automatically request and receive additional resources during demand spikes without manual intervention. The system autonomously provisions resources, manages load distribution, and scales capacity based on real-time monitoring of workload conditions, maintaining productivity during peaks while avoiding unnecessary permanent resource commitment.
3Quantity of substance
If physical hardware scaling is used, then capacity increases, but time consumption and costs increase
Solution Approach 1:
The patent uses virtual machine replication and cloning capabilities to rapidly create additional computing instances from standardized templates. Instead of physically provisioning and configuring new hardware, the system creates virtual copies of proven configurations, enabling capacity expansion to occur in minutes rather than days, thus dramatically reducing scaling time while increasing system capacity.
Solution Approach 2:
The system replaces physical hardware provisioning and configuration with virtualized software-based resource creation and allocation. Virtual machine instantiation, resource allocation, and system configuration are performed through software operations rather than physical assembly and setup, enabling near-instantaneous capacity expansion without the time-consuming mechanical processes of physical hardware deployment.
4Productivity
If network traffic is unrestricted, then application performance is improved, but quality of service deteriorates due to traffic bursts
Solution Approach 1:
The system implements preliminary traffic shaping and rate limiting mechanisms that proactively manage network traffic before bursts can degrade service quality. By pre-configuring bandwidth allocation, priority queuing, and traffic policing rules, the system ensures that critical applications receive guaranteed bandwidth while limiting the impact of traffic bursts, thus maintaining both application performance and overall quality of service during high-demand periods.
Data Source
AI summary
Processes for the control of traffic and Quality of Service (QoS) over a switch fabric network comprised of application servers and virtual I/O servers. In the embodiment, an application server includes virtual device interfaces, a QoS module, and a network port controlled by a packet scheduler. When the QoS module receives a packet from a virtual device interface, the QoS module stores the packet in a queue. The QoS module removes the packet from the queue and transmits it to the packet scheduler, in accordance with a hierarchical token bucket that allocates bandwidth for the port among the virtual device interfaces in the application server. In the embodiment, the port is the root of the hierarchy for the hierarchical token bucket and the virtual device interfaces are the leaves. The packet scheduler uses round-round arbitration to transmit the packet it receives to the port.