Distributed Traffic Controller for Horizontal Scaling Load Balance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Horizontally scaled processing systems face challenges in managing traffic distribution due to assumptions based on single-instance architectures, leading to uneven utilization of application instances and inefficient state synchronization, resulting in load imbalances and increased network bandwidth costs.
Innovation Solution
Implementing a distributed traffic controller at each application instance to classify and regulate traffic flows based on Service Level Agreements (SLAs), using efficient information propagation protocols to calculate global demand values and apply utilization limits, thereby preventing disproportionate utilization and optimizing resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If round-robin load distribution is used to distribute traffic to application instances, then traffic is distributed across all instances, but sticky connections accumulate at one or more instances causing uneven utilization
Solution Approach 1:
The patent implements a feedback mechanism where application instances report their current utilization state and sticky connection counts to a load balancer. The load balancer uses this feedback information to dynamically adjust traffic distribution decisions, avoiding instances that have accumulated excessive sticky connections and preventing further uneven utilization.
Solution Approach 2:
The load distribution strategy transitions from static round-robin to dynamic adaptation. The system continuously monitors utilization metrics and adjusts traffic routing in real-time based on current instance states, allowing the distribution pattern to evolve and adapt to changing conditions rather than following a fixed sequence.
2Device complexity
If centralized traffic control is implemented in a single instance, then traffic can be observed and controlled at a single point, but the system does not scale to horizontally scaled architectures where instances come and go
Solution Approach 1:
The centralized control function is segmented and distributed to individual application instances. Each instance runs its own traffic control logic and maintains local state information, eliminating the single point of control and enabling the system to scale horizontally as instances are added or removed without affecting overall control functionality.
Solution Approach 2:
The load balancer serves as an intermediary that coordinates between distributed application instances. It collects state information from instances, makes routing decisions based on this information, and directs traffic accordingly, enabling decentralized control while maintaining coherent system-wide traffic management.
3Reliability
If state synchronization is implemented among peer application instances, then traffic control parameters can be kept consistent, but network bandwidth consumption and message overhead increase
Solution Approach 1:
Each application instance maintains its own local state information about traffic control parameters and utilization metrics rather than continuously synchronizing with all other instances. This local quality approach reduces network communication overhead while still enabling coordinated traffic management through the load balancer's global view.
Solution Approach 2:
Instead of full state synchronization, instances share specific state information copies with the load balancer, which then uses this information for routing decisions. This selective copying of relevant state data minimizes network bandwidth consumption compared to complete state replication among all instances.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention comprises an apparatus and method for distributed traffic control in a horizontally scaled application, in which a software-based application is implemented as a number of peer application instances that each provides a portion of the application's overall capability or capacity. An apparatus that includes a distributed traffic controller is instantiated or otherwise implemented at each application instance, and these apparatuses collectively operate to limit the overall utilization of the application by individual clients or affiliated groups of clients according to, e.g., Service Level Agreements or SLAs, and further operate to prevent disproportionate utilization of any one of the application instances. Advantageously, such operations are accomplished according to the teachings herein using efficient information propagation protocols between the distributed traffic controllers.