Gateway Event-Loop Connection Pooling to Reduce Network Churn
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for reducing network connection churn in gateway services lead to high connection counts and difficulty in balancing network traffic, especially with increasing use of streaming applications and mTLS authentication, resulting in inefficient resource allocation and scalability issues.
Innovation Solution
Implementing a network gateway service that dynamically subsets and balances load by creating a ring of nodes, assigning service requests to event loops, and using connection pooling to reduce connections through multiplexing and load balancing, ensuring optimal replication and subset sizes based on environmental factors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional connection pooling is implemented for each event loop to reduce contention, then event loop performance is improved, but the number of network connections increases exponentially
Solution Approach 1:
The patent segments the ring of nodes into multiple node subsets, where each event loop is assigned to a specific node subset. This segmentation allows event loops to connect only to their assigned subset rather than all nodes, reducing the total number of connections while maintaining performance. The ring topology is divided into manageable segments that can be independently managed.
Solution Approach 2:
Different event loops are assigned to different node subsets based on local conditions and requirements. Each event loop has its own dedicated connection pool to its assigned node subset, creating localized optimization. This local quality approach ensures that each event loop has direct access to its required resources without competing for connections with other event loops.
2Adaptability or versatility
If the number of event loops, servers, and gateway instances increases to handle more traffic, then system capacity is improved, but connection churn increases
Solution Approach 1:
The patent implements dynamic scaling of node subsets based on system conditions. When servers are added or removed from the ring, the node subsets are dynamically reconfigured and reassigned to event loops. This dynamic adaptation allows the system to scale capacity while maintaining stable connections, as event loops maintain their assignments to subsets rather than requiring reconnection to individual nodes.
Solution Approach 2:
Event loops are pre-assigned to specific node subsets before runtime operations. This preliminary assignment establishes stable connections in advance, so that when traffic increases or system configuration changes, the connection structure remains stable. The subset assignments are determined beforehand based on expected load patterns and system configuration.
3Productivity
If more connections are created to handle increased network traffic, then traffic handling capability is improved, but load balancing complexity increases
Solution Approach 1:
The load balancing complexity is segmented and distributed across multiple independent event loops, each managing its own connection pool to a specific node subset. Rather than one complex centralized load balancer managing all connections, the system divides the load balancing function across many simpler event loop instances, each handling only its assigned subset. This segmentation reduces the complexity of individual load balancing decisions while maintaining overall system capability.
Data Source
AI summary
The disclosed computer-implemented method may include receiving, by a network gateway service, a service request from a client device. The method may also include assigning, by the network gateway service, the service request to an event loop. Additionally, the method may include determining, by the network gateway service, a destination of the event loop based on one or more resources of one or more nodes in a ring of nodes. Furthermore, the method may include creating, by the network gateway service, a connection pool for the event loop to select a node subset of the ring of nodes based on a mapping of event loops to node subsets. Finally, the method may include performing, by the network gateway service, load balancing of the connection pool to execute the service request. Various other methods, systems, and computer-readable media are also disclosed.