Multi-Core Intermediary Packet Routing for Datacenter Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional load balancing methods in datacenters with limited public IP addresses result in high latency and bandwidth consumption, degrading overall performance due to inefficient routing of network packets between client devices and backend servers.
Innovation Solution
Implementing a system with two sets of multi-core intermediary devices, where a client-side and server-side multi-core intermediary device pair use a flow distributor, hash calculator, core selector, key retriever, and packet modifier to identify and assign the correct cores for processing network packets based on RSS hash configuration, reducing intermediary-to-intermediary and core-to-core communications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional load balancing methods are used to route network packets between client devices and backend servers with limited public IP addresses, then bandwidth utilization is improved, but latency increases and overall performance degrades
Solution Approach 1:
The system segments the load balancing function into two distinct sets of intermediaries: client-side intermediaries that receive packets from clients and server-side intermediaries that forward packets to servers. This segmentation allows each set to specialize in specific routing tasks, reducing the complexity of packet routing decisions and decreasing latency while maintaining efficient bandwidth utilization through coordinated operation between the two sets.
Solution Approach 2:
The patent introduces multi-core intermediaries as mediator devices between clients and servers, where each intermediary contains multiple cores that can independently process packets. These intermediaries act as intelligent mediators that maintain flow state information and make optimized routing decisions, reducing the need for complex coordination and thereby decreasing latency while efficiently utilizing available bandwidth.
2Loss of time
If multi-core intermediaries are deployed to improve packet routing efficiency, then latency decreases, but device complexity increases
Solution Approach 1:
The system divides the intermediary functionality into two separate sets (client-side and server-side), each with simplified responsibilities. Client-side intermediaries focus on receiving packets from clients and forwarding to appropriate server-side intermediaries, while server-side intermediaries focus on delivering packets to backend servers. This segmentation reduces the complexity of individual devices while maintaining low latency through coordinated operation.
Solution Approach 2:
Each intermediary device is designed with multiple cores that can handle multiple packet flows simultaneously, providing multi-functionality within a single device type. This universal design allows the same hardware platform to be used across both client-side and server-side deployments, reducing overall system complexity while maintaining high performance through parallel processing.
3Measurement precision
If flow state information is maintained across multiple intermediaries to enable accurate packet routing, then routing precision improves, but memory resources are consumed
Solution Approach 1:
The flow state information is segmented and distributed across the two sets of intermediaries rather than being集中ized in a single location. Client-side intermediaries maintain flow state for incoming client connections, while server-side intermediaries maintain flow state for outgoing server connections. This segmentation reduces the memory burden on any single intermediary while maintaining routing precision through coordinated state management between the two sets.
Data Source
AI summary
The present disclosure is directed towards systems and methods routing network packets between multi-core intermediaries. A processor of a plurality of processors on a client-side intermediary device may receive a packet from a client device. The processor may be identified by a core identifier. The processor may calculate a first set of source port addresses based on a first key and the core identifier. The processor may identify a target server-side intermediary device and a target processor based on data received with the packet or metadata received from the target server-side intermediary device. The processor may calculate a second set of port addresses based on a second key and the target core identifier. The processor may identify a port address common to both the first set and second set of port addresses. The processor may replace the original source port address in the packet with the identified port address.