Service Mesh Data Plane Offload for Low-Latency HTTP/3 Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Service meshes in cloud-native computing suffer from performance and latency issues due to slow L7 software proxies and multiple network stack traversals, which hinder developers with high performance needs from utilizing their benefits.
Innovation Solution
The solution involves separating the control plane and data plane, offloading data processing to network interface controllers (NICs) via PCIe switches, and maintaining decision-making processes in the CPU, thereby bypassing CPU cache during data streaming.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If L7 software proxies are used for service mesh traffic handling, then service-to-service communication control is improved, but processing speed deteriorates
Solution Approach 1:
The patent segments the service mesh functionality into control plane (CPU-based decision making) and data plane (NIC-based data handling). The L7 proxies are separated from the data processing path, allowing traffic to be handled by specialized hardware NICs that perform packet processing without involving the full L7 software stack, thus resolving the contradiction between control capability and processing speed.
Solution Approach 2:
The patent introduces network interface controllers (NICs) as intermediary hardware components between the CPU and the network data path. These NICs act as mediators that handle packet processing, routing, and data transmission directly, bypassing the need for L7 software proxies to process every packet through the full application layer stack, thereby improving speed while maintaining control through the control plane.
2Adaptability or versatility
If multiple L7 proxies and gateways are deployed, then service mesh functionality is improved, but communication latency increases
Solution Approach 1:
The patent extracts the data processing functions from the L7 proxy layer and relocates them to dedicated NIC hardware. By taking out the packet processing workload from the software proxy chain, the system eliminates multiple latency-inducing hops through L7 proxies while maintaining service mesh functionality through the control plane's routing decisions.
Solution Approach 2:
The patent replaces the mechanical/software-based L7 proxy processing with hardware-based NIC processing. Instead of using software processes to handle packet routing and data transmission, the system uses dedicated network interface controllers that perform these functions at hardware speed, dramatically reducing the time spent on packet processing and inter-proxy communication.
3Ease of operation
If TCP/IP network stack traversals are increased, then network routing control is improved, but processing overhead increases
Solution Approach 1:
The patent performs preliminary routing decisions in the control plane before data processing begins. The control plane establishes routing rules and network paths in advance, allowing the data plane NICs to execute pre-computed routing instructions without needing to traverse the full TCP/IP stack multiple times. This preliminary action reduces the complexity of repeated stack traversals while maintaining routing control flexibility.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of managing data streaming processes may include at a processing device, computing hypertext transmission protocol version 3 (HTTP/3) header of a first message received at a first network interface controller (NIC) into a first control message and a second control message, and transmitting the first control message to the first NIC associated with the processing device. The method may further include transmitting the second control message to a second NIC associated with the processing device, and with the first control message and the second control message, transmitting data directly between the first NIC and the second NIC.