QUIC Scalable Sockets for UDP Locking and Fragmentation Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current UDP processing systems face performance issues due to shared sockets and synchronization problems, leading to bottlenecks and poor scalability, as well as inefficiencies in handling small packet transmissions and receptions.
Innovation Solution
Implementing batched UDP packet processing and scalable sockets using QUIC, which involves creating separate sockets for each client connection, synchronizing send and receive buffers, and utilizing batch APIs to process multiple packets as a single unit, thereby avoiding IP fragmentation and improving throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If UDP applications post one small send at a time to avoid IP fragmentation, then IP fragmentation is avoided, but performance deteriorates due to executing the entire data path for each small packet
Solution Approach 1:
The patent combines multiple small UDP packets into a single batched send operation. Instead of posting one small send at a time, the application can post multiple packets in a single batch, which are then transmitted together. This merging approach maintains avoidance of IP fragmentation while improving performance by reducing the number of times the entire data path is executed.
Solution Approach 2:
The patent implements preliminary batching of UDP packets before transmission. Applications can prepare and batch multiple packets in advance, then post them as a single unit to the network stack. This preliminary action allows the system to optimize the data path execution by processing multiple packets together rather than individually, thus improving throughput without causing fragmentation.
2Reliability
If UDP is implemented as a message-oriented protocol with single socket per server, then message boundaries are preserved, but scale bottlenecks occur due to locking and synchronization issues
Solution Approach 1:
The patent segments the single UDP socket into multiple sockets, with each socket handling a specific connection or flow. This segmentation eliminates the need for locking and synchronization when multiple connections are processed simultaneously, as each socket operates independently. Message boundaries are still preserved within each socket's context, maintaining reliability while improving scalability.
Solution Approach 2:
The patent implements per-connection sockets rather than a single shared socket. This excessive action of creating more sockets than the minimum one goes against the traditional message-oriented UDP approach, but it eliminates synchronization bottlenecks by providing dedicated buffers and processing paths for each connection, thus improving server scalability without sacrificing message integrity.
3Device complexity
If all incoming connection requests share the same socket, then socket resource usage is minimized, but performance issues occur due to fate sharing where one connection can stall others
Solution Approach 1:
The patent divides the single shared socket into multiple independent sockets, each dedicated to a specific connection or flow. This segmentation ensures that one connection cannot stall others, as each has its own buffer and processing path. While this increases the number of socket objects, it dramatically improves throughput by eliminating fate sharing and synchronization overhead.
Solution Approach 2:
The patent assigns different properties to different sockets based on their specific connection requirements. Each socket has its own send and receive buffers tailored to the characteristics of its associated connection. This local quality approach allows optimal performance for each connection while maintaining overall system efficiency, resolving the trade-off between resource usage and throughput.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system having scalable sockets to support User Datagram Protocol (UDP) connections identifies a plurality of UDP connections, wherein a plurality of remote clients connect to corresponding ones of the plurality of UDP connections. Each one of a plurality of UDP sockets is associated with a corresponding one of the plurality of UDP connections. A network stack lookup for UDP packets in network traffic is performed using a network stack to identify the UDP socket corresponding to the remote client associated with each of the UDP packet. The UDP packets are buffered with a send buffer and a receive buffer for the UDP socket corresponding to the remote client associated with the UDP packets as determined by the network stack lookup to support communication over the plurality of UDP connections using the plurality of UDP sockets. The system thereby operates more efficiently and/or is more scalable.