Transmit Queue Selection via Reverse Hash Indirection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtualization software faces challenges in determining the appropriate hardware transmit queue for outbound data messages when multiple hardware receive queues are used for load balancing, as the information to select the corresponding transmit queue is not available due to inbound messages being processed by multiple different receive queues.
Innovation Solution
The method involves identifying the paired receive queue for an inbound data message corresponding to an outbound data message by calculating a hash from packet header values and using an indirection table or flow cache to determine the transmit queue, ensuring that both inbound and outbound data messages of a flow are processed by the same CPU, thereby enhancing throughput and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple hardware receive queues are used for load balancing to increase data throughput, then throughput is improved, but the ability to determine which transmit queue to select for outbound messages is lost
Solution Approach 1:
The system pre-calculates and stores the reverse hash values for all possible inbound flows in an indirection table before outbound message processing occurs. This preliminary action enables the virtualization software to quickly determine the correct paired transmit queue for outbound messages without losing queue selection information, even when multiple receive queues are used for load balancing.
Solution Approach 2:
The indirection table acts as an intermediary data structure that stores the mapping between hash values and receive queue identifiers. By introducing this intermediary, the system preserves the relationship between inbound and outbound queue pairs even when multiple receive queues are used, allowing the virtualization software to correctly select transmit queues for outbound messages.
2Productivity
If multiple hardware receive queues are used to process inbound messages for a DCN, then processing capacity is improved, but the pairing information needed for efficient outbound processing is lost
Solution Approach 1:
The system pre-calculates reverse hash values for all possible symmetric inbound flows and stores them in the indirection table before outbound message processing. This preliminary calculation ensures that when outbound messages need to be processed, the correct paired receive queue (and thus the correct transmit queue) can be reliably determined even though multiple receive queues are actively processing inbound messages.
Solution Approach 2:
The system uses hash functions to create a deterministic mapping between outbound messages and their corresponding inbound message queues. By calculating the reverse hash of outbound message parameters and looking up the result in the indirection table, the system receives feedback about which receive queue originally processed the corresponding inbound message, thereby maintaining reliable queue pairing information.
3Reliability
If a single receive queue is used for a DCN, then queue pairing information is maintained, but throughput requirements cannot be met
Solution Approach 1:
The system pre-calculates and stores reverse hash values in the indirection table, enabling the virtualization software to determine the correct paired transmit queue for outbound messages even when multiple receive queues are used. This preliminary action allows the system to maintain queue pairing information while using multiple receive queues to meet high throughput requirements.
Solution Approach 2:
The indirection table serves as an intermediary that preserves the relationship between hash values and receive queue identifiers. This intermediary structure enables the system to use multiple receive queues for high throughput while still maintaining the ability to correctly pair transmit and receive queues for efficient processing, thus resolving the contradiction between throughput and queue pairing information.
Data Source
AI summary
Some embodiments provide a method for selecting a transmit queue of a network interface card (NIC) of a host computer for an outbound data message. The NIC includes multiple transmit queues and multiple receive queues. Each of the transmit queues is individually associated with a different receive queue, and the MC performs a load balancing operation to distribute inbound data messages among multiple receive queues. The method extracts a set of header values from a header of the outbound data message. The method uses the extracted set of header values to identify a receive queue which the MC would select for a corresponding inbound data message upon which the NIC performed the load balancing operation. The method selects a transmit queue associated with the identified receive queue to process the outbound data message.