Network card performance tuning method in cloud network environment

Abstract

The invention discloses a network card performance tuning method in a cloud network environment, and relates to the technical field of cloud network management. Comprising the steps of 1, setting IRQ affinity, 2, adjusting interruption regulation and control of a network card, 3, setting the size of an annular buffer area of the network card, 4, adjusting rx hash fields of RSS, setting the hash fields of the RSS, selecting queues of the network card according to hash of a source IP and a destination IP, dispersing incoming network data packets into different receiving queues, modifying RSS hash values of a TCP and a UDP, and sending the modified RSS hash values to the network card. And step 5, using iperf3 to carry out network testing on each item of setting, and evaluating and further optimizing each item of setting according to a test result.

Description

Technical Field

[0001] The present invention discloses a method for optimizing network card performance in a cloud network environment, and relates to the technical field of cloud network management. Background Art

[0002] With the rapid development of cloud computing technology, its applications in cloud network environments are becoming more and more widespread. In a cloud network environment, servers need to process a large number of network requests, so the network performance of the server becomes one of the key factors affecting the performance of the entire system. In order to improve the network performance of the server, researchers have proposed many methods, such as network protocol optimization, network topology optimization, load balancing, etc. However, these methods often require major changes to the entire network and are difficult to implement.

[0003] The network card is an important component for the server to exchange data with the external network, and its performance directly affects the network performance of the server. In recent years, with the continuous development of network card technology, the speed of network cards has become faster and faster, but it has also brought new challenges. Existing network card configurations usually rely on the default settings of the operating system, which are designed to meet general purposes rather than optimized for specific applications or workloads. Therefore, in a cloud network environment, these default settings may not fully utilize the performance of the network card, resulting in increased network latency and inefficient data transmission. In addition, in the case of high network traffic, the existing network card processing method may cause data packet loss or untimely processing, further affecting network performance. The network protocol stack of the operating system is designed for a general network environment and cannot be optimized for a specific cloud network environment. Summary of the invention

[0004] The present invention aims to solve the problems of the prior art and provide a method for tuning the performance of a network card in a cloud network environment, thereby solving the deficiencies of the prior art in processing high network traffic, reducing latency and achieving load balancing. Changing the network card configuration by performance tuning can significantly improve the performance of the cloud network environment.

[0005] The specific scheme proposed by the present invention is:

[0006] The present invention provides a method for optimizing network card performance in a cloud network environment, comprising:

[0007] Step 1: Set IRQ affinity: Bind interrupt requests from different network card queues to different CPU cores and specify which CPU cores handle specific interrupt requests.

[0008] Step 2: Adjust the interrupt control of the network card: According to the interrupt merging or interrupt delay function supported by the network card, set the number of interrupts per second according to the usage scenario to balance power consumption and performance.

[0009] Step 3: Set the ring buffer size of the network card: Set at least two ring buffers for each network interface of the network card, namely the receive buffer and the send buffer. Use the command ethtool-SethX to observe the value of rx_dropped or estimate the cache pressure when activating multiple queues to set the ring buffer size.

[0010] Step 4: Adjust the rx hash field of RSS: Set the hash field of RSS, select the queue of the network card according to the hash of the source IP and destination IP, and distribute the incoming network data packets to different receiving queues.

[0011] And modify the RSS hash value of both TCP and UDP to disperse the network communication data between virtual machines.

[0012] Step 5: Use iperf3 to perform network tests on each setting, and evaluate and further optimize each setting based on the test results.

[0013] Furthermore, when setting the IRQ affinity in step 1 of the method for tuning the performance of a network card in a cloud network environment, the method includes: first stopping the iqrbalance service, then using the set_irq_affinity script or manually searching for the processor core bound to each node, searching for the IQR currently assigned to the network card, and writing the SMP affinity value into the corresponding IRQ entry, and then modifying the IQR according to the actual use effect.

[0014] Furthermore, in step 2 of the method for tuning the performance of a network card in a cloud network environment, an interrupt control command is used to adjust the interrupt control of the network card to be turned on or off, and a fixed interrupt frequency of the interrupt control is set through the interrupt control command.

[0015] Furthermore, in step 3 of the method for optimizing the performance of a network card in a cloud network environment, the size of the receiving buffer is set to store data packets received from the network. When a data packet arrives, the data packet is placed in the receiving buffer through the network card, and an interrupt request is triggered to notify the CPU that a new data packet needs to be processed;

[0016] Set the send buffer size to store the data packets to be sent through the network interface. When a data packet needs to be sent, the CPU puts the data packet into the send buffer and notifies the network card. The network card takes the data packet out of the send buffer and sends it through the network.

[0017] Furthermore, in step 5 of the method for optimizing network card performance in a cloud network environment, a physical machine is deployed and the performance mode is turned on for testing, and both TCP and UDP are tested during the test. In the cloud network environment, multiple virtual machines in the same network but different physical nodes are created to perform flow testing.

[0018] Use multi-process iperf3 to test the network limit when performing flow testing, or use the iperf3 command to perform network testing by simulating the situation where multiple nodes and multiple network card queues are interacting with the network at the same time.

[0019] The present invention also provides a device for tuning network card performance in a cloud network environment, including a setting adjustment module and a test tuning module.

[0020] Set the IRQ affinity of the adjustment module: bind the interrupt requests of different network card queues to different CPU cores, and specify which CPU cores handle specific interrupt requests.

[0021] Adjust the interrupt control of the network card: According to the interrupt merging or interrupt delay function supported by the network card, set the number of interrupts per second according to the usage scenario to balance power consumption and performance.

[0022] Set the ring buffer size of the network card: Set at least two ring buffers for each network interface of the network card, namely the receive buffer and the send buffer. Use the command ethtool -S ethX to observe the value of rx_dropped or estimate the cache pressure when activating multiple queues to set the ring buffer size.

[0023] Adjust the rx hash field of RSS: Set the hash field of RSS to select the queue of the network card according to the hash of the source IP and destination IP, and distribute the incoming network data packets to different receiving queues.

[0024] And modify the RSS hash value of both TCP and UDP to disperse the network communication data between virtual machines;

[0025] The test and tuning module uses iperf3 to perform network tests on each setting, and further optimizes each setting based on the test results.

[0026] Furthermore, when the setting adjustment module of the network card performance tuning device in a cloud network environment sets the IRQ affinity, it includes: first stopping the iqrbalance service, then using the set_irq_affinity script or manually searching for the processor core bound to each node, searching for the IQR currently assigned to the network card, and writing the SMP affinity value into the corresponding IRQ entry, and then modifying the IQR according to the actual use effect.

[0027] The setting regulation module uses the interrupt regulation command to adjust the interrupt regulation of the network card to be turned on or off, and also sets the fixed interrupt frequency of the interrupt regulation through the interrupt regulation command.

[0028] Furthermore, the setting adjustment module of the device for optimizing the performance of a network card in a cloud network environment sets the size of a receiving buffer for storing data packets received from the network. When a data packet arrives, the data packet is placed in the receiving buffer through the network card, and an interrupt request is triggered to notify the CPU that a new data packet needs to be processed;

[0029] Set the send buffer size to store the data packets to be sent through the network interface. When a data packet needs to be sent, the CPU puts the data packet into the send buffer and notifies the network card. The network card takes the data packet out of the send buffer and sends it through the network.

[0030] Furthermore, the test and tuning module of the network card performance tuning device in a cloud network environment deploys a physical machine and turns on the performance mode for testing, and tests both TCP and UDP during the test. In the cloud network environment, multiple virtual machines in the same network but different physical nodes are created to perform flow testing.

[0031] Use multi-process iperf3 to test the network limit when performing flow testing, or use the iperf3 command to perform network testing by simulating the situation where multiple nodes and multiple network card queues are interacting with the network at the same time.

[0032] The benefits of the present invention are:

[0033] Improve network performance in cloud network environments, and enable each CPU of the host machine to participate in network processing based on interrupt load balancing and queue load balancing. Avoid the situation where a single CPU cannot handle large-scale network traffic;

[0034] Set the frequency of network interruption to find a balance between low latency and low power consumption;

[0035] Set an appropriate ringbuffer value to avoid packet loss due to traffic processing failure. Changing the network card configuration through performance tuning can significantly improve the performance of the cloud network environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic flow chart of the method of the present invention. DETAILED DESCRIPTION

[0037] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0038] Example 1

[0039] The present invention provides a method for optimizing network card performance in a cloud network environment, comprising:

[0040] Step 1: Set IRQ affinity: Bind interrupt requests from different network card queues to different CPU cores and specify which CPU cores handle specific interrupt requests.

[0041] IRQ affinity is a concept involved in computer operating systems. In a multi-processor system, it refers to binding one or more interrupt requests (IRQs) to a specific CPU core to optimize performance and resource utilization. In a network interface card, by setting IRQ affinity, you can specify which CPU cores handle specific interrupt requests and bind interrupts from different network card queues to different CPU cores.

[0042] Furthermore, to set IRQ affinity, you must first stop the iqrbalance service, then use the set_irq_affinity script or enable it manually.

[0043] set_irq_affinity is available in the source code packages of many network cards, such as / scripts / set_irq_affinity in the X710 network card package; you can also use manual commands to set it: 1. numactl-hardware finds the processor core bound to each node; 2. Find the IQR currently assigned to the network card:

[0044] lspci|grep-i ethernetls-l / sys / bus / pci / devices / PCI_ADDRESS / irq; 3. Write the SMP affinity value to the corresponding IRQ entry. You need to write the corresponding affinity value for each IRQ entry: echo 1> / proc / irq / irq_num / smp_affinity; echo 2> / proc / irq / irq_num / smp_affinity. When setting the affinity, you need to modify it according to the actual use effect; you can specify all CPU cores, or exclude some commonly used CPUs, such as CPU0, because CPU0 is used for timer tasks.

[0045] Step 2: Adjust the interrupt control of the network card: According to the interrupt merging or interrupt delay function supported by the network card, set the number of interrupts per second according to the usage scenario to balance power consumption and performance.

[0046] The network card supports a function called interrupt coalescing or interrupt delay, which allows the network card to accumulate multiple network events over a period of time before generating an interrupt. Set a reasonable number of interrupts per second according to the usage scenario. The specific commands can be referred to as follows:

[0047] Disable the adaptive interrupt control function ethtool -C NIC name adaptive-rx off adaptive-tx off

[0048] Enable the adaptive interrupt control function ethtool -C NIC name adaptive-rx on adaptive-tx on

[0049] Disable interrupt control function ethtool -C NIC name adaptive-rx off adaptive-tx off rx-usecs 0 tx-usecs 0

[0050] Set the interrupt control to a fixed interrupt frequency of 62us, about 16,000 interrupts per second:

[0051] ethtool -C NIC name adaptive-rx off adaptive-tx off rx-usecs 62tx-usecs 62

[0052] Step 3: Set the ring buffer size of the network card: Set at least two ring buffers for each network interface of the network card, namely the receiving buffer and the sending buffer. Use the command ethtool-S ethX to observe the value of rx_dropped or estimate the pressure of the cache when activating multiple queues to set the ring buffer size. For example, the default value is 512, and the maximum value is 4096. The command is: ethtool-G network card name rx 256tx 256 If you suspect that the packet loss is caused by insufficient cache at the current interrupt frequency, you can try to use the maximum value first, and then try to use the minimum value. Then you can use the binary search method to try different values ​​until the optimal performance is achieved.

[0053] The receive buffer (RX Ring Buffer) is used to store data packets received from the network. When data packets arrive, the network card puts them into the receive buffer and triggers an interrupt request to notify the CPU that there are new data packets to be processed. The CPU then processes these packets and passes them to the network protocol stack. The send buffer (TX RingBuffer) is used to store data packets to be sent through the network interface. When a data packet needs to be sent, the CPU puts the data packet into the send buffer and notifies the network card. The network card takes the data packet out of the send buffer and sends it over the network. The management of the ring buffer is part of the network driver, and they usually interact with the CPU through polling or interrupt-driven mechanisms in the driver. The size and configuration of these buffers have a significant impact on network performance. If the buffer is too small, it may cause data packet loss when receiving or inefficient sending when sending. If the buffer is too large, it may occupy too much memory resources.

[0054] Step 4: Adjust the rx hash field of RSS: Set the hash field of RSS, select the queue of the network card according to the hash of the source IP and destination IP, disperse the incoming network data packets to different receiving queues, and modify the RSS hash value of both TCP and UDP to disperse the network communication data between virtual machines.

[0055] The network card queue is selected based on the hash of the source IP and the destination IP. At this time, the network traffic is very likely to be in a network card queue, thereby interrupting the fixed CPU and affecting network performance. The RSS hash can be modified to add the source port and the destination port to implement the hash of the 4-tuple. The code reference is as follows:

[0056] sudo ethtool-n eth0 rx-flow-hash udp4

[0057] UDP over IPV4 flows use these fields for computing Hash flow key:

[0058] IP SA

[0059] IP DA

[0060] The udp hash found by the above command is the source IP and destination IP. Use the following command to add the source port and destination port:

[0061] sudo ethtool -N NIC name rx-flow-hash udp4 sdfn

[0062] It is recommended to modify the RSS hash value for both TCP and UDP, because most of the network communication between virtual machines interacts through the VXLAN network and depends on the UDP of the environment.

[0063] Step 5: Use iperf3 to perform network tests on each setting, and evaluate and further optimize each setting based on the test results.

[0064] You can deploy a physical machine and enable performance mode for testing. During the test, both TCP and UDP are tested. In the cloud network environment, create multiple virtual machines on the same network but different physical nodes to perform traffic testing.

[0065] Use multi-process iperf3 to test the network limit when performing flow testing, or use the iperf3 command to perform network testing by simulating the situation where multiple nodes and multiple network card queues are interacting with the network at the same time.

[0066] Example 2

[0067] The present invention also provides a device for tuning network card performance in a cloud network environment, including a setting adjustment module and a test tuning module.

[0068] Set the IRQ affinity of the adjustment module: bind the interrupt requests of different network card queues to different CPU cores, and specify which CPU cores handle specific interrupt requests.

[0069] Adjust the interrupt control of the network card: According to the interrupt merging or interrupt delay function supported by the network card, set the number of interrupts per second according to the usage scenario to balance power consumption and performance.

[0070] Set the ring buffer size of the network card: Set at least two ring buffers for each network interface of the network card, namely the receive buffer and the send buffer. Use the command ethtool -S ethX to observe the value of rx_dropped or estimate the cache pressure when activating multiple queues to set the ring buffer size.

[0071] Adjust the rx hash field of RSS: Set the hash field of RSS to select the queue of the network card according to the hash of the source IP and destination IP, and distribute the incoming network data packets to different receiving queues.

[0072] And modify the RSS hash value of both TCP and UDP to disperse the network communication data between virtual machines;

[0073] The test and tuning module uses iperf3 to perform network tests on each setting, and further optimizes each setting based on the test results.

[0074] As the information interaction and execution process between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.

[0075] Similarly, the device of the present invention improves the network performance in the cloud network environment, and enables each CPU of the host machine to participate in network processing based on interrupt load balancing and queue load balancing, thus avoiding the problem that a single CPU cannot handle large-scale network traffic;

[0076] Set the frequency of network interruption to find a balance between low latency and low power consumption;

[0077] Set an appropriate ringbuffer value to avoid packet loss due to traffic processing failure. Changing the network card configuration through performance tuning can significantly improve the performance of the cloud network environment.

[0078] It should be noted that not all steps and modules in the above-mentioned processes and device structures are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0079] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A method for optimizing network card performance in a cloud network environment, characterized in that include: Step 1: Set IRQ affinity: Bind interrupt requests from different network card queues to different CPU cores and specify which CPU cores handle specific interrupt requests. Step 2: Adjust the interrupt control of the network card: According to the interrupt merging or interrupt delay function supported by the network card, set the number of interrupts per second according to the usage scenario to balance power consumption and performance. Step 3: Set the ring buffer size of the network card: Set at least two ring buffers for each network interface of the network card, namely the receive buffer and the send buffer. Use the command ethtool-SethX to observe the value of rx_dropped or estimate the cache pressure when activating multiple queues to set the ring buffer size. Step 4: Adjust the rx hash field of RSS: Set the hash field of RSS, select the queue of the network card according to the hash of the source IP and destination IP, and distribute the incoming network data packets to different receiving queues. And modify the RSS hash value of both TCP and UDP to disperse the network communication data between virtual machines. Step 5: Use iperf3 to perform network tests on each setting, and evaluate and further optimize each setting based on the test results.

2. The method for optimizing network card performance in a cloud network environment according to claim 1, characterized in that When setting the affinity of IRQ in step 1, it includes: first stopping the iqrbalance service, then using the set_irq_affinity script or manually searching for the processor core bound to each node, searching for the IQR currently assigned to the network card, and writing the SMP affinity value to the corresponding IRQ entry, and then modifying the IQR according to the actual use effect.

3. The method for optimizing network card performance in a cloud network environment according to claim 1, characterized in that In step 2, the interrupt control command is used to adjust the interrupt control of the network card to be turned on or off, and the fixed interrupt frequency of the interrupt control is also set through the interrupt control command.

4. The method for optimizing network card performance in a cloud network environment according to claim 1, wherein: In step 3, the size of the receiving buffer is set to store the data packets received from the network. When the data packet arrives, the network card puts the data packet into the receiving buffer and triggers an interrupt request to notify the CPU that there is a new data packet to be processed; Set the send buffer size to store the data packets to be sent through the network interface. When a data packet needs to be sent, the CPU puts the data packet into the send buffer and notifies the network card. The network card takes the data packet out of the send buffer and sends it through the network.

5. The method for optimizing network card performance in a cloud network environment according to claim 1, characterized in that In step 5, deploy a physical machine and enable performance mode for testing. During the test, both TCP and UDP are tested. In the cloud network environment, create multiple virtual machines on the same network but different physical nodes to perform flow testing. Use multi-process iperf3 to test the network limit when performing flow testing, or use the iperf3 command to perform network testing by simulating the situation where multiple nodes and multiple network card queues are interacting with the network at the same time.

6. A device for optimizing network card performance in a cloud network environment, characterized in that Including setting adjustment module and test tuning module, Set the IRQ affinity of the adjustment module: bind the interrupt requests of different network card queues to different CPU cores, and specify which CPU cores handle specific interrupt requests. Adjust the interrupt control of the network card: According to the interrupt merging or interrupt delay function supported by the network card, set the number of interrupts per second according to the usage scenario to balance power consumption and performance. Set the ring buffer size of the network card: Set at least two ring buffers for each network interface of the network card, namely the receive buffer and the send buffer. Use the command ethtool -S ethX to observe the value of rx_dropped or estimate the cache pressure when activating multiple queues to set the ring buffer size. Adjust the rx hash field of RSS: Set the hash field of RSS to select the queue of the network card according to the hash of the source IP and destination IP, and distribute the incoming network data packets to different receiving queues. And modify the RSS hash value of both TCP and UDP to disperse the network communication data between virtual machines; The test and tuning module uses iperf3 to perform network tests on each setting, and further optimizes each setting based on the test results.

7. The device for optimizing network card performance in a cloud network environment according to claim 6, characterized in that When setting the IRQ affinity of the adjustment module, it includes: first stop the iqrbalance service, then use the set_irq_affinity script or manually find the processor core bound to each node, find the IQR currently assigned to the network card, and write the SMP affinity value to the corresponding IRQ entry. Later, modify the IQR according to the actual usage effect.

8. The device for optimizing network card performance in a cloud network environment according to claim 6, characterized in that The setting regulation module uses the interrupt regulation command to regulate the opening or closing of the interrupt regulation of the network card, and also sets the fixed interrupt frequency of the interrupt regulation through the interrupt regulation command.

9. The device for optimizing network card performance in a cloud network environment according to claim 6, characterized in that The setting adjustment module sets the size of the receiving buffer, which is used to store the data packets received from the network. When the data packet arrives, the data packet is placed in the receiving buffer through the network card and an interrupt request is triggered to notify the CPU that there is a new data packet to be processed; Set the send buffer size to store the data packets to be sent through the network interface. When a data packet needs to be sent, the CPU puts the data packet into the send buffer and notifies the network card. The network card takes the data packet out of the send buffer and sends it through the network.

10. The device for optimizing network card performance in a cloud network environment according to claim 6, characterized in that The test and tuning module deploys a physical machine and turns on the performance mode for testing. During the test, both TCP and UDP are tested. In the cloud network environment, multiple virtual machines with the same network but different physical nodes are created to perform flow testing. Use multi-process iperf3 to test the network limit when performing flow testing, or use the iperf3 command to perform network testing by simulating the situation where multiple nodes and multiple network card queues are interacting with the network at the same time.

Images