Data packet receiving method and device, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses a data packet receiving method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of computer networks. The method comprises the following steps: acquiring data packet receiving state parameters in real time; dynamically adjusting data packet receiving configuration according to the data packet receiving state parameters; and receiving and processing the data packet according to the adjusted data packet receiving configuration. According to the scheme, the data packet receiving configuration can be dynamically adjusted.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of computer network technologies, and more particularly to a method, apparatus, electronic device, and computer-readable storage medium for receiving data packets. Background Art

[0002] With the rapid growth of network traffic, the efficiency of receiving and distributing data packets faces severe challenges. Related technologies have introduced various optimization mechanisms such as New API (NAPI), Receive Side Scaling (RSS), and Receive Packet Steering (RPS). However, these optimization mechanisms have the problem of insufficient dynamicity in high-traffic scenarios, which may lead to excessive load on some Central Processing Units (CPUs). Summary of the Invention

[0003] Embodiments of the present application provide a method, apparatus, electronic device, and computer-readable storage medium for receiving data packets, which can dynamically adjust the data packet receiving configuration.

[0004] In a first aspect, embodiments of the present application provide a method for receiving data packets, including: Obtaining real-time data packet reception status parameters; Dynamically adjusting the data packet receiving configuration according to the data packet reception status parameters; Receiving and processing data packets according to the adjusted data packet receiving configuration.

[0005] In one embodiment, the data packet reception status parameters include: the number of data packets to be processed in the data packet reception queue and the total number of data packets in the data packet reception queue; The dynamically adjusting the data packet receiving configuration according to the data packet reception status parameters includes: Determining the retention rate of the data packet reception queue as the ratio of the number of data packets to be processed to the total number of data packets; Obtaining a queue expansion threshold and a queue contraction threshold; Expanding the data packet reception queue when the retention rate of the data packet reception queue is greater than the queue expansion threshold; Contracting the data packet reception queue when the retention rate of the data packet reception queue is less than the queue contraction threshold.

[0006] In one embodiment, the packet reception status parameter includes: the total number of packet reception queues, the five-tuple information of the target packet, and the real-time loads of the multiple packet reception queues respectively; The dynamically adjusting the packet reception configuration according to the packet reception status parameter includes: Calculating a hash value of the target packet according to the five-tuple information of the target packet; Obtaining the weights of the multiple packet reception queues respectively; Adjusting the weights of the multiple packet reception queues respectively according to the real-time loads of the multiple packet reception queues respectively to obtain the adjusted weights; Adjusting the target packet reception queue for receiving the target packet according to the hash value of the target packet and the adjusted weights of the multiple packet reception queues respectively.

[0007] In one embodiment, the packet reception status parameter includes: soft interrupt real-time statistical data; The dynamically adjusting the packet reception configuration according to the packet reception status parameter includes: Determining the real-time loads of the multiple CPUs respectively according to the soft interrupt real-time statistical data; Modifying the binding relationship between the packet reception queue and the multiple CPUs according to the real-time loads of the multiple CPUs respectively.

[0008] In one embodiment, the packet reception status parameter includes: the current network traffic rate; The dynamically adjusting the packet reception configuration according to the packet reception status parameter includes: Obtaining a network traffic rate threshold; Increasing the polling budget when the current network traffic rate is greater than the network traffic rate threshold; Reducing the polling budget when the current network traffic rate is less than the network traffic rate threshold.

[0009] In one embodiment, the dynamically adjusting the packet reception configuration according to the packet reception status parameter includes: Obtaining the packet processing delay; Obtaining a delay threshold corresponding to the packet processing delay; Dynamically adjusting the polling budget when the packet processing delay is not greater than the delay threshold.

[0010] In one embodiment, the method further includes: Obtaining a packet merging threshold; Merge the multiple received data packets according to the data packet merging threshold to obtain a merged data packet; Deliver the merged data packet.

[0011] In a second aspect, an embodiment of the present application provides a data packet receiving device, including: An acquisition module for acquiring data packet reception status parameters in real time; An adjustment module for dynamically adjusting the data packet reception configuration according to the data packet reception status parameters; A receiving module for receiving and processing data packets according to the adjusted data packet reception configuration.

[0012] In one embodiment, the data packet reception status parameters include: the number of data packets to be processed in the data packet reception queue and the total number of data packets in the data packet reception queue; the adjustment module includes: A retention rate determination unit for determining the ratio of the number of data packets to be processed to the total number of data packets as the retention rate of the data packet reception queue; A queue threshold acquisition unit for acquiring a queue expansion threshold and a queue contraction threshold; An expansion unit for expanding the data packet reception queue when the retention rate of the data packet reception queue is greater than the queue expansion threshold; A contraction unit for contracting the data packet reception queue when the retention rate of the data packet reception queue is less than the queue contraction threshold.

[0013] In one embodiment, the data packet reception status parameters include: the total number of data packet reception queues, the five-tuple information of the target data packet, and the respective real-time loads of the multiple data packet reception queues; the adjustment module includes: A hash value calculation unit for calculating the hash value of the target data packet according to the five-tuple information of the target data packet; A weight acquisition unit for acquiring the respective weights of the multiple data packet reception queues; A weight adjustment unit for adjusting the respective weights of the multiple data packet reception queues according to the respective real-time loads of the multiple data packet reception queues to obtain adjusted weights; A receiving queue adjustment unit for adjusting the target data packet reception queue for receiving the target data packet according to the hash value of the target data packet and the respective adjusted weights of the multiple data packet reception queues.

[0014] In one embodiment, the data packet reception status parameters include: soft interrupt real-time statistical data; the adjustment module includes: A real-time load determination unit for determining the real-time load of each of multiple CPUs according to the real-time statistical data of the soft interrupt; A relationship modification unit for modifying the binding relationship between the packet reception queue and multiple CPUs according to the real-time load of each of the multiple CPUs.

[0015] In one embodiment, the packet reception status parameter includes: the current network traffic rate; the adjustment module includes: A rate threshold acquisition unit for acquiring a network traffic rate threshold; A budget increase unit for increasing the polling budget when the current network traffic rate is greater than the network traffic rate threshold; A budget decrease unit for decreasing the polling budget when the current network traffic rate is less than the network traffic rate threshold.

[0016] In one embodiment, the adjustment module includes: A delay acquisition unit for acquiring the packet processing delay; A delay threshold acquisition unit for acquiring the delay threshold corresponding to the packet processing delay; A budget adjustment unit for dynamically adjusting the polling budget when the packet processing delay is not greater than the delay threshold.

[0017] In one embodiment, the device further includes: A merge threshold acquisition module for acquiring a packet merge threshold; A packet merge module for merging multiple received packets according to the packet merge threshold to obtain a merged packet; A delivery module for delivering the merged packet.

[0018] In a third aspect, an embodiment of the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps in the above-mentioned packet reception method are implemented.

[0019] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by the processor, the steps in the above-mentioned packet reception method are implemented.

[0020] Fifth aspect, an embodiment of the present application further provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various optional implementation manners described in the embodiments of the present application.

[0021] The embodiments of the present application have the following beneficial effects: The packet reception configuration can be dynamically adjusted according to the packet reception status parameters obtained in real time, so that the packets are received and processed according to the adjusted packet reception configuration. In this way, the dynamic adjustment of the packet reception configuration can cope with different scenarios, reduce the packet loss rate during traffic bursts, improve the packet reception efficiency, and achieve the load balancing of the CPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 It is a schematic diagram of the steps of a method for receiving packets provided by an embodiment of the present application; Figure 2 It is a schematic diagram of the steps of another method for receiving packets provided by an embodiment of the present application; Figure 3 It is a schematic diagram of the structure of a packet receiving device provided by an embodiment of the present application; Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] The following will clearly and completely describe the technical solutions in the present application with reference to the drawings in the present application. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

[0025] In one embodiment, as Figure 1As shown, a method for receiving data packets is provided. Although the logical order is shown in the step schematic diagram, in some cases, the steps shown or described can be executed in an order different from that shown in the accompanying drawings. Specifically, the method for receiving data packets can be applied to a data packet receiving end, which can include, but is not limited to, one or more of a smart phone, a tablet computer, a portable computer, a desktop computer, and a vehicle-mounted computer.

[0026] The following will be described in detail respectively. It should be noted that the description order of the following embodiments does not limit the priority order of the embodiments.

[0027] According to Figure 1 the method for receiving data packets shown, the method at least includes steps S110 to S130, which are introduced in detail as follows: In step S110, obtain the data packet reception status parameter in real time.

[0028] The data packet reception status parameter refers to a parameter related to receiving data packets.

[0029] The data packet reception status parameter can include, but is not limited to, one or more of the number of data packets to be processed in the data packet reception queue, the total number of data packets in the data packet reception queue, the total number of the data packet reception queue, the five-tuple information of the target data packet, the real-time load of the data packet reception queue, the real-time statistics data of soft interrupts, and the current network traffic rate.

[0030] In step S120, dynamically adjust the data packet reception configuration according to the data packet reception status parameter.

[0031] The data packet reception configuration is a configuration related to receiving data packets. The data packet reception configuration can include, but is not limited to, one or more of the number of data packet reception queues, the weight of the data packet reception queue, the binding relationship between the data packet reception queue and the CPU, and the polling budget.

[0032] According to the data packet reception status parameters in multiple dimensions, the data packet reception situation can be determined from multiple dimensions. According to the data packet reception situations in multiple dimensions, it can be determined whether it is necessary to adjust the data packet reception configuration for each dimension.

[0033] Optionally, the retention rate of the data packet reception queue can be determined according to the number of data packets to be processed in the data packet reception queue and the total number of data packets in the data packet reception queue, so as to determine whether to expand or contract the data packet reception queue according to the retention rate of the data packet reception queue.

[0034] Optionally, the weight of the data packet receiving queue can be adjusted according to the real-time load of the data packet receiving queue, and then the data packet receiving queue corresponding to the data packet can be determined according to the adjusted weight of the data packet receiving queue.

[0035] In step S130, the data packet is received and processed according to the adjusted data packet receiving configuration.

[0036] After the data packet receiving configuration is adjusted in real time, the data packet can be received according to the adjusted data packet receiving configuration in real time, and the received data packet can be processed.

[0037] By adopting the technical solution of the embodiment of the present application, the data packet receiving configuration can be dynamically adjusted according to the data packet receiving status parameters obtained in real time, so that the data packet is received and processed according to the adjusted data packet receiving configuration. In this way, the dynamic adjustment of the data packet receiving configuration can cope with different scenarios, reduce the packet loss rate during traffic bursts, improve the data packet receiving efficiency, and achieve the load balancing of the CPU.

[0038] On the basis of the above technical solution, as an embodiment, the data packet receiving status parameters include: the number of data packets to be processed in the data packet receiving queue and the total number of data packets in the data packet receiving queue.

[0039] Dynamically adjusting the data packet receiving configuration according to the data packet receiving status parameters may include: determining the ratio of the number of data packets to be processed to the total number of data packets as the retention rate of the data packet receiving queue; obtaining the queue expansion threshold and the queue contraction threshold; expanding the data packet receiving queue when the retention rate of the data packet receiving queue is greater than the queue expansion threshold; and contracting the data packet receiving queue when the retention rate of the data packet receiving queue is less than the queue contraction threshold.

[0040] The data packet receiving queue is a queue for temporarily storing received data packets. During the data receiving process, data packets may arrive at a relatively fast speed, but the processing program may not be able to process the data packets immediately. The data packet receiving queue can temporarily store the received data packets to avoid data packet loss, so that the subsequent processing program can obtain the data packets from the data packet receiving queue for processing. The data packet receiving queue can be a Ring Buffer queue.

[0041] The data packets to be processed in the data packet receiving queue are the data packets that have been received in the data packet receiving queue and are waiting for the processing program to process. The total number of data packets in the data packet receiving queue is the total number of various data packets.

[0042] Optionally, the ratio of the number of packets to be processed in the packet reception queue to the total number of packets in the packet reception queue can be determined as the retention rate of the packet reception queue.

[0043] Optionally, the packet loss statistics information (such as rx_queue_X_drops and rx_fifo_errors) of the packet reception queue in / proc / net / dev can also be periodically checked, and the retention rate of the packet reception queue can be determined according to the packet loss statistics information of the packet reception queue. The packet loss rate of the packet reception queue can be determined according to the packet loss statistics information of the packet reception queue, and the retention rate of the packet reception queue is proportional to the packet loss rate of the packet reception queue.

[0044] The preset queue expansion threshold and queue contraction threshold can be obtained, and the queue expansion threshold is greater than the queue contraction threshold. When the retention rate of the packet reception queue is greater than the queue expansion threshold, the packet reception queue is triggered to expand. Optionally, the expansion capacity can be a fixed value. Optionally, the expansion capacity can be determined according to the difference between the retention rate and the queue expansion threshold, and the expansion capacity is proportional to the difference.

[0045] When the retention rate of the packet reception queue is less than the queue contraction threshold, the packet reception queue is triggered to contract. Optionally, the contraction capacity can be a fixed value. Optionally, the contraction capacity can be determined according to the difference between the queue contraction threshold and the retention rate, and the contraction capacity is proportional to the difference.

[0046] By adopting the technical solution of the embodiment of the present application, the retention rate of the packet reception queue can be obtained in real time, and the capacity of the packet reception queue can be adjusted in real time according to the retention rate of the packet reception queue, so as to ensure that the capacity of the packet reception queue is within a suitable range, avoid the excessive capacity of the packet reception queue from occupying a large amount of storage space, and avoid the newly arrived packets from being discarded due to the too small capacity of the packet reception queue.

[0047] On the basis of the above technical solution, as an embodiment, the packet reception status parameters include: the total number of packet reception queues, the five-tuple information of the target packet, and the real-time loads of multiple packet reception queues respectively.

[0048] Dynamically adjusting the packet reception configuration according to the packet reception status parameters may include: calculating the hash value of the target packet according to the five-tuple information of the target packet; obtaining the weights of multiple packet reception queues respectively; adjusting the weights of multiple packet reception queues respectively according to the real-time loads of multiple packet reception queues respectively to obtain the adjusted weights; and adjusting the target packet reception queue for receiving the target packet according to the hash value of the target packet and the adjusted weights of multiple packet reception queues respectively.

[0049] The target data packet can be any data packet to be stored in the queue. The five-tuple information of the data packet is the key information used to identify and distinguish different network connections or data packet flows, including the source Internet Protocol (IP) address, destination IP address, source port number, destination port number, and transport layer protocol. The source IP address refers to the IP address of the device or network node that sends the data packet; the destination IP address is the IP address of the target device or network node to which the data packet is to be sent; the source port number is used to identify the port used by the application or process that sends the data packet on the source device; the destination port number specifies the port used by the application or process on the target device to which the data packet is to be sent; the transport layer protocol refers to the transport layer protocol used by the data packet, such as the Transmission Control Protocol (TCP) and User Datagram Protocol (UDP), etc.

[0050] A preset hash function can be obtained, and the hash function is used to perform a hash calculation on the five-tuple information of the target data packet to obtain the hash value of the target data packet. Among them, the hash value of the target data packet can be obtained by performing a hash calculation on one or more pieces of information in the five-tuple information of the target data packet.

[0051] In the related art, the queue to which the data packet belongs is statically determined directly by the remainder of dividing the hash value of the data packet by the total number of data packet receiving queues. For example, multiple data packet receiving queues can be numbered in advance. When there are four data packet receiving queues, the numbers of the four data packet receiving queues can be 0, 1, 2, and 3 respectively; if the remainder of dividing the hash value of the data packet by the total number of data packet receiving queues is 2, then the data packet can be stored in the data packet receiving queue numbered 2.

[0052] In an embodiment of the present application, weights are set for each data packet receiving queue, and the weights can be dynamically adjusted according to the real-time load of the data packet receiving queue. When determining the data packet receiving queue to which the target data packet belongs, the hash value of the target data packet and the weights of the data packet receiving queue are comprehensively considered, so as to balance the load of the data packet receiving queue.

[0053] In one embodiment, the target data packet receiving queue for receiving the target data packet can be determined according to the remainder of dividing the hash value of the target data packet by the total number of data packet receiving queues and the weights of the data packet receiving queue.

[0054] For example, the weights of two packet reception queues numbered 0 and 1 before adjustment are both 0.5. Since the weights are equal, if the remainder of the hash value of a packet divided by the total number of packet reception queues is 0, the packet can be stored in packet reception queue 0; if the remainder is 1, the packet can be stored in packet reception queue 1. When it is found that the load of packet reception queue 0 is high and the load of packet reception queue 1 is low, the weight of packet reception queue 0 can be adjusted to 0.6, and the weight of packet reception queue 1 can be adjusted to 0.4. According to the modified weights, 20% (i.e., 0.6 - 0.4) of the packets with a remainder of 1 will be assigned to packet reception queue 0.

[0055] In another embodiment, the target packet reception queue for receiving the target packet can be directly determined according to the hash value of the target packet and the weights of the packet reception queues.

[0056] For example, the weights of four packet reception queues before adjustment are all 0.25, and the adjusted weights are 0.1, 0.15, 0.35, and 0.4 in sequence; perform a hash calculation on the five-tuple information of the target packet to obtain a relatively large range of hash values, and then allocate the packet reception queues according to the new weights. Assume the hash value range is 0 - 99, then according to the adjusted weights, packets with hash values in the range of 0 - 9 can be allocated to packet reception queue 0; packets with hash values in the range of 10 - 24 can be allocated to packet reception queue 1; packets with hash values in the range of 25 - 59 can be allocated to packet reception queue 2; packets with hash values in the range of 60 - 99 can be allocated to packet reception queue 3.

[0057] In one of the embodiments, the real-time load of the packet reception queue can be determined according to the ratio of the number of packets in the packet reception queue to the capacity of the packet reception queue. According to the real-time load of the packet reception queue, the weight of the packet reception queue is dynamically adjusted. The load and weight of the packet reception queue are inversely proportional. The greater the real-time load of the packet reception queue, the lower the weight of the packet reception queue; the smaller the real-time load of the packet reception queue, the higher the weight of the packet reception queue.

[0058] Optionally, the weight of the packet reception queue can be dynamically adjusted through the ethtool–X command. The ethtool-X command is a sub-command of the ethtool tool in the Linux system, mainly used to configure the packet reception queue of network devices.

[0059] Adopting the technical solution of the embodiment of the present application, determining the target data packet receiving queue statically according to the remainder is changed to determining the target data packet receiving queue according to the hash value of the target data packet and the dynamic weight. Since the weight of the data packet receiving queue is adjusted in real time according to the real-time load of the data packet receiving queue, it is possible to guide the data packets to the data packet receiving queue with a lower load, thereby realizing the load balancing of the data packet receiving queue.

[0060] On the basis of the above technical solution, as an embodiment, the data packet receiving status parameter includes: real-time soft interrupt statistical data. Dynamically adjusting the data packet receiving configuration according to the data packet receiving status parameter may include: determining the respective real-time loads of multiple central processing units (CPUs) according to the real-time soft interrupt statistical data; modifying the binding relationship between the data packet receiving queue and the multiple CPUs according to the respective real-time loads of the multiple CPUs.

[0061] The RPS mechanism can distribute data packets to different CPU cores for processing. By reasonably distributing data packets to different CPU cores for processing, the competition between CPUs and cache misses are reduced, and the processing speed of network data packets and the overall network performance of the system are improved. Especially in the case of processing high-concurrency network connections and a large number of network data packets, the throughput and response speed of the system can be significantly improved.

[0062] In the related art RPS mechanism, after receiving a data packet, according to the static binding relationship between the data packet receiving queue where the data packet is located and the CPU, the CPU statically bound by the data packet receiving queue where the data packet is located processes the data packet.

[0063] In an embodiment of the present application, the respective real-time loads of multiple CPUs can be determined according to the real-time soft interrupt statistical data, and according to the real-time load of the CPU, the binding relationship between the data packet receiving queue and the CPU can be dynamically modified, so that the data packet is processed by the CPU with a lower load.

[0064] The real-time soft interrupt statistical data can be obtained by monitoring the distribution of network receive (NET_RX) soft interrupts in / proc / softirqs in real time. The real-time soft interrupt statistical data may include the number of data packets to be processed by each of multiple central processing units (CPUs) and the rate of each CPU processing data packets. Optionally, the load of the CPU can be determined according to the number of data packets to be processed by the CPU and the rate of processing data packets. Optionally, the load of the CPU can be obtained in real time directly through a command-line tool.

[0065] By modifying the binding relationship between multiple packet reception queues and multiple CPUs, when the load of a CPU is higher, the number of packet reception queues bound to the CPU can be made smaller, and when the load of the CPU is lower, the number of packet reception queues bound to the CPU can be made larger. Optionally, it can be achieved by dynamically modifying / sys / class / net / <device>The / queues / rx-X / rps_cpus file is modified to change the binding relationship between the data packet receiving queue where the data packet is located and the CPU.

[0066] When a data packet is received, the binding relationship between the data packet receiving queue and the CPU can be dynamically modified so that the data packet is assigned to the CPU with the lowest real-time load.

[0067] Adopting the technical solution of the embodiment of the present application, the allocation path of data packets can be dynamically adjusted according to the real-time load of the CPU, realizing the load balancing of the CPU, giving full play to the advantages of the multi-core processor, accelerating the data processing and operation speed, avoiding the single-core bottleneck, improving the system performance, increasing the system stability, and improving the resource utilization rate.

[0068] Based on the above technical solution, as an embodiment, the data packet reception status parameter includes: the current network traffic rate. Dynamically adjusting the data packet reception configuration according to the data packet reception status parameter can include: obtaining the network traffic rate threshold; increasing the polling budget when the current network traffic rate is greater than the network traffic rate threshold; and reducing the polling budget when the current network traffic rate is less than the network traffic rate threshold.

[0069] The NAPI mechanism is a technology used to improve the efficiency of network data packet reception in the Linux network subsystem. The NAPI mechanism adopts a combination of interrupts and polling. When a new data packet arrives, the kernel is first notified through an interrupt. After the kernel responds to the interrupt, it will batch obtain data packets from the device receiving queue in a polling manner for processing until the data packets in the queue are processed or the preset polling budget is reached, and then the interrupt is restarted to wait for the next data packet to arrive.

[0070] The polling budget of the NAPI mechanism in the related technology is fixed. In an embodiment of the present application, whether to modify the polling budget can be determined according to the magnitude relationship between the current network traffic rate and the network traffic rate threshold.

[0071] The current network traffic rate can be obtained through a command-line tool, network monitoring software, or system built-in function. The preset network traffic rate threshold can be obtained, and the network traffic rate threshold can be set according to actual requirements.

[0072] Increase the polling budget when the current network traffic rate is greater than the network traffic rate threshold; reduce the polling budget when the current network traffic rate is less than the network traffic rate threshold. Optionally, the increased or decreased polling budget can be determined according to the difference between the current network traffic rate and the network traffic rate threshold, and the increased or decreased polling budget is proportional to the difference. Optionally, the polling budget can be dynamically adjusted through the sysctl -w net.core.netdev_budget command.

[0073] Among them, the polling budget can be the number of data packets batch-processed by the polling packet receiving function (poll function) of the NAPI mechanism. By calling the poll function of the NAPI mechanism, data packets can be batch-extracted from the data packet receiving queue and processed.

[0074] A budget threshold can be set. When the increased polling budget is greater than the budget threshold, the polling budget is adjusted to the budget threshold to control the upper limit of the growth of the polling budget.

[0075] By adopting the technical solution of the embodiment of the present application, the receiving efficiency in high-traffic scenarios can be optimized by dynamically adjusting the polling budget of NAPI; appropriately increasing the polling budget in high-traffic scenarios can reduce the soft interrupt trigger frequency and improve the throughput; appropriately reducing the polling budget in low-traffic scenarios can reduce the CPU occupancy.

[0076] Based on the above technical solution, as an embodiment, dynamically adjusting the data packet receiving configuration according to the data packet receiving status parameter may include: obtaining the data packet processing delay; obtaining the delay threshold corresponding to the data packet processing delay; and dynamically adjusting the polling budget when the data packet processing delay is not greater than the delay threshold.

[0077] Since adjusting the polling budget of NAPI may increase the scheduling delay under high traffic. Therefore, the polling budget may be dynamically adjusted only when the data packet processing delay is not greater than the delay threshold. The delay threshold can be set according to actual requirements.

[0078] The data packet processing delay is the time interval from when the data packet enters a network device (such as a router, switch, server, etc.) until the device finishes processing it and forwards or transmits it. Optionally, the data packet processing delay can be obtained through a network test tool or network monitoring software.

[0079] By adopting the technical solution of the embodiment of the present application, by monitoring the data processing delay, the adjustment frequency of the polling budget can be dynamically adjusted, thereby avoiding the impact of the scheduling delay caused by dynamically adjusting the polling budget on data packet reception and processing.

[0080] Based on the above technical solution, as an embodiment, as Figure 2 shown, the data packet receiving method may further include steps S210 to S230.

[0081] In step S210, obtain the data packet merging threshold.

[0082] In step S220, according to the data packet merging threshold, merge the received multiple data packets to obtain the merged data packet.

[0083] In step S230, the merged data packets are delivered.

[0084] The Generic Receive Offload (GRO) mechanism can be used to merge data packets, reducing the number of packets processed by the upper network stack. The data packet merging threshold can be set according to actual requirements. When delivering data packets, multiple data packets can be merged, and the number of data packets merged each time is not greater than the data packet merging threshold.

[0085] Optionally, the napi_gro_receive function can be called to merge multiple data packets into a large data packet, improving the processing efficiency of the network stack. After the merging is completed, the merged data packet is delivered to the upper IP layer through the netif_receive_skb function.

[0086] Adopting the technical solution of the embodiment of the present application to merge data packets and then deliver them can improve the transmission efficiency, reduce network congestion, reduce resource occupancy, and enhance data reliability.

[0087] In the initialization stage, configure the network card to support the RSS and NAPI mechanisms, and start the dynamic path allocation module and the dynamic queue monitoring module. The dynamic path allocation module is used to determine the data packet reception queue for receiving data packets and determine the binding relationship between the data packet reception queue and the CPU; the dynamic queue monitoring module is used to expand and contract the data packet reception queue. Configure the initial size and maximum capacity of the data packet reception queue through the ethtool - G command.

[0088] In the real - time monitoring stage, read the / proc / net / dev and / proc / softirqs data through the dynamic queue monitoring module, and analyze the queue retention rate and soft - interrupt distribution. The dynamic path allocation module adjusts the RSS and RPS configurations according to the traffic characteristics (the real - time load of the data packet reception queue and the current network traffic rate) and the CPU load.

[0089] In the soft - interrupt processing stage, when the network card receives data, a soft - interrupt is triggered; the poll function of NAPI is called to batch - extract data packets from the data packet reception queue; the napi_gro_receive function is used to merge data packets, reducing the processing burden of the network stack.

[0090] In the dynamic optimization stage, according to the real - time monitoring results, dynamically adjust the number of queues, queue weights, and NAPI budgets; after the adjustment is completed, reload the network card configuration to make the dynamic adjustment strategy take effect.

[0091] The packet receiving method proposed in an embodiment of this application can be applied to data center load balancing. In a high-traffic environment, this packet receiving method is applicable to data center multi-path traffic management and high-throughput scenarios through dynamic path allocation and queue optimization.

[0092] The packet receiving method proposed in an embodiment of this application can also be applied to the edge computing environment. For data transmission optimization between edge nodes, dynamic queue adjustment can effectively handle burst traffic.

[0093] The packet receiving method proposed in an embodiment of this application can also be applied to cloud computing network optimization. This packet receiving method can be used as the basis for internal network scheduling of cloud service providers. By dynamically selecting paths and load balancing, the network performance between virtual machines and containers can be improved.

[0094] The packet receiving method proposed in an embodiment of this application can also be applied to industrial Internet of Things scenarios. In industrial devices that support Ethernet communication, this packet receiving method can be used to optimize high-concurrency data transmission and reduce network latency.

[0095] In 5G base stations, the dynamic monitoring mechanism of the packet receiving queue can be combined with 5G network slicing to optimize multi-path transmission between base stations.

[0096] In the in-vehicle network of autonomous vehicles, the dynamic adjustment of network path allocation and queues can be combined with the low-latency requirements of sensor data.

[0097] In a distributed storage system, the efficiency of big data transmission can be improved by optimizing NAPI budgets and queue allocations.

[0098] Optionally, the packet receiving method proposed in an embodiment of this application can also combine machine learning algorithms to predict network traffic rates, and adjust queue configurations and modify polling budgets in advance based on the predicted network traffic rates to further reduce the packet loss rate during traffic bursts.

[0099] Optionally, for the packet receiving method proposed in an embodiment of this application, on a NIC that supports hardware acceleration, it can also combine hardware RPS and software path selection mechanisms to further optimize path allocation and CPU utilization.

[0100] Optionally, for the packet receiving method proposed in an embodiment of this application, in scenarios with high traffic delay requirements, a path allocation and queue management algorithm based on packet priorities can also be designed. The priorities of each packet can be determined, and packets with different priorities can be assigned to different packet receiving queues.

[0101] The method for receiving data packets proposed in an embodiment of this application can adjust one or more of the data packet receiving configurations simultaneously. The adjustment methods of these multiple data packet receiving configurations can cooperate with each other and work together to improve the data packet receiving performance.

[0102] In one embodiment, the expansion / contraction of the data packet receiving queue can be coordinated with the dynamic adjustment in the RSS mechanism and / or the RPS mechanism to optimize load balancing. Among them, expanding / contracting the data packet receiving queue according to the traffic load can avoid queue overflow or resource waste. The RSS mechanism can distribute data packets to multiple data packet receiving queues through hardware hashing, and combined with the expansion / contraction of the data packet receiving queue, it can effectively divert traffic. The RPS mechanism further optimizes CPU load balancing at the software level and dynamically selects the most idle CPU to process data packets.

[0103] The expansion / contraction of the data packet receiving queue and the dynamic adjustment in the RSS mechanism and / or the RPS mechanism are coordinated, which has the advantage of consistent dynamics. The expansion / contraction of the data packet receiving queue and the dynamic adjustment in the RSS mechanism and / or the RPS mechanism both rely on real-time monitoring (such as queue retention rate, CPU load) and dynamically adjust the configuration according to the feedback.

[0104] The expansion / contraction of the data packet receiving queue and the dynamic adjustment in the RSS mechanism and / or the RPS mechanism have a cooperative relationship in terms of resources. After the data packet receiving queue is expanded, RSS can effectively utilize the newly added data packet receiving queue; when the data packet receiving queue is contracted, RPS reallocates the processing tasks to avoid performance degradation.

[0105] The expansion / contraction of the data packet receiving queue and the dynamic adjustment in the RSS mechanism and / or the RPS mechanism can form a load balancing chain: through the coordination of the RSS mechanism to queue adjustment and then to the RPS mechanism, a multi-level load balancing from hardware to queue and then to CPU can be constructed.

[0106] In one embodiment, the coordination of adjusting the polling budget of NAPI and data packet merging can reduce the CPU overhead. Increasing the polling budget allows the CPU to process more data packets at one time and reduces context switching; decreasing the polling budget can prevent the CPU from being overloaded. Merging data packets at the Data Plane Development Kit (DPDK) or network card level can reduce the number of interrupts that the CPU needs to process.

[0107] The coordination of adjusting the polling budget of NAPI and data packet merging can optimize the CPU load. Data packet merging reduces the interrupt frequency, and together with the polling budget adjustment, it reduces the CPU's interrupt processing overhead.

[0108] Adjusting the polling budget of NAPI and packet coalescing has a dynamic match. In high-traffic scenarios, packet coalescing and a high NAPI budget work together to maximize CPU processing efficiency.

[0109] Adjusting the polling budget of NAPI and packet coalescing can achieve hardware cooperation. The coalescing capabilities of network cards or DPDK and the software optimization of NAPI complement each other, forming a low-overhead processing link.

[0110] In one embodiment, the overall coordination of dynamically adjusting the packet reception queue expansion and / or contraction, the RSS mechanism and / or the RPS mechanism, and adjusting the polling budget of NAPI and packet coalescing can optimize the entire process of packet reception and processing. Specifically, hardware shunting can be achieved through the RSS mechanism. Packets are distributed to multiple queues through hashing to relieve the pressure on a single queue, laying the foundation for subsequent optimization. Dynamically expanding / contracting the packet reception queue can dynamically adjust the number of queues according to the traffic distribution of RSS to ensure maximum resource utilization efficiency. The RPS mechanism can achieve CPU balance, dynamically allocating queue tasks to the most idle CPU at the software level to avoid single-core overload. Adjusting the polling budget of NAPI and packet coalescing can reduce overhead, improve CPU processing efficiency, reduce interrupts and context switches, and enhance overall throughput.

[0111] In one embodiment, in high-traffic burst scenarios, the RSS mechanism can disperse traffic to multiple queues and dynamically expand the queues at the same time. The RPS mechanism can select idle CPUs for processing. Increasing the NAPI budget and packet coalescing can reduce CPU overhead.

[0112] In one embodiment, in low-traffic scenarios, queues can be contracted to save resources, and the polling budget can be reduced to avoid waste of idle CPU.

[0113] To facilitate better implementation of the packet reception method of the present application, the present application also provides a packet reception device based on the above packet reception method. The meanings of the nouns are the same as those in the above packet reception method, and the specific implementation details can refer to the description in the method embodiments.

[0114] Please refer to Figure 3 , Figure 3 which is a schematic structural diagram of the packet reception device provided by the embodiments of the present application. The packet reception device includes: An acquisition module 301, configured to acquire packet reception status parameters in real time; An adjustment module 302, configured to dynamically adjust packet reception configuration according to the packet reception status parameters; A reception module 303, configured to receive and process packets according to the adjusted packet reception configuration.

[0115] In one embodiment, the data packet reception status parameter includes: the number of data packets to be processed in the data packet reception queue and the total number of data packets in the data packet reception queue; the adjustment module 302 includes: A retention rate determination unit, configured to determine the ratio of the number of data packets to be processed to the total number of data packets as the retention rate of the data packet reception queue; A queue threshold acquisition unit, configured to acquire a queue expansion threshold and a queue contraction threshold; An expansion unit, configured to expand the data packet reception queue when the retention rate of the data packet reception queue is greater than the queue expansion threshold; A contraction unit, configured to contract the data packet reception queue when the retention rate of the data packet reception queue is less than the queue contraction threshold.

[0116] In one embodiment, the data packet reception status parameter includes: the total number of data packet reception queues, the five-tuple information of the target data packet, and the respective real-time loads of multiple data packet reception queues; the adjustment module 302 includes: A hash value calculation unit, configured to calculate a hash value of the target data packet according to the five-tuple information of the target data packet; A weight acquisition unit, configured to acquire the respective weights of multiple data packet reception queues; A weight adjustment unit, configured to adjust the respective weights of multiple data packet reception queues according to the respective real-time loads of multiple data packet reception queues to obtain adjusted weights; A reception queue adjustment unit, configured to adjust the target data packet reception queue for receiving the target data packet according to the hash value of the target data packet and the respective adjusted weights of multiple data packet reception queues.

[0117] In one embodiment, the data packet reception status parameter includes: soft interrupt real-time statistical data; the adjustment module 302 includes: A real-time load determination unit, configured to determine the respective real-time loads of multiple central processing units (CPUs) according to the soft interrupt real-time statistical data; A relationship modification unit, configured to modify the binding relationship between the data packet reception queue and multiple CPUs according to the respective real-time loads of multiple CPUs.

[0118] In one embodiment, the data packet reception status parameter includes: the current network traffic rate; the adjustment module 302 includes: A rate threshold acquisition unit, configured to acquire a network traffic rate threshold; A budget increase unit for increasing the polling budget when the current network traffic rate is greater than the network traffic rate threshold; A budget decrease unit for decreasing the polling budget when the current network traffic rate is less than the network traffic rate threshold.

[0119] In one embodiment, the adjustment module 302 includes: A delay acquisition unit for acquiring the packet processing delay; A delay threshold acquisition unit for acquiring the delay threshold corresponding to the packet processing delay; A budget adjustment unit for dynamically adjusting the polling budget when the packet processing delay is not greater than the delay threshold.

[0120] In one embodiment, the device further includes: A merge threshold acquisition module for acquiring the packet merge threshold; A packet merge module for merging multiple received packets according to the packet merge threshold to obtain a merged packet; A delivery module for delivering the merged packet.

[0121] By adopting the technical solution of the embodiment of the present application, the packet reception configuration can be dynamically adjusted according to the packet reception status parameters obtained in real time, so that the packets are received and processed according to the adjusted packet reception configuration. In this way, the dynamic adjustment of the packet reception configuration can cope with different scenarios, reduce the packet loss rate during traffic bursts, improve the packet reception efficiency, and achieve the load balancing of the CPU.

[0122] For the specific limitations on the packet receiving device, reference can be made to the limitations on the packet receiving method in the above text, which will not be elaborated here. Each module in the above packet receiving device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in hardware form or independent of it, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0123] In addition, the present application also provides an electronic device, as Figure 4 shown, which shows the structural schematic diagram of the electronic device involved in the present application. Specifically: The electronic device may include a processor 401 with one or more processing cores and a memory 402 with one or more computer-readable storage media and other components. Those skilled in the art can understand, Figure 4 The electronic device structure shown does not limit the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Among them: The processor 401 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 402, and calling the data stored in the memory 402, it executes various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 401 either.

[0124] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, image playback function, etc.); the data storage area can store data created according to the use of the electronic device. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0125] In one embodiment, the electronic device further includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 403 may also include any components such as one or more DC or AC power supplies, a recharge system, a power device debugging circuit, a power converter or inverter, and a power status indicator.

[0126] In one embodiment, the electronic device may further include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0127] Although not shown, the electronic device may further include a display unit and the like, which will not be elaborated herein. Specifically, in this embodiment, the processor 401 in the electronic device will, according to the following instructions, load the executable files corresponding to the processes of one or more application programs into the memory 402, and the processor 401 will run the application programs stored in the memory 402, so as to implement the steps in any of the packet receiving methods provided by the embodiments of the present application.

[0128] Those skilled in the art can understand that Figure 4 the structure shown in

[0129] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0130] In one embodiment, an electronic device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the method described in any embodiment of the present application is implemented.

[0131] In some embodiments, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in any embodiment of the present application is implemented.

[0132] The specific implementation of each of the above operations can be seen in the previous embodiments and will not be elaborated herein.

[0133] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by instructions controlling relevant hardware. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0134] Therefore, the present application provides a computer-readable storage medium, on which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any of the packet receiving methods provided by the present application.

[0135] The specific implementation of each of the above operations can be seen in the previous embodiments and will not be elaborated herein.

[0136] Among them, the computer-readable storage medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc.

[0137] Since the instructions stored in the computer-readable storage medium can execute the steps in any of the packet receiving methods provided by this application, the beneficial effects achievable by any of the packet receiving methods provided by this application can be realized. For details, see the previous embodiments and will not be elaborated here.

[0138] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the said element.

[0139] The above has introduced in detail a packet receiving method, device, electronic device and computer-readable storage medium provided by this application. Specific examples are used in this article to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.< / device>

Claims

1. A method for receiving a data packet, characterized in that: include: Obtain data packet receiving status parameters in real time; Dynamically adjusting the data packet receiving configuration according to the data packet receiving state parameter; The data packet is received and processed according to the adjusted data packet receiving configuration.

2. The method according to claim 1, characterized in that The data packet receiving state parameters include: the number of data packets to be processed in the data packet receiving queue and the total number of data packets in the data packet receiving queue; The dynamically adjusting the data packet receiving configuration according to the data packet receiving state parameter comprises: Determine the ratio of the number of the data packets to be processed to the total number of the data packets as the retention rate of the data packet receiving queue; Get the queue expansion threshold and queue contraction threshold; When the retention rate of the data packet receiving queue is greater than the queue expansion threshold, expanding the data packet receiving queue; When the retention rate of the data packet receiving queue is less than the queue shrinking threshold, the data packet receiving queue is shrunk.

3. The method according to claim 1, characterized in that The data packet receiving state parameters include: the total number of data packet receiving queues, the five-tuple information of the target data packet and the real-time load of each of the plurality of data packet receiving queues; The dynamically adjusting the data packet receiving configuration according to the data packet receiving state parameter comprises: Calculate the hash value of the target data packet according to the five-tuple information of the target data packet; Obtaining respective weights of the plurality of data packet receiving queues; According to the real-time loads of the plurality of data packet receiving queues, the weights of the plurality of data packet receiving queues are adjusted to obtain adjusted weights; According to the hash value of the target data packet and the adjusted weights of the plurality of data packet receiving queues, the target data packet receiving queue for receiving the target data packet is adjusted.

4. The method according to claim 1, characterized in that: The data packet receiving status parameters include: real-time statistics of soft interrupts; The dynamically adjusting the data packet receiving configuration according to the data packet receiving state parameter comprises: Determine the real-time load of each of the plurality of central processing units (CPUs) according to the real-time statistical data of the soft interrupt; The binding relationship between the data packet receiving queue and the multiple CPUs is modified according to the real-time loads of the multiple CPUs respectively.

5. The method according to claim 1, characterized in that The data packet receiving state parameters include: current network traffic rate; The dynamically adjusting the data packet receiving configuration according to the data packet receiving state parameter comprises: Get the network traffic rate threshold; When the current network traffic rate is greater than the network traffic rate threshold, increasing the polling budget; When the current network traffic rate is less than the network traffic rate threshold, the polling budget is reduced.

6. The method according to claim 5, characterized in that The dynamically adjusting the data packet receiving configuration according to the data packet receiving state parameter comprises: Get packet processing delay; Obtaining a delay threshold corresponding to the data packet processing delay; When the data packet processing delay is not greater than the delay threshold, the polling budget is dynamically adjusted.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Get the packet merging threshold; According to the data packet merging threshold, merging the received multiple data packets to obtain a merged data packet; The merged data packet is delivered.

8. A device for receiving a data packet, characterized in that: include: An acquisition module is used to acquire data packet receiving status parameters in real time; An adjustment module, used for dynamically adjusting the data packet receiving configuration according to the data packet receiving state parameter; The receiving module is used to receive and process the data packet according to the adjusted data packet receiving configuration.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the method for receiving a data packet as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method for receiving a data packet as described in any one of claims 1 to 7 are implemented.

Images