Method and device for packet sending based on pre-occupancy credit dynamic flow control, equipment and storage medium
By using a pre-allocated credit dynamic flow control method, credits are dynamically allocated and compensated, solving the problem of insufficient packet sending engine cache, improving PCIe utilization efficiency and packet sending performance, and reducing resource overhead.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2026-03-31
AI Technical Summary
In existing technologies, packet sending engines are prone to insufficient cache space when accessing memory via PCIe, which affects the efficiency of PCIe usage and blocks other modules. Furthermore, buffer resources are expensive and their evaluation is inaccurate, making it difficult to ensure packet sending performance while reducing resource overhead.
A pre-allocated credit dynamic flow control method is adopted. The credit management module dynamically allocates credits based on the type of packet request and the amount of data, ensuring that the internal cache of the packet sending engine does not overflow, and compensates the credits after the downstream module has finished processing the request, thereby optimizing packet sending performance.
It effectively saves DMA cache resources, improves the efficiency of the packet sending engine in utilizing PCIe, enhances packet sending performance, and reduces resource overhead.
Smart Images
Figure CN118900251B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of network transmission, and specifically relates to packet sending optimization methods, devices, and storage media based on pre-occupied credit dynamic flow control. Background Technology
[0002] The main function of the packet sending engine is to receive commands related to data packet transmission, convert them into data packet header commands and data read commands, then use the data read commands to retrieve main memory data through the memory access module, and finally concatenate the data packet header with the main memory data before sending it. The internal structure of the packet sending engine includes a data packet transmission command parsing module, a data packet generation module, a main memory data caching module, and a data packet header and main memory data concatenation and sending module, as follows: Figure 1 As shown, the capacity calculation of the main memory data cache module is mostly related to the latency of the memory access module and the currently expected bandwidth and line speed.
[0003] Retrieving main memory data via the memory access module requires using PCIe memory access. If the read data cannot be sent in a timely manner, and the packet sending engine's internal data cache space is insufficient, backpressure on PCIe will severely impact PCIe's efficiency and block other modules from using PCIe.
[0004] Furthermore, implementing a buffer within the packet sending engine to hold all read-back memory access data and cache data returned from the PCIe side would obviously incur significant resource overhead. Moreover, PCIe latency is not constant, leading to inaccurate buffer evaluation. Therefore, only by reducing resource overhead while ensuring unaffected packet sending performance can the packet sending engine's PCIe utilization efficiency be improved. Summary of the Invention
[0005] The purpose of this application is to provide a packet sending optimization method and apparatus based on pre-occupied credit dynamic flow control, which aims to reduce the overhead of accessing memory resources.
[0006] According to a first aspect of this application, a packet sending optimization method based on pre-allocated credit dynamic flow control is provided, comprising:
[0007] When the packet sending engine receives a packet sending request, if the packet sending request is a DMA memory access request, it applies for packet sending credit from the credit management module according to the physical page information carried in the packet sending request; if the packet sending request is not a DMA memory access request, it applies for packet sending credit from the credit management module according to the software configuration information.
[0008] When the credit management module determines that the packet credit of the DMA memory access request is not greater than the currently available credit, it allocates the packet credit to the downstream module, sends the packet request to the downstream module, and updates the currently available credit.
[0009] When the downstream module finishes processing the packet request, it will compensate the corresponding packet credit to the current available credit of the credit management module.
[0010] In an optional implementation, the step of requesting a package credit from the credit management module based on software configuration information further includes:
[0011] If the packet request is not a DMA memory access request, then a packet credit is requested from the credit management module according to the software configuration information. After the credit for the ongoing DMA memory access request is approved, and all DMA commands are issued, a credit is issued for the current non-memory access request.
[0012] In an optional implementation, applying for packet credit from the credit management module based on the physical page information carried in the packet request further includes:
[0013] The number of data packets carried by the physical page of the current memory access request is calculated based on the data length of the DMA memory access request. Then, the number of data packets is multiplied by the credit consumption amount configured by the software to obtain the packet credit of the current DMA memory access request, and the packet credit is requested from the credit management module.
[0014] In an optional implementation, updating the currently available credit further includes:
[0015] The outsourcing credit for the application will be deducted from the currently available credit.
[0016] In an optional implementation, the step of compensating the corresponding packet-issuing credit to the currently available credit of the credit management module further includes:
[0017] For completed DMA memory access requests, determine whether the current DMA data is abnormal;
[0018] If the current DMA data is normal, credit compensation is performed based on the packet credit allocated by the upstream, and the remaining credit of the DMA memory access request is maintained.
[0019] If the current DMA data is abnormal, credit compensation will be performed based on the remaining credit of the DMA memory access request.
[0020] According to a second aspect of this application, a packet sending optimization device based on pre-allocated credit dynamic flow control is provided, comprising:
[0021] The credit application unit is used to apply for a packet credit from the credit management module based on the physical page information carried in the packet request when the packet engine receives a packet request. If the packet request is a DMA memory access request, the unit applies for a packet credit from the credit management module based on the software configuration information.
[0022] The packet sending unit is used to allocate the packet sending credit to the downstream module and send the packet sending request to the downstream module when the credit management module determines that the packet sending credit of the DMA memory access request is not greater than the current available credit, and at the same time update the current available credit.
[0023] The credit compensation unit is used to compensate the corresponding packet sending credit to the current available credit of the credit management module when the downstream module finishes processing the packet sending request.
[0024] A third aspect of this application provides an electronic device including a processor and a memory, the memory storing a plurality of instructions, and the processor being configured to read the instructions and execute the method of the first aspect described above.
[0025] A fourth aspect of this application provides a computer-readable storage medium storing a plurality of instructions that can be read by a processor and used to execute the method described in the first aspect.
[0026] Compared with related technologies, the technical solution of this application has the following advantages:
[0027] This application saves DMA cache resources, improves the efficiency of the packet sending engine in utilizing PCIe, and refines the request data volume according to packet granularity, thereby reducing resource overhead and improving packet sending performance.
[0028] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures and processes shown in the description and the accompanying drawings. Attached Figure Description
[0029] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0030] Figure 1 This is a schematic diagram of the packet sending engine structure based on relevant technologies.
[0031] Figure 2 This is a flowchart of a packet sending optimization method based on pre-occupied credit dynamic flow control according to an exemplary embodiment of this application.
[0032] Figure 3 This is a schematic diagram of a packet sending engine logic employing pre-reserved credits according to an exemplary embodiment of this application.
[0033] Figure 4 This is a schematic diagram of credit management logic according to an exemplary embodiment of this application.
[0034] Figure 5 This is a schematic diagram of a credit application logic for a non-memory access request according to an exemplary embodiment of this application.
[0035] Figure 6 This is a schematic diagram of the credit application logic for a memory access (DMA) request according to an exemplary embodiment of this application.
[0036] Figure 7 This is a schematic diagram of credit compensation logic according to an exemplary embodiment of this application. Detailed Implementation
[0037] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0038] The method provided in this application can be implemented in a terminal environment that may include one or more of the following components: a processor, a memory, and a display screen. The memory stores at least one instruction, which is loaded and executed by the processor to implement the method described in the following embodiments.
[0039] A processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts of the terminal, and performs various functions and processes data by running or executing instructions, programs, code sets or instruction sets stored in memory, and by calling data stored in memory.
[0040] Memory can include random access memory (RAM) or read-only memory (ROM). Memory can be used to store instructions, programs, code, code sets, or instructions.
[0041] The display screen is used to show the user interface of each application.
[0042] In addition, those skilled in the art will understand that the structure of the terminal described above does not constitute a limitation on the terminal. The terminal may include more or fewer components, or combine certain components, or have different component arrangements. For example, the terminal may also include radio frequency circuits, input units, sensors, audio circuits, power supplies, and other components, which will not be described in detail here.
[0043] Based on the above analysis, this application proposes a packet sending optimization method and apparatus based on pre-allocated credit dynamic flow control. It employs a credit pre-allocation method to determine whether the current request can be sent, ensuring both that the internal DMA buffer does not overflow and that packet sending performance is guaranteed. The credit can be the clock cycles consumed by the packet sending exit when processing the request, excluding the time spent reading the load. For requests carrying a load, credit is requested at the granularity of the physical page carried in the request. For requests without a load, credit can only be requested after all read DMA commands for preceding load-carrying requests have been issued. Credit is compensated to the credit management module after each request is processed by the downstream module or after each data packet is sent.
[0044] See Figure 2 The flowchart provided in this application shows the packet sending optimization method based on pre-occupied credit dynamic flow control, which includes:
[0045] Step 201: When the packet sending engine receives a packet sending request, if the packet sending request is a DMA memory access request, it applies for packet sending credit from the credit management module according to the physical page information carried in the packet sending request. If the packet sending request is not a DMA memory access request, it applies for packet sending credit from the credit management module according to the software configuration information.
[0046] This application uses a pre-allocated credit packet sending engine, such as Figure 3 As shown, a credit management module is added to a regular packet sending engine. When the request parsing module receives a packet sending request, it requests credit from the credit management module to use for sending data packets.
[0047] For example, the credit application logic for non-DMA requests is as follows: Figure 5 As shown, the system waits for the credit requests of ongoing memory access requests to be approved before requesting credits based on the software-configured request credits. Only after the credit requests are approved, i.e., all DMA commands are issued, are credits allocated for non-memory access requests. The credits for non-memory access requests are the request credits pre-configured by the software. For example, the credit request logic for memory access (DMA) requests is as follows: Figure 6 As shown, the number of data packets pkt carried by the current memory access request physical page is calculated based on the path maximum transmission unit (PMTU) and the memory access request data length. _`num` (pkt_num = length of data requested for memory access / PMTU) is incremented by 1 if the length of the data carried by the physical page is not an integer multiple of PMTU. Then, `pkt_num` is multiplied by the credit consumption amount configured in the software to obtain the credit value requested for the current memory access request. This credit is then requested from the credit module. Only after obtaining the credit can the memory access command be issued. Optionally, when a physical page exception occurs, credit is requested based on the number of remaining packets in the request to ensure that the credit is not leaked.
[0048] Step 202: When the credit management module determines that the packet credit of the DMA memory access request is not greater than the current available credit, it allocates the packet credit to the downstream module, sends the packet request to the downstream module, and updates the current available credit.
[0049] In a preferred embodiment, the credit calculation method available in the initial state is as follows:
[0050] total _ credit = host-side data read bandwidth latency × line speed × shortest packet header processing cycle.
[0051] The logic of the credit management module is as follows: Figure 4 As shown, the system first determines if credit is currently available. If no credit is available, the credit application is rejected, and the system waits for downstream credit compensation. When credit is available, if the requested credit is less than or equal to the current available credit, the credit application is approved, and the requested credit is deducted from the current available credit. After the credit application is approved, the generated control information and physical pages are distributed to downstream modules. In a preferred embodiment, a data packet splicing module is used to splice and send the data packets.
[0052] Step 203: When the downstream module finishes processing the packet request, it compensates the corresponding packet credit to the current available credit of the credit management module.
[0053] Credit compensation logic is as follows Figure 7 As shown, after each request is processed, the credit compensation for that request is sent back to the credit management module. Specifically, it is determined whether the currently processed request is a memory access request. If it is a memory access request, it is determined whether the current DMA data is abnormal. If normal, credit compensation is performed based on the packet credit allocated upstream, and the remaining credit for that request is maintained. If abnormal, credit compensation is performed based on the remaining credit for that request. If it is not a memory access request, compensation is performed directly based on the credit allocated upstream.
[0054] As can be seen, the packet optimization method based on pre-allocated credit dynamic flow control proposed in this application has the following advantages compared with related technologies: it saves DMA cache resources, improves the utilization efficiency of the packet sending engine for PCIe, refines the request data volume according to packet granularity, and improves packet sending performance while reducing resource overhead. The scheme in this application adopts a software-configurable total credit and an independent credit allocation method based on request type, which is more software-friendly. This scheme can quickly replenish credit in abnormal scenarios, optimizing the processing performance of the packet sending engine. This scheme statistically analyzes the time period consumed by each request at the packet sending egress point, and then feeds back the credit suggestion value for each request to the software, improving the rationality of credit configuration. Taking a PCIe latency of 2000ns, PCIe bandwidth of 400Gbps, line speed of 100Mpps, minimum packet length of 64B, and a non-data processing cycle of 4T for each packet sent by the data packet egress module as an example. Experiments show that in traditional methods, the amount of data that needs to be cached every 2000ns is approximately 2000ns × 100Mpps × 4T × 64B ≈ 800 × 64B. Due to the cumulative effect, this amount of data will continue to accumulate, and no matter how large the buffer is, backpressure on the PCIe cannot be avoided. After applying the scheme in this application, the amount of DMA data read every 2000ns is controlled to be 2000ns × 100Mpps × 64B + 64 × 64 (ensuring 100Mpps), and the cache data depth is set to 64. This saves up to 90% of DMA cache resources.
[0055] Accordingly, in a second aspect, this application provides a packet optimization apparatus based on pre-allocated credit dynamic flow control, comprising:
[0056] The credit application unit is used to apply for a packet credit from the credit management module based on the physical page information carried in the packet request when the packet engine receives a packet request. If the packet request is a DMA memory access request, the unit applies for a packet credit from the credit management module based on the software configuration information.
[0057] The packet sending unit is used to allocate the packet sending credit to the downstream module and send the packet sending request to the downstream module when the credit management module determines that the packet sending credit of the DMA memory access request is not greater than the current available credit, and at the same time update the current available credit.
[0058] The credit compensation unit is used to compensate the corresponding packet sending credit to the current available credit of the credit management module when the downstream module finishes processing the packet sending request.
[0059] The above-mentioned device can be implemented by the packet sending optimization method based on pre-occupied credit dynamic flow control provided in the first aspect of the embodiment. For the specific implementation method, please refer to the description in the first aspect of the embodiment, which will not be repeated here.
[0060] It is understood that the module structures, names, and parameters described in the above embodiments are merely examples. Those skilled in the art can also make readily conceived combinations and adjustments to the structural features of the above embodiments according to their needs, and the concept of this application should not be limited to the specific details of the above examples.
[0061] This application also provides an electronic device, including a processor and a memory, wherein the memory stores multiple instructions, and the processor is used to read the instructions and execute any of the methods described in the first aspect above. The processor and memory can be connected via a bus or other means, taking a bus connection as an example. The processor can be a Central Processing Unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above-mentioned types of chips.
[0062] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the methods in the above-described method embodiments.
[0063] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0064] This application also provides an electronic device including a processor and a memory connected to the processor, the memory storing a plurality of instructions that can be loaded and executed by the processor to enable the processor to perform any of the methods described in the first aspect above. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.
[0065] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A method for packet sending optimization based on pre-occupancy credit dynamic flow control, characterized in that, The application comprises the following steps: When the packet sending engine receives a packet sending request, if the packet sending request is a DMA access request, the credit management module is applied for packet sending credit according to the physical page information carried by the packet sending request; if the packet sending request is not a DMA access request, the credit management module is applied for packet sending credit according to software configuration information; When the credit management module judges that the packet sending credit applied by the DMA access request is not greater than the current available credit, the packet sending credit is allocated to a downstream module, the packet sending request is sent to the downstream module, and the current available credit is updated; When the downstream module processes the packet sending request, the corresponding packet sending credit is compensated to the current available credit of the credit management module; The credit management module is applied for packet sending credit according to the physical page information carried by the packet sending request, and further comprises the following steps: The number of data packets carried by the physical page of the current access request is calculated according to the data length of the DMA access request, and then the number of data packets is multiplied by the software configured credit consumption to obtain the packet sending credit of the current DMA access request, and the credit management module is applied for the packet sending credit; The number of data packets is the ratio of the data length of the access request to the path maximum transmission unit (PMTU).
2. The method of claim 1, wherein, The credit management module is applied for packet sending credit according to software configuration information, and further comprises the following steps: If the packet sending request is not a DMA access request, the credit management module is applied for packet sending credit according to software configuration information, and waits for the credit of the ongoing DMA access request to pass through, and then the credit of the current non-access request is sent after all DMA commands are sent out.
3. The method of claim 1, wherein the method further comprises: The current available credit is updated, and further comprises the following steps: The applied packet sending credit is deducted from the current available credit.
4. The method of claim 1, wherein the method further comprises: The corresponding packet sending credit is compensated to the current available credit of the credit management module, and further comprises the following steps: For the processed DMA access request, it is determined whether the current DMA data is abnormal; If the current DMA data is normal, the credit compensation is performed according to the packet sending credit allocated by the upstream, and the remaining credit of the DMA access request is maintained; If the current DMA data is abnormal, the credit compensation is performed according to the remaining credit of the DMA access request.
5. A packet sending optimization device based on pre-occupancy credit dynamic flow control, characterized in that, The application comprises the following steps: The credit application unit is configured to, when the packet sending engine receives a packet sending request, if the packet sending request is a DMA access request, apply for packet sending credit to the credit management module according to the physical page information carried by the packet sending request; if the packet sending request is not a DMA access request, apply for packet sending credit to the credit management module according to software configuration information; The packet sending unit is configured to, when the credit management module judges that the packet sending credit applied by the DMA access request is not greater than the current available credit, allocate the packet sending credit to a downstream module, send the packet sending request to the downstream module, and update the current available credit; The credit compensation unit is configured to, when the downstream module processes the packet sending request, compensate the corresponding packet sending credit to the current available credit of the credit management module; The credit application unit is further configured to: According to a data length of the DMA access request, a number of data packets carried by a physical page of a current access request is calculated, and then the number of data packets is multiplied by a software-configured credit consumption to obtain packet sending credit of the current DMA access request, and the credit management module is applied for the packet sending credit; The number of data packets is a ratio of the data length of the access request to a path maximum transmission unit (PMTU).
6. The apparatus for packet sending optimization based on pre-occupancy credit dynamic flow control according to claim 5, wherein, The credit application unit is further configured to: If the packet sending request is not a DMA access request, the credit management module is applied for packet sending credit according to software configuration information, and after the credit of the ongoing DMA access request is passed and all DMA commands are sent, credit for the current non-access request is sent.
7. The apparatus for packet sending optimization based on pre-occupancy credit dynamic flow control according to claim 5, wherein, The packet sending unit is further configured to: The applied packet sending credit is deducted from the current available credit.
8. The apparatus for packet sending optimization based on pre-occupancy credit dynamic flow control according to claim 5, wherein, The credit compensation unit is further configured to: For a processed DMA access request, it is determined whether current DMA data is abnormal; If the current DMA data is normal, credit compensation is performed according to the packet sending credit allocated by an upstream, and remaining credit of the DMA access request is maintained; If the current DMA data is abnormal, credit compensation is performed according to the remaining credit of the DMA access request.
9. An electronic device, comprising: The computer readable storage medium stores a plurality of instructions, and the plurality of instructions can be read and executed by the processor to perform the packet sending optimization method based on the pre-occupied credit dynamic flow control.
10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a plurality of instructions, and the plurality of instructions can be read and executed by the processor to perform the packet sending optimization method based on the pre-occupied credit dynamic flow control.
Citation Information
Patent Citations
Transporting operations of arbitrary size over remote direct memory access
US20130007180A1