Packet rearrangement method, device, chip, equipment, storage medium and program product

Abstract

The application relates to a message rearrangement method, device, equipment, storage medium and program product. The method comprises the following steps: receiving an out-of-order message sent by a request end, wherein the out-of-order message comprises a message sequence number and a packet sequence number; determining whether the out-of-order message is in a current ordering delivery segment of a hardware processing engine of a network device according to the message sequence number and the packet sequence number; if the out-of-order message is in the current ordering delivery segment, determining an ordering position of the out-of-order message in a bitmap of a queue management context corresponding to the current ordering delivery segment according to the message sequence number and the packet sequence number; and rearranging the out-of-order message by adding mark information at the ordering position. The method can support large-range out-of-order correction of messages.

Description

TECHNICAL FIELD

[0001] The present application relates to the chip technical field, in particular to a message rearrangement method and device, chip, equipment, storage medium and program product. BACKGROUND

[0002] In emerging scenarios such as artificial intelligence and machine learning, network throughput is more important than network delay, and the number of working central processing units and image processing units is small, but the single traffic is large, so it is required to have low tail delay under single load. Since Remote Direct Memory Access over Converged Ethernet (ROCEV) 2 does not support out-of-order sending and rearrangement, ROCEV 2 cannot solve the problem of high throughput and low tail delay.

[0003] In related technologies, part of the software can complete the out-of-order rearrangement of the same queue pair (QP) of Remote Direct Memory Access (RDMA) through a processor or a coprocessor.

[0004] However, since the processor or the coprocessor needs to use the Real-Time Clock (RTC) to complete the processing of the sequence, it is impossible to use multi-threading to complete the sequence for a single QP, which leads to the inability to support large-scale out-of-order correction of messages. SUMMARY

[0005] Therefore, it is necessary to provide a message rearrangement method, device, equipment, storage medium and program product capable of supporting large-scale out-of-order correction of messages to solve the above technical problems.

[0006] In a first aspect, the present application provides a message rearrangement method. The method comprises:

[0007] receiving an out-of-order message sent by a request end, wherein the out-of-order message comprises a message sequence number and a packet sequence number;

[0008] determining whether the out-of-order message is within a current sequence delivery segment of a hardware processing engine of a network device according to the message sequence number and the packet sequence number;

[0009] if the out-of-order message is within the current sequence delivery segment, determining a sequence position of the out-of-order message in a bitmap of a queue management context corresponding to the current sequence delivery segment according to the message sequence number and the packet sequence number;

[0010] rearranging the out-of-order message by adding mark information at the sequence position.

[0011] In one embodiment, the determining whether the out-of-order packet is in the current ordered delivery segment of the hardware processing engine of the network device according to the message sequence number and the packet sequence number comprises:

[0012] If the difference between the message sequence number and the expected message sequence number is less than or equal to the maintainable message number of the hardware processing engine and the difference between the packet sequence number and the expected packet sequence number is less than or equal to the maintainable packet number of the hardware processing engine, it is determined that the out-of-order packet is in the current ordered delivery segment of the hardware processing engine.

[0013] If the difference between the message sequence number and the expected message sequence number is greater than the maintainable message number of the hardware processing engine or the difference between the packet sequence number and the expected packet sequence number is greater than the maintainable packet number of the hardware processing engine, it is determined that the out-of-order packet is not in the current ordered delivery segment of the hardware processing engine.

[0014] In one embodiment, the expected message sequence number is a message sequence number corresponding to a first ordered position in the bitmap of the queue management context, and the expected packet sequence number is a packet sequence number corresponding to the first ordered position in the bitmap of the queue management context.

[0015] In one embodiment, after the determining whether the out-of-order packet is in the current ordered delivery segment of the hardware processing engine of the network device according to the message sequence number and the packet sequence number, the method further comprises:

[0016] If the out-of-order packet is not in the current ordered delivery segment of the hardware processing engine, a first out-of-order event message is reported to a micro-engine pool, the first out-of-order event message including the out-of-order packet, and the micro-engine pool being configured to invoke host memory ordering of the out-of-order packet.

[0017] In one embodiment, after the rearranging the out-of-order packet by adding the mark information at the ordered position, the method further comprises:

[0018] moving the bitmap of the queue management context according to the added mark information in the bitmap of the queue management context;

[0019] If the bitmap of the queue management context is moved from the current ordered delivery segment to a next ordered delivery segment and the first out-of-order event message is reported, a second out-of-order event message is reported to the micro-engine pool, the second out-of-order event message being configured to request the micro-engine pool to release first ordering information of the out-of-order packet corresponding to the next ordered delivery segment.

[0020] In one of the embodiments, after the reporting the second out-of-order event message to the micro-engine pool, the method further comprises:

[0021] receiving the first ordering information sent by the micro-engine pool;

[0022] performing a bitwise OR operation between the first ordering information and second ordering information in the bitmap of the queue management context corresponding to the next ordering delivery segment to update the bitmap of the queue management context corresponding to the next ordering delivery segment.

[0023] In a second aspect, the present application provides a packet rearrangement device. The device comprises:

[0024] a receiving module configured to receive an out-of-order packet sent by a requesting end, wherein the out-of-order packet comprises a message sequence number and a packet sequence number;

[0025] a processing module configured to determine whether the out-of-order packet is within a current ordering delivery segment of a hardware processing engine of a network device according to the message sequence number and the packet sequence number, and determine an ordering position of the out-of-order packet in a bitmap of a queue management context corresponding to the current ordering delivery segment according to the message sequence number and the packet sequence number if the out-of-order packet is within the current ordering delivery segment.

[0026] a rearrangement module configured to rearrange the out-of-order packet by adding marking information at the ordering position.

[0027] In one of the embodiments, the processing module is further configured to determine that the out-of-order packet is within the current ordering delivery segment of the hardware processing engine if a difference between the message sequence number and an expected message sequence number is less than or equal to a maintainable message number of the hardware processing engine and a difference between the packet sequence number and an expected packet sequence number is less than or equal to a maintainable packet number of the hardware processing engine, and determine that the out-of-order packet is not within the current ordering delivery segment of the hardware processing engine if the difference between the message sequence number and the expected message sequence number is greater than the maintainable message number of the hardware processing engine or the difference between the packet sequence number and the expected packet sequence number is greater than the maintainable packet number of the hardware processing engine.

[0028] In one of the embodiments, the expected message sequence number is a message sequence number corresponding to a first ordering position in the bitmap of the queue management context, and the expected packet sequence number is a packet sequence number corresponding to the first ordering position in the bitmap of the queue management context.

[0029] In one of the embodiments, the processing module is further configured to report a first out-of-order event message to the micro-engine pool if the out-of-order packet is not in the current order delivery segment of the hardware processing engine, and the first out-of-order event message includes the out-of-order packet.

[0030] In one of the embodiments, the processing module is further configured to move the bitmap of the queue management context according to the added flag information in the bitmap of the queue management context, and report a second out-of-order event message to the micro-engine pool if the bitmap of the queue management context is moved from the current order delivery segment to a next order delivery segment and the first out-of-order event message has been reported, and the second out-of-order event message is used to request the micro-engine pool to release the first order information of the out-of-order packet corresponding to the next order delivery segment.

[0031] In one of the embodiments, the receiving module is further configured to receive the first order information sent by the micro-engine pool.

[0032] The processing module is further configured to perform an or operation on the first order information and second order information in the bitmap of the queue management context corresponding to the next order delivery segment bit by bit to update the bitmap of the queue management context corresponding to the next order delivery segment.

[0033] In a third aspect, a chip is provided. The chip includes a memory and a processor. The memory stores a computer program. The processor implements the packet rearrangement method of the first aspect when executing the computer program.

[0034] In a fourth aspect, a computer device is provided. The computer device includes a processor and the chip of the third aspect. The chip is configured to schedule packets to the processor or the chip itself for processing. The processor is configured to process the packets scheduled by the chip.

[0035] In a fifth aspect, a computer readable storage medium is provided. The computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the packet rearrangement method.

[0036] In a sixth aspect, a computer program product is provided. The computer program product includes a computer program. The computer program is executed by a processor to implement the packet rearrangement method.

[0037] The packet rearrangement method, device, equipment, storage medium and program product provided by the application first receive a disordered packet sent by a request end, wherein the disordered packet comprises a message sequence number and a packet sequence number. Then, whether the disordered packet is in a current ordering delivery segment of a hardware processing engine of a network device is determined according to the message sequence number and the packet sequence number. If the disordered packet is in the current ordering delivery segment, the ordering position of the disordered packet in a bit map of a queue management context corresponding to the current ordering delivery segment is determined according to the message sequence number and the packet sequence number. Finally, the disordered packet is rearranged by adding mark information at the ordering position. Since the hardware segment ordering is adopted, the hardware overhead is reduced, the packet rearrangement performance is improved, and the packet large-scale disorder correction can be supported. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 An application environment diagram of a packet rearrangement method provided by an embodiment of the application;

[0039] Figure 2 A flowchart of a packet rearrangement method provided by an embodiment of the application;

[0040] Figure 3 A diagram for realizing packet-by-packet balancing provided by an embodiment of the application;

[0041] Figure 4 A bit map corresponding to a micro-engine pool before updating provided by an embodiment of the application;

[0042] Figure 5 A bit map corresponding to a micro-engine pool after updating provided by an embodiment of the application;

[0043] Figure 6 A bit map of a queue management context provided by an embodiment of the application;

[0044] Figure 7 Another bit map of a queue management context provided by an embodiment of the application;

[0045] Figure 8 Another bit map corresponding to a micro-engine pool provided by an embodiment of the application;

[0046] Figure 9 A bit map corresponding to a next ordering delivery segment of a micro-engine pool provided by an embodiment of the application;

[0047] Figure 10 Still another bit map of a queue management context provided by an embodiment of the application;

[0048] Figure 11A bitmap of another queue management context provided in the embodiments of this application;

[0049] Figure 12 A schematic diagram illustrating the principle of message reordering provided in an embodiment of this application;

[0050] Figure 13 An interactive schematic diagram of message rearrangement provided in an embodiment of this application;

[0051] Figure 14 A flowchart illustrating another message rearrangement method provided in this application embodiment;

[0052] Figure 15 A structural diagram of a message rearrangement apparatus provided in an embodiment of this application;

[0053] Figure 16 This is an internal structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] The following section will first explain the application environment of the message rearrangement method.

[0056] The message rearrangement method provided in this application embodiment can be applied to network devices. Figure 1 This diagram illustrates the application environment of a message rearrangement method provided in an embodiment of this application. Figure 1 As shown, a network device may include a requesting end and a receiving end. A requesting end 101 on one network device can send an out-of-order message to a receiving end 102 on another network device. The out-of-order message includes a message sequence number and a packet sequence number. Then, the receiving end 102 determines whether the out-of-order message is within the current ordering delivery segment of the network device's hardware processing engine based on the message sequence number and packet sequence number. If the out-of-order message is within the current ordering delivery segment, the receiving end 102 determines the ordering position of the out-of-order message in the bitmap of the queue management context corresponding to the current ordering delivery segment based on the message sequence number and packet sequence number. Then, the receiving end 102 rearranges the out-of-order message by adding marker information to the ordering position. If the out-of-order message is not within the current ordering delivery segment of the hardware processing engine, the receiving end 102 reports a first out-of-order event message to the network device's micro-engine pool 103. The first out-of-order event message includes the out-of-order message, and the micro-engine pool is used to call the host memory to order the out-of-order message. The micro-engine pool may be a processor module of the network device. A host can be the system (e.g., operating system) of a network device.

[0057] It should be understood that the embodiments of the present application do not limit the network device, and in some embodiments, the network device can include a network interface card (NIC) and the like.

[0058] In one embodiment, as shown in Figure 2 , a packet rearrangement method is provided, which is applied to the receiving end in Figure 1 for example, including S201-S204:

[0059] S201, receiving the out-of-order packet sent by the request end, the out-of-order packet including a message sequence number and a packet sequence number.

[0060] In the present application, if the request end (Requester) supports sending out-of-order packets, the request end can send out-of-order packets to the receiving end (Responder) so that the receiving end can rearrange the out-of-order packets.

[0061] Wherein, the message sequence number (MSN) and the packet sequence number (PSN) are both descriptors of the out-of-order packet.

[0062] It should be understood that in RDMA, there is a concept of message (MESSAGE), and a MESSAGE can include one or more packet (Packet, PKT). Accordingly, the request end can attach a message sequence number (Message Sequence Number, MSN) and a packet sequence number (Packet Sequence Number, PSN) when sending out-of-order packets, so that the receiving end can sort the out-of-order packets based on the MSN and PSN.

[0063] In some embodiments, the out-of-order packet is formed when the network card or the switch exchanges the source packet to multiple transmission paths through dynamic load balancing.

[0064] It should be understood that dynamic load balancing is a core network solution for distributing traffic among multiple servers in a server farm. Load balancing can improve the availability and responsiveness of an application and prevent server overload. The load balancing service is located between the client device and the backend server, receives incoming requests, and then distributes them to any available server that can meet these requests.

[0065] For example, a network server usually has multiple workers (e.g., processes or threads). If multiple clients are connected to the same worker, the worker will become busy and cause huge tail latency, while other workers will run in an idle state. In order to reduce the overall utilization of resources and the performance requirements of the network server, dynamic load balancing can be used to avoid multiple clients connecting to the same worker.

[0066] Figure 3 An implementation of per-packet balancing provided by an embodiment of the present application is shown. As shown in the figure, Figure 3 In the present application, the source message can be switched to multiple transmission paths by the network card or switch through dynamic load balancing, which reduces the tail latency of single flow and greatly shortens the single flow transmission completion time.

[0067] For example, when dynamic load balancing through a switch is used to implement per-packet balancing, the transmission path (PATH) of the out-of-order sending can be determined by the switch through a key HASH factor. The key HASH factor can include the source IP address, the source IP address + destination IP address, the source IP address + destination IP address + source port number + destination port number, etc.

[0068] For example, when dynamic load balancing through a network card is used to implement per-packet balancing, the source port number (e.g., UDP.SPORT) of the User Datagram Protocol (UDP) can be modified, and the low 16 bits of the packet sequence number (PSN) can be used as the source port number of the UDP, so that the source message is switched to different PATHs through the modified source port number of the UDP.

[0069] In the present application, based on the per-packet distribution and ordering, the network card or switch can better use various dynamic load balancing algorithms for different types of traffic, and avoid the influence of unbalanced single large flow or multi-flow equal-cost multi-path routing (ECMP) in the network.

[0070] S202, according to the message sequence number and the packet sequence number, determine whether the out-of-order message is in the current ordering delivery segment of the hardware processing engine of the network device.

[0071] In this step, when the receiving end receives the out-of-order message sent by the requesting end, the message sequence number and the packet sequence number can be used to determine whether the out-of-order message is in the current ordering delivery segment of the hardware processing engine of the network device.

[0072] The hardware processing engine can be an RDMA hardware processing engine (RPE).

[0073] The current ordering delivery segment is a packet sequence currently being processed for ordering.

[0074] In some embodiments, the alignment of the out-of-order packets corresponding to the next ordering delivery segment is performed after the alignment of the out-of-order packets corresponding to the current ordering delivery segment. That is, at each ordering delivery segment, the hardware processing engine only maintains the out-of-order packets expected to be received.

[0075] It should be understood that the embodiments of the present application do not limit how to determine whether the out-of-order packet is within the current ordering delivery segment of the hardware processing engine of the network device. In some embodiments, if the difference between the message sequence number and the expected message sequence number is less than or equal to the number of maintainable messages of the hardware processing engine and the difference between the packet sequence number and the expected packet sequence number is less than or equal to the number of maintainable packets of the hardware processing engine, it is determined that the out-of-order packet is within the current ordering delivery segment of the hardware processing engine.

[0076] For example, if MSN-Expected Message Sequence Number (EPSN)≤X and PSN-Expected Packet Sequence Number (EPSN)≤n, the receiving end can determine that the out-of-order packet is within the current ordering delivery segment of the hardware processing engine. Correspondingly, the ordering position of the out-of-order packet can be determined in the bitmap of the queue pair context (QPC) corresponding to the current ordering delivery segment. X is the number of maintainable messages of the hardware processing engine, and n is the number of maintainable packets of the hardware processing engine.

[0077] In other embodiments, if the difference between the message sequence number and the expected message sequence number is greater than the number of maintainable messages of the hardware processing engine or the difference between the packet sequence number and the expected packet sequence number is greater than the number of maintainable packets of the hardware processing engine, it is determined that the out-of-order packet is not within the current ordering delivery segment of the hardware processing engine.

[0078] For example, if MSN-EMSN>X or PSN-EPSN>n, the receiving end can determine that the out-of-order packet is not within the current ordering delivery segment of the hardware processing engine. X is the number of maintainable messages of the hardware processing engine, and n is the number of maintainable packets of the hardware processing engine.

[0079] It should be understood that the embodiments of the present application do not limit how the expected message sequence number and the expected packet sequence number are determined. In some embodiments, the expected message sequence number is the message sequence number corresponding to the first ordering position in the bitmap of the queue management context, and the expected packet sequence number is the packet sequence number corresponding to the first ordering position in the bitmap of the queue management context.

[0080] It should be understood that the embodiments of the present application also do not limit the maintainable message number of the hardware processing engine and the maintainable packet number of the hardware processing engine. The maintainable packet number of the hardware processing engine can be specifically set according to hardware resources and / or system Round-Trip Time (RTT).

[0081] For example, 64 or 128 or 256 can be selected as the maintainable packet number of the hardware processing engine based on hardware resources and system RTT. For example, 2 or 4 or 8 or 16 can be selected as the maintainable message number of the hardware processing engine based on hardware resources. For example, the ordering delivery segment can be further divided into sub-segments, for example, an ordering delivery segment with a length of n can be divided into m sub-segments, and m is 4 or 8.

[0082] S203, if the out-of-order packet is in the current ordering delivery segment, the ordering position of the out-of-order packet in the bitmap of the queue management context corresponding to the current ordering delivery segment is determined according to the message sequence number and the packet sequence number.

[0083] In this step, when it is determined that the out-of-order packet is in the current ordering delivery segment, the ordering position of the out-of-order packet in the bitmap of the queue management context corresponding to the current ordering delivery segment is determined according to the message sequence number and the packet sequence number.

[0084] In some embodiments, the ordering position of the out-of-order packet can be determined by first locating the position of the corresponding message in the bitmap based on the message sequence number, and then locating the position of the corresponding packet in the bitmap based on the packet sequence number.

[0085] For example, if the current ordering delivery segment is MSN 0 and iPSN 1 to 15. The out-of-order packets with MSN 0 and iPSN 1 to 15 do not need to be sent to the micro-engine pool for processing, but only need to determine the corresponding bit in the bitmap of the QPC, and mark the corresponding bit position 1 to indicate that the out-of-order packet has been received and sorted.

[0086] In other embodiments, if the out-of-order packet is not in the current ordering delivery segment of the hardware processing engine, a first out-of-order event message is reported to the micro-engine pool, the first out-of-order event message includes the out-of-order packet, and the micro-engine pool is used to call the host memory to sort the out-of-order packet.

[0087] In some embodiments, a bitmap table can also be maintained in the micro-engine pool, and when the micro-engine pool receives the first out-of-order event message, the micro-engine pool can determine the ordering position of the out-of-order packet in the first out-of-order event message in the bitmap table corresponding to the micro-engine pool, and then set the bit position of the ordering position to 1 to complete the ordering.

[0088] For example, if the current ordering delivery segment is MSN corresponding to 0 and iPSN is 1 to 15, when the received out-of-order packet has MSN equal to 0 and iPSN greater than 16, or has MSN not equal to 0, the hardware processing engine cannot perform ordering. At this time, the hardware processing engine can send a first out-of-order event message to the micro-engine pool to make the micro-engine pool order the out-of-order packet.

[0089] The packet rearrangement method provided by the embodiments of the present application can realize the soft and hard combined reordering scheme of the micro-engine pool and the hardware processing engine through the bitmap table maintained by the micro-engine pool. Compared with the related art, the soft and hard combined reordering scheme is used for reordering, which not only provides the ordering processing capability of the line-speed hardware processing engine, but also provides the flexible ordering expansion capability based on the microcode of the micro-engine pool, so that the resources and costs of the hardware can be effectively controlled.

[0090] In addition, compared with the existing scheme, the hardware processing engine at the reordering end aggregates the received out-of-order packets, and after receiving the complete out-of-order packets in the current ordering delivery segment, interacts with the micro-engine pool once. The micro-engine pool delivers the first ordering information of the out-of-order packet corresponding to the next ordering delivery segment recorded by the micro-engine pool to the hardware processing engine before the ordering degree of the hardware processing engine exceeds the ordering degree, so that the hardware processing engine and the second ordering information in the bitmap of the queue management context corresponding to the next ordering delivery segment perform bitwise OR operation, so as to reorder the out-of-order packet corresponding to the next ordering delivery segment, thereby speeding up the ordering speed of the out-of-order packet.

[0091] Figure 4 A bitmap before updating corresponding to the micro-engine pool provided by the embodiments of the present application, Figure 5 A bitmap after updating corresponding to the micro-engine pool provided by the embodiments of the present application. As shown in Figure 4 and 5 If the current ordering delivery segment is MSN corresponding to 0 and iPSN is 1 to 15, the out-of-order packets of MSN corresponding to 0 and iPSN corresponding to 17, MSN corresponding to 0 and iPSN corresponding to 32, and MSN corresponding to 1 and iPSN corresponding to 0 are all not in the current ordering delivery segment, and corresponding first out-of-order event messages can be generated and reported to the micro-engine pool for ordering update in the bitmap table corresponding to the micro-engine pool, so that the bitmap table corresponding to the micro-engine pool is updated to Figure 4 as shown in Figure 5The micro-engine pool corresponds to a bitmap table.

[0092] For example, for out-of-order packets exceeding the expected receiving, the micro-engine pool can be used to sort the out-of-order packets, and the maximum bitmap that the micro-engine pool can maintain is z. Wherein, msn-emsn < y.

[0093] The bitmap that the engine pool can maintain is a system double data rate (DDR) memory. z is usually greater than n, for example, 256, 384, etc. y is the total number of messages that the memory can maintain, for example, 8, 16, 32.

[0094] For example, if msn-emsn ≥ y or psn-epsn ≥ z, the out-of-order packet exceeds the processing capacity of the system, and can be directly discarded, and then recovered through a related protocol (for example, go-back-n protocol).

[0095] In some embodiments, after rearranging the out-of-order packet by adding the mark information at the sorting position, the bitmap of the queue management context can also be moved according to the mark information that has been added in the bitmap of the queue management context.

[0096] For example, the bitmap of the queue management context can be moved according to the length of the sub-segment in the current sorting delivery segment. If the length of the current sorting delivery segment is 16, the current sorting delivery segment can be divided into 4 sub-segments, and each sub-segment corresponds to 4 bits. When the 4 bits starting from the 1st bit are all marked by setting 1, the bitmap of the queue management context can be moved 4 bits.

[0097] For example, Figure 6 A bitmap of a queue management context provided by an embodiment of the present application, Figure 7 Another bitmap of a queue management context provided by an embodiment of the present application. When the hardware processing engine receives an out-of-order packet with MSN 0 and iPSN 0, the bitmap of the queue management context is as shown in the following figure: Figure 6 At this time, the bit position of MSN 0 and iPSN 0 in the bitmap of the queue management context can be set to 1, so that the 4 bits starting from the 1st bit are all set to 1. At this time, the bitmap of the queue management context can be moved to the left by 4 as shown in the following figure: Figure 7

[0098] ​It should be noted that, due to the moving of the bitmap of the queue management context, the message sequence number and the packet sequence number corresponding to the first ordering position of the bitmap of the queue management context also change. Correspondingly, the out-of-order packet originally not in the current ordering delivery segment may change to be in the current ordering delivery segment.

[0099] For example, after the message sequence number and the packet sequence number corresponding to the first ordering position are changed from MSN being 0 and iPSN being 0 to MSN being 0 and iPSN being 4, the out-of-order packets with MSN being 0 and iPSN being 16-19 change from not being in the current ordering delivery segment to being in the current ordering delivery segment.

[0100] In some embodiments, if the bitmap of the queue management context is moved from the current ordering delivery segment to the next ordering delivery segment and the first out-of-order event message is reported, a second out-of-order event message is reported to the micro-engine pool, and the second out-of-order event message is used to request the micro-engine pool to release the first ordering information of the out-of-order packet corresponding to the next ordering delivery segment.

[0101] For example, if the current ordering delivery segment corresponds to the out-of-order packets with MSN being 0 and iPSN being 1-15, when the current ordering delivery segment is moved to MSN being 0 and iPSN being 1-16, the next ordering delivery segment corresponding to the out-of-order packets with MSN being 0 and iPSN being 16-31 can be entered. At this time, if the bitmap corresponding to the micro-engine pool is as shown in Figure 8 , the bitmap of the next ordering delivery segment corresponding to MSN being 0 and iPSN being 16-31 as shown in Figure 9 may be sent to the hardware processing engine.

[0102] In other embodiments, after the second out-of-order event message is reported to the micro-engine pool, the first ordering information sent by the micro-engine pool can be received. Then, the first ordering information and the second ordering information in the bitmap of the queue management context corresponding to the next ordering delivery segment are operated by bitwise OR operation to update the bitmap of the queue management context corresponding to the next ordering delivery segment.

[0103] For example, if the micro-engine pool sends the bitmap of the next ordering delivery segment corresponding to MSN being 0 and iPSN being 16-31 as shown in Figure 9 to the hardware processing engine, the hardware processing engine can perform bitwise OR operation on the bitmap of the queue management context as shown in Figure 10 and the bitmap sent by the micro-engine pool to obtain the bitmap of the queue management context as shown in Figure 11 .

[0104] Figure 12 A principle diagram of packet rearrangement provided by the embodiments of the present application is as shown in Figure 12As shown, the packet rearrangement method provided by the embodiments of the present application greatly reduces the number of packets sent to the micro-engine pool for processing by using hardware segment sorting, and reduces the hardware overhead by using the memory of the system of the micro-engine pool, thereby achieving the ability to improve system performance while achieving the ability to implement system line-speed sorting.

[0105] S204, rearranging the out-of-order packets by adding mark information at the sorting position.

[0106] In this step, after determining the sorting position of the out-of-order packets, the out-of-order packets can be rearranged by adding mark information at the sorting position.

[0107] It should be understood that, when rearranging the out-of-order packets, the data of the out-of-order packets can be directly written to the cache of the host through the address written by the packet header, and only the descriptors of the packets need to be sorted. That is, the sorting position corresponding to the descriptor is added with mark information in the bitmap, and the mark information can be, for example, 1 at the bit position of the sorting position.

[0108] Continuing to refer to Figure 12 When the hardware processing engine completes the sorting of the out-of-order packets, the hardware processing engine can return the confirmation information to the request end according to the maintenance information. In addition, the receiving end can report the completion queue entry (CQE) to the host based on the operation code.

[0109] The interaction events in the packet rearrangement process will be described below. Figure 13 An interaction diagram of the packet rearrangement provided by the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, Figure 13 After the request end sends the out-of-order packets to the receiving end, if the out-of-order packets are not in the current sorting delivery segment, the hardware processing engine can report a first out-of-order event message to the micro-engine pool. If the first out-of-order event message has been reported and the bitmap of the queue management context moves to the next sorting delivery segment, the hardware processing engine can report a second out-of-order event message to the micro-engine pool. The formats of the first and second out-of-order event messages can be as shown in Table 1.

[0110] Table 1

[0111]

[0112] Subsequently, the micro-engine can send a response message after receiving the second out-of-order event message, and the first sorting information of the out-of-order packets corresponding to the next sorting delivery segment can be released to the hardware processing engine in the response message to synchronize the bitmap corresponding to the micro-engine. The format of the response message can be as shown in Table 2.

[0113] Table 2

[0114]

[0115] In the present application, based on the hardware processing engine sorting ability, the fastest can support one sorting for each sorting delivery segment processing, the performance is far more than the sorting based on the processor core type, meets the sorting ability requirement of various data centers at present and in the future, provides the effective bandwidth of the whole data center. Meanwhile, based on the sorting scheme of hardware processing engine and micro engine pool, the large range out-of-order correction ability is provided, and a certain flexibility is provided, and the cost of the whole reordering scheme is reduced.

[0116] The packet reordering method provided by the embodiment of the present application first receives the out-of-order packet sent by the request end, and the out-of-order packet includes the message sequence number and the packet sequence number. Then, according to the message sequence number and the packet sequence number, it is determined whether the out-of-order packet is in the current sorting delivery segment of the hardware processing engine of the network device. If the out-of-order packet is in the current sorting delivery segment, then according to the message sequence number and the packet sequence number, the sorting position of the out-of-order packet in the bit map of the queue management context corresponding to the current sorting delivery segment is determined. Finally, the out-of-order packet is reordered by adding the marking information at the sorting position. Since the hardware segmented sorting is adopted, the number of packets sent to the micro processing pool for processing is greatly reduced, and the hardware overhead is also reduced, so that the packet reordering performance is improved while the packet reordering realizability is achieved, thereby supporting the large range out-of-order correction of the packet.

[0117] Figure 14 The flowchart of another packet reordering method provided by the embodiment of the present application is shown in FIG. 3, which includes S301-S310. Figure 14

[0118] S301, receiving the out-of-order packet sent by the request end, and the out-of-order packet includes the message sequence number and the packet sequence number.

[0119] In some embodiments, the out-of-order packet is formed when the source packet is exchanged to multiple transmission paths by dynamic load balancing of the network card or the switch.

[0120] S302, according to the message sequence number and the packet sequence number, it is determined whether the out-of-order packet is in the current sorting delivery segment of the hardware processing engine of the network device.

[0121] If yes, S304 is executed, and if no, S303 is executed.

[0122] In some embodiments, if the difference between the message sequence number and the expected message sequence number is less than or equal to the maintainable message number of the hardware processing engine, and the difference between the packet sequence number and the expected packet sequence number is less than or equal to the maintainable packet number of the hardware processing engine, it is determined that the out-of-order packet is in the current sorting delivery segment of the hardware processing engine.

[0123] ​In some embodiments, if the difference between the message sequence number and the expected message sequence number is greater than the maintainable message number of the hardware processing engine or the difference between the packet sequence number and the expected packet sequence number is greater than the maintainable packet number of the hardware processing engine, it is determined that the out-of-order packet is not in the current ordering delivery segment of the hardware processing engine.

[0124] S303, report the first out-of-order event message to the micro-engine pool, the first out-of-order event message including the out-of-order packet, and the micro-engine pool being used to call the host memory to order the out-of-order packet.

[0125] S304, determine the ordering position of the out-of-order packet in the bitmap of the queue management context corresponding to the current ordering delivery segment according to the message sequence number and the packet sequence number.

[0126] S305, reorder the out-of-order packet by adding mark information at the ordering position.

[0127] S306, move the bitmap of the queue management context according to the mark information that has been added in the bitmap of the queue management context.

[0128] S307, determine whether the bitmap of the queue management context has been moved from the current ordering delivery segment to the next ordering delivery segment and the first out-of-order event message has been reported.

[0129] If yes, perform S301, and if no, perform S308.

[0130] S308, report the second out-of-order event message to the micro-engine pool, the second out-of-order event message being used to request the micro-engine pool to release the first ordering information of the out-of-order packet corresponding to the next ordering delivery segment.

[0131] S309, receive the first ordering information sent by the micro-engine pool.

[0132] S310, perform bitwise OR operation between the first ordering information and the second ordering information in the bitmap of the queue management context corresponding to the next ordering delivery segment to update the bitmap of the queue management context corresponding to the next ordering delivery segment.

[0133] The message rearrangement method provided by the embodiments of the present application first receives the out-of-order message sent by the request end, and the out-of-order message includes a message sequence number and a packet sequence number. Then, whether the out-of-order message is in the current ordering delivery segment of the hardware processing engine of the network device is determined according to the message sequence number and the packet sequence number. If the out-of-order message is in the current ordering delivery segment, the ordering position of the out-of-order message in the bitmap of the queue management context corresponding to the current ordering delivery segment is determined according to the message sequence number and the packet sequence number. Finally, the out-of-order message is rearranged by adding mark information at the ordering position. Since the hardware segmented ordering is adopted, the number of messages sent to the micro processing pool for processing is greatly reduced, and the hardware overhead is also reduced, so that the realizability of message rearrangement is achieved while the performance of message rearrangement is improved, thereby supporting large-scale out-of-order correction of messages.

[0134] It should be understood that, although each step in the flowchart involved in the above embodiments is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0135] Based on the same inventive concept, the embodiments of the present application also provide a message rearrangement device for implementing the above-mentioned message rearrangement method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more message rearrangement device embodiments provided below can refer to the limitations of the message rearrangement method in the above text, and will not be repeated here.

[0136] In one embodiment, as shown in Figure 15 A message rearrangement device 400 is provided, which includes a receiving module 401, a processing module 402 and a rearrangement module 403.

[0137] The receiving module 401 is configured to receive the out-of-order message sent by the request end, and the out-of-order message includes a message sequence number and a packet sequence number.

[0138] The processing module 402 is configured to determine, according to the message sequence number and the packet sequence number, whether the out-of-order packet is in a current ordering delivery segment of a hardware processing engine of the network device; and if the out-of-order packet is in the current ordering delivery segment, determine the ordering position of the out-of-order packet in a bitmap of a queue management context corresponding to the current ordering delivery segment according to the message sequence number and the packet sequence number.

[0139] The rearranging module 403 is configured to rearrange the out-of-order packet by adding the marking information at the ordering position.

[0140] In one of the embodiments, the processing module 402 is further configured to determine that the out-of-order packet is in the current ordering delivery segment of the hardware processing engine if a difference between the message sequence number and an expected message sequence number is less than or equal to a maintainable message number of the hardware processing engine and a difference between the packet sequence number and an expected packet sequence number is less than or equal to a maintainable packet number of the hardware processing engine; and determine that the out-of-order packet is not in the current ordering delivery segment of the hardware processing engine if the difference between the message sequence number and the expected message sequence number is greater than the maintainable message number of the hardware processing engine or the difference between the packet sequence number and the expected packet sequence number is greater than the maintainable packet number of the hardware processing engine.

[0141] In one of the embodiments, the expected message sequence number is a message sequence number corresponding to a first ordering position in the bitmap of the queue management context, and the expected packet sequence number is a packet sequence number corresponding to the first ordering position in the bitmap of the queue management context.

[0142] In one of the embodiments, the processing module 402 is further configured to report, to a micro-engine pool, a first out-of-order event message if the out-of-order packet is not in the current ordering delivery segment of the hardware processing engine, the first out-of-order event message including the out-of-order packet, and the micro-engine pool being configured to call a host memory to reorder the out-of-order packet.

[0143] In one of the embodiments, the processing module 402 is further configured to move the bitmap of the queue management context according to the marking information that has been added in the bitmap of the queue management context; and report, to the micro-engine pool, a second out-of-order event message if the bitmap of the queue management context is moved from the current ordering delivery segment to a next ordering delivery segment and the first out-of-order event message has been reported, the second out-of-order event message being configured to request the micro-engine pool to release first ordering information of the out-of-order packet corresponding to the next ordering delivery segment.

[0144] In one of the embodiments, the receiving module 401 is further configured to receive the first ordering information sent by the micro-engine pool.

[0145] The processing module 402 is further configured to perform a bitwise OR operation on the first ordering information and second ordering information in the bitmap of the queue management context corresponding to the next ordering delivery segment, to update the bitmap of the queue management context corresponding to the next ordering delivery segment.

[0146] The modules in the packet rearrangement apparatus can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so as to be invoked by the processor to perform the operations corresponding to the modules.

[0147] In an embodiment, a computer device is provided, which can be a server. The server is plugged with a network interface card, and the network interface card includes a chip. An internal structure diagram of the server can be as shown in Figure 16 The computer device includes a processor, a memory, an input / output interface, and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. Those skilled in the art can understand that the structure shown in Figure 16 The structure shown in the above embodiment is only a block diagram of part of the structure related to the scheme of the present application, and does not limit the computer device to which the scheme of the present application is applied. Specifically, the computer device can include more or fewer components than those shown in the diagram, or combine certain components, or have a different arrangement of components.

[0148] In an embodiment, a chip is provided. The chip includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the packet rearrangement method. The chip can be a data processing unit (DPU) chip.

[0149] In an embodiment, a computer device is provided. The computer device includes a processor and the chip provided in the foregoing embodiment. The chip is configured to schedule packets to the processor or the chip itself to process the packets. The processor is configured to process the packets scheduled by the chip. The computer device can be a server, a server cluster, or the like.

[0150] In an embodiment, a computer readable storage medium is provided. The computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the packet rearrangement method.

[0151] In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the packet rearrangement method described above.

[0152] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0153] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present disclosure.

[0154] The above embodiments only express several implementation ways of the present application, and the description is specific and detailed, but it should not be understood as a limitation to the patent scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A message rearrangement method, characterized in that, The method includes: Receive out-of-order messages sent by the requesting end, wherein the out-of-order messages include message sequence number and packet sequence number; Based on the message sequence number and the packet sequence number, determine whether the out-of-order message is within the current sorting delivery segment of the network device's hardware processing engine; If the out-of-order message is within the current order delivery segment, the order position of the out-of-order message is determined in the bitmap of the queue management context corresponding to the current order delivery segment based on the message sequence number and packet sequence number. The out-of-order messages are rearranged by adding marker information at the sorting positions.

2. The method according to claim 1, characterized in that, The step of determining whether the out-of-order packet is within the current ordering delivery segment of the network device's hardware processing engine based on the message sequence number and packet sequence number includes: If the difference between the message sequence number and the expected message sequence number is less than or equal to the number of maintainable messages of the hardware processing engine and the difference between the packet sequence number and the expected packet sequence number is less than or equal to the number of maintainable messages of the hardware processing engine, then the out-of-order message is determined to be within the current sorting delivery segment of the hardware processing engine. If the difference between the message sequence number and the expected message sequence number is greater than the number of messages that the hardware processing engine can maintain, or if the difference between the packet sequence number and the expected packet sequence number is greater than the number of messages that the hardware processing engine can maintain, then it is determined that the out-of-order message is not in the current sorting delivery segment of the hardware processing engine.

3. The method according to claim 2, characterized in that, The expected message sequence number is the message sequence number corresponding to the first sorting position in the bitmap of the queue management context, and the expected packet sequence number is the packet sequence number corresponding to the first sorting position in the bitmap of the queue management context.

4. The method according to claim 1, characterized in that, After determining whether the out-of-order packet is within the current ordering delivery segment of the network device's hardware processing engine based on the message sequence number and packet sequence number, the method further includes: If the out-of-order message is not within the current sorting delivery segment of the hardware processing engine, a first out-of-order event message is reported to the micro-engine pool. The first out-of-order event message includes the out-of-order message, and the micro-engine pool is used to call the host memory to sort the out-of-order message.

5. The method according to claim 4, characterized in that, After rearranging the out-of-order packets by adding marker information at the sorting position, the method further includes: The bitmap of the queue management context is moved according to the added marker information in the bitmap of the queue management context; If the bitmap of the queue management context moves from the current sorting delivery segment to the next sorting delivery segment and has reported the first out-of-order event message, then a second out-of-order event message is reported to the micro-engine pool. The second out-of-order event message is used to request the micro-engine pool to release the first sorting information of the out-of-order packets corresponding to the next sorting delivery segment.

6. The method according to claim 5, characterized in that, After reporting the second out-of-order event message to the micro-engine pool, the method further includes: Receive the first sorting information sent by the micro-engine pool; The first sorting information is bitwise ORed with the second sorting information in the bitmap of the queue management context corresponding to the next sorting delivery segment to update the bitmap of the queue management context corresponding to the next sorting delivery segment.

7. A message rearrangement apparatus, characterized in that, The device includes: The receiving module is used to receive out-of-order messages sent by the requesting end, wherein the out-of-order messages include message sequence numbers and packet sequence numbers; The processing module is configured to determine whether the out-of-order packet is within the current sorting and delivery segment of the hardware processing engine of the network device based on the message sequence number and the packet sequence number; if the out-of-order packet is within the current sorting and delivery segment, the module determines the sorting position of the out-of-order packet in the bitmap of the queue management context corresponding to the current sorting and delivery segment based on the message sequence number and the packet sequence number. The rearrangement module is used to rearrange the out-of-order packets by adding marker information at the sorting positions.

8. A chip, characterized in that, The method includes a memory and a processor, the memory storing a computer program, characterized in that the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer device comprising a processor and the chip of claim 8, characterized in that, The chip is used to schedule messages to the processor or to process them on its own, and the processor is used to process messages scheduled by the chip.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images