Data sending and data receiving methods, and apparatus, device and storage medium

Abstract

The present application belongs to the technical field of communications. Provided are data sending and data receiving methods, and an apparatus, a device and a storage medium. The method comprises: sending a first data packet group on a first egress port of a sending device, wherein the first data packet group belongs to a first data stream, and the first data packet group comprises a plurality of first data packets; determining a second egress port of the first data stream on the sending device, wherein the first egress port is different from the second egress port; determining a second data packet group on the basis of packet group granularity information, wherein the packet group granularity information is used for dividing the first data stream into a plurality of packet groups, the second data packet group belongs to the first data stream, and the second data packet group comprises a plurality of second data packets; and sending the second data packet group on the second egress port. The use of the solution of the present application can improve the load balancing effect.

Description

Methods, apparatus, devices and storage media for transmitting and receiving data

[0001] This application claims priority to Chinese patent application filed on July 19, 2024, with application number 202410980609.3, entitled "Method, Apparatus, Device and Storage Medium for Data Transmission and Data Reception", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of network communication technology, and in particular to a method, apparatus, device and storage medium for data transmission and data reception. Background Technology

[0003] Load balancing is crucial in networks to prevent congestion and packet loss. Currently, sending devices often have multiple ports transmitting data packets to receiving devices. To ensure load balancing across these ports, equal cost multiple path (ECMP) load balancing is employed. The scheme involves the sending device calculating the hash value of the 5-tuple of a data packet, determining the output port based on this hash value, and then using that port to send the data packet. This ensures that all data packets with the same 5-tuple are forwarded along the same path, while data packets with different 5-tuples are forwarded through multiple paths in the network before being received by the receiving device.

[0004] In a network, the number of data packets in different 5-tuples may be different. The sending device cannot evenly distribute the received data packets to the sending device's ports, resulting in unbalanced link load. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for data transmission and reception, which can improve the link load balancing effect. The technical solution adopted is as follows:

[0006] In a first aspect, this application provides a data transmission method applied to a transmitting device. The method includes: transmitting a first data packet group on a first output port of the transmitting device, the first data packet group belonging to a first data stream and including multiple first data packets; determining a second output port of the first data stream on the transmitting device, the first output port being different from the second output port; determining a second data packet group based on packet group granularity information, the packet group granularity information being used to divide the first data stream into multiple packet groups, the second data packet group belonging to the first data stream and including multiple second data packets; and transmitting the second data packet group on the second output port.

[0007] In the scheme shown in this application, when the transmitting device sends a data stream, it re-determines the output port of the data stream after sending each data packet group. The re-determined output port is different from the previously determined output port. Furthermore, it uses packet group granularity information to determine the next data packet group to be sent and uses the re-determined output port to send that data packet group. In this way, data packets in a data stream can be sent using multiple output ports instead of using a single port, thus improving the load balancing effect.

[0008] In one alternative approach, the number of first data packets included in the first data packet group is such that the outgoing port of the transmission traffic is switched to prevent short-circuit switching; the number of first data packets included in the first data packet group is greater than the number of second data packets included in the second data packet group; the method further includes: determining a third outgoing port of the first data stream on the transmitting device, the third outgoing port being different from both the first and second outgoing ports; determining a third data packet group based on packet group granularity information, the third data packet group belonging to the first data stream, the third data packet group including multiple third data packets; and transmitting the third data packet group on the third outgoing port.

[0009] In the scheme shown in this application, when the data stream is a short stream, it is not desirable for the data stream to be divided into data packet groups, otherwise it will increase the sorting burden of the receiving device. When the data stream is a large stream, the data stream is divided into data packet groups for load balancing and transmission, which can achieve load balancing when sending each data stream and improve the load balancing effect.

[0010] In one alternative approach, before transmitting the first data packet group at the first output port of the transmitting device, the method further includes: transmitting a third data packet group at the first output port, the third data packet group belonging to the first data stream, the third data packet group comprising a plurality of third data packets.

[0011] In the scheme shown in this application, the first and second data packet groups in the data stream can be sent without switching the output port, but instead, the same output port can be used for transmission.

[0012] In one alternative approach, the number of first data packets included in the first data packet group is equal to the number of second data packets included in the second data packet group.

[0013] In one alternative approach, the number of first data packets in the first data packet group is determined based on historical transmission traffic statistics of the sending device. This method of determining the number of first data packets based on historical transmission traffic information allows for a more accurate definition of whether the data stream transmitted by the sending device is a short-lived stream.

[0014] In one alternative approach, packet granularity information includes data volume and / or transmission cycle.

[0015] In one alternative approach, the packet granularity information includes the data volume; the data volume is determined based on the bandwidth error between the output ports of the transmitting device; or, the data volume is determined based on the round-trip time (RTT) and connection bandwidth between the transmitting device and the receiving device of the first data stream; or, the data volume is determined based on the input equalization target. This provides several possible ways to determine the data volume.

[0016] In one alternative approach, before transmitting the first data packet group on the first output port of the transmitting device, the method further includes: a first congestion state sent by the receiving device receiving the first data stream, the first congestion state indicating the congestion state of the receiving device; and determining, based on the first congestion state, that the receiving device is not congested.

[0017] In the scheme shown in this application, the transmitting device can also transmit data streams based on the congestion information fed back by the receiving device, thereby improving the reliability of data stream transmission.

[0018] In an alternative approach, the method further includes: if it is determined that the receiving device is congested based on a first congestion state, then switching the mode of sending the first data stream to a target load balancing mode, wherein the data packets of the first data stream are sent in the target load balancing mode without being out of order at the receiving device.

[0019] In the scheme shown in this application, when the receiving device reports congestion, the transmitting device can adjust the load balancing strategy so that the receiving device does not need to perform order preservation processing, thereby reducing the possibility of congestion of the receiving device.

[0020] In one alternative approach, the method further includes: a second congestion state sent by a receiving device receiving the first data stream, the second congestion state indicating the congestion state of the sending device at each ingress port of the receiving device; determining a second egress port of the first data stream on the sending device, including: determining an egress port on the sending device that is not congested ingress port to the receiving device based on the second congestion state; and selecting a second egress port that is different from the first egress port from the determined egress ports.

[0021] In the scheme shown in this application, when the receiving device reports congestion at the ingress port, the sending device may not send data packets to the congested ingress port, thereby reducing the possibility of congestion at that ingress port.

[0022] In one alternative approach, multiple first data packets in a first data packet group are transmitted to the receiving device of the first data stream via the same path; and multiple second data packets in a second data packet group are transmitted to the receiving device of the first data stream via the same path.

[0023] In the scheme shown in this application, all data packets in a single data packet group are sent to the receiving device via the same path, which ensures that the data packets in a single data packet group are not out of order. The receiving device does not need to sort the data packets in the data packet group, thus saving resources.

[0024] Secondly, this application provides a data receiving method applied to a receiving device. The method includes: receiving multiple data packets of a first data stream sent by a transmitting device, the multiple data packets forming multiple data packet groups, wherein a first data packet group and a second data packet group are sent at a first output port and a second output port of the transmitting device, respectively, the first output port and the second output port being different, the first data packet group being sent before the second data packet group, the first data packet group including multiple first data packets, the second data packet group including multiple second data packets, the second data packet group being determined according to packet group granularity information, the packet group granularity information being used to divide the first data stream into multiple packet groups; determining the data packet group to which each data packet belongs, performing order-preserving processing on the data packet groups of the first data stream to obtain an order-preserving first data stream; and outputting the order-preserving first data stream.

[0025] In the scheme shown in this application, when the sending device sends data packets of the data stream according to the data packet group, the receiving device performs order preservation processing on the data packet group to ensure that the data packets in the data stream are in the correct order.

[0026] In one alternative approach, for each of the multiple packet groups, the paths taken by each packet in the packet group to the receiving device are the same.

[0027] In the scheme shown in this application, all data packets in a single data packet group are sent to the receiving device via the same path, which ensures that the data packets in a single data packet group are not out of order. The receiving device does not need to sort the data packets in the data packet group, thus saving resources.

[0028] In an alternative approach, the method further includes: generating a first congestion state of the receiving device based on the out-of-order status of the data packet groups received by the receiving device; and sending the first congestion state to the sending device.

[0029] In the scheme shown in this application, the sending device is provided with its own congestion information, which enables the sending device to adjust its load balancing strategy and improve the reliability of data transmission.

[0030] In one alternative approach, the method further includes: generating a second congestion state based on the out-of-order status of data packet groups received by the receiving device from the sending device at each input port, the second congestion state including the congestion state of the sending device at each input port; and sending the second congestion state to the sending device.

[0031] In the scheme shown in this application, the sending device is provided with congestion information of its own ingress port, so that the sending device can adjust the sending outgress port and improve the reliability of data transmission.

[0032] Thirdly, this application provides a data transmission apparatus having the functionality to implement the first aspect or any optional method of the first aspect described above. The apparatus includes at least one module for implementing the data transmission method provided by the first aspect or any optional method of the first aspect.

[0033] In some embodiments, the modules in the data transmission device are implemented in software, specifically program modules. In other embodiments, the modules in the data transmission device are implemented in hardware or firmware.

[0034] Fourthly, this application provides a data receiving apparatus having the functionality to implement the second aspect or any optional method of the second aspect described above. The apparatus includes at least one module for implementing the data receiving method provided by the second aspect or any optional method of the second aspect.

[0035] In some embodiments, the modules in the data receiving device are implemented in software, specifically program modules. In other embodiments, the modules in the data receiving device are implemented in hardware or firmware.

[0036] Fifthly, this application provides a transmission device, the security protection device including a memory and a processor connected to the memory;

[0037] The memory is used to store program instructions;

[0038] The processor is used to execute the program instructions to enable the transmitting device to implement the data transmission method provided by the first aspect or any alternative method of the first aspect.

[0039] Sixthly, this application provides a receiving device, the security protection device including a memory and a processor connected to the memory;

[0040] The memory is used to store program instructions;

[0041] The processor is configured to execute the program instructions to enable the receiving device to implement the data receiving method provided by the second aspect or any alternative method of the second aspect.

[0042] In a seventh aspect, this application provides a computer-readable storage medium storing at least one computer instruction that is read by a processor to cause a transmitting device to perform the data transmission method provided in the first aspect or any alternative method of the first aspect.

[0043] Eighthly, this application provides a computer-readable storage medium storing at least one computer instruction that is read by a processor to cause a receiving device to perform the data receiving method provided in the second aspect or any alternative method of the second aspect.

[0044] Ninthly, this application provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a transmitting device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the transmitting device to perform the data transmission method provided in the first aspect or any alternative method of the first aspect.

[0045] In a tenth aspect, this application provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a receiving device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the receiving device to perform the data receiving method provided in the second aspect or any alternative method of the second aspect.

[0046] In one aspect, this application provides a communication system, which includes a transmitting device and a receiving device. The transmitting device is used to perform the data transmission method provided by the first aspect or any optional method of the first aspect, and the receiving device is used to perform the data reception method provided by the second aspect or any optional method of the second aspect. Attached Figure Description

[0047] Figure 1 is a schematic diagram of the architecture of a two-layer network provided in an exemplary embodiment of this application;

[0048] Figure 2 is a schematic diagram of the architecture of a three-layer network provided in an exemplary embodiment of this application;

[0049] Figure 3 is a schematic flowchart of a data transmission method provided in an exemplary embodiment of this application;

[0050] Figure 4 is a schematic diagram of the historical data volume of a transmitting device provided in an exemplary embodiment of this application;

[0051] Figure 5 is a schematic diagram of a load balancing object provided in an exemplary embodiment of this application;

[0052] Figure 6 is a schematic diagram of a transmission data packet group provided in an exemplary embodiment of this application;

[0053] Figure 7 is a schematic diagram of a data transmission and reception method flow provided in an exemplary embodiment of this application;

[0054] Figure 8 is a schematic diagram of the constituent data flow provided in an exemplary embodiment of this application;

[0055] Figure 9 is a schematic diagram of a spine node forwarding a group of data packets provided in an exemplary embodiment of this application;

[0056] Figure 10 is a schematic diagram of a receiving device performing order-preserving processing on data packet groups according to an exemplary embodiment of this application;

[0057] Figure 11 is a schematic diagram of the switching of load balancing modes provided in an exemplary embodiment of this application;

[0058] Figure 12 is a schematic diagram of a sorting cache and a congestion cache provided in an exemplary embodiment of this application;

[0059] Figure 13 is a schematic diagram of a sorting cache and a congestion cache provided in another exemplary embodiment of this application;

[0060] Figure 14 is a schematic diagram of the status information obtained by the receiving device according to an exemplary embodiment of this application;

[0061] Figure 15 is a schematic diagram of a receiving device feeding back information to a transmitting device according to an exemplary embodiment of this application;

[0062] Figure 16 is a schematic diagram of a transmitting device updating a status table based on received feedback information according to an exemplary embodiment of this application;

[0063] Figure 17 is a schematic diagram of the back pressure of the transmitting device provided in an exemplary embodiment of this application;

[0064] Figure 18 is a schematic diagram of a link provided in an exemplary embodiment of this application;

[0065] Figure 19 is a schematic diagram of a transmitting device transmitting a data stream according to an exemplary embodiment of this application;

[0066] Figure 20 is a schematic diagram of data flow switching under multiple load balancing modes provided in an embodiment of this application;

[0067] Figure 21 is a schematic diagram of the processing flow of a receiving device provided in an embodiment of this application;

[0068] Figure 22 is a schematic diagram of the structure of a data transmission apparatus provided in one embodiment of this application;

[0069] Figure 23 is a schematic diagram of the structure of a data receiving device provided in an embodiment of this application;

[0070] Figure 24 is a schematic diagram of the structure of a device provided in an embodiment of this application;

[0071] Figure 25 is a schematic diagram of the structure of a device provided in another embodiment of this application. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0073] Currently, in multi-layer switched networks, ECMP load balancing technology using five-tuples is employed to achieve load balancing. The scheme works as follows: the sending device receives a data packet from the server, calculates the hash value of the five-tuple, maps it to the sending port based on this hash value, and then sends it. In this way, all data packets with the same five-tuple are forwarded along the same forwarding path, while data packets with different five-tuples are forwarded through multiple paths in the network and are received by the receiving device, ultimately being forwarded to the destination server. However, the number of data packets with different five-tuples may vary in the network, making it impossible for the sending device to evenly distribute the received data packets across its ports, resulting in unbalanced link load. Alternatively, the sending device can treat consecutive packets of a specific five-tuple flow as a load balancing object. If the arrival interval between consecutive packets in a five-tuple flow exceeds a time threshold, subsequent packets of that five-tuple are allowed to undergo routing. Routing can be performed in various ways, such as random selection or hash selection. While this solution can achieve load balancing in certain scenarios, in artificial intelligence (AI) data centers, the same quintuple represents a continuous large flow. The interval between messages before and after the same quintuple is very small, making it difficult to segment. This results in the continuous large flow not being sent evenly from multiple ports, which may lead to the inability to achieve load balancing.

[0074] Based on this, this application provides a data transmission method in which, for each data stream, after sending a group of data packets for the data stream, the port for the data stream is re-determined. In this way, the data packets of a single data stream can be sent evenly from multiple ports, achieving finer-grained load balancing and improving load balancing effect and link utilization.

[0075] The following describes the methods for sending and receiving data, and the entities that perform these methods.

[0076] The main body executing the data transmission method is the data transmission device. Optionally, the data transmission device is a hardware device, such as a transmitting device, which can be a switch or a router, etc. Optionally, the data transmission device is a software device, such as a software program running on the hardware device.

[0077] The main body executing the data receiving method is the receiving device. Optionally, the data receiving device is a hardware device, such as a receiving equipment, which can be a switch or a router. Optionally, the data receiving device is a software device, such as a software program running on the hardware device.

[0078] The following describes the system architecture provided in the embodiments of this application. This system architecture can provide network services for server clusters in a data center.

[0079] Figure 1 illustrates a two-layer network architecture. This architecture includes multiple top-of-rack (TOR) switches and multiple spine nodes. Each TOR is connected to all spine nodes, and each TOR is connected to at least one server. Different TORs are connected to different servers. Servers connected to the same TOR communicate directly through the TOR, while servers connected to different TORs communicate through both the TOR and the spine nodes.

[0080] Figure 2 illustrates a three-layer network architecture. This architecture includes multiple TORs, multiple leaf nodes, and multiple spine nodes. Each TOR is connected to one leaf node, different leaf nodes are connected to some TORs, each leaf node is connected to all spine nodes, and each TOR is connected to at least one server. Different TORs are connected to different servers. Servers connected to the same TOR communicate directly through the TOR, while servers connected to different TORs communicate through the TOR, leaf nodes, and spine nodes.

[0081] In Figures 1 and 2, the transmitting device can be any TOR from among multiple TORs, and the receiving device can be any TOR from among multiple TORs. The server includes a general-purpose computing server, an artificial intelligence training server, and a storage server. The general-purpose computing server is a central processing unit (CPU) server, the artificial intelligence training server is a graphics processing unit (GPU) server, and the storage server is a solid-state drive (SSD) server or a hard disk drive (HDD) server.

[0082] The following describes the data transmission method flow, see steps 301 to 304 in Figure 3.

[0083] Step 301: Send a first data packet group on the first output port of the sending device. The first data packet group belongs to the first data stream and includes multiple first data packets.

[0084] The first data stream is any data stream sent by the sending device.

[0085] In this embodiment, when the transmitting device transmits the first data stream, it determines the output port for transmitting the first data stream, referred to as the first output port. Data packets of the first data stream are transmitted on the first output port, and the transmitted data packets belong to the first data packet group. Here, it is not necessary to wait until all the first data packet groups are received before transmitting the first data packet group together. The transmission of the first data packet group is determined to be complete when the number of data packets transmitted on the first output port equals the number of first data packets.

[0086] In this process, the sending device uses the information of the first data packet in the first data packet group to perform a hash calculation, obtains a hash value, and maps this hash value to an output port, which is called the first output port. The sending device can store this first output port, and subsequent first data packets in the first data packet group will all be sent using the first output port. Alternatively, the sending device can perform a hash calculation on the information of each data packet before sending it, obtain a hash value, and map this hash value to an output port, which is also called the first output port. Here, since the data packet information is the same and the selectable output ports during mapping are also the same, the first data packet in the first data packet group can all be mapped to the first output port.

[0087] It should be understood that in the embodiments of this application, data packets can also be referred to as messages.

[0088] In one alternative approach, the first data packet group is the first data packet group of the first data stream. When the first data stream is a short stream (also known as a mouse stream), it is not desirable to switch out of the port. When the first data stream is a large stream (also known as an elephant stream or a continuous large stream), the port is switched out. Data packets in the large stream are sent evenly from multiple ports to achieve load balancing, thereby achieving finer-grained load balancing. Therefore, we need to consider the number of first data packets in the first data packet group so that the port is not switched out when the first data stream is a short stream, but is switched out when the first data stream is a large stream. Considering this reason, the number of first data packets in the first data packet group is greater than the number of data packets in subsequent data packet groups, that is, the first number is greater than the second number. The first number is the number of first data packets included in the first data packet group, and the second number is the number of second data packets included in the second data packet group. The second data packet group is the data packet group after the first data packet group.

[0089] Optionally, data streams with a data volume less than a certain threshold are classified as short streams, while data streams with a data volume greater than the certain threshold are classified as large streams. The number of first data packets included in the first data packet group is configured manually based on experience. For example, the number of first data packets may be 100 or 200.

[0090] Alternatively, short and large flows are related to the sending device and are determined based on the historical transmission traffic statistics of the sending device. That is, the number of the first data packets is determined based on the historical transmission traffic statistics of the sending device. For example, as shown in Figure 4, the total amount of data transmitted by each data stream within a certain period is statistically analyzed to obtain the statistical probability distribution of the total transmitted data volume. The top X% of the data streams in the total transmitted data volume are identified as large flows. The value of X is set based on experience; assuming X is 10, the judgment threshold is 1M, the size of each data packet is 1KB, 1MB / 1KB = 1000, and the number of first data packets in the first data packet group is 100.

[0091] In one alternative approach, the transmitting device does not switch its output port when sending the first two data packet groups. It switches its output port starting from the third data packet group. The third data packet group is the first data packet group of the first data stream and includes multiple third data packets. The first data packet group is the second data packet group of the first data stream. The transmitting device sends the third data packet group from the first output port. The number of third data packets may be the same as or different from the number of first data packets, and the size of the third data packets may be the same as or different from the size of the first data packets.

[0092] In one alternative approach, data streams are differentiated based on load balancing objects, which include, but are not limited to, the following five categories.

[0093] 1. Five-tuple object.

[0094] A 5-tuple includes the source address, source port, destination address, destination port, and transport layer protocol. The source and destination addresses can be Internet Protocol (IP) addresses or Media Access Control (MAC) addresses. When the load balancing object is a 5-tuple object, the sequence number of the data stream can be the value obtained by hashing the 5-tuple.

[0095] The sending device identifies the header information of each data packet and obtains the 5-tuple of the data packet. The source address in the 5-tuple is the address of the server that sent the data packet, and the destination address in the 5-tuple is the address of the server that received the data packet.

[0096] 2. An object consisting of a transmitting device and a receiving device.

[0097] The sending device identifies the header of each data packet and obtains the destination address from it. This destination address is the address of the server receiving the data packet, which can be an IP address or a MAC address, etc. The sending device uses this server address to find the corresponding receiving device, thereby identifying the receiving device to which the data packet was sent. For example, each device stores a mapping relationship between the destination address and the receiving device. This mapping relationship can be statically configured in the device at the initial stage of network setup and updated during use. This mapping relationship is shown in Table 1, which uses TOR as an example. In the scenarios of Figures 1 and 2, the destination address is the server address, and the receiving device is TOR.

[0098] Table 1

[0099] 3. An object consisting of the processing channel of the transmitting device and the processing channel of the receiving device.

[0100] Transmitting and receiving devices include network chips. The processing channels of both the transmitting and receiving devices are the packet processing planes of the network chips they encompass. Within high-bandwidth network chips, data packets are typically processed by multiple processing channels, each responsible for handling different ports. For example, a 12Tbps network chip might include four 3Tbps processing channels. The transmitting device might have 100 ports, with each processing channel handling data packets for 25 ports.

[0101] The sending device determines the ingress port for each data packet received from the server. Based on the mapping between ingress ports and processing channels, it determines the processing channel corresponding to the ingress port of each data packet, thus identifying the processing channel for that data packet on the sending device. The sending device also stores a mapping between the processing channels of the receiving device and destination addresses. In this mapping, the destination address is either the destination IP address or the destination MAC address. The sending device determines the destination address of each data packet and, based on the mapping between the processing channels and destination addresses of the receiving device, determines the processing channel for that data packet on the receiving device.

[0102] Alternatively, the transmitting device can also use information such as a virtual extended local area network (VxLAN) tunnel to determine the processing channel of the receiving device. For example, the transmitting device stores a mapping relationship between VxLAN tunnels and the processing channels of the receiving device, and the transmitting device determines the corresponding processing channel of the receiving device based on the VxLAN tunnel corresponding to the destination address.

[0103] 4. An object consisting of the processing channels of the transmitting and receiving devices.

[0104] 5. An object consisting of the processing channel of the transmitting device and the receiving device.

[0105] The mode for determining the load balancing object to which the data packet belongs in steps 4 and 5 is described in step 3 and will not be repeated here.

[0106] Regarding load balancing objects, this application embodiment also provides a comparative schematic diagram of various load balancing objects as shown in Figure 5 to better understand the dimensions of various load balancing objects. As shown in Figure 5, the quintuple object has the smallest dimension, followed by the object composed of the processing channels of the sending device and the receiving device, and finally the object composed of the processing channels of the sending device and the receiving device, or the object composed of the processing channels of the sending device and the receiving device.

[0107] Step 302: Determine the second output port of the first data stream on the sending device. The first output port and the second output port are different.

[0108] In this embodiment, after sending the first data packet group, or after sending the first data packet group, a second output port for the first data stream on the sending device is determined from among the output ports other than the first output port. Specifically, the sending device uses information from the data packets to be sent next (i.e., the first data packet of the second data packet group) to perform a hash calculation, obtains a hash value, and maps this hash value to an output port, which is called the second output port. During mapping, the first output port is excluded, and the least frequently used output port is preferentially selected.

[0109] Step 303: Determine the second data packet group based on the packet group granularity information. The packet group granularity information is used to divide the first data stream into multiple packet groups. The second data packet group belongs to the first data stream and includes multiple second data packets.

[0110] In this embodiment, packet group granularity information is used to divide the first data stream into multiple packet groups. The sending device determines the second data packet group of the first data stream based on the packet group granularity information. The second data packet group is adjacent to or not adjacent to the first data packet group. The second data packet group includes multiple second data packets. The size of the second data packet is the same as or different from the size of the first data packet.

[0111] In one alternative approach, the packet group granularity information includes the data volume and / or the transmission period. The transmitting device groups data packets of the first data volume into a second packet group. For example, the first data volume is the number of data packets, which is 100, and 100 consecutive data packets constitute the second packet group. Alternatively, the transmitting device groups data packets transmitted within the transmission period into the second packet group. For example, if the transmission period is 1 second, all data packets transmitted within that 1 second belong to the second packet group. Alternatively, the transmitting device determines the second packet group based on the first data volume and the transmission period. For example, the minimum value between the number of data packets corresponding to the first data volume and the number of data packets corresponding to the transmission period is used as the packet group granularity. Another example is using the average value between the number of data packets corresponding to the first data volume and the number of data packets corresponding to the transmission period as the packet group granularity.

[0112] Optionally, when the packet granularity information includes the data volume, there are multiple ways to determine the data volume. Two feasible methods are provided below.

[0113] Method 1: Direct configuration.

[0114] a) The data volume is configured by the user based on experience and with load balancing as the goal. For example, the data volume is expressed using the number of data packets, such as 4, 8, 16, 32, or 64.

[0115] b. The network controller connects to each device in the network. The user inputs the desired data balance target into the network controller. The network controller stores the mapping between the data balance target and the data volume. The network controller finds the corresponding data volume for the data balance target in this mapping and sends it to the sending device. For example, the data balance target is a link throughput balance error of less than 0.1%, 0.5%, 1%, 2%, and 5%, which directly map to 4, 8, 16, 32, and 64 data packets, respectively.

[0116] Option 2: Automatic determination.

[0117] a. The network controller measures the average bandwidth of the outgoing port links of the transmitting devices in the network, determines the maximum difference between the bandwidth of the outgoing port link and the average bandwidth, and divides this maximum difference by the average bandwidth to obtain the actual link balancing error *r* between the outgoing port links. Based on the difference between the actual link balancing error *r* and the target link balancing error, negative feedback control is implemented to obtain the data volume. For example, if *r* = 3%, the target link balancing error is 1%, and the current data volume is 32, to achieve the 1% target link balancing error, the size of the balancing data packet group is reduced to lower the target link balancing error, so the data volume is adjusted to 16. That is, when the actual link balancing error is greater than the target link balancing error + 0.5%, the current data volume is reduced; when the difference between the actual link balancing error and the target link balancing error is less than or equal to 0.5%, the current data volume is maintained; and when the actual link balancing error is less than the difference between the target link balancing error and 0.5%, the current data volume is increased. The specific increase or decrease can be configured empirical values. Alternatively, when the actual link balancing error is greater than the target link balancing error + 0.5%, the current data volume is reduced; otherwise, the current data volume is maintained. Once the data volume update is confirmed, the network controller can send the updated data volume to each sending device.

[0118] b. The network controller controls the sending devices to send probe packets to the receiving devices, measuring the average RTT (Round-Trip Time) and connection bandwidth C between the sending and receiving devices. Then, it determines the data volume based on the RTT*C value. For example, RTT*C = 5µs * 100Gbps = 500Kbit = 64KB = 48 * 1.5KB, where 1.5KB is the size of a single data packet, and the data volume is 48. The network controller then allocates 48 to each sending device. Here, the connection bandwidth can be the maximum amount of data that a sending device can simultaneously send to the receiving device, or it can be a pre-configured fixed value.

[0119] In an alternative approach, the first data packet group is also determined at the packet group granularity, and the number of first data packets in the first data packet group is equal to the number of second data packets in the second data packet group. In this way, adjacent data packet groups in the first data stream are sent using different output ports.

[0120] Step 304: Send the second data packet group on the second output port.

[0121] In this embodiment, after the second output port is determined, the data packets of the first data stream are sent. The data packets sent belong to the second data packet group, and it is not necessary to wait until all the second data packet groups are received before sending the second data packet group together. After the number of data packets sent reaches the packet group granularity, it is determined that the second data packet group has been sent.

[0122] Using the process shown in Figure 3, when the data stream is a large flow, the data stream is divided into data packet groups. The data packets of the data stream can be output evenly from multiple ports to achieve load balancing. However, when the data stream is a short flow, it is not desirable to divide the data stream. This is because if it is divided, the receiving device needs to perform order preservation processing on the data packet groups. Thus, when the data stream is short, it is not divided to reduce the resources of the receiving device for order preservation processing.

[0123] In one alternative approach, the number of first data packets in the first data packet group is determined to prevent short-circuit switching of the transmission port. The first data packet group is the first data packet group of the first data stream. After the second data packet group is sent, the third output port of the first data stream on the transmitting device is determined from the output ports other than the first and second output ports. Then, the data packets of the first data stream are sent, and the sent data packets belong to the third data packet group. The third data packet group is sent together without waiting for all of them to be received. The transmission of the third data packet group is considered complete when the number of sent data packets reaches the packet group granularity. For example, as shown in Figure 6, in the system architecture shown in Figure 1, the first data packet of the first data packet group is output from the first output port, the second data packet of the second data packet group is output from the second output port, the third data packet of the third data packet group is output from the third output port, and so on, sending all data packets. Adjacent data packet groups use different output ports.

[0124] In one alternative approach, all data packets within a single packet group are sent to the receiving device via the same path, ensuring that the data packets within the group are not out of order. This eliminates the need for the receiving device to sort the data packets within the group, conserving resources. For example, for each data packet, after determining its packet group and data stream, the sending device adds the packet's sequence number and the data stream's sequence number to the packet. This allows the forwarding switch to identify packets belonging to the same packet group and output them from a single output port, while also enabling the receiving device to perform order-preserving processing on the data stream.

[0125] The principle of data transmission has been described above. Next, we will explain it from the perspective of data transmission and reception. Here, we will take the two-layer network architecture shown in Figure 1 as an example. See steps 701 to 706 in Figure 7.

[0126] Step 701: The sending device receives the data packet sent by the server.

[0127] In this embodiment, when the server (i.e., the source server) sends data to other servers, the source server packages the data into data packets and sends these data packets to a connected switch, which is called the sending device. The sending device receives the data packets sent by the server.

[0128] Step 702: The sending device determines the data stream to which the received data packet belongs based on the load balancing object.

[0129] In this embodiment, the sending device stores a load balancing object. Based on this load balancing object, it identifies whether the received data packet belongs to a data stream already existing in the data flow status table or a new data stream. The data flow status table records relevant information about the data stream sent by the sending device. If it belongs to an existing data stream, the sequence number and other information of the data packets in that data stream are updated in the data flow status table. If it does not belong to an existing data stream, a new data stream is added to the data flow status table, and the sequence number and other information of the data packets in that data stream are recorded. For example, as shown in Figure 8, the load balancing object is an object composed of the sending device and the receiving device. The sending device (TOR2) receives data packets destined for the receiving device (TORi) from server 1, receives data packets destined for the receiving device (TORi) from server 2, and receives data packets destined for the receiving device (TORi) from server 3. These data packets constitute a data stream.

[0130] Step 703: Send the first data packet group on the first output port of the sending device, determine the second output port of the first data stream on the sending device, the first output port and the second output port are different; determine the second data packet group according to the packet group granularity information, and send the second data packet group on the second output port.

[0131] The processing procedure for step 703 is shown in Figure 3 and will not be repeated here.

[0132] In one alternative approach, after sending the data stream, the sending device can also record relevant information about the data stream, as shown in Table 2, which is a data stream status table.

[0133] Table 2

[0134] In Table 2, the sequence number of a data stream represents the number of an existing data stream, used to distinguish it from existing data streams. Upon identifying a new data stream, a sequence number is added to the new data stream to differentiate it from existing data streams, and the relevant information of the new data stream is added to Table 2. For each data stream, the sequence number of the last data packet group is the sequence number of the last data packet group sent in that data stream, the sequence number of the last data packet is the sequence number of the last data packet sent in that data stream, and the final outgoing port is the outgoing port used by the last data packet sent in that data stream.

[0135] Step 704: The receiving device receives multiple data packets of the first data stream sent by the sending device.

[0136] In this embodiment, data packets are transmitted in the network, and the data packets in a data packet group arrive at the receiving device through the same transmission path. For example, as shown in FIG9, a data packet group including data packets 1 to 4 is transmitted to spine node 3. Spine node 3 inputs each data packet of the data packet group from the same ingress port and outputs it from the same egress port. The small boxes in FIG9 represent ports.

[0137] Step 705: The receiving device determines the data packet group to which each data packet belongs, performs order-preserving processing on the data packet groups of the first data stream, and obtains the first data stream after order-preserving processing.

[0138] In this embodiment, each data packet includes a sequence number of the data packet group. The receiving device uses the sequence number of the data packet group in each data packet to determine data packets belonging to the same data packet group. The receiving device uses the sequence number of the data stream in the data packet to identify different data packet groups belonging to the same data stream. Although the data packets in a data packet group will not be out of order, because the transmission paths of different data packet groups may be different, out-of-order packets may occur at the receiving device. The receiving device performs order-preserving processing on the data packet groups belonging to the first data stream to obtain the first data stream with order preservation.

[0139] Step 706: The receiving device outputs the first data stream after sequence preservation processing.

[0140] In this embodiment, the receiving device determines the destination address from the data packets of the first data stream after the order-preserving processing. Based on the mapping relationship between the destination address and the output port, the receiving device determines the output port corresponding to the destination address of the order-preserving data stream and outputs the order-preserving data stream from that output port. For example, as shown in Figure 10, in a data stream, the first data packet group includes data packets with sequence numbers 1 to 5, and the second data packet group includes data packets with sequence numbers 6 to 9. The data packets in the first data packet group and the data packets in the second data packet group are sorted to obtain data packets with sequence numbers 1 to 9, and these data packets with sequence numbers 1 to 9 are sent to the destination server.

[0141] In the process shown in Figure 7, all data packets in a single data packet group are sent to the receiving device via the same path, ensuring that the data packets in a single data packet group are not out of order. The receiving device only needs to perform order preservation processing on the data packet group, without needing to perform order preservation processing on the data packets within the data packet group, which can save resources.

[0142] In this embodiment, to improve load balancing performance in the network, the receiving device can send feedback information to the sending device, which then adjusts its load balancing strategy based on the feedback information. For example, as shown in Figure 11, the load balancing mode is switched based on network congestion. When network congestion occurs and / or the receiving device's resources are insufficient, the load balancing mode switches from fine-grained load balancing (the load balancing mode used in Figures 3 and 7) to a target load balancing mode. Data streams sent using the target load balancing mode are received by the receiving device without being out of order. Target load balancing modes include, but are not limited to, ECMP mode and flow let mode. When network congestion is cleared and / or the receiving device's resources are sufficient, the load balancing mode switches back from the target load balancing mode to the fine-grained load balancing mode. Thus, switching the load balancing mode based on network conditions or the receiving device's situation can improve data transmission performance.

[0143] Feedback information is determined based on the state of the links maintained by the receiving device and its own state. The state of the links maintained by the receiving device and its own state includes end-to-end link quality information and the state of the receiving device itself. The following describes the end-to-end link quality information and the state of the receiving device, as well as the method for determining the feedback information to be sent to the transmitting device, and the processing performed by the transmitting device based on this feedback information.

[0144] 1. End-to-end link quality information.

[0145] Each data packet carries an explicit congestion notification (ECN) tag, which is included in the packet header and added by the devices the packet passes through when congestion is buffered. This allows the receiving device to check the packet header for an ECN tag; if an ECN tag is present, link congestion is determined. Alternatively, each data packet may also include an extension field, which includes the outgoing port queue congestion length of devices along the transmission path and / or the link utilization rate. The link utilization rate is the proportion of time with data within a statistical period to the total time. If the outgoing port queue congestion length exceeds a length threshold, and / or the link utilization rate exceeds a utilization threshold, link congestion is determined. The length and utilization thresholds can be set according to actual needs, and are not limited in this embodiment.

[0146] 2. The status of the receiving device itself.

[0147] Before describing the state of the receiving device itself, we first describe its sort buffer and congestion buffer. The sort buffer is used to receive out-of-order packets in fine-grained load balancing mode, while the congestion buffer stores congested packets after order preservation processing. When the network experiences sudden congestion or large link gaps, the receiving device's sort buffer and congestion buffer can be consumed, or even exhausted. The receiving device can then report the congestion status to the sending devices, notifying all sending devices to switch load balancing modes to the target load balancing mode. In this way, since the target load balancing mode does not involve out-of-order packets, there is no consumption of the sort buffer and congestion buffer.

[0148] The ordering cache and congestion cache occupy independent physical storage resources in the receiving device, as shown in Figure 12, or the ordering cache and congestion cache share physical storage resources in the receiving device, as shown in Figure 13. In addition, as shown in Figures 12 and 13, the receiving device also has an ordering process and an order-preserving process. The ordering process is used to manage the ordering cache, and the order-preserving process is used to manage the congestion cache.

[0149] As shown in Figures 14, 15, and 16, the state of the receiving device includes information characterizing the overall congestion state of the receiving device itself (i.e., a and b), information characterizing the congestion state of the local port of the receiving device (i.e., c, d, and e), and information characterizing the congestion state of the switching path (i.e., f, g, and h).

[0150] a. The amount of space occupied by the sorting cache.

[0151] The sorting cache occupancy is the amount of data stored in the received out-of-order data packet groups, as shown in Table 3, where the occupancy is 1234kB at a certain moment.

[0152] Table 3

[0153] The receiving device determines whether the occupancy of the sorting buffer exceeds a first threshold. If it exceeds the first threshold, it determines a first congestion state indicating congestion; if it does not exceed the first threshold, it determines a first congestion state indicating no congestion. The first threshold can be set based on experience, and may vary for different sorting buffer capacities.

[0154] The receiving device sends a first congestion status to each transmitting device. In addition to indicating whether there is congestion, the first congestion status also indicates the identifier of the receiving device to which the first congestion status belongs. Thus, the first congestion status includes the identifier of the receiving device and the identifier of whether there is congestion. The identifier of the receiving device can be indicated using 12 bits, and whether there is congestion can be indicated using 1 bit.

[0155] Each receiving device sends the first congestion status to the sending device. Each sending device can receive the first congestion status of multiple receiving devices, as shown in Table 4. The network includes K receiving devices, and Table 4 records the first congestion status of each receiving device.

[0156] Table 4

[0157] For each transmitting device, if it receives a first congestion status indication from a receiving device indicating congestion, the load balancing mode of the data stream sent to that receiving device is switched to the target load balancing mode. If the first congestion status indication is not congested, the load balancing mode is not switched, and the fine-grained load balancing mode is still used.

[0158] Optionally, if the first congestion state indicates congestion, the load balancing mode of the existing data stream destined for the receiving device is switched to a pre-exit load balancing state. In the pre-exit load balancing state, when sending data packets, the previously used output port is used, and a timer begins. After the aging time is reached, the load balancing mode of the existing data stream is switched to the target load balancing mode. Furthermore, if the first congestion state indicates congestion, the load balancing mode of the newly received data stream is switched to the target load balancing mode. In this way, new data streams do not need to occupy the sorting buffer; only existing data streams occupy the sorting buffer, reducing the amount of sorting buffer usage.

[0159] Optionally, after the load balancing mode of the data stream sent by the sending device to the receiving device is switched to the target load balancing mode, if the first congestion status indicated by the receiving device is congested, the target load balancing mode is continued to be used to send data. If the first congestion status indicates no congestion, the load balancing mode is switched to the fine-grained load balancing mode, and the congestion status of the sorted cache in Table 4 is updated.

[0160] Optionally, if the first congestion state indicates no congestion, the load balancing mode can be switched to the pre-switching state and a timer can be started. After the aging time for switching is reached, the load balancing mode can be switched to fine-grained load balancing mode. The aging time for switching here can be the same as or different from the aging time for switching in the pre-exit state.

[0161] Optionally, when the load balancing mode can be switched, the data flow status table also includes the current load balancing mode of the data flow. After switching the load balancing mode, the load balancing mode in the data status table is updated, see Table 5.

[0162] Table 5

[0163] Optionally, the aging-off time can also be recorded in Table 5.

[0164] b. The amount of congestion cache occupied.

[0165] The congestion buffer occupancy refers to the amount of data stored in the ordered data packet group, as shown in Table 6, where the occupancy is 4567kB at a certain moment.

[0166] Table 6

[0167] The receiving device determines whether the occupancy of the congestion buffer exceeds a second threshold. If it does, the congestion buffer is determined to be congested; otherwise, it is determined to be uncongested. The second threshold can be set based on experience, and may vary depending on the capacity of the congestion buffer.

[0168] Each receiving device sends the congestion buffer status to each sending device. Each sending device can receive the congestion buffer status of multiple receiving devices and obtain the congestion status of the congestion buffers of multiple receiving devices, as shown in Table 7. The network includes K receiving devices, and Table 7 records the congestion status of the congestion buffers corresponding to each receiving device.

[0169] Table 7

[0170] As shown in Figure 17, for each transmitting device, if it receives a message indicating that the congestion buffer of a receiving device is congested, it applies backpressure to the data destined for that receiving device. This can be achieved by reducing the rate at which data is sent to the receiving device, pausing data transmission, or sending a notification to the server containing the data destined for the receiving device, instructing it to suspend data transmission to that server. Subsequently, if the receiving device's congestion buffer is no longer congested, it no longer applies backpressure to the data destined for the destination exchange.

[0171] Optionally, when sending the congestion status of the congestion buffer, the receiving device uses a 12-bit indication, and the congestion status uses a 1-bit indication.

[0172] c. The amount of space occupied by the sorting buffer for each ingress port.

[0173] For each ingress port of the receiving device, the occupancy of the sorting buffer for that ingress port is equal to the amount of out-of-order data packets received at that ingress port. As shown in Table 8, the receiving device includes O1 ingress ports, and the occupancy of the sorting buffers for ingress port 1, ingress port 2, and ingress port O1 are 30kB, 25kB, and 10kB, respectively.

[0174] Table 8

[0175] The receiving device stores multiple thresholds to classify the reception congestion level of ingress ports. The number and values ​​of these thresholds are set according to actual needs. For example, multiple thresholds may include three thresholds, allowing ingress ports to be divided into four reception congestion levels: TH0, TH1, and TH2. For each ingress port, if the sorting buffer occupancy is less than TH0, the reception congestion level is 0; if the sorting buffer occupancy is between TH0 and TH1, the reception congestion level is 1; if the sorting buffer occupancy is between TH1 and TH2, the reception congestion level is 2; and if the sorting buffer occupancy is greater than TH2, the reception congestion level is 3. Based on this classification method, ingress ports are divided into multiple ingress port groups with different reception congestion levels, resulting in approximately load balancing within each ingress port group.

[0176] The receiving device sends the receive congestion level of each of its input ports to each transmitting device. The transmitting device receives the receive congestion level of each input port. For each receiving device, the transmitting device determines the congestion level of the output port leading to that input port based on the received receive congestion level and updates the congestion level of each output port. For example, for transmitting device 1, the receive congestion level of input port R1 of receiving device 2 is 0. Output port R2 of transmitting device 1 is connected to input port R1 of receiving device 2 via Spine node 8, and the congestion level of output port R2 of transmitting device 1 is also 0.

[0177] Table 9 shows the congestion levels of each output port of the transmitting device leading to a receiving device.

[0178] Table 9

[0179] When a sending device sends a group of data packets to a receiving device, data packets belonging to the same data stream are sent through the same congestion level output port. The output port with the lowest congestion level is selected for transmission. The lower the congestion level, the less congested the link where the output port is located.

[0180] Optionally, when the sending device has multiple output ports, if some output ports indicate congestion, then when sending data packet groups to the receiving device, data packet groups belonging to the same data stream are sent through output ports with the same congestion level. If no port indicates congestion, then data packets are not sent according to the congestion level.

[0181] Optionally, when sending the congestion level, the ingress port uses a 10-bit indicator, and the congestion level uses a 2-bit indicator. Here, in the network structure shown in Figure 1, the ingress port can be a ridge node.

[0182] It should be noted that the above-mentioned ingress port is the port for receiving data packets sent by the sending device. In fact, the ingress port can also be used as the egress port of the receiving device to output data packets.

[0183] d. The occupancy of the sorting buffer for each downlink output port. The downlink output port is used to connect to the next level node. For example, in Figure 1 and Figure 2, the receiving device is TOR, and the next level node is a server, etc.

[0184] For each downlink output port of the receiving device, the occupancy of the sorting buffer for that downlink output port is the amount of data in the out-of-order packet group destined for that downlink output port. As shown in Table 10, the receiving device includes O2 downlink output ports, and the occupancy of the sorting buffer for downlink output port 1, downlink output port 2, and downlink output port O2 are 50kB, 40kB, and 105kB, respectively.

[0185] Table 10

[0186] For each downlink output port of the receiving device, the receiving device determines whether the occupancy of the sorting buffer for that downlink output port exceeds a third threshold. If it exceeds the third threshold, the sorting buffer for that downlink output port is determined to be congested; if it does not exceed the third threshold, the sorting buffer for that downlink output port is determined to be uncongested. The third threshold can be set based on empirical values.

[0187] The receiving device sends the congestion status of its sorting buffers for each downlink egress port to each transmitting device, and each transmitting device receives the congestion status of its sorting buffers for each downlink egress port. Thus, since each receiving device sends congestion status to the transmitting devices, each transmitting device can receive the congestion status of the sorting buffers for each downlink egress port from multiple receiving devices, obtain the congestion status of the sorting buffers for each downlink egress port from multiple receiving devices, and update the congestion status of each downlink egress port. As shown in Table 11, the network includes K receiving devices, each receiving device has O2 downlink egress ports, and Table 11 records the congestion status of the sorting buffers for each downlink egress port of each receiving device.

[0188] Table 11

[0189] For each transmitting device, if the congestion status of the sorting buffer of the first downlink outgoing port of the receiving device is determined to be congested, the load balancing mode of the data stream sent to the first downlink outgoing port is switched to the target load balancing mode. If the congestion status of the sorting buffer of the first downlink outgoing port is not congested, the load balancing mode is not switched, and the fine-grained load balancing mode is still used.

[0190] Optionally, if the sorting buffer of the first downlink egress port is congested, the load balancing mode of the existing data stream sent to the first downlink egress port is switched to the load balancing pre-exit state, and a timer is set. After the timer reaches the switching aging time, the load balancing mode of the existing data stream is switched to the target load balancing mode. If the sorting buffer of the first downlink egress port is congested, the load balancing mode of the newly received data stream is switched to the target load balancing mode.

[0191] After switching to target load balancing mode, if the congestion status of the sorting buffer of the first downlink outgoing port is not congested, the data stream sent to the downlink outgoing port will be switched to fine-grained load balancing mode.

[0192] Optionally, before switching to fine-grained load balancing mode, switch to the pre-switching state, start timing, and switch to fine-grained load balancing mode after the aging time is reached.

[0193] Optionally, when sending the congestion status of the sorting buffer, the receiving device uses a 12-bit indication, the downlink port uses a 7-bit indication, and the congestion status uses a 1-bit indication.

[0194] e. The congestion buffer usage of each downlink output port of the receiving device. The downlink output port is used to connect to the next level node. For example, in Figure 1 and Figure 2, the receiving device is TOR, and the next level node is a server, etc.

[0195] For each downlink output port of the receiving device, the congestion buffer occupancy of that downlink output port is equal to the data size of the ordered data packets destined for that downlink output port. As shown in Table 12, the receiving device includes O2 downlink output ports, and the congestion buffer occupancy of downlink output port 1, downlink output port 2, and downlink output port O2 are 500kB, 100kB, and 900kB, respectively.

[0196] Table 12

[0197] The receiving device determines the relationship between the congestion buffer occupancy of each downlink egress port and a fourth threshold. For a given downlink egress port, if the congestion buffer occupancy exceeds the fourth threshold, the transmission congestion status of that downlink egress port is determined to be congested; otherwise, the transmission congestion status is determined to be non-congested. The fourth threshold is set based on empirical values. The receiving device sends the transmission congestion status of each downlink egress port to each transmitting device. Thus, since each receiving device sends its own transmission congestion status for each downlink egress port to the transmitting devices, each transmitting device can receive the transmission congestion status of each downlink egress port from multiple receiving devices, obtain the transmission congestion status of each downlink egress port from multiple receiving devices, and update the recorded transmission congestion status. As shown in Table 13, the network includes K receiving devices, each receiving device has O2 downlink egress ports, and Table 13 records the transmission congestion status of the downlink egress ports of each receiving device.

[0198] Table Thirteen

[0199] For each transmitting device, if it receives a message indicating that the first downlink outgoing port of a receiving device is congested, it applies backpressure to the data destined for that port. This can be achieved by reducing the rate at which data is sent to the first downlink outgoing port, pausing data transmission to the first downlink outgoing port, or sending an indication message to a connected server to suspend data transmission to a specified server (the server connected to the congested downlink outgoing port), as shown in Figure 17. If the first downlink outgoing port is congested, backpressure is applied to the data destined for it. If the first downlink outgoing port is subsequently found to be uncongested, backpressure is no longer applied to the data destined for it.

[0200] f. The amount of space occupied by each balanced path in the sorting cache.

[0201] In the two-layer network structure shown in Figure 1, for a destination switch and a source switch, a path (i.e., a link) is the path from an outgoing port of the source switch to an incoming port of the destination switch, as shown in Figure 18. Since the outgoing port of the transmitting device is fixed, the connected spine node is also fixed, and the spine node's connection to the designated receiving device's incoming port is also fixed. Therefore, from the perspective of a designated receiving device, each balanced path can be the path between the receiving device's incoming port and each transmitting device. As shown in Table 14, for a receiving device, there are R incoming ports, and K transmitting devices in the network. Table 14 shows the occupancy of the sorting buffer for each outgoing port connected to each transmitting device. This occupancy is the number of data packet groups received by the incoming port.

[0202] Table 14

[0203] It should be noted that the above explanation is based on a two-layer structure. In a three-layer or higher structure, the forwarding capability of Layer 3 and higher switches is strong and the load is balanced. Therefore, the path of the sending and receiving devices to the upper-layer switch is also considered.

[0204] g. Congestion status assessment of each equilibrium path.

[0205] The definitions of each balanced path are described in f and will not be repeated here. The congestion status of each balanced path includes both congested and non-congested states. The congestion status of each balanced path is determined based on the content in Table 14 and / or end-to-end link quality status information. For example, the relationship between the occupancy of the sorting buffer of each ingress port and a preset threshold is determined. For an ingress port of a balanced path, if the occupancy of the sorting buffer of the ingress port is greater than the preset threshold, and / or the end-to-end link quality status information indicates congestion, then the receiving congestion status of the ingress port is determined to be congested; otherwise, the receiving congestion status of the ingress port is determined to be non-congested. As shown in Table 15, for a receiving device, there are R ingress ports, and there are W transmitting devices in the network. Table 15 shows the receiving congestion status of each ingress port connected to each transmitting device.

[0206] Table 15

[0207] In the two-layer structure, the states shown in Table 15 are actually end-to-end congestion states.

[0208] The receiving device obtains the end-to-end link status information with each sending device based on the information in Table 15. For sending device 1, the row containing sending device 1 in Table 15 contains the end-to-end link status information. For each sending device, each receiving device sends its end-to-end link status information to that sending device; this is called the second congestion state. The sending device receives end-to-end link status information from multiple receiving devices and updates the existing end-to-end link status information, as shown in Table 16.

[0209] Table 16

[0210] For each transmitting device, when sending a data stream to a receiving device, the congestion status of each input port of the receiving device is used to determine the uncongested input ports of the receiving device. Using the uncongested input ports of the receiving device, the output ports of the transmitting device are then searched for an output port that can be routed to the uncongested input port, and the data stream is sent using the found output port. For example, for transmitting device 1, the input port R1 of receiving device 2 is uncongested, and the output port R2 of transmitting device 1 is connected to the input port R1 of receiving device 2 via Spine node 8. When transmitting device 1 sends a data stream, it selects output port R2 to send the data stream to Spine node 8, and Spine node 8 then sends the data stream to the input port R1 of receiving device 2.

[0211] Optionally, when sending congestion status assessment, the sending device uses a 12-bit indication, the receiving device uses a 12-bit indication, the ingress port uses a 7-bit indication, and the congestion status uses a 1-bit indication.

[0212] h. The degree of out-of-order delivery of data packets from each data stream to the receiving device.

[0213] For each data stream, the expected sequence number and the maximum received sequence number of the data packets in that data stream are periodically obtained. The expected sequence number of the data packet group is the sequence number of the data packet group currently expected to be received (e.g., if the data packet group with sequence number 19 is currently received, the expected data packet group with sequence number 20 is expected to be received). The maximum received sequence number is the maximum sequence number of the data packet group currently received. Based on these two sequence numbers, the out-of-order degree of the data packet group to the receiving device in that data stream is determined. As shown in Table 17, there are a total of Q data streams. In data stream 1, the expected sequence number and the maximum received sequence number of the data packet group are 1 and 10, respectively. In data stream 2, the expected sequence number and the maximum received sequence number of the data packet group are 30 and 120, respectively. In data stream Q, the expected sequence number and the maximum received sequence number of the data packet group are 3 and 4, respectively.

[0214] Table 17

[0215] For each data stream received by the receiving device, the receiving device calculates the distance between the expected sequence number of the data packet group and the maximum received sequence number of the data packet group. If this distance is greater than a fifth threshold, the data packets of that data stream are determined to be out of order, and a first congestion state indicates congestion. Otherwise, the data packets of that data stream are determined to be in order, and a first congestion state indicates no congestion. The fifth threshold can be set based on empirical values. The receiving device sends the first congestion state to the sending device that sent the data stream, and the sending device receives the first congestion state from the receiving device. Here, when the distance between the expected sequence number of the data packet group and the maximum received sequence number of the data packet group is relatively large, later data packets are received first, while earlier data packets have not yet been received, indicating a high degree of out-of-order delivery.

[0216] After the transmitting device receives the first congestion status, the process it performs is described in section a, and will not be repeated here.

[0217] Optionally, the out-of-order degree can be indicated using 1 bit, and the transmitting device can use 12 bits. If the out-of-order degree corresponds to a processing channel of the transmitting device, then the processing channel can use 14 bits.

[0218] It should be noted that 'a' and 'f' can send the first congestion state independently, or they can be generated uniformly based on the out-of-order state of the data stream sent by the sending device and / or the occupancy of the sorting buffer of the receiving device. For example, if the out-of-order state of the data stream sent by the sending device is high and / or the occupancy of the sorting buffer of the receiving device exceeds a first threshold, the generated first congestion state indicates congestion; if the out-of-order state of the data stream sent by the sending device is low and / or the occupancy of the sorting buffer of the receiving device does not exceed the first threshold, the generated first congestion state indicates no congestion.

[0219] It should also be noted that the receiving device can periodically determine the above information; the determination periods for different pieces of information can be the same or different. When determining information using the same period, they can be sent together, and the sent message can carry an identifier indicating feedback information.

[0220] In this embodiment of the application, in order to better understand the scheme, the processing flow of the transmitting device is also provided, as shown in Figure 19, steps s1 to s6.

[0221] Step s1: Obtain the data stream.

[0222] Step s2, based on Table 4, determine whether to adopt the fine-grained load balancing mode.

[0223] Step s3: If fine-grained load balancing mode is adopted, the data stream is sent according to the fine-grained load balancing mode, and Tables 9 and 16 are used to select the output port for the data packet group during transmission.

[0224] Step s4: If the fine-grained load balancing mode is not used, the data stream is sent according to the target load balancing mode.

[0225] Step s5: Receive feedback information from the receiving device.

[0226] Step s6: Update and maintain the status tables based on the feedback information. For example, update and maintain Tables 4, 9, and 16.

[0227] The following is a schematic diagram of the sending device switching load balancing mode, as shown in Figure 20, steps a1 to a17.

[0228] Step a1: Obtain the data stream.

[0229] Step a2: Determine whether the fine-grained load balancing mode is enabled.

[0230] Step a3: If enabled, determine from the data flow status table whether the data flow is in fine-grained load balancing mode.

[0231] Step a4: If in fine-grained load balancing mode, then read Table 4.

[0232] Step a5: Determine if the receiving device is congested.

[0233] Step a6: Read Table 16, select the uplink port for the data packet, and send the data packet.

[0234] Step a7: Update the data flow status table.

[0235] In step a7, update the data stream status table (Table 5) with information such as the sequence number of the data packet and the last uplink port to be sent.

[0236] Step a8: In step a3, if the load balancing mode is not in fine-grained mode, determine whether the load balancing pre-exit state is in.

[0237] Step a9: If the system is in a pre-exit balanced state, determine whether the switching aging time has expired.

[0238] If step a10 times out, switch to the target load balancing mode to send data packets.

[0239] In step a11, if the timeout has not occurred, the data stream will be sent using the previously used output port.

[0240] Step a12: If not in the pre-exit balanced state, determine whether it is in the target load balancing mode.

[0241] Step a13: If the target load balancing mode is in progress, determine whether the receiving device is not congested.

[0242] Step a14: If the receiving device is not congested, switch to the ready-to-switch state and proceed to step a16.

[0243] Step a15: If not in the target load balancing mode, determine whether it is in the pre-switching state.

[0244] Step a16: If the system is in the pre-switching state, determine whether the switching aging time has expired.

[0245] Step a17: If the aging time has not expired, the data stream is sent in the target load balancing mode; otherwise, return to step a3.

[0246] The processing flow for the receiving device is then provided, as shown in Figure 21, steps b1 to b5.

[0247] Step b1: Receive data packets.

[0248] Step b2: Determine whether the data packet group to which the data packet belongs is the expected data packet group.

[0249] In step b3, if it is the expected packet group, it is bypassed to the congestion buffer and the next sequence number of the packet group is searched in the sorting buffer. If found, the found packet group is stored in the congestion buffer.

[0250] In step b4, if the packet is not the expected packet group, the packet is stored in the sorting cache.

[0251] Step b5: Send data packets from the congestion buffer to the destination server.

[0252] In this embodiment, fine-grained load balancing technology for data packet groups is used when sending data streams, improving load balancing efficiency and link utilization. Furthermore, data packets belonging to the same data packet group share the same transmission path, reducing the sorting overhead of the receiving device and achieving a good balance between cost and overhead. In addition, this application monitors and provides feedback on the network status and its own switch status through the receiving device, enabling timely coordination with the sending device to adjust the sending device's load balancing strategy, such as output port selection and load balancing mode selection. This significantly improves system reliability and avoids situations where the system's load balancing strategy fails or even severely impairs link utilization in extreme cases.

[0253] Especially when applied to AI training network scenarios, most of which involve continuous large flows, the use of fine-grained load balancing mode results in better load balancing between and within large flows. This improves the load balancing effect, reliability, and system robustness of AI training network data, thereby enhancing the overall training efficiency of AI training and reducing training time and costs.

[0254] Figure 22 is a structural diagram of a data transmission apparatus provided in an embodiment of this application. This apparatus can be implemented as part or all of a device through software, hardware, or a combination of both, and is applied to a transmission device. The apparatus provided in this embodiment can implement part of the process described in Figure 3 of this embodiment. The apparatus includes: a transmission module 2210 and a determination module 2220, wherein:

[0255] The sending module 2210 is used to send a first data packet group on the first output port of the sending device. The first data packet group belongs to a first data stream and includes multiple first data packets. Specifically, it can be used to implement the sending function of step 301 and execute the implicit steps included in step 301.

[0256] The determining module 2220 is configured to: determine a second output port of the first data stream on the transmitting device, wherein the first output port and the second output port are not the same;

[0257] The second data packet group is determined based on the packet group granularity information. The packet group granularity information is used to divide the first data stream into multiple packet groups. The second data packet group belongs to the first data stream. The second data packet group includes multiple second data packets. Specifically, it can be used to implement the determination function of steps 302 and 303 and to execute the implicit steps included in steps 302 and 303.

[0258] The sending module 2210 is also used to send the second data packet group on the second output port, specifically to implement the sending function of step 304 and to execute the implicit steps included in step 304.

[0259] In one alternative approach, the number of first data packets included in the first data packet group is greater than the number of second data packets included in the second data packet group to prevent short-circuit switching of the outgoing port for transmitting traffic;

[0260] The determining module 2220 is further configured to:

[0261] Determine a third output port of the first data stream on the transmitting device, wherein the third output port is different from both the first output port and the second output port;

[0262] A third data packet group is determined based on the packet group granularity information. The third data packet group belongs to the first data stream and includes multiple third data packets.

[0263] The sending module 2210 is also used to send a third data packet group on the third output port.

[0264] In an alternative embodiment, the sending module 2210 is further configured to:

[0265] Before sending the first data packet group on the first output port of the sending device, a third data packet group is sent on the first output port. The third data packet group belongs to the first data stream and includes multiple third data packets.

[0266] In one alternative, the number of first data packets included in the first data packet group is equal to the number of second data packets included in the second data packet group.

[0267] In one alternative approach, the number of the first data packets in the first data packet group is determined based on the historical transmission traffic statistics of the transmitting device.

[0268] In one alternative approach, the packet group granularity information includes the data volume and / or transmission cycle.

[0269] In one alternative approach, the packet group granularity information includes the data volume;

[0270] The amount of data is determined based on the bandwidth error between the output ports of the transmitting device; or...

[0271] The amount of data is determined based on the RTT and connection bandwidth between the transmitting device and the receiving device of the first data stream; or,

[0272] The amount of data is determined based on the input balance target.

[0273] In an alternative embodiment, the apparatus further includes: a receiving module configured to receive a first congestion status sent by a receiving device of the first data stream before transmitting the first data packet group on a first output port of the transmitting device, the first congestion status being used to indicate the congestion status of the receiving device;

[0274] The determining module 2220 is further configured to determine, based on the first congestion state, that the receiving device is not congested.

[0275] In an alternative embodiment, the determining module 2220 is further configured to:

[0276] If the receiving device is determined to be congested based on the first congestion state, the method of sending the first data stream is switched to the target load balancing mode, wherein the data packets of the first data stream are sent in the target load balancing mode without being out of order at the receiving device.

[0277] In an alternative embodiment, the apparatus further includes: a receiving module, configured to receive a second congestion status sent by a receiving device of the first data stream, the second congestion status indicating the congestion status of the sending device at each ingress port of the receiving device;

[0278] The determining module 2220 is used for:

[0279] Based on the second congestion state, determine the non-congested input port of the transmitting device that is sent to the receiving device.

[0280] Among the determined output ports, select the second output port, which is different from the first output port.

[0281] In one alternative approach, the multiple first data packets in the first data packet group are transmitted to the receiving device of the first data stream via the same path;

[0282] The multiple second data packets in the second data packet group are transmitted to the receiving device of the first data stream via the same path.

[0283] Figure 23 is a structural diagram of the data receiving device provided in an embodiment of this application. This device can be implemented as part or all of the device through software, hardware, or a combination of both, and is applied to a receiving device. The device provided in this embodiment can implement part of the process described in Figure 7 of this embodiment. The device includes: a receiving module 2310, a first determining module 2320, and an output module 2330, wherein:

[0284] The receiving module 2310 is used to receive multiple data packets of a first data stream sent by the sending device. The multiple data packets form multiple data packet groups. The first data packet group and the second data packet group are sent to the first output port and the second output port of the sending device, respectively. The first output port and the second output port are different. The first data packet group is sent before the second data packet group. The first data packet group includes multiple first data packets. The second data packet group includes multiple second data packets. The second data packet group is determined according to packet group granularity information. The packet group granularity information is used to divide the first data stream into multiple packet groups. Specifically, it can be used to implement the sending function of step 704 and execute the implicit steps included in step 704.

[0285] The first determining module 2320 is used to determine the data packet group to which each data packet belongs, and to perform order-preserving processing on the data packet groups of the first data stream to obtain the first data stream after order-preserving processing. Specifically, it can be used to implement the determining function of step 705 and to execute the implicit steps included in step 705.

[0286] The output module 2330 is used to output the first data stream after the order-preserving processing. Specifically, it can be used to implement the output function of step 706 and execute the implicit steps included in step 706.

[0287] In one alternative approach, for each of the plurality of data packet groups, the paths by which each data packet in the data packet group is transmitted to the receiving device are the same.

[0288] In an alternative embodiment, the first determining module 2320 is further configured to generate a first congestion state of the receiving device based on the out-of-order status of the data packet groups received by the receiving device.

[0289] The output module 2330 is also used to send the first congestion status to the sending device.

[0290] In an alternative embodiment, the first determining module 2320 is configured to generate a second congestion state based on the out-of-order status of data packet groups sent by the transmitting device received at each input port of the receiving device. The second congestion state includes the congestion state of the transmitting device at each input port.

[0291] The output module 2330 is also used to send the second congestion status to the sending device.

[0292] The detailed process of data transmission by the data transmission device shown in Figure 22 is described in the preceding embodiments and will not be repeated here. The detailed process of data reception by the data reception device shown in Figure 23 is also described in the preceding embodiments and will not be repeated here. This module division is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be indirect couplings or communication connections through interfaces, devices, or modules, or they may be electrical, mechanical, or other forms of connection.

[0293] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0294] Furthermore, the modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or in software.

[0295] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0296] The hardware structure of the device is described below. This device can be either a transmitting or receiving device.

[0297] In one alternative embodiment, as shown in Figure 24, the device includes a main control board 2410 and a first interface board 2430.

[0298] The main control board 2410, also known as the main processing unit (MPU) or route processor card, is used to control and manage various components in the device, including route calculation, device management, device maintenance, and protocol processing functions. The main control board 2410 includes a central processing unit 2411 and a memory 2412.

[0299] The first interface board 2430 is also called a line processing unit (LPU), line card, or service board. The first interface board 2430 provides various service interfaces and implements packet forwarding. Service interfaces include, but are not limited to, Ethernet interfaces, POS (packet over SONET / SDH) interfaces, etc., with Ethernet interfaces including, for example, flexible Ethernet clients (FlexE clients). The first interface board 2430 includes: a first central processing unit 2431, a first network processor 2432, a first forwarding table entry memory 2434, and a first physical interface card (PIC) 2433.

[0300] The first central processing unit 2431 on the first interface board 2430 is used to control and manage the first interface board 2430 and communicate with the central processing unit 2411 on the main control board 2410.

[0301] The first network processor 2432 is used to implement packet forwarding processing. The first network processor 2432 may be in the form of a forwarding chip. Specifically, the first network processor 2432 forwards received packets based on the forwarding table stored in the first forwarding table entry memory 2434. If the destination address of the packet is the address of a device, the packet is sent to the CPU (such as the central processing unit 2411) for processing; if the destination address of the packet is not the address of a device, the next hop and outgoing interface corresponding to the destination address are looked up in the forwarding table according to the destination address, and the packet is forwarded to the outgoing interface corresponding to the destination address. Uplink packet processing includes: packet ingress interface processing, forwarding table lookup; downlink packet processing includes: forwarding table lookup, etc.

[0302] The first physical interface card 2433 is used to implement physical layer interfacing functions. Raw traffic enters the first interface board 2430 through this card, and processed packets are sent out from the first physical interface card 2433. The first physical interface card 2433, also called a daughter card, can be installed on the first interface board 2430. It is responsible for converting photoelectric signals into packets, performing validity checks on the packets, and forwarding them to the first network processor 2432 for processing. In some embodiments, the central processing unit can also perform the functions of the first network processor 2432, such as implementing software forwarding based on a general-purpose CPU, thus eliminating the need for the first network processor 2432 in the first physical interface card 2433.

[0303] Optionally, the device includes multiple interface boards, such as a second interface board 2440, which includes a second central processing unit 2441, a second network processor 2442, a second forwarding table entry memory 2444, and a second physical interface card 2443.

[0304] Optionally, the device also includes a switching fabric board 2420. The switching fabric board 2420 is also referred to, for example, as a switch fabric unit (SFU). In the case where the network device has multiple first interface boards 2430, the switching fabric board 2420 is used to perform data exchange between the interface boards. For example, the first interface board 2430 and the second interface board 2440 communicate, for example, through the switching fabric board 2420.

[0305] The main control board 2410 and the first interface board 2430 are coupled. For example, the main control board 2410, the first interface board 2430, the second interface board 2440, and the switching network board 2420 are interconnected via a system bus and a system backplane. In an alternative embodiment, an inter-process communication (IPC) channel is established between the main control board 2410 and the first interface board 2430, and the main control board 2410 and the first interface board 2430 communicate through the IPC channel.

[0306] Logically, the device includes a control plane and a forwarding plane. The control plane includes a main control board 2410 and a first central processing unit 2431, while the forwarding plane includes various components that perform forwarding, such as a first forwarding table entry memory 2434, a first physical interface card 2433, and a first network processor 2432. The control plane performs functions such as router operation, generating forwarding tables, processing signaling and protocol messages, and configuring and maintaining the device's status. The control plane distributes the generated forwarding tables to the forwarding plane. In the forwarding plane, the first network processor 2432 looks up and forwards messages received by the first physical interface card 2433 based on the forwarding tables distributed by the control plane. The forwarding tables distributed by the control plane are stored, for example, in the first forwarding table entry memory 2434. In some embodiments, the control plane and the forwarding plane are completely separate and not on the same device.

[0307] The operation on the second interface board 2440 is the same as that on the first interface board 2430, and will not be described again for simplicity. The device in this embodiment can correspond to the data transmission method or data reception method in the above-described method embodiments. The main control board 2410, the first interface board 2430 and / or the second interface card 2440 in the device, for example, implement the functions and / or various steps of the data transmission method or data reception method in the above-described method embodiments, and will not be described again for simplicity.

[0308] It's worth noting that a device may have one or more main control boards, including primary and backup boards. It may also have one or more interface boards; the more powerful the device's data processing capabilities, the more interface boards it provides. Each interface board may also have one or more physical interface cards. A switching network board may or may not exist; multiple boards can share the load and provide redundancy. In a centralized forwarding architecture, the device may not need a switching network board, as the interface boards handle the entire system's business data processing. In a distributed forwarding architecture, the device can have at least one switching network board, enabling data exchange between multiple interface boards and providing high-capacity data exchange and processing capabilities. Therefore, the data access and processing capabilities of a distributed architecture device are greater than those of a centralized architecture device. Alternatively, the device can also consist of a single board, without a switching network board. The functions of the interface board and the main control board are integrated on this one board. In this case, the central processing unit (CPU) on the interface board and the CPU on the main control board can be combined into a single CPU to perform the combined functions. This type of device has lower data exchange and processing capabilities (e.g., low-end switches or routers). The specific architecture adopted depends on the specific network deployment scenario, and no restrictions are imposed here.

[0309] In another alternative embodiment, as shown in Figure 25, the device may optionally be implemented using a general bus architecture. The device includes a bus 2502, a processor 2504, a memory 2506, and a communication interface 2508. The processor 2504, memory 2506, and communication interface 2508 communicate with each other via the bus 2502. The device may be a switch or a router. It should be understood that this application does not limit the number of processors or memories in the device.

[0310] Bus 2502 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 25, but this does not imply that there is only one bus or one type of bus. Bus 2504 can include pathways for transmitting information between various components of the device (e.g., memory 2506, processor 2504, communication interface 2508).

[0311] Processor 2504 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0312] The memory 2506 may include volatile memory, such as random access memory (RAM). The memory 2506 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0313] The memory 2506 stores executable program code, which the processor 2504 executes to implement the aforementioned data transmission or data reception method. That is, the memory 2506 stores instructions for performing the data transmission or data reception method.

[0314] The communication interface 2508 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the device and other devices or communication networks.

[0315] In an alternative embodiment, this application also provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor in the device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the device to perform the processes shown in Figures 3 and 7.

[0316] In this application, the terms "first" and "second," etc., are used to distinguish identical or similar items that have substantially the same function and purpose. It should be understood that there is no logical or temporal dependency between "first" and "second," nor does it limit the quantity or execution order. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another. For example, without departing from the scope of various examples, a first data packet group can be referred to as a second data packet group, and similarly, a second data packet group can be referred to as a first data packet group. Both the first and second data packet groups can be data packet groups, and in some cases, they can be separate and different data packet groups.

[0317] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of data transmission, characterized by, The method is applied to a sending device, and comprises the following steps: sending a first data packet group on a first egress port of the sending device, the first data packet group belonging to a first data flow, and the first data packet group comprising a plurality of first data packets; determining a second egress port of the first data flow on the sending device, the first egress port being different from the second egress port; determining a second data packet group according to packet group granularity information, the packet group granularity information being used to divide the first data flow into a plurality of packet groups, the second data packet group belonging to the first data flow, and the second data packet group comprising a plurality of second data packets; sending the second data packet group on the second egress port.

2. The method of claim 1, wherein, The number of the first data packets comprised in the first data packet group is greater than the number of the second data packets comprised in the second data packet group, so as to prevent the egress port of the short flow switching sending traffic. The method further comprises the following steps: determining a third egress port of the first data flow on the sending device, the third egress port being different from the first egress port and the second egress port; determining a third data packet group according to the packet group granularity information, the third data packet group belonging to the first data flow, and the third data packet group comprising a plurality of third data packets; sending the third data packet group on the third egress port.

3. The method of claim 1, wherein, Before the step of sending the first data packet group on the first egress port of the sending device, the method further comprises the following step: sending a third data packet group on the first egress port, the third data packet group belonging to the first data flow, and the third data packet group comprising a plurality of third data packets.

4. The method according to any one of claims 1 to 3, characterized in that, The number of the first data packets comprised in the first data packet group is equal to the number of the second data packets comprised in the second data packet group.

5. The method according to claim 1 or 2, characterized in that, The number of the first data packets in the first data packet group is determined according to historical sending traffic statistical information of the sending device.

6. The method according to any one of claims 1 to 5, characterized in that, The packet group granularity information comprises a data amount and / or a sending period.

7. The method of claim 6, wherein, The packet group granularity information comprises the data amount. The data amount is determined according to a bandwidth error between egress ports of the sending device; or The data amount is determined according to a round-trip time (RTT) and a connection bandwidth between the sending device and a receiving device of the first data flow; or The data amount is determined according to an input equalization degree target.

8. The method according to any one of claims 1 to 7, characterized in that, Before the step of sending the first data packet group on the first egress port of the sending device, the method further comprises the following steps: receiving a first congestion state sent by a receiving device of the first data flow, the first congestion state being used to indicate a congestion state of the receiving device; determining, based on the first congestion state, that the receiving device is not congested.

9. The method of claim 8, wherein, The method further comprises the following steps: if it is determined, based on the first congestion state, that the receiving device is congested, switching a mode of sending the first data flow to a target load balancing mode, wherein the data packets of the first data flow are not out of order at the receiving device according to the target load balancing mode.

10. The method according to any one of claims 1 to 9, characterized in that, The method further comprises the following steps: The receiving device receiving the first data stream sends a second congestion state, which is used to indicate the congestion state of the sending device corresponding to each ingress port of the receiving device; The determination of the second egress port of the first data stream on the sending device comprises: Based on the second congestion state, the egress port of the sending device to the non-congestion ingress port of the receiving device is determined; Among the determined egress ports, the second egress port different from the first egress port is selected.

11. The method according to any one of claims 1 to 10, characterized in that, The paths of the multiple first data packets in the first data packet group to the receiving device of the first data stream are the same; The paths of the multiple second data packets in the second data packet group to the receiving device are the same.

12. A method of data reception, characterized by, The method applied to the receiving device comprises: The receiving device receiving the first data stream sent by the sending device comprises multiple data packets, which form multiple data packet groups, the first data packet group and the second data packet group in the multiple data packet groups are sent from the first egress port and the second egress port of the sending device respectively, the first data packet group is sent before the second data packet group, the first data packet group comprises multiple first data packets, the second data packet group comprises multiple second data packets, the second data packet group is determined according to packet group granularity information, and the packet group granularity information is used to divide the first data stream into multiple packet groups; Each data packet belongs to a data packet group, and the data packet groups of the first data stream are processed in sequence to obtain a first data stream processed in sequence; The first data stream processed in sequence is output.

13. The method of claim 12, wherein, For each data packet group in the multiple data packet groups, the paths of the multiple data packets in the data packet group to the receiving device are the same.

14. The method according to claim 12 or 13, characterized in that, The method further comprises: Based on the out-of-order situation of the receiving device receiving the data packet group, a first congestion state of the receiving device is generated; The first congestion state is sent to the sending device.

15. The method according to any one of claims 12 to 14, characterized in that, The method further comprises: Based on the out-of-order situation of each ingress port of the receiving device receiving the data packet group sent by the sending device, a second congestion state is generated, and the second congestion state comprises the congestion state of the sending device corresponding to each ingress port; The second congestion state is sent to the sending device.

16. An apparatus for data transmission, the apparatus comprising: The apparatus applied to the sending device comprises: A sending module is configured to send a first data packet group on a first egress port of the sending device, the first data packet group belongs to a first data stream, and the first data packet group comprises multiple first data packets; A determination module is configured to determine a second egress port of the first data stream on the sending device, and the first egress port is different from the second egress port; A second data packet group is determined according to packet group granularity information, the packet group granularity information is used to divide the first data stream into multiple packet groups, the second data packet group belongs to the first data stream, and the second data packet group comprises multiple second data packets; The sending module is further configured to send the second data packet group on the second egress port.

17. The apparatus of claim 16, wherein, The number of the first data packets included in the first data packet group is determined according to historical sending traffic statistical information of the sending device. The determining module is further configured to: determine a third out port of the sending device for the first data flow, the third out port being different from the first out port and the second out port; determine a third data packet group according to the packet group granularity information, the third data packet group belonging to the first data flow, and the third data packet group including a plurality of third data packets; The sending module is further configured to send the third data packet group on the third out port.

18. The apparatus of claim 16 or 17, wherein, The number of the first data packets in the first data packet group is determined according to historical sending traffic statistical information of the sending device.

19. The apparatus of any one of claims 16 to 18, wherein, The packet group granularity information includes a data amount and / or a sending period.

20. The apparatus of claim 19, wherein, The packet group granularity information includes the data amount. The data amount is determined according to a bandwidth error between out ports of the sending device; or The data amount is determined according to a round trip time (RTT) and a connection bandwidth between the sending device and a receiving device of the first data flow; or The data amount is determined according to an input balance target.

21. The apparatus of any one of claims 16 to 20, wherein, The apparatus further includes a receiving module configured to receive a first congestion state sent by a receiving device of the first data flow before the sending module sends the first data packet group on the first out port of the sending device, the first congestion state being used to indicate a congestion state of the receiving device. The determining module is further configured to determine that the receiving device is not congested based on the first congestion state.

22. The apparatus of claim 21, wherein, The determining module is further configured to: if it is determined that the receiving device is congested based on the first congestion state, switch a mode of sending the first data flow to a target load balancing mode, wherein the first data flow is sent according to the target load balancing mode, and data packets of the first data flow are not out of order at the receiving device.

23. An apparatus for data reception, the apparatus comprising: The apparatus is applied to a receiving device and includes: a receiving module configured to receive a plurality of data packets of a first data flow sent by a sending device, the plurality of data packets forming a plurality of data packet groups, a first data packet group and a second data packet group in the plurality of data packet groups being sent on a first out port and a second out port of the sending device respectively, the first data packet group being sent before the second data packet group, the first data packet group including a plurality of first data packets, the second data packet group including a plurality of second data packets, and the second data packet group being determined according to packet group granularity information, the packet group granularity information being used to divide the first data flow into a plurality of packet groups; a first determining module configured to determine a data packet group to which each data packet belongs, and to perform in-order processing on the data packet groups of the first data flow to obtain an in-order processed first data flow; an output module configured to output the in-order processed first data flow.

24. A transmitting device, comprising: The sending device includes a processor and a memory, and the processor is configured to execute computer instructions stored in the memory, so that the sending device performs the method according to any one of claims 1 to 11.

25. A receiving device, comprising: The receiving device comprises a processor and a memory, the processor being configured to execute computer instructions stored in the memory, so that the receiving device performs the method according to any one of claims 12-15.

26. A computer-readable storage medium, characterized in that, The storage medium stores at least one computer instruction, which is read by the processor to enable the sending device to perform the method according to any one of claims 1-11.

27. A computer-readable storage medium, characterized in that, The storage medium stores at least one computer instruction, which is read by the processor to enable the receiving device to perform the method according to any one of claims 12-15.

28. A communication system, characterized by The communication system comprises a sending device and a receiving device; The sending device is configured to perform the method according to any one of claims 1-11; The receiving device is configured to perform the method according to any one of claims 12-15.

Images