Message forwarding method, communication node, communication system and related product

Abstract

The present application belongs to the technical field of networks. Disclosed are a message forwarding method, a communication node, a communication system and a related product. The method is applied to a first network device in a communication network. The method comprises: on the basis of indication information carried in a data message to be forwarded, determining a plurality of first paths, wherein the indication information indicates a second network device that is the last to forward the data message in a communication network, and the plurality of first paths refer to message transmission paths from a first network device to the second network device; and on the basis of global path congestion states respectively corresponding to the plurality of first paths, selecting a target path from among the plurality of first paths, wherein the global path congestion states indicate the global congestion conditions of the corresponding first paths, and the target path is used for forwarding the data message to the second network device. In this way, when a first network device forwards a data message, the congestion condition of an entire message transmission path is taken into comprehensive consideration, thereby improving the accuracy of target path selection.

Description

Message forwarding methods, communication nodes, communication systems and related products

[0001] This application claims priority to Chinese Patent Application No. 202410514055.8, filed on April 23, 2024, entitled "Message Forwarding Method, Communication Node, Communication System and Related Products", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of network technology, and in particular to a message forwarding method, communication node, communication system and related products. Background Technology

[0003] In data centers, the Clos architecture (a network architecture named after a person) is widely used to build high-performance, high-reliability network architectures to meet the ever-increasing demands for data processing and transmission. The Clos architecture typically employs a multi-layered switch connection to form a highly interconnected network topology. As shown in Figure 1, taking a two-layer Clos architecture as an example, the top-of-rack (TOR) switch, directly connected to the servers, serves as the first layer in the network architecture, providing network access services to the connected servers. Simultaneously, each TOR switch in the first layer is also connected to a spine switch (also known as a spine switch) in the second layer, enabling communication between servers through the two-layer Clos architecture.

[0004] In related technologies, for TOR switches connected to multiple servers, when forwarding data streams sent by multiple servers, the TOR switch needs to perform load balancing on multiple data streams to determine the network transmission path of each data stream in the Clos architecture. TOR switches typically employ packet-level load balancing for data stream forwarding. That is, the TOR switch does not distinguish the data stream to which a data packet belongs. When forwarding each data packet, the TOR switch determines the target egress port for sending each data packet from its egress ports based on the packet backlog and congestion status of the egress ports connected to each Spine switch, thus distributing each data packet packet across multiple egress ports for forwarding.

[0005] However, in the above packet-level load balancing schemes, the TOR switch has low accuracy in determining the packet transmission path of data packets. Summary of the Invention

[0006] This application provides a message forwarding method, a communication node, a communication system, and related products, which can improve the accuracy of packet-by-packet routing when network devices forward data packets. The technical solution is as follows:

[0007] Firstly, a message forwarding method is provided, applied to a first network device in a communication network, the method comprising:

[0008] Based on the indication information carried in the data packet to be forwarded, multiple first paths are determined. The indication information indicates the second network device in the communication network that is the last to forward the data packet. The multiple first paths refer to the packet transmission paths from the first network device to the second network device. Based on the global path congestion status corresponding to the multiple first paths, a target path is selected from the multiple first paths. The global path congestion status indicates the global congestion situation of the corresponding first path. The target path is used to forward the data packet to the second network device.

[0009] Therefore, for a data packet to be forwarded, the first network device first determines the second network device in the communication network that was the last to forward the data packet based on the indication information carried in the data packet, and then determines the global path congestion status corresponding to the multiple packet transmission paths (i.e., first paths) from the first network device to the second network device. For each first path, since the global path congestion status indicates the overall congestion situation of the entire first path and reflects the congestion situation of all network devices on that first path when forwarding data packets, the first network device selects the target path for forwarding the data packet from the multiple first paths based on the global path congestion status corresponding to each of the multiple first paths. In other words, when forwarding data packets, the first network device comprehensively considers the congestion situation of the entire packet transmission path, improving the accuracy of target path selection.

[0010] Furthermore, when the first network device is forwarding multiple data packets, the packet forwarding method provided in this application can improve the accuracy of packet-by-packet routing, thereby making the first network device perform better in load balancing when forwarding multiple data packets.

[0011] Optionally, before selecting a target path from the plurality of first paths, the method further includes: obtaining the remote path congestion status corresponding to each of the plurality of first paths, wherein the remote path congestion status indicates the congestion status of network devices other than the first network device on the corresponding first path; and determining the global path congestion status corresponding to each of the plurality of first paths based on the remote path congestion status corresponding to each of the plurality of first paths.

[0012] Since the remote path congestion status of the first path indicates the congestion status of network devices other than the first network device on that first path, after determining the remote path congestion status of the first path, it can be directly determined as the global path congestion status of the corresponding first path. That is, the remote path congestion status of the first path can be equivalent to the global path congestion status of the first path.

[0013] It should be noted that when determining the global path congestion status of the first path, in addition to the remote path congestion status, the first network device can also combine other information to comprehensively analyze the global path congestion situation of the first path, thereby determining the global path congestion status of the first path.

[0014] Optionally, determining the global path congestion status corresponding to each of the multiple first paths based on the remote path congestion status corresponding to each of the multiple first paths includes: determining the port congestion status corresponding to at least one outgoing port in the first network device, wherein the at least one outgoing port is a port connected to the multiple first paths, and the port congestion status indicates the congestion situation of the corresponding outgoing port; and determining the global path congestion status corresponding to each of the multiple first paths based on the port congestion status corresponding to the at least one outgoing port and the remote path congestion status corresponding to each of the multiple first paths.

[0015] In other words, when determining the global path congestion status for each first path, the congestion status of at least one outgoing port connected to the first network device and multiple first paths is first determined. Then, the congestion status of network devices other than the first network device on each first path is obtained to obtain the remote path congestion status. Finally, the global path congestion status for each of the multiple first paths is determined by combining the port congestion status corresponding to at least one outgoing port and the remote path congestion status corresponding to each of the multiple first paths. Thus, by comprehensively analyzing the global path congestion status corresponding to each of the multiple first paths based on the port congestion status corresponding to at least one outgoing port and the remote path congestion status corresponding to each of the multiple first paths, the accuracy of determining the global path congestion status is improved.

[0016] It should be noted that the operations of determining the port congestion status corresponding to at least one output port and determining the remote path congestion status of each first path can be performed simultaneously or sequentially. This application does not restrict the execution order.

[0017] If the port congestion status of the outgoing port connected to the first path indicates that the corresponding outgoing port is congested, and / or the congestion status of the remote path corresponding to the first path indicates that the network devices on the first path other than the first network device are congested, then it is determined that the first path is congested; otherwise, it is determined that the first path is not congested.

[0018] Optionally, obtaining the remote path congestion status corresponding to the plurality of first paths includes: receiving an announcement message, the announcement message carrying the port congestion status corresponding to at least one ingress port in the second network device, the at least one ingress port being a port connected to the plurality of first paths, the port congestion status indicating the congestion status of the corresponding ingress port; and determining the remote path congestion status corresponding to the plurality of first paths based on the port congestion status corresponding to the at least one ingress port.

[0019] Optionally, the notification message includes a port congestion status field, which carries the port congestion status corresponding to the at least one ingress port.

[0020] That is, the first network device determines the remote path congestion status of multiple first paths connected to the at least one ingress port based on the port congestion status corresponding to at least one ingress port of the second network device carried in the notification message.

[0021] Optionally, the notification message also carries an intra-flow packet out-of-order status, which indicates the out-of-order status of packets that have arrived at the second network device in the data flow to which the data packet belongs; determining the remote path congestion status corresponding to the plurality of first paths based on the port congestion status corresponding to the at least one ingress port includes: determining the remote path congestion status corresponding to the plurality of first paths based on the port congestion status corresponding to the at least one ingress port and the intra-flow packet out-of-order status.

[0022] Optionally, the notification message includes a port congestion status field and an out-of-order status field. The port congestion status field is used to carry the port congestion status corresponding to the at least one inbound port, and the out-of-order status field is used to carry the out-of-order degree of the packets within the flow.

[0023] In other words, when the notification message carries the port congestion status and intra-flow packet out-of-order degree of at least one ingress port of the second network device receiving data packets, since the at least one ingress port is a port connected to multiple first paths, the port congestion status of the at least one ingress port can reflect the congestion situation during the transmission of data packets received by the ingress port from the first network device to the second network device; at the same time, since the intra-flow packet out-of-order degree can also reflect the congestion situation during the transmission of data packets from the first network device to the second network device to a certain extent, after receiving the port congestion status and intra-flow packet out-of-order degree of the at least one ingress port, the first network device can comprehensively analyze the congestion situation of the packet transmission path from the first network device to the second network device, thereby obtaining the remote path congestion status corresponding to each of the multiple first paths.

[0024] Therefore, when analyzing the remote path congestion status of the first path, the first network device not only considers the data packets received by the second network device, but also the out-of-order status of the data packets themselves, thereby improving the accuracy and reliability of determining the remote path congestion status of the first path.

[0025] Optionally, obtaining the remote path congestion status corresponding to the plurality of first paths includes: receiving a notification message, the notification message carrying the remote path congestion status corresponding to the plurality of first paths.

[0026] Optionally, the notification message includes a path congestion status field, which carries the congestion status of the remote paths corresponding to the plurality of first paths.

[0027] Therefore, when the notification message carries the congestion status of the remote paths corresponding to multiple first paths, the first network device can directly obtain the congestion status of the remote paths corresponding to each first path by parsing the notification message. Since the second network device directly reports the congestion status of the remote paths corresponding to multiple first paths, the first network device can quickly determine the congestion status of the remote paths of each first path based on the notification message, thus improving the efficiency of the first network device in determining the congestion status of the remote paths of multiple first paths.

[0028] Optionally, selecting a target path from the multiple first paths based on the global path congestion status corresponding to each of the multiple first paths includes: determining the out-of-order type of the data stream to which the data packet belongs; determining a path selection strategy for the data packet based on the out-of-order type; and selecting the target path from the multiple first paths based on the path selection strategy.

[0029] Optionally, the out-of-order flow type includes a first type of out-of-order flow and a second type of out-of-order flow; determining the out-of-order flow type of the data flow to which the data packet belongs includes: if the intra-flow packet out-of-order degree of the data flow to which the data packet belongs is greater than the packet out-of-order threshold, then the out-of-order flow type of the data flow to which the data packet belongs is determined to be the first type of out-of-order flow; if the intra-flow packet out-of-order degree of the data flow to which the data packet belongs is less than or equal to the packet out-of-order threshold, then the out-of-order flow type of the data flow to which the data packet belongs is determined to be the second type of out-of-order flow.

[0030] Optionally, the path selection strategy includes a first selection strategy and a second selection strategy. The first selection strategy indicates that the data packet is transmitted using a first type of path, which is a congested path. The second selection strategy indicates that the data packet is transmitted using a second type of path, which is a non-congested path. Determining the path selection strategy for the data packet based on the out-of-order flow type includes: if the out-of-order flow type is the first type of out-of-order flow, then the path selection strategy for the data packet is determined to be the first selection strategy; if the out-of-order flow type is the second type of out-of-order flow, then the path selection strategy for the data packet is determined to be the second selection strategy.

[0031] Considering that after a data packet is transmitted to the second network device, the second network device needs to reorder the data packets within the data stream to which the data packet belongs, when the data packets in a certain data stream are severely out of order, even if the first path without congestion is used to forward the data packets in that data stream to the second network device, the second network device cannot reorder them in time. It needs to wait until all the out-of-order data packets before that data packet have arrived before it can reorder the received data packets according to the packet sequence number.

[0032] Therefore, in order to improve the out-of-order reordering efficiency of the second network device, when the first network device performs packet-by-packet routing for data packets destined for the second network device, it can adopt different path selection strategies to select the target path for the data packet based on the out-of-order status of the packets within the data stream to which the data packet belongs.

[0033] Optionally, before determining the global path congestion status corresponding to the multiple first paths, the method includes: determining the packet container size of each outgoing port in the first network device, wherein the packet container size indicates the total number of bytes of packets sent by the corresponding outgoing port in the current statistical period; and determining the at least one outgoing port from the outgoing ports whose packet container size is less than the container threshold.

[0034] Therefore, for the first network device, based on the packet container size of each outgoing port, at least one outgoing port capable of sending the data packet can be pre-selected from those whose packet container size is less than the container threshold before forwarding the data packet. In other words, each time the first network device sends a data packet, it can select at least one outgoing port capable of sending the current data packet based on the data packets already sent on each outgoing port. Thus, during the data packet sending process, the first network device can ensure that each outgoing port is used evenly, making effective use of the bandwidth of each outgoing port, thereby improving the bandwidth utilization rate of each outgoing port in the first network device.

[0035] Optionally, the data stream to which the data packet belongs is a first type of data stream or a second type of data stream; wherein, the five-tuple information of each data packet in the first type of data stream is the same, and each data packet in the second type of data stream corresponds to the same source network device and the same destination network device in the communication network.

[0036] Therefore, when forwarding multiple data packets, the first network device can first determine whether the multiple data packets belong to a single data stream or multiple data streams, and then use different load balancing granularities for packet-by-packet forwarding. In other words, when implementing the packet forwarding method provided in this application, the first network device can use different load balancing granularities to perform packet-by-packet routing and forwarding according to actual needs, improving the flexibility of the first network device when performing packet-level load balancing.

[0037] Secondly, a message forwarding method is provided, applied to a target device, the method comprising:

[0038] The congestion status of at least one ingress port corresponding to the received data packet is determined, wherein the port congestion status indicates the congestion condition of the corresponding ingress port; if an announcement message trigger condition is detected, an announcement message is sent to a first network device in the communication network based on the port congestion status corresponding to the at least one ingress port, so as to instruct the first network device to determine the global path congestion status corresponding to multiple first paths based on the announcement message, wherein the first network device is a network device in the communication network that forwards the data packet before a second network device, and the multiple first paths refer to the packet transmission paths from the first network device to the second network device, and the global path congestion status indicates the global congestion condition of the corresponding first path.

[0039] Therefore, when the target device receives a data packet, it can determine the port congestion status corresponding to at least one ingress port of the received data packet. Based on the port congestion status corresponding to each of these ingress ports, and upon detecting a notification message trigger, the target device sends a notification message to the first network device. This instructs the first network device to determine the global path congestion status of the packet transmission path from the first network device to the target device based on the information carried in the notification message. Thus, by having the target device send notification messages to the first network device, the first network device is assisted in determining the global path congestion status of the packet transmission path, thereby more efficiently determining the target path for forwarding each data packet and improving the accuracy of packet-by-packet routing when forwarding data packets.

[0040] Optionally, the method further includes: determining the in-flow packet out-of-order degree of the data stream to which the data packet belongs, wherein the in-flow packet out-of-order degree indicates the out-of-order status of packets that have arrived at the target device in the data stream to which the data packet belongs; and sending an announcement message to the first network device based on the port congestion status corresponding to the at least one ingress port, comprising: sending an announcement message to the first network device based on the port congestion status corresponding to the at least one ingress port and the in-flow packet out-of-order degree.

[0041] Optionally, the in-stream message out-of-order degree is the difference between the number of data packets that have not arrived between the message sequence number of the currently arriving data packet and the message sequence number of the expected data packet, and the number of data packets that arrived earlier than the expected data packet.

[0042] Optionally, determining the port congestion status corresponding to at least one ingress port includes: determining the proportion of congested packets corresponding to the at least one ingress port, wherein the proportion of congested packets is the ratio between the total number of data packets received in the corresponding ingress port and the number of data packets carrying congestion flags in the data packets received in the ingress port; if the proportion of congested packets is greater than a port congestion threshold, then the corresponding ingress port is determined to be congested; if the proportion of congested packets is less than or equal to the port congestion threshold, then the corresponding ingress port is determined not to be congested.

[0043] The congestion flag carried in the data packet can be an explicit congestion notification (ECN) flag.

[0044] In other words, for a data packet forwarded from the first network device, when the data packet is forwarded by other network devices, those devices can tag the data packet based on their own congestion levels when receiving the data packet, thus indicating the congestion situation at each intermediate network device. Therefore, when the target device receives the data packet, it can parse the ECN tag from it and determine the congestion situation when the data packet was forwarded at other network devices after being sent from the first network device.

[0045] Optionally, the notification message triggering condition includes at least one of the following:

[0046] The out-of-order degree of the intra-stream packets is greater than the out-of-order threshold;

[0047] Congestion exists in any one of the at least one inlet ports;

[0048] The time interval since the last sending of the notification message has reached the notification interval duration.

[0049] Optionally, the notification message carries the port congestion status corresponding to the at least one ingress port.

[0050] Optionally, the notification message includes a port congestion status field, which carries the port congestion status corresponding to the at least one ingress port.

[0051] That is, when the notification message carries the port congestion status corresponding to at least one ingress port, the first network device analyzes the remote path congestion status of the first path connected to the at least one ingress port based on the port congestion status corresponding to the at least one ingress port, thereby determining the global path congestion status of the first path.

[0052] Optionally, the notification message carries the port congestion status corresponding to the at least one ingress port and the out-of-order status of the in-flow packets.

[0053] Optionally, the notification message includes a port congestion status field and an out-of-order status field. The port congestion status field is used to carry the port congestion status corresponding to the at least one inbound port, and the out-of-order status field is used to carry the out-of-order degree of the packets within the flow.

[0054] In other words, when the notification message carries the port congestion status and in-flow packet out-of-order degree corresponding to at least one inbound port, the first network device determines the remote path congestion status of the first path connected to the at least one inbound port based on the port congestion status and in-flow packet out-of-order degree, and then determines the global path congestion status of the first path. Since determining the remote path congestion status of the first path comprehensively considers both the port congestion status and in-flow packet out-of-order degree corresponding to at least one inbound port, the accuracy of determining the remote path congestion status is improved.

[0055] Optionally, the notification message carries the remote path congestion status of multiple first paths corresponding to the data packet, and the remote path congestion status indicates the congestion status of network devices other than the first network device on the corresponding first path; before sending the notification message to the first network device, the method further includes: determining the remote path congestion status corresponding to the multiple first paths based on the port congestion status corresponding to the at least one ingress port respectively.

[0056] Optionally, the notification message carries the remote path congestion status of multiple first paths corresponding to the data packet, and the remote path congestion status indicates the congestion status of network devices other than the first network device on the corresponding first path; before sending the notification message to the first network device, the method further includes: determining the remote path congestion status corresponding to the multiple first paths based on the out-of-order degree of the intra-flow packets and the port congestion status corresponding to the at least one ingress port respectively.

[0057] Optionally, the notification message includes a path congestion status field, which carries the congestion status of the remote paths corresponding to the plurality of first paths.

[0058] Therefore, the target device can autonomously calculate the remote path congestion status of the first path and send it to the first network device through an announcement message. This allows the first network device to quickly determine the remote path congestion status of each first path based on the announcement message, thereby improving the efficiency of the first network device in determining the remote path congestion status of multiple first paths.

[0059] Optionally, the data stream to which the data packet belongs is a first type of data stream or a second type of data stream; wherein, the five-tuple information of each data packet in the first data stream is the same, and each data packet in the second type of data stream corresponds to the same source network device and the same destination network device in the communication network.

[0060] Specifically, when the target device is a second network device, the data stream to which the data packet belongs can be either a first-type data stream or a second-type data stream; when the target device is a second computing node, the data stream to which the data packet belongs is a first-type data stream.

[0061] Thirdly, a message forwarding apparatus is provided, which has the function of implementing the message forwarding method behavior described in the first aspect above. The message forwarding apparatus includes at least one module, which is used to implement the message forwarding method provided in the first aspect above.

[0062] Fourthly, a message forwarding apparatus is provided, which has the function of implementing the message forwarding method behavior described in the second aspect above. The message forwarding apparatus includes at least one module for implementing the message forwarding method provided in the second aspect above.

[0063] Fifthly, a network device is provided, the network device including a processor and a memory, the memory being used to store a computer program for executing the packet forwarding method provided in the first aspect (or second aspect). The processor is configured to execute the computer program stored in the memory to implement the steps of the packet forwarding method described in the first aspect (or second aspect).

[0064] Optionally, the network device may further include a communication bus for establishing a connection between the processor and the memory.

[0065] In a sixth aspect, a computer device is provided, the computer device including a processor and a memory, the memory being used to store a computer program for executing the packet forwarding method provided in the second aspect. The processor is configured to execute the computer program stored in the memory to implement the steps of the packet forwarding method described in the second aspect.

[0066] Optionally, the computer device may further include a communication bus for establishing a connection between the processor and the memory.

[0067] In a seventh aspect, a communication node is provided, the communication node including a processor, the processor being configured to perform the steps of the message forwarding method described in the first aspect, or to perform the steps of the message forwarding method described in the second aspect.

[0068] Eighthly, a communication system is provided, the communication system including a first communication node and a second communication node, the first communication node being configured to perform the steps of the message forwarding method described in the first aspect, and the second communication node being configured to perform the steps of the message forwarding method described in the second aspect.

[0069] In a ninth aspect, a chip system is provided, the chip system including a processor, the processor being configured to support a communication node in implementing the steps of the message forwarding method described in the first aspect, or in implementing the steps of the message forwarding method described in the second aspect.

[0070] Optionally, the chip system also includes a memory for storing necessary program instructions and data for the communication node.

[0071] Optionally, the chip system may consist of chips or may include chips and other discrete devices.

[0072] In a tenth aspect, a computer-readable storage medium is provided, wherein the storage medium stores instructions that, when executed on a network device, cause the network device to perform the steps of the packet forwarding method described in the first aspect, or to perform the steps of the packet forwarding method described in the second aspect.

[0073] In an eleventh aspect, a computer-readable storage medium is provided, wherein the storage medium stores instructions that, when executed on a computing node, cause the computing node to perform the steps of the message forwarding method described in the second aspect above.

[0074] In a twelfth aspect, a computer program product containing instructions is provided, which, when executed on a network device, cause the network device to perform the steps of the packet forwarding method described in the first aspect, or to perform the steps of the packet forwarding method described in the second aspect.

[0075] Alternatively, a computer program is provided that, when run on a network device, causes the network device to perform the steps of the packet forwarding method described in the first aspect, or to perform the steps of the packet forwarding method described in the second aspect.

[0076] In a thirteenth aspect, a computer program product containing instructions is provided, which, when executed on a computing node, cause the computing node to perform the steps of the message forwarding method described in the second aspect above.

[0077] Alternatively, a computer program is provided that, when run on a computing node, causes the computing node to perform the steps of the message forwarding method described in the second aspect above.

[0078] The technical effects achieved by the third to thirteenth aspects mentioned above are similar to the technical effects achieved by the corresponding technical means in the first or second aspects mentioned above, and will not be repeated here. Attached Figure Description

[0079] Figure 1 is a schematic diagram of a two-layer Clos architecture provided in an embodiment of this application;

[0080] Figure 2 is a schematic diagram of the process of a network device forwarding data packets in the related technology provided in the embodiments of this application;

[0081] Figure 3 is a schematic diagram of data packet transmission in a network architecture provided in an embodiment of this application;

[0082] Figure 4 is a schematic diagram of a data packet forwarding process at a load balancing granularity provided in an embodiment of this application;

[0083] Figure 5 is a schematic diagram of another data packet forwarding process at a load balancing granularity provided in an embodiment of this application;

[0084] Figure 6 is a schematic diagram of another data packet forwarding process at a load balancing granularity provided in an embodiment of this application;

[0085] Figure 7 is a flowchart illustrating a message forwarding method provided in an embodiment of this application;

[0086] Figure 8 is a flowchart illustrating a method for determining global path congestion status according to an embodiment of this application;

[0087] Figure 9 is a flowchart illustrating the selection of a target path according to an embodiment of this application;

[0088] Figure 10 is a schematic diagram of another process for selecting a target path provided in an embodiment of this application;

[0089] Figure 11 is a flowchart illustrating another message forwarding method provided in an embodiment of this application;

[0090] Figure 12 is a schematic diagram of the triggering logic of an announcement message provided in an embodiment of this application;

[0091] Figure 13 is a schematic diagram of the format of a notification message provided in an embodiment of this application;

[0092] Figure 14 is a schematic diagram comparing the load balancing effects provided in an embodiment of this application;

[0093] Figure 15 is a schematic diagram of a message forwarding device provided in an embodiment of this application;

[0094] Figure 16 is a schematic diagram of another message forwarding device provided in an embodiment of this application;

[0095] Figure 17 is a schematic diagram of the structure of a network device provided in an embodiment of this application;

[0096] Figure 18 is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0097] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0098] To facilitate understanding, before providing a detailed explanation of the message forwarding method provided in the embodiments of this application, the relevant background and implementation environment involved in the embodiments of this application will be introduced first.

[0099] First, the relevant background information related to the embodiments of this application will be introduced.

[0100] With the rapid development of artificial intelligence (AI) technology, the parameters of AI models, the scale of computing clusters, and data traffic are also growing rapidly. This has led to increasingly higher demands on the network size of computing clusters and the communication bandwidth of nodes during the training process of AI models. Currently, the network architecture of computing clusters typically adopts a flat network topology. Among them, the Clos architecture, as an optimal topology for high-performance networks, is the most commonly used network topology in computing clusters for training AI models.

[0101] Referring to Figure 1, in a two-layer Clos architecture communication network, the top-of-rack (TOR) switch, directly connected to the server, serves as the first layer of the network architecture, providing network access services to multiple connected servers. In this case, the TOR0-TOR3 switches in the first layer can also be referred to as access layer network devices. Each TOR switch in the first layer is also connected to the spine switch (also called a Spine switch) in the second layer. The Spine0 and Spine1 switches in the second layer can also be referred to as core layer network devices. Thus, through the interconnection of multiple switches in the communication network, multiple servers can achieve communication connections through the two-layer Clos architecture.

[0102] It should be noted that Figure 1 is only used as an example to illustrate the network including four TOR switches and two Spine switches, and does not constitute a limitation on the number of network devices in the communication network. That is, the embodiments of this application do not limit the number of network devices in the communication network, which may include more or fewer switches, as long as the communication connection between multiple servers in the computing cluster can be realized.

[0103] Furthermore, in AI model training scenarios, the data traffic transmitted in the network is characterized by a small number of streams, large bandwidth per stream, bursts and synchronization of traffic, and periodic traffic. Moreover, as the scale of the computing cluster increases, the linearity between the overall computing power of the computing cluster and the number of processors included in the computing cluster decreases significantly. That is, when the number of processors included in the computing cluster increases to a certain extent, the computing power of the computing cluster stagnates. The main reason for this is that as the scale of the computing cluster increases, the possibility of uneven load distribution when the network devices in the network architecture of the computing cluster forward data traffic also increases.

[0104] Therefore, for each network device in the Clos architecture that forms a computing cluster, load balancing is required when forwarding the data traffic generated by each server in the AI ​​model training scenario.

[0105] Load balancing schemes for data traffic in a network can be categorized into three types based on their load balancing granularity, from largest to smallest: flow-level, sub-flow-level, and packet-level. Currently, when performing load balancing on data traffic in a network, access layer network devices connected to servers typically use packet-level load balancing to forward received data packets packet by packet.

[0106] In related technologies, in communication networks using a Clos architecture, when a TOR switch, acting as an access layer network device, forwards data packets sent by multiple servers, it determines the target output port for each data packet from the output ports of the TOR switch based on the packet backlog at its output ports connected to each Spine switch in the communication network, as well as the packet congestion at each Spine switch, so as to distribute each data packet packet by packet across multiple output ports for forwarding.

[0107] Referring to Figure 2, taking a two-layer Clos architecture communication network as an example, the first and second servers are connected to the first switch to access the network through the first switch, and the third and fourth servers are connected to the second switch to access the network through the second switch. The first and second switches can be connected via four switches (e.g., the third, fourth, fifth, and sixth switches). Specifically, data stream A, which the first server needs to send to the third server via the network, includes four data packets: a1, a2, a3, and a4. Similarly, data stream B, which the second server needs to send to the fourth server via the network, also includes four data packets: b1, b2, b3, and b4.

[0108] In related technologies, the process of the first switch forwarding data packets packet by packet is as follows: The first switch determines the port packet backlog status of its uplink outgoing ports connected to the third, fourth, fifth, and sixth switches. Simultaneously, the first switch interacts with the third, fourth, fifth, and sixth switches to obtain their respective packet congestion status. Then, based on the port packet backlog status of each of its own uplink outgoing ports and the packet congestion status of the adjacent switches (also known as next-hop switches) connected to those uplink outgoing ports, the first switch determines the uplink outgoing port for forwarding each data packet and forwards the data packet to the adjacent network layer next-hop switch through the corresponding uplink outgoing port. This allows the next-hop switch to continue transmitting the data packet until it is forwarded to the corresponding server.

[0109] Referring to Figure 2, after packet-by-packet routing, the first switch sends data packets b4 and a2 through the uplink output port connected to the third switch to forward them to the third switch. The third switch then forwards data packets b4 and a2 to the second switch, which finally forwards data packet a2 to the third server and data packet b4 to the fourth server.

[0110] Furthermore, after the second switch forwards the data packets to the third and fourth servers in the order of reception, the third and fourth servers use direct data placement (DDP) technology to receive and reorder the data packets in the data stream out of order. Finally, the third server receives all the data packets of data stream A, and the fourth server receives all the data packets of data stream B.

[0111] However, in the aforementioned related technologies, when the first switch performs packet-by-packet routing, it needs to communicate with the next-hop switches in the adjacent network layers, namely the third, fourth, fifth, and sixth switches, to obtain the packet congestion status of the switches in the adjacent network layers. This results in weak versatility of the solution. For example, when the types of switches in the adjacent network layers are different, the switches in the adjacent network layers may not be able to transmit packet congestion information, causing the first switch to be unable to obtain the packet congestion status of the next-hop switches, which seriously affects the load balancing effect of the first switch when forwarding data packets.

[0112] Moreover, when the first switch selects the uplink output port for forwarding data packets, it only considers the packet congestion of the switches adjacent to the first switch in the communication network (i.e., the third, fourth, fifth, and sixth switches). However, each packet transmission path between the first switch and the second switch may include multiple switches. Considering only the packet congestion of adjacent switches will result in low accuracy of the first switch in selecting routes on a packet-by-packet basis, thus leading to poor packet-level load balancing performance.

[0113] Based on this, embodiments of this application provide a packet forwarding method. After receiving a data packet, a first network device in a communication network, for each data packet to be forwarded, can determine the second network device that last forwarded the data packet in the communication network based on the indication information carried in the network packet, and determine multiple first paths that can forward the data packet from the first network device to the second network device. Then, based on the global path congestion status of the multiple first paths, a target path is selected from the multiple first paths to forward the data packet to the second network device through the target path. For each first path, since the global path congestion status indicates the overall congestion situation of the entire first path, it can reflect the congestion situation of all network devices on the first path when forwarding data packets. Therefore, selecting the target path for transmitting data packets based on the global path congestion status of the first path comprehensively considers the congestion situation of the entire first path, improves the accuracy of packet-by-packet routing by the network device, and makes the load balancing effect of the network device when forwarding data packets packet by packet better.

[0114] It should be understood that the training scenario of the AI ​​model described above is merely an exemplary scenario, intended to more clearly illustrate the technical solution of the embodiments of this application, and does not constitute a limitation on the technical solution provided by the embodiments of this application. The message forwarding method provided by the embodiments of this application can also be applied to other scenarios in which multiple computing nodes communicate through network devices, or in other communication networks. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solution provided by the embodiments of this application is also applicable to similar technical problems.

[0115] Secondly, the implementation environment involved in the embodiments of this application will be introduced.

[0116] The message forwarding method provided in this application is applied in a communication network, which includes multiple network devices. When multiple network devices establish communication connections, they can be connected using a single-layer network topology or a multi-layer network topology; this application does not impose any limitation on this.

[0117] Optionally, when the communication network operates on a data center / computing cluster, multiple network devices in the communication network can be connected using a multi-layered network topology, such as a Clos architecture. In this way, multiple computing nodes within the data center / computing cluster can communicate and connect through multiple network devices.

[0118] As an example, network devices can be devices with data packet forwarding functions such as switches, gateways, and routers, and computing nodes can be devices / devices / chips with computing and communication functions such as terminals, servers, and network cards. This application does not limit these aspects.

[0119] In one possible implementation, when the communication network architecture is a Clos architecture, each computing node in the data center can connect to a network device to access the network through that network device. Multiple network devices are interconnected through the Clos architecture, so that after each computing node is connected to a network device, it can communicate with other computing nodes in the entire data center through that network device, or that network device and other network devices in the Clos architecture.

[0120] As an example, as shown in Figure 3, a first network device in a communication network is connected to multiple first computing nodes, and a second network device is connected to multiple second computing nodes. When a first computing node needs to communicate with a second computing node through the first network device to send a data packet to the second computing node, after receiving the data packet sent by the first computing node, the first network device determines the global path congestion status corresponding to the multiple first paths, and then selects a target path from the multiple first paths based on the global path congestion status, and forwards the data packet according to the target path to the second network device, and then forwards the data packet to the second computing node through the second network device.

[0121] Among them, multiple first paths refer to the message transmission paths from the first network device to the second network device, and the global congestion status of the first path indicates the global congestion situation of the corresponding first path.

[0122] It should be noted that for each first path, which also includes at least one network device other than the first network device and the second network device, the global congestion status of the first path is determined by the port congestion status of the output port of the first network device, the congestion status of at least one network device other than the first network device and the second network device on the first path, and the port congestion status of the input port of the second network device, and can reflect the overall congestion status of the first path.

[0123] In some embodiments, the packet forwarding method provided in this application embodiment can be executed by a second network device. That is, when a data packet is forwarded to the second network device, the second network device reorders the data packet and forwards the reordered data packet to the corresponding second computing node. Simultaneously, the second network device also determines the port congestion status corresponding to at least one of its own ingress ports. When an announcement message trigger condition is detected, the second network device sends an announcement message to the first network device based on the port congestion status corresponding to at least one of its own ingress ports, instructing the first network device to determine the global path congestion status corresponding to multiple first paths based on the announcement message.

[0124] At least one ingress port is the port through which the second network device receives data packets, and the port congestion status of the ingress port indicates the congestion situation of the corresponding ingress port.

[0125] Optionally, after receiving a data packet, the second network device further determines the in-flow packet out-of-order degree of the data stream to which the data packet belongs. When an announcement message trigger condition is detected, the second network device sends an announcement message to the first network device based on the aforementioned in-flow packet out-of-order degree and the port congestion status corresponding to at least one of its own ingress ports, so as to instruct the first network device to determine the global path congestion status corresponding to multiple first paths based on the announcement message.

[0126] Among them, the out-of-order status of intra-flow packets indicates the out-of-order status of packets that have arrived at the second network device in the data stream to which the data packet belongs.

[0127] It should be understood that Figure 3 is only an example where each message transmission path between the first and second network devices includes one network device. Without limiting the topology of the communication network, the message transmission path between the first and second network devices may include more network devices, and this embodiment does not impose any limitations on this. Furthermore, this embodiment does not limit the number and type of computing nodes connected to the first and second network devices.

[0128] In some embodiments, the packet forwarding method provided in this application can be executed by a second computing node. That is, after the second computing node receives a data packet, it reorders the data packet and determines the port congestion status corresponding to at least one of its own ingress ports. When an announcement message trigger condition is detected, the second computing node sends an announcement message to the first network device based on the port congestion status corresponding to at least one of its own ingress ports, instructing the first network device to determine the global path congestion status corresponding to multiple first paths based on the announcement message.

[0129] At least one ingress port is the port through which the second computing node receives data packets, and the port congestion status of the ingress port indicates the congestion situation of the corresponding ingress port.

[0130] Optionally, after receiving a data packet, the second computing node also determines the in-flow packet out-of-order degree of the data stream to which the data packet belongs. When an announcement message trigger condition is detected, the second computing node sends an announcement message to the first network device based on the in-flow packet out-of-order degree and the port congestion status corresponding to at least one of its own ingress ports, so as to instruct the first network device to determine the global path congestion status corresponding to multiple first paths based on the announcement message.

[0131] Among them, the out-of-order status of intra-stream packets indicates the out-of-order status of packets that have arrived at the second computing node in the data stream to which the data packet belongs.

[0132] In summary, the message forwarding method provided in this application can be executed by a first network device and a target device. The first network device is the network device that forwards data packets before a second network device in the communication network. It can be the first network device to receive the data packet, or it can be a network device that subsequently forwards data packets before the second network device; this application does not impose any restrictions on this. The target device can be a second network or a second computing node. The second network device is the last network device in the communication network to forward data packets, and the second computing node is a computing node connected to the second network device that receives the data packet.

[0133] It should be noted that when the second computing node executes the packet forwarding method provided in this application embodiment, its specific implementation logic is similar to the steps executed by the second network device. You can refer to the method embodiment corresponding to the second network device. This will not be elaborated in this application embodiment.

[0134] Furthermore, the packet forwarding method provided in this application embodiment is a packet-level load balancing solution. The load balancing granularity of the first network device when forwarding data packets includes two types: first type data flow level load balancing and second type data flow level load balancing.

[0135] In the first type of data stream, all data packets share the same 5-tuple information. In the second type of data stream, each data packet corresponds to the same source network device and the same destination network device across multiple network devices. That is, for a data packet sent from a source network device to a destination network device, the data stream to which the data packet belongs can be either the first type of data stream or the second type of data stream.

[0136] The data packet carries information such as the source Internet Protocol (IP) address, source port, destination IP address, destination port, and protocol number. For example, assuming a first computing node sends a data packet to a second computing node via a communication network, the source IP address and source port are the IP address and port of the first computing node, the destination IP address and destination port are the IP address and port of the second computing node, and the protocol number is the number of the transport protocol used by the first and second computing nodes to communicate. A data stream can include one or more data packets, and these data packets must have the same 5-tuple; in other words, one or more data packets with the same 5-tuple can constitute a data stream.

[0137] Therefore, when performing packet-level load balancing on data packets to be forwarded, the source network device can perform packet-by-packet routing and forwarding based on a single data stream, according to the packet forwarding method provided in the embodiments of this application. Alternatively, the source network device can perform packet-by-packet routing and forwarding based on multiple data streams transmitted between the source network device and the destination network device, according to the packet forwarding method provided in the embodiments of this application, without distinguishing the data stream to which the data packets destined for the destination network device belong.

[0138] In this embodiment, the source network device can be the first network device shown in Figure 3, and the destination network device can be the second network device shown in Figure 3. Next, the implementation schemes for the two load balancing granularities will be explained in conjunction with the packet forwarding method in this embodiment.

[0139] (1) When the data stream to which the data message belongs is a type 1 data stream, the data message carries a 5-tuple information and a message sequence number.

[0140] Referring to Figure 4, for a data packet to be forwarded, the first network device determines that the data packet needs to be forwarded to the second network device based on the five-tuple information of the data packet. The first network device determines multiple first paths that can forward the data packet, and then selects a target path from the multiple first paths according to the global path congestion status corresponding to each of the multiple first paths. Then, it sends the data packet to the intermediate network device of the next hop according to the target path. The intermediate network device continues to forward the data packet according to the target path, and finally sends the data packet to the second network device connected to the second computing node.

[0141] After the second network device receives a data packet, if the second network device executes the packet forwarding method provided in the embodiments of this application, the second network device calculates the intra-flow packet out-of-order degree of the data stream to which the data packet belongs based on the received data packet, determines the port congestion state corresponding to at least one ingress port of the received data packet, and then, when the notification message triggering condition is detected, sends a notification message to the first network device based on the port congestion state corresponding to at least one ingress port, or based on the port congestion state corresponding to at least one ingress port and the intra-flow packet out-of-order degree, to instruct the first network device to determine the global path congestion state corresponding to multiple first paths based on the notification message.

[0142] After the second network device receives the data packet, if the second computing node executes the packet forwarding method provided in this application embodiment, the second network device will not perform the above calculation processing and will directly forward the data packet to the second computing node.

[0143] Referring to Figure 5, after receiving a data packet forwarded by the second network device, the second computing node calculates the out-of-order degree of the intra-flow packets of the data stream to which the data packet belongs, determines the port congestion status corresponding to at least one ingress port of the received data packet, and then, when the notification message trigger condition is detected, sends a notification message to the first network device based on the port congestion status corresponding to at least one ingress port, or based on the port congestion status corresponding to at least one ingress port and the intra-flow packet out-of-order degree, so as to instruct the first network device to determine the global path congestion status corresponding to multiple first paths based on the notification message.

[0144] (2) When the data stream to which the data packet belongs is a second type of data stream, the data packet carries the device identifier of the first network device, the device identifier of the second network device, and the message sequence number.

[0145] Referring to Figure 6, for a data packet to be forwarded, the first network device does not distinguish the data stream to which the data packet belongs. Based on the device identifier of the second network device carried in the data packet, the first network device determines that the data packet needs to be forwarded to the second network device. The first network device determines multiple first paths that can forward the data packet, and then selects a target path from the multiple first paths according to the global path congestion status corresponding to each of the multiple first paths. Then, it sends the data packet to the next-hop intermediate network device according to the target path. The intermediate network device continues to forward the data packet according to the target path, and finally sends the data packet to the second network device connected to the second computing node.

[0146] It should be noted that when the data stream to which the data packet belongs is a second type of data stream, all data packets will converge at the second network device. At this time, the second network device will execute the packet forwarding method provided in the embodiments of this application.

[0147] Therefore, the second network device calculates the out-of-order status of the intra-flow packets of the data stream to which the data packets belong based on the received data packets, determines the port congestion status corresponding to at least one ingress port of the received data packets, and then sends an announcement message to the first network device based on the port congestion status corresponding to at least one ingress port, or based on the port congestion status corresponding to at least one ingress port and the intra-flow packet out-of-order status, when an announcement message is detected, instructing the first network device to determine the global path congestion status corresponding to multiple first paths based on the announcement message.

[0148] That is, when the target device for executing the packet forwarding method provided in this application embodiment is a second computing node, the data packets transmitted between the first network device and the second computing node are first type data streams; when the target device for executing the packet forwarding method provided in this application embodiment is a second network device, the data packets transmitted between the first network device and the second network device can be either first type data streams or second type data streams.

[0149] In summary, when forwarding multiple data packets, the first network device in this embodiment can first determine whether the multiple data packets belong to a single data stream or multiple data streams, and then use different load balancing granularities for packet-by-packet forwarding. That is, when implementing the packet forwarding method provided in this embodiment, the first network device can use different load balancing granularities to perform packet-by-packet routing and forwarding according to actual needs, improving the flexibility of the first network device when performing packet-level load balancing.

[0150] Next, the message forwarding method provided in the embodiments of this application will be explained in detail with reference to the accompanying drawings.

[0151] Please refer to Figure 7, which is a flowchart of a message forwarding method provided in an embodiment of this application. The method is illustrated using a first network device in a communication network as an example. As shown in Figure 7, when the first network device executes the message forwarding method provided in this embodiment, it includes the following steps.

[0152] Step 701: Based on the indication information carried in the data packet to be forwarded, determine multiple first paths. The indication information indicates the second network device that is the last to forward the data packet in the communication network. The multiple first paths refer to the packet transmission path from the first network device to the second network device.

[0153] The indication information carried in the data packet can be either the device identifier of the destination network device or the node identifier of the destination computing node. The destination network device is the last network device in the communication network to forward the data packet; the destination computing node is the computing node that receives the data packet through the communication network, that is, the receiving node of the forwarded data packet in the communication network.

[0154] In step 701 above, the indication information carried in the data packet can be the device identifier of the second network device or the node identifier of the second computing node. Since the second computing node is connected to the second network device, data packets sent to the second computing node through this communication network will inevitably be forwarded by the second network device. That is, the second network device is the last device to forward the data packet to the second computing node.

[0155] Among them, the device identifier can be the device number, device identity document (ID), etc., and the node identifier can be the node ID, node IP address, node port, etc., and this application embodiment does not limit this.

[0156] As described above, the data stream to which the data packet to be forwarded belongs can be either a first-type data stream or a second-type data stream; this application embodiment does not impose any restrictions on this. Specifically, in a first-type data stream, the five-tuple information of each data packet is identical, while in a second-type data stream, each data packet corresponds to the same source network device and the same destination network device among multiple network devices.

[0157] That is, for a data packet to be forwarded, regardless of whether the data stream to which the data packet belongs is a first type of data stream or a second type of data stream, the first network device can perform the following step 702 based on the data packet to determine the target path of the data packet and forward the data packet according to the target path.

[0158] For the first network device, it corresponds to multiple message transmission paths in the entire communication network. Different message transmission paths may connect to different destination network devices. When performing the above step 701, when it is determined that the data packet needs to be sent to the second computing node based on the indication information carried by the data packet, or when it is determined that the data packet needs to be sent to the second network device based on the device identifier of the destination network device carried by the data packet, the first network device selects multiple first paths from its multiple message transmission paths that can forward the data packet to the second network device, that is, selects the message transmission path from the first network device to the second network device.

[0159] Among them, the multiple first paths may be all the message transmission paths of the first network device, or they may be some of the message transmission paths of the first network device, depending on the network architecture of the communication network. This application embodiment does not limit this.

[0160] Furthermore, after determining multiple first paths, the first network device can analyze the global path congestion status of these multiple first paths, and then execute step 702 according to the global path congestion status corresponding to each of the multiple first paths. Next, with reference to Figure 8, the implementation process of the first network device determining the global path congestion status corresponding to each of the multiple first paths will be described.

[0161] Please refer to Figure 8. The first network device determines the global path congestion status corresponding to multiple first paths, including the following steps.

[0162] Step 801: Obtain the remote path congestion status corresponding to each of the multiple first paths. The remote path congestion status indicates the congestion status of network devices other than the first network device on the corresponding first path.

[0163] Optionally, the remote path congestion status of the first path may include whether there is congestion at the remote end or no congestion at the remote end. For example, the remote path congestion status of the first path is represented by a status flag value. When the status flag value of the remote path congestion status of the first path is 1, it indicates that there is congestion at the remote end of the first path; when the status flag value of the remote path congestion status of the first path is 0, it indicates that there is no congestion at the remote end of the first path.

[0164] In some embodiments, the remote path congestion status of the first path is determined based on an announcement message sent by the second network device or the second computing node. When the content and format of the announcement message differ, the process of determining the respective remote path congestion status of multiple first paths also differs slightly. Next, taking an announcement message sent by the second network device as an example, the process of the first network device determining the respective remote path congestion status of multiple first paths will be explained in conjunction with the content and format of the announcement message.

[0165] In the first scenario, the first network device receives an announcement message carrying the port congestion status corresponding to at least one ingress port in the second network device. The at least one ingress port is a port connected to multiple first paths, and the port congestion status indicates the congestion status of the corresponding ingress port. Based on the port congestion status corresponding to the at least one ingress port, the remote path congestion status corresponding to the multiple first paths is determined.

[0166] In this first case, the notification message includes a port congestion status field, which carries the port congestion status of the at least one inbound port.

[0167] For the second network device, an ingress port may connect to one first path or multiple first paths, depending on the specific architecture of the communication network in which the first and second network devices are located. This application embodiment does not impose any restrictions on this.

[0168] In one possible implementation, since the at least one ingress port is a port connected to multiple first paths, after the first network device receives the port congestion status of the at least one ingress port, it can directly determine the port congestion status of the at least one ingress port as the remote path congestion status of the corresponding first path.

[0169] In other words, the port congestion state of the ingress port of the second network device can be equivalent to the remote path congestion state of the first path connected to that ingress port.

[0170] Optionally, after receiving the port congestion status of at least one ingress port, the first network device may also combine other information to comprehensively analyze the remote path congestion situation of the first path, thereby determining the remote path congestion status corresponding to the first path. This application embodiment does not impose any limitations on this.

[0171] In the second scenario, the first network device receives an announcement message carrying the in-flow packet out-of-order degree and the port congestion status corresponding to at least one ingress port of the second network device. The in-flow packet out-of-order degree indicates the out-of-order status of packets that have arrived at the second network device in the data flow to which the data packet belongs. The at least one ingress port is a port connected to multiple first paths, and the port congestion status indicates the congestion status of the corresponding ingress port. Based on the port congestion status of at least one ingress port and the in-flow packet out-of-order degree, the remote path congestion status corresponding to the multiple first paths is determined.

[0172] Regarding the port congestion status corresponding to at least one ingress port in the second network, please refer to the explanation in the first case above, which will not be repeated here.

[0173] In this second scenario, the notification message includes a port congestion status field and an out-of-order status field. The port congestion status field is used to carry the port congestion status of at least one inbound port, and the out-of-order status field is used to carry the out-of-order degree of packets within the flow.

[0174] It should be noted that when there is severe out-of-order delivery of packets that have reached the second network device in the data stream to which the data packet belongs, it indicates that there is congestion on the packet transmission path between the first and second network devices, causing the data packets to fail to arrive at the second network device in the order of their sequence numbers. Similarly, when there is a small amount of out-of-order delivery or no out-of-order delivery of packets that have reached the second network device in the data stream to which the data packet belongs, it indicates that there is basically no congestion on the packet transmission path between the first and second network devices, and the data packets can basically arrive at the second network device in the order of their sequence numbers.

[0175] Therefore, when determining the congestion status of the remote path corresponding to the first path, based on the first case mentioned above, the out-of-order status of the data packets within the data stream to which the data packet belongs can be combined to comprehensively analyze the congestion status of the remote path corresponding to the first path.

[0176] Taking a first path as an example, in one possible implementation, when the port congestion status of the ingress port connected to the first path indicates that the ingress port is congested, or when the out-of-order degree of the intra-flow packets of the data stream to which the data packet belongs is greater than the out-of-order degree threshold, the remote path congestion status of the first path is determined to be remote congestion.

[0177] The disorder threshold is a preset value, and the specific value of it is not limited in this embodiment.

[0178] As an example, the calculation logic for determining the congestion status of remote paths corresponding to multiple first paths based on the port congestion status of at least one ingress port and the out-of-order packet order within the flow is shown in Table 1 below.

[0179] Table 1

[0180] In Table 1, both the port congestion status of the ingress port and the remote path congestion status are represented by status flag values. When the status flag value of the port congestion status of the ingress port is 1, it indicates that the ingress port is congested; when the status flag value of the port congestion status of the ingress port is 0, it indicates that the ingress port is not congested. Similarly, when the status flag value of the remote path congestion status is 1, it indicates that the remote end of the first path is congested; when the status flag value of the remote path congestion status is 0, it indicates that the remote end of the first path is not congested.

[0181] In Table 1, the out-of-order status of intra-flow packets can also be represented using status flag values. When the status flag value of the intra-flow packet out-of-order status is 1, it indicates that the out-of-order status of the intra-flow packets to which the data packet belongs is greater than the out-of-order threshold; when the status flag value of the intra-flow packet out-of-order status is 0, it indicates that the out-of-order status of the intra-flow packets to which the data packet belongs is less than or equal to the out-of-order threshold.

[0182] As shown in Table 1, for any first path, when the port congestion status flag of the ingress port of the second network device connected to the first path is 1, and / or the out-of-order status flag of the intra-flow packets is 1, the first network device determines that the remote path congestion status flag of the first path is also 1, that is, there is congestion at the remote end of the first path.

[0183] In summary, when the notification message carries the port congestion status and intra-flow packet out-of-order degree of at least one ingress port of the second network device receiving data packets, since the at least one ingress port is a port connected to multiple first paths, the port congestion status of the at least one ingress port can reflect the congestion situation during the transmission of data packets received at that ingress port from the first network device to the second network device. At the same time, since the intra-flow packet out-of-order degree can also reflect the congestion situation during the transmission of data packets from the first network device to the second network device to a certain extent, after receiving the port congestion status and intra-flow packet out-of-order degree of the at least one ingress port, the first network device can comprehensively analyze the congestion situation of the packet transmission path from the first network device to the second network device, thereby obtaining the remote path congestion status corresponding to each of the multiple first paths.

[0184] Therefore, when analyzing the remote path congestion status of the first path, the first network device not only considers the data packets received by the second network device, but also the out-of-order status of the data packets themselves, thereby improving the accuracy and reliability of determining the remote path congestion status of the first path.

[0185] It should be noted that in both scenarios above, after the second network device determines the port congestion status and intra-flow packet out-of-order rate of at least one ingress port based on the data packet reception status, it directly feeds back the port congestion status of at least one ingress port, or the port congestion status and intra-flow packet out-of-order rate of at least one ingress port, to the first network device via an announcement message. The first network device then determines the remote path congestion status corresponding to each of the multiple first paths according to the aforementioned calculation logic. In the process of determining the remote path congestion status of the first paths, the second network device does not need to calculate the remote path congestion status for each first path, allowing it to quickly feed back the announcement message and improving the efficiency of the announcement message feedback.

[0186] In the third scenario, the first network device receives an announcement message that carries the congestion status of the remote paths corresponding to multiple first paths. In this way, the first network device directly obtains the remote path status of each of the multiple first paths from the announcement message.

[0187] In this third scenario, the notification message includes a path congestion status field, which carries the congestion status of the remote paths corresponding to the multiple first paths.

[0188] It should be noted that, referring to the first scenario above, the remote path congestion status of the first path can be determined based on the port congestion status corresponding to at least one ingress port in the second network device; referring to the second scenario above, the remote path congestion status of the first path can be determined based on the port congestion status corresponding to at least one ingress port in the second network device and the in-flow packet out-of-order degree. In the third scenario, the embodiments of this application do not limit the implementation method of the second network device determining the remote path congestion status of the first path; it can be the method shown in the first scenario above, or the method shown in the second scenario above.

[0189] That is, based on the reception of data packets transmitted from the first network device, the second network device first determines the port congestion status and in-flow packet out-of-order degree of at least one ingress port in the second network device, and then determines the remote path congestion status corresponding to multiple first paths according to the port congestion status and in-flow packet out-of-order degree of the at least one ingress port, and then feeds back the remote path congestion status corresponding to multiple first paths to the first network device through an announcement message.

[0190] Therefore, in the third scenario, after receiving the notification message, the first network device can directly determine the congestion status of the remote paths corresponding to multiple first paths by parsing the message. Since the second network device directly reports the congestion status of the remote paths corresponding to multiple first paths, the first network device can quickly determine the congestion status of the remote paths for each first path based on the notification message, thus improving the efficiency of the first network device in determining the congestion status of the remote paths for multiple first paths.

[0191] Step 802: Based on the remote path congestion status corresponding to each of the multiple first paths, determine the global path congestion status corresponding to each of the multiple first paths.

[0192] Optionally, the global path congestion status of the first path may include whether the first path is congested or not. For example, the global path congestion status of the first path is represented by a status flag value. When the status flag value of the global path congestion status of the first path is 1, it indicates that the first path is congested; when the status flag value of the global path congestion status of the first path is 0, it indicates that the first path is not congested.

[0193] In one possible implementation, since the remote path congestion status of the first path indicates the congestion status of network devices other than the first network device on the first path, after determining the remote path congestion status of the first path, the remote path congestion status can be directly determined as the global path congestion status of the corresponding first path.

[0194] In other words, the congestion status of the remote path of the first path can be equated with the global path congestion status of the first path.

[0195] It should be noted that when determining the global path congestion status of the first path, in addition to the remote path congestion status, the first network device can also combine other information to comprehensively analyze the global path congestion situation of the first path, thereby determining the global path congestion status of the first path. This application embodiment does not impose any limitations on this.

[0196] In another possible implementation, the process of implementing step 802 above may include the following steps (1)-(2).

[0197] (1) Determine the port congestion status corresponding to at least one outgoing port in the first network device, wherein the at least one outgoing port is a port connected to multiple first paths, and the port congestion status indicates the congestion status of the corresponding outgoing port.

[0198] It should be understood that for the first network device, an outgoing port may connect to one first path or multiple first paths, depending on the specific architecture of the communication network in which the first network device and the second network device are located. This application embodiment does not impose any restrictions on this.

[0199] Optionally, for the first network device, the port congestion status corresponding to the outgoing port may include whether the outgoing port is congested or not. For example, the port congestion status of the incoming port is represented by a status flag value. When the status flag value of the incoming port congestion status is 1, it indicates that the incoming port is congested; when the status flag value of the incoming port congestion status is 0, it indicates that the incoming port is not congested.

[0200] In one possible implementation, the process of determining the port congestion state corresponding to at least one output port in the first network device can be as follows: determining the packet queue depth corresponding to at least one output port in the first network device, wherein the packet queue depth indicates the backlog of data packets in the corresponding output port; and determining the port congestion state corresponding to at least one output port based on the packet queue depth corresponding to at least one output port.

[0201] The message queue depth can be the total number of data packets accumulated in the corresponding outgoing port, or the total number of bytes of data packets accumulated in the outgoing port. This application embodiment does not limit this.

[0202] As an example, when the packet queue depth corresponding to the outgoing port is greater than the backlog threshold, it is determined that the outgoing port is congested; when the packet queue depth corresponding to the outgoing port is less than the backlog threshold, it is determined that the outgoing port is not congested. When the packet queue depth corresponding to the outgoing port is equal to the backlog threshold, it can be determined that the outgoing port is congested or not; this embodiment of the application does not impose any limitations on this.

[0203] Correspondingly, the backlog threshold is a preset maximum number of backlogged packets on the port, or the maximum number of bytes of backlogged packets on the port. This backlog threshold can be flexibly set and adjusted according to actual needs, and this application embodiment does not impose any restrictions on it.

[0204] In some embodiments, the first network device can periodically monitor the port congestion status of each of its own output ports, store the port congestion status of each output port in the first network device through a local port congestion status record table, and update the local port congestion status record table after each monitoring of the port congestion status of the corresponding output port.

[0205] In this way, the first network device can quickly determine the current port congestion status of each outgoing port by querying its own stored outgoing port congestion status record table.

[0206] Assuming the first network device includes 16 outgoing ports, Table 2 below provides an example outgoing port congestion status record table.

[0207] Table 2

[0208] In Table 2, the port identifier of the output port is represented by the port number, and the port congestion status of the corresponding output port is represented by the status flag value. As explained above, when the status flag value of the output port congestion status is 0, it means that the output port is not congested; when the status flag value of the output port congestion status is 1, it means that the output port is congested.

[0209] It should be understood that the port identifier and port congestion status in Table 2 can also be represented in other ways. The embodiments of this application are only exemplified by the above methods and do not constitute a limitation on their representation.

[0210] In some embodiments, when there are multiple outgoing ports in the first network device that are connected to multiple first paths, in order to ensure that the bandwidth of each outgoing port in the first network device can be effectively utilized, some outgoing ports can be pre-selected from multiple outgoing ports based on the data packets sent by each outgoing port in the current statistical period to obtain at least one outgoing port, and then the port congestion status of the at least one outgoing port can be determined.

[0211] In one possible implementation, the process of selecting at least one outgoing port from the outgoing ports of the first network device can be as follows: determining the packet container size of each outgoing port in the first network device, wherein the packet container size indicates the total number of bytes of packets sent by the corresponding outgoing port in the current statistical period; and determining the at least one outgoing port from the outgoing ports whose packet container size is less than the container threshold.

[0212] The statistical period is a preset statistical interval, and the container threshold is a preset maximum number of message bytes. The specific values ​​of the statistical period and the container threshold can be flexibly set and adjusted according to the actual situation. This application embodiment does not limit their specific values.

[0213] Optionally, the first network device can periodically monitor the packet container size of each of its own outgoing ports according to a set statistical period, store the packet container size of each outgoing port in a local port forwarding record table, and update the local port forwarding record table after each monitoring of the packet container size of the outgoing port.

[0214] Assuming the first network device has 16 outgoing ports, Table 3 below provides an example of a local port forwarding record table.

[0215] Table 3

[0216] It should be understood that the values ​​of the message container size for each output port in Table 3 are merely examples, and the port identifiers in Table 3 can also be represented in other ways. This embodiment of the application only uses the port identifier as the port number as an example, and does not constitute a limitation on its representation.

[0217] Optionally, if multiple first paths are connected to one output port, Table 3 may also include the path identifiers of the multiple first paths corresponding to that output port for each output port.

[0218] It should be noted that during the process of calculating the packet container size of each outgoing port, if the packet container size of the outgoing port is found to be equal to the container threshold, then the packet container size of that outgoing port needs to be normalized or zeroed out. In the current data packet transmission, the outgoing port is not considered. However, when the next data packet is transmitted, since the packet container after that outgoing port has been normalized or zeroed out, its packet container size will inevitably be less than the container threshold. Therefore, when the next data packet is transmitted, the outgoing port can be selected and the steps to determine the port congestion status corresponding to that outgoing port can be performed.

[0219] Therefore, for the first network device, based on the packet container size of each outgoing port, at least one outgoing port capable of sending the data packet can be pre-selected from those whose packet container size is less than the container threshold before forwarding the data packet. In other words, each time the first network device sends a data packet, it can select at least one outgoing port capable of sending the current data packet based on the data packets already sent on each outgoing port. Thus, during the data packet sending process, the first network device can ensure that each outgoing port is used evenly, making effective use of the bandwidth of each outgoing port, thereby improving the bandwidth utilization rate of each outgoing port in the first network device.

[0220] (2) Based on the port congestion status corresponding to at least one outgoing port and the remote path congestion status corresponding to multiple first paths, determine the global path congestion status corresponding to multiple first paths.

[0221] In other words, when determining the global path congestion status for each first path, the congestion status of at least one outgoing port connected to the first network device and multiple first paths is first determined. Then, the congestion status of network devices other than the first network device on each first path is obtained to obtain the remote path congestion status. Finally, the global path congestion status for each of the multiple first paths is determined by combining the port congestion status corresponding to at least one outgoing port and the remote path congestion status corresponding to each of the multiple first paths. Thus, by comprehensively analyzing the global path congestion status corresponding to each of the multiple first paths based on the port congestion status corresponding to at least one outgoing port and the remote path congestion status corresponding to each of the multiple first paths, the accuracy of determining the global path congestion status is improved.

[0222] It should be noted that the operations of determining the port congestion status corresponding to at least one output port and determining the remote path congestion status of each first path can be performed simultaneously or sequentially. The execution order is not limited in this embodiment.

[0223] In one possible implementation, if the port congestion status of the outgoing port connected to the first path indicates that the corresponding outgoing port is congested, and / or the congestion status of the remote path corresponding to the first path indicates that the network devices on the first path other than the first network device are congested, then it is determined that the first path is congested; otherwise, it is determined that the first path is not congested.

[0224] As an example, if the port congestion status flag of the outgoing port is 1, it indicates that the outgoing port is congested; if the port congestion status flag of the corresponding outgoing port is 0, it indicates that the outgoing port is not congested. If the port congestion status flag of the remote path corresponding to the first path is 1, it indicates that the network devices on the first path other than the first network device are congested; if the port congestion status flag of the remote path corresponding to the first path is 0, it indicates that the network devices on the first path other than the first network device are not congested. Then, when the port congestion status flag of the outgoing port corresponding to the first path is 1, and / or the port congestion status flag of the remote path corresponding to the first path is 1, the global path congestion status flag of the first path is determined to be 1 to indicate that the first path is congested; otherwise, the global path congestion status flag of the first path is determined to be 0 to indicate that the first path is not congested.

[0225] In some embodiments, the first network device may store and maintain a global path congestion status record table to record the global path congestion status of the message transmission path corresponding to each outgoing port in the first network device, and update the global path congestion status record table after each notification message is received.

[0226] As an example, Table 4 below provides a model global path congestion status record table.

[0227] Table 4

[0228] The global path congestion state is determined based on the remote path congestion state and the port congestion state of the outgoing port in the first network device. In other words, the global path congestion state is the sum of the remote path congestion state and the port congestion state corresponding to the outgoing port in the first network device.

[0229] After the first network device determines the remote path congestion status corresponding to the first path based on the announcement message, it first updates the remote path congestion status of the first path in Table 4. Then, based on the updated remote path congestion status of the first path and the port congestion status of the outgoing port in the first network device, it updates the global path congestion status of the first path, thereby updating the global path congestion status record table.

[0230] It should be understood that, for the first network device, considering that the first network device communicates with multiple network devices in the communication network, and the first path is the message transmission path between the first network device and the second network device, the message transmission path shown in Table 4 above not only includes the message transmission path between the first network device and the second network device, but may also include the message transmission path between the first network device and other network devices. This application embodiment does not limit this.

[0231] As an example, the process of the first network device updating the global path congestion status record table of the first path can be as follows: the global path congestion status of multiple message transmission paths maintained by the first network device is initialized to the port congestion status of the corresponding outgoing port in the first network device. After receiving the announcement message sent by the second network device, the remote path congestion status of multiple first paths between the first network device and the second network device is first determined based on the information carried in the announcement message, and the remote path congestion status of the first path in Table 4 above is updated; then, based on the remote path congestion status of the first path, the global path congestion status of the first path in Table 4 above is updated.

[0232] As mentioned above, when updating the global path congestion status of the first path, the update can be based on the remote path congestion status of the first path, or it can be combined with the remote path congestion status of the first path and the port congestion status of at least one outgoing port in the first network device. This application embodiment does not limit this.

[0233] Taking the updating of the global path congestion state of the first path by combining the remote path congestion state of the first path and the port congestion state of at least one outgoing port in the first network device as an example, the update strategy for the global path congestion state can be as follows: if the status flag value of the outgoing port's congestion state changes from 0 to 1, the status flag value of the global path congestion state is incremented by 1; if the status flag value of the outgoing port's congestion state changes from 1 to 0, the status flag value of the global path congestion state is decremented by 1. Similarly, if the status flag value of the remote path congestion state changes from 0 to 1, the status flag value of the global path congestion state is incremented by 1; if the status flag value of the remote path congestion state changes from 1 to 0, the status flag value of the global path congestion state is decremented by 1. For the calculated global path congestion state, when its status flag value is greater than 1, it is uniformly represented by 1.

[0234] In summary, after determining multiple first paths based on the indication information carried in the data packet to be forwarded, and analyzing the global path congestion status corresponding to each of the multiple first paths, the first network device can execute the following step 702 to select the target path for forwarding the data packet from the multiple first paths.

[0235] Step 702: Based on the global path congestion status corresponding to the multiple first paths, select a target path from the multiple first paths. The global path congestion status indicates the global congestion situation of the corresponding first path. The target path is used to forward data packets to the second network device.

[0236] As mentioned earlier, the global path congestion status of the first path can reflect the congestion situation of the entire message transmission path from the first network device to the second network device.

[0237] In one possible implementation, step 702 can be implemented as follows: based on the global path congestion status corresponding to multiple first paths, select the first path without congestion from the multiple first paths, and determine the first path without congestion as the target path.

[0238] When there are multiple first paths without congestion, one of these first paths without congestion can be randomly selected as the target path. This application does not impose any limitations on this.

[0239] Optionally, considering that after a data packet is transmitted to the second network device, the second network device needs to reorder the data packets within the data stream to which the data packet belongs, when the data packets in a certain data stream are severely out of order, even if the first path without congestion is used to forward the data packets in that data stream to the second network device, the second network device cannot reorder them in time. It needs to wait until all the out-of-order data packets before that data packet have arrived before it can reorder the received data packets according to the packet sequence number.

[0240] Therefore, in order to improve the out-of-order reordering efficiency of the second network device, when the first network device performs packet-by-packet routing for data packets destined for the second network device, it can adopt different path selection strategies to select the target path for the data packet based on the out-of-order status of the packets within the data stream to which the data packet belongs.

[0241] In another possible implementation, as shown in Figure 9, the implementation process of step 702 above may include the following steps.

[0242] Step 7021: Determine the out-of-order type of the data stream to which the data packet belongs.

[0243] Among them, the out-of-order flow types include the first type of out-of-order flow and the second type of out-of-order flow.

[0244] In one possible implementation, if the out-of-order degree of the intra-flow packets in the data stream to which the data packet belongs is greater than the out-of-order threshold, then the out-of-order type of the data stream to which the data packet belongs is determined to be a first-type out-of-order flow; if the out-of-order degree of the intra-flow packets in the data stream to which the data packet belongs is less than or equal to the out-of-order threshold, then the out-of-order type of the data stream to which the data packet belongs is determined to be a second-type out-of-order flow.

[0245] The disorder threshold is a preset value, and the specific value of this application embodiment is not limited.

[0246] Optionally, the first network device may store and maintain a message sequence record table to record the out-of-order type of the data stream to which each forwarded data packet belongs.

[0247] As an example, Table 5 below provides a model message sequence record table.

[0248] Table 5

[0249] The data flow identifier can be determined by the message transmission path between the first network device and the destination network device, as well as the device identifier of the corresponding destination network device. That is, Table 5 above records the message sequence number and out-of-order type when the first network device transmits data flows with multiple network devices, including the second network device, and may also include other network devices besides the second network device.

[0250] Step 7022: Determine the path selection strategy for data packets based on the out-of-order flow type.

[0251] The path selection strategy includes a first selection strategy and a second selection strategy. The first selection strategy indicates that a first type of path is used to transmit data packets, which is a path that is congested. The second selection strategy indicates that a second type of path is used to transmit data packets, which is a path that is not congested.

[0252] As an example, if the out-of-order type of the data stream to which the data packet belongs is a first type of out-of-order flow, then the path selection strategy for the data packet is determined to be the first selection strategy; if the out-of-order type of the data stream to which the data packet belongs is a second type of out-of-order flow, then the path selection strategy for the data packet is determined to be the second selection strategy.

[0253] Step 7023: Based on the path selection strategy, select the target path from multiple first paths.

[0254] That is, when the path selection strategy of the data packet is the first selection strategy, the congested path is selected from multiple first paths as the target path; when the path selection strategy of the data packet is the second selection strategy, the non-congested path is selected from multiple first paths as the target path.

[0255] To facilitate understanding, in conjunction with Figure 10, an example is provided to illustrate the process of the first network device selecting the target path for the data packet to be forwarded.

[0256] Referring to Figure 10, for a data packet to be forwarded, all outgoing ports of the first network device are polled first to determine whether the packet container size corresponding to each outgoing port is less than a container threshold. Outgoing ports with packet container sizes smaller than the container threshold are then selected from multiple outgoing ports. For outgoing ports with packet container sizes smaller than the container threshold, based on the first path connected to the outgoing port and the out-of-order flow type of the data stream to which the data packet belongs, the target path for forwarding the data packet is selected from multiple first paths.

[0257] When the out-of-order flow type of the data stream to which the data packet belongs is Type I out-of-order flow, the process of selecting a target path from multiple first paths for an outgoing port whose packet container size is less than the container threshold can be as follows: In the first round of path polling, for the currently polled outgoing port, based on the global path congestion status corresponding to the multiple first paths connected to the outgoing port, a congested path is selected as the target path. If none of the multiple first paths connected to the outgoing port are congested, the next outgoing port is polled, and the above steps are repeated to select a congested path as the target path from the multiple first paths connected to the polled outgoing port. If, after polling all outgoing ports whose packet container size is less than the container threshold, no congested first path is found, a second round of path polling is performed. In the second round of path polling, for the currently polled outgoing port, based on the global path congestion status corresponding to the multiple first paths connected to the outgoing port, a non-congested path is selected as the target path.

[0258] During the aforementioned path polling process, if no congested first path is found in the first round, and no non-congested first path is found in the second round, the first network device forwards the data packet using the same outgoing port as the previous forwarding port, or the first network device randomly selects an outgoing port to forward the data packet and determines the target path from the multiple first paths connected to that outgoing port. This application embodiment does not impose any limitations on this.

[0259] In other words, when the out-of-order type of the data stream to which the data packet belongs is the first type of out-of-order flow, the first path with congestion is selected as the target path first; when none of the first paths are congested, the first path without congestion is selected as the target path, or the first path connected to the output port of the last forwarded data packet is selected as the target path, or a first path is randomly selected as the target path.

[0260] When the out-of-order flow type of the data stream to which the data packet belongs is Type II out-of-order flow, the process of selecting a target path from multiple first paths for an outgoing port whose packet container size is less than the container threshold can be as follows: In the first round of path polling, for the currently polled outgoing port, based on the global path congestion status corresponding to the multiple first paths connected to the outgoing port, a path without congestion is selected as the target path. If all multiple first paths connected to the outgoing port are congested, the next outgoing port is polled, and the above steps are repeated to select a path without congestion as the target path from the multiple first paths connected to the polled outgoing port. If, after polling all outgoing ports whose packet container size is less than the container threshold, no first path without congestion is found, a second round of path polling is executed. In the second round of path polling, for the currently polled outgoing port, based on the global path congestion status corresponding to the multiple first paths connected to the outgoing port, a path with congestion is selected as the target path from the multiple first paths.

[0261] Similarly, during the above path polling process, if no congested first path is found in the first round and no congested first path is found in the second round, the first network device will forward the data packet using the same outgoing port as the previous forwarding port, or the first network device will randomly select an outgoing port to forward the data packet and determine the target path from the multiple first paths connected to that outgoing port. This application embodiment does not impose any limitations on this.

[0262] In other words, when the out-of-order type of the data stream to which the data packet belongs is the second type of out-of-order flow, the first path without congestion is selected as the target path first; when all first paths are congested, the first path with congestion is selected as the target path, or the first path connected to the output port of the last forwarded data packet is selected as the target path, or a first path is randomly selected as the target path.

[0263] Optionally, after sending a data packet at the corresponding outgoing port, the packet container size of that outgoing port is updated based on the size of the forwarded data packet. For example, Table 3 above is updated based on the outgoing port that forwarded the data packet and the size of the data packet.

[0264] In some embodiments, after determining the target path for forwarding the data packet, the packet forwarding method provided in this application further includes: transmitting the data packet according to the target path to forward the data packet to a second network device in the communication network, and the second network device finally forwarding the data packet to the corresponding second computing node.

[0265] In other words, when the first network device sends a data packet, it carries the path information of the target path, so that after the next-hop network device receives the data packet, it can continue to transmit the data packet according to the target path until the data packet is forwarded to the second network device, which then sends the data packet to the second computing node.

[0266] After receiving a data packet, the second network device can reorder the data packet according to the data stream to which the data packet belongs and the packet sequence number. Based on the reordering result, the data packets destined for the second computing node are forwarded to the second computing node in sequence.

[0267] Optionally, after receiving a data packet, the second network device can also forward the data packet directly to the second computing node based on the destination IP carried in the data packet, and the second computing node can then reorder the received data packet based on the packet sequence number.

[0268] In summary, in this embodiment, for a data packet to be forwarded, the first network device first determines the second network device in the communication network that is the last to forward the data packet based on the indication information carried in the data packet, and determines the global path congestion status corresponding to each of the multiple packet transmission paths (i.e., first paths) from the first network device to the second network device. For each first path, since the global path congestion status indicates the overall congestion situation of the entire first path and can reflect the congestion situation of all network devices on the first path when forwarding data packets, the first network device selects the target path for forwarding the data packet from the multiple first paths based on the global path congestion status corresponding to each of the multiple first paths. Therefore, it can be seen that the first network device comprehensively considers the congestion situation of the entire packet transmission path during the forwarding process, improving the accuracy of target path selection.

[0269] Furthermore, when the first network device is forwarding multiple data packets, the packet forwarding method provided in this application embodiment can improve the accuracy of packet-by-packet routing, thereby making the first network device perform better in load balancing when forwarding multiple data packets.

[0270] Please refer to Figure 11, which is a flowchart illustrating another message forwarding method provided in this application embodiment. The method is described using a target device as an example. As explained above, the communication network includes multiple network devices with communication connections. The target device can be a second network device in the communication network, or a second computing node connected to the second network device. As shown in Figure 11, when the target device executes the message forwarding method provided in this application embodiment, it includes the following steps.

[0271] Step 1101: Determine the port congestion status corresponding to at least one ingress port of the received data packet. The port congestion status indicates the congestion situation of the corresponding ingress port.

[0272] As explained above, the data stream to which this data packet belongs is either a first-class data stream or a second-class data stream; in the first-class data stream, the five-tuple information of each data packet is the same, while in the second-class data stream, each data packet corresponds to the same source network device and the same destination network device among multiple network devices.

[0273] It should be noted that when the target device is a second network device, the data stream to which the data packet belongs can be either a first-class data stream or a second-class data stream; when the target device is a second computing node, the data stream to which the data packet belongs is a first-class data stream.

[0274] Optionally, the data packet may carry indication information, which may be the device identifier of the destination network device or the node identifier of the destination computing node. The destination network device is the last network device in the communication network to forward the data packet; the destination computing node is the computing node that receives the data packet through the communication network, that is, the receiving node of the forwarded data packet in the communication network.

[0275] In step 1101 above, the indication information carried in the data packet can be the device identifier of the second network device or the node identifier of the second computing node. Since the second computing node is connected to the second network device, data packets sent to the second computing node through this communication network will inevitably be forwarded by the second network device. That is, the second network device is the last device to forward the data packet to the second computing node.

[0276] The device identifier can be a device number, device ID, etc., and the node identifier can be a node ID, node IP address, node port, etc. This application embodiment does not impose any restrictions on this.

[0277] When the target device is the second computing node, it receives data packets from the second network device. At this time, the target device receives data packets through only one ingress port, which is the ingress port connected to the second network device. The target device can determine the port congestion status for this ingress port.

[0278] When the target device is a second network device, if there are multiple data packets, the number of ingress ports for receiving data packets may be one or more. In this case, the target device can determine the port congestion status of at least one ingress port for receiving data packets.

[0279] Optionally, the port congestion status of the ingress port can include whether the ingress port is congested or not. For example, the port congestion status of the ingress port can be represented by a status flag value. When the status flag value of the ingress port is 1, it indicates that the ingress port is congested; when the status flag value of the corresponding port congestion status is 0, it indicates that the ingress port is not congested.

[0280] In one possible implementation, step 1101 can be implemented as follows: determining the port congestion status of the ingress port based on the proportion of congested packets corresponding to at least one ingress port and the port congestion threshold. Specifically, if the proportion of congested packets is greater than the port congestion threshold, it is determined that the corresponding ingress port is congested; if the proportion of congested packets is less than or equal to the port congestion threshold, it is determined that the corresponding ingress port is not congested.

[0281] The congestion percentage is the ratio between the total number of data packets received at the corresponding ingress port and the number of data packets carrying congestion flags received at that ingress port. The port congestion threshold is a preset value, and this embodiment does not limit its specific value.

[0282] As an example, the congestion flag carried by the data packet can be an explicit congestion notification (ECN) flag.

[0283] In other words, for a data packet forwarded from the first network device, when the data packet is forwarded by other network devices, those devices can tag the data packet based on their own congestion levels when receiving the data packet, thus indicating the congestion situation at each intermediate network device. Therefore, when the target device receives the data packet, it can parse the ECN tag from it and determine the congestion situation when the data packet was forwarded at other network devices after being sent from the first network device.

[0284] It should be noted that when the target device is the second computing node, the ECN tag carried in the data packet may be marked by other network devices between the first and second network devices, or it may be marked by the second network device itself. When the ECN tag carried in the data packet is marked by the second network device, the second computing node cannot distinguish the congestion status of the remote path of each first path, or in other words, the port congestion status of at least one ingress port determined by the second computing node based on the proportion of congestion packets is the same.

[0285] Optionally, the target device may maintain an ingress port congestion status record table based on all its ingress ports to record the number of packets carrying ECN tags and the total number of packets received by each ingress port.

[0286] As an example, Table 6 below provides an exemplary ingress port congestion status record table.

[0287] Table 6

[0288] For any ingress port in the target device, the ratio between the number of packets carrying ECN tags in that ingress port and the total number of packets can be calculated based on Table 6, thereby obtaining the congestion packet ratio of that ingress port. Based on the congestion packet ratio, it can be determined whether the ingress port is congested.

[0289] It should be understood that the number of packets carrying ECN tags and the total number of packets in each ingress port in Table 6 are merely examples. The port identifiers of the ingress ports in Table 6 can also be represented in other ways. This embodiment of the application only uses the port identifier as the port number as an example and does not constitute a limitation on its representation method.

[0290] Step 1102: If the notification message trigger condition is detected, a notification message is sent to the first network device in the communication network based on the port congestion status corresponding to at least one ingress port, so as to instruct the first network device to determine the global path congestion status corresponding to multiple first paths based on the notification message.

[0291] Here, the first network device is the network device in the communication network that forwards the data packet before the second network device. Multiple first paths refer to the packet transmission paths from the first network device to the second network device. The global path congestion status indicates the global congestion status of the corresponding first path.

[0292] Referring to the previous explanation regarding the first network device receiving the notification message, the target device also has the following situations when sending the notification message to the first network device, which will be introduced separately below.

[0293] In the first case, the notification message carries the port congestion status corresponding to the at least one ingress port.

[0294] Optionally, the notification message includes a port congestion status field, which carries the port congestion status corresponding to at least one ingress port.

[0295] In this first scenario, since the port congestion status of the at least one ingress port can reflect the congestion situation during the transmission of the data packet received at the ingress port from the first network device to the target device, and the second network device forwards the data packet at the last hop in the communication network, the first network device can determine the congestion situation of the packet transmission path from the first network device to the second network device after receiving the port congestion status of the at least one ingress port, thereby obtaining the remote path congestion status corresponding to the multiple first paths respectively.

[0296] In some embodiments, after receiving a data packet, the target device is further configured to determine the intra-flow packet out-of-order degree of the data flow to which the data packet belongs, and then send an announcement message to the first network device in combination with the intra-flow packet out-of-order degree, so as to instruct the first network device to determine the global path congestion state corresponding to multiple first paths based on the port congestion state corresponding to at least one ingress port and the intra-flow packet out-of-order degree.

[0297] Among them, the in-stream message out-of-order status indicates the out-of-order status of messages that have arrived at the target device in the data stream to which the data packet belongs.

[0298] In the second scenario, if the target device also determines the out-of-order status of the data stream to which the data packet belongs, the notification message can carry at least one port congestion status and the out-of-order status of the data stream corresponding to the ingress port.

[0299] Optionally, the notification message includes a port congestion status field and an out-of-order status field. The port congestion status field is used to carry the port congestion status corresponding to at least one inbound port, and the out-of-order status field is used to carry the out-of-order degree of packets within the flow.

[0300] In one possible implementation, the in-stream message out-of-order degree is the difference between the number of data packets that have not arrived between the sequence number of the currently arriving data packet and the sequence number of the expected data packet, and the number of data packets that arrived earlier than the expected data packet.

[0301] As an example, the out-of-order rate of packets within a flow can be calculated using the following formula (1): δ=cp-ep-Δ (1)

[0302] Where Δ is the number of out-of-order packets that have arrived, ep is the sequence number of the expected data packet, and cp is the sequence number of the currently arriving data packet.

[0303] For example, assuming the expected data packet sequence number is 0, and the data packets arrive in the order of data packet number 8 and data packet number 10, then the out-of-order degree of the intra-flow packets calculated based on the above formula (1) is 8 and 9 respectively.

[0304] In the third scenario, when the target device determines the port congestion status corresponding to at least one ingress port and the out-of-order status of intra-flow packets in the data stream to which the data packet belongs, the notification message may carry the remote path congestion status of multiple first paths corresponding to the data packet. The remote path congestion status indicates the congestion status of network devices other than the first network device on the corresponding first path.

[0305] Optionally, the notification message includes a path congestion status field, which carries the congestion status of the remote paths corresponding to the multiple first paths.

[0306] In one possible implementation, the target device determines the remote path congestion status of multiple first paths based on the port congestion status corresponding to at least one ingress port.

[0307] In another possible implementation, the target device determines the remote path congestion status corresponding to multiple first paths based on the out-of-order status of packets within the flow and the port congestion status corresponding to at least one ingress port.

[0308] It should be noted that the specific implementation process of the target device determining the remote path congestion status of the first path can be referred to in the method embodiment of Figure 7 above. The implementation process of the first network device determining the remote path congestion status of the first path is the same, and the implementation logic is the same. This application embodiment will not be repeated here.

[0309] Based on the above three scenarios, when the target device performs step 1102, the triggering conditions for the detected notification message include at least one of the following:

[0310] (1) The out-of-order degree of the in-flow packets is greater than the out-of-order degree threshold.

[0311] (2) At least one of the input ports is congested.

[0312] (3) The time interval since the last notification message was sent has reached the notification interval duration.

[0313] The notification interval can be any preset value, such as round-trip time (RTT). In actual implementation, the value of the notification interval can also be adjusted, and this application embodiment does not limit this.

[0314] Optionally, the target device may also, at preset intervals, clear the ECN tag message statistics (i.e., the last two columns of Table 6) of each ingress port in Table 6 above, based on the port congestion status of at least one ingress port. Furthermore, regardless of whether an announcement message has been sent within the preset interval, an announcement message is sent to the first network device based on the proportion of congested messages at each ingress port before clearing, to inform the first network device of the remote path congestion status on the message transmission path between the target device and the target device.

[0315] The preset period is a preset value, for example, the preset period can be 0.5*RTT; and in actual implementation, the value of the preset period can also be adjusted, and this application embodiment does not limit it.

[0316] To facilitate understanding, the following will take the target device as the second network device as an example and, with reference to Figure 12, illustrate the implementation process of the target device sending an announcement message to the first network device based on the data packet reception status.

[0317] Referring to Figure 12, when forwarding data packets of data stream 1 and data stream 2, the first network device uses the packet forwarding method provided in this application embodiment to determine the target path of each data packet and forward the corresponding data packets according to the target path. Specifically, the target path corresponding to data packets 11, 22, 13, and 23 is path 1, i.e., the path shown by the solid line in Figure 12, which is also the packet transmission path shown as "first network device - intermediate network device 1 - second network device"; the target path corresponding to data packets 21, 12, 24, and 14 is path 2, i.e., the path shown by the dashed line in Figure 12, which is also the packet transmission path shown as "first network device - intermediate network device 2 - second network device".

[0318] After the first network device sends the above 8 data packets one by one, the data packets received by the second network device are also out of order due to the different congestion conditions of the two packet transmission paths. As shown in Figure 12, assuming that the second network device receives data packet 13 first on path 1 and data packet 24 first on path 2, the second network device can send an announcement message to the first network device based on the received data packets. This message instructs the first network device to analyze the congestion status of the far-end paths of path 1 and path 2 based on the information carried in the announcement message when forwarding data packets in the future. That is, it can determine the congestion status of intermediate network devices other than the first network device and the second network device on path 1 and path 2.

[0319] As an example, the second network device will construct a co-announcement message to indicate to the first network device the congestion status of the remote paths corresponding to path 1 and path 2, respectively, when any of the following conditions are met.

[0320] (1) The out-of-order degree of the data stream to which the arrived data packet belongs is greater than the out-of-order degree threshold, and no data packet in the data stream triggers an announcement message within the announcement interval.

[0321] As shown in Figure 12, when the second network device receives data packet 24 from data stream 1, the in-stream packet out-of-order degree is 3. Assuming the out-of-order degree threshold is 2, it is determined that the current in-stream packet out-of-order degree of data stream 1 to which data packet 24 belongs is greater than the out-of-order degree threshold. At this time, if data stream 1 does not trigger an announcement message within the announcement interval, the second network device generates an announcement message based on the in-stream packet out-of-order degree of data stream 1 and sends the announcement message to the first network device.

[0322] The notification message carries the out-of-order status of intra-stream messages in data stream 1 and the port congestion status of the ingress port receiving the data message 24.

[0323] (2) The proportion of congested packets in any one of the ingress ports of the second network device receiving data packets is greater than the port congestion threshold, and no data packets in the data stream to which the data packet belongs trigger an announcement message within the announcement interval.

[0324] As shown in Figure 12, when the second network device receives data packet 13 from data stream 2, the data packet carries an ECN tag. If the second network device determines that the proportion of congested packets on the ingress port that received the data packet 13 is greater than the port congestion threshold, and data stream 2 does not trigger an announcement packet within the announcement interval, then the second network device determines that the ingress port is congested. Based on the port congestion status of the ingress port, the second network device generates an announcement packet and sends the announcement packet to the first network device.

[0325] The notification message carries the out-of-order status of intra-stream messages in data stream 2 and the port congestion status of the ingress port receiving the data message 13.

[0326] (3) The time interval since the last notification message was sent has reached the notification interval duration.

[0327] As an example, referring to Figure 13, the notification message may include a port congestion status field and an out-of-order status field. The port congestion status field is used to carry the port congestion status of at least one inbound port, and the out-of-order status field is used to carry the out-of-order degree of packets within the flow.

[0328] Optionally, the notification message may also include a source IP field corresponding to the data packet that triggered the notification message and a destination IP field of the destination device that sent the notification message. In the example shown in Figure 12, the source IP field is used to carry the source IP address of data packet 24 or the source IP address of data packet 13; the destination IP field is used to carry the device IP address of the second network device.

[0329] Optionally, the notification message may include a message type field that carries the message type of the notification message to indicate that the notification message is used by the first network device to detect remote path congestion status when forwarding data packets packet by packet during load balancing.

[0330] Optionally, the announcement message also carries path information for the data packets. In a specific network topology (e.g., a Layer 2 Clos architecture), this path information can also be represented by the port identifier of the outgoing port of the first network device that sends the data packets.

[0331] In summary, in this embodiment, when the target device receives a data packet, it can determine the port congestion status corresponding to at least one ingress port of the received data packet. Based on the port congestion status corresponding to each of the at least one ingress port, and upon detecting a notification message trigger, the target device sends a notification message to the first network device. This instructs the first network device to determine the global path congestion status of the packet transmission path from the first network device to the target device based on the information carried in the notification message. Thus, by sending notification messages from the target device to the first network device, the first network device is assisted in determining the global path congestion status of the packet transmission path, thereby more efficiently determining the target path for forwarding each data packet and improving the accuracy of packet-by-packet routing when forwarding data packets.

[0332] Based on the message forwarding methods shown in Figures 3-13 above, taking All-to-All and All-reduce communication scenarios as examples, the packet-by-packet load balancing scheme based on packet forwarding provided in this application embodiment is compared with the traditional equal-cost multi-path routing (ECMP) load balancing scheme based on hash. The comparison results are shown in Figure 14.

[0333] In distributed or parallel computing, "All to All" typically refers to the process where each computing node needs to communicate with all other nodes. This communication model ensures global coordination and data exchange, enabling nodes to share information or perform specific collaborative tasks. "All reduce," on the other hand, is a commonly used data communication and synchronization mechanism in distributed computing. Its function is to aggregate data from various computing nodes and broadcast the results to all nodes to achieve global data synchronization. This facilitates efficient computation and coordination in distributed systems, especially in scenarios where the results of various subtasks need to be merged to arrive at a final result.

[0334] Referring to Figure 14, assuming a path latency jitter of 10µs, in the All-to-All communication scenario, 128 compute nodes communicate using a Layer 2 Clos architecture communication network, and in the All-reduce communication scenario, 2048 compute nodes communicate using a Layer 2 Clos architecture communication network. As shown in Figure 14, in the All-to-All communication scenario, the packet-by-packet load balancing scheme of this application can improve throughput by more than 50%; in the All-reduce communication scenario, the packet-by-packet load balancing scheme of this application can improve throughput by more than 100%.

[0335] Therefore, the packet forwarding method provided in this application embodiment can not only accurately and effectively determine the forwarding path of each data packet, but also significantly improve the load balancing effect of the first network device when forwarding multiple data packets.

[0336] Figure 15 is a schematic diagram of a message forwarding device provided in an embodiment of this application. This message forwarding device can be implemented by software, hardware, or a combination of both as part or all of a first network device in a communication network. The first network device can be the first network device shown in Figures 3-13 above. Referring to Figure 15, the message forwarding device 1500 includes: a path determination module 1501 and a path selection module 1502.

[0337] The path determination module 1501 is used to determine multiple first paths based on the indication information carried in the data packet to be forwarded. The indication information indicates the second network device that last forwards the data packet in the communication network. The multiple first paths refer to the packet transmission path from the first network device to the second network device.

[0338] The path selection module 1502 is used to select a target path from multiple first paths according to the global path congestion status corresponding to each of the multiple first paths. The global path congestion status indicates the global congestion situation of the corresponding first path, and the target path is used to forward data packets to the second network device.

[0339] Optionally, the message forwarding device 1500 further includes:

[0340] The remote status acquisition module is used to acquire the remote path congestion status corresponding to multiple first paths. The remote path congestion status indicates the congestion status of network devices other than the first network device on the corresponding first path.

[0341] The global state determination module is used to determine the global path congestion status of multiple first paths based on the remote path congestion status of each first path.

[0342] Optionally, the global state determination module is specifically used for:

[0343] Determine the port congestion status corresponding to at least one outgoing port in the first network device, wherein at least one outgoing port is a port connected to multiple first paths, and the port congestion status indicates the congestion status of the corresponding outgoing port;

[0344] Based on the port congestion status corresponding to at least one outgoing port and the remote path congestion status corresponding to multiple first paths, the global path congestion status corresponding to multiple first paths is determined.

[0345] Optionally, the remote status acquisition module is also used for:

[0346] Receive notification messages, which carry the port congestion status corresponding to at least one ingress port in the second network device. At least one ingress port is a port connected to multiple first paths, and the port congestion status indicates the congestion status of the corresponding ingress port.

[0347] Based on the port congestion status corresponding to at least one ingress port, determine the remote path congestion status corresponding to multiple first paths.

[0348] Optionally, the notification message includes a port congestion status field, which carries the port congestion status corresponding to at least one ingress port.

[0349] Optionally, the notification message also carries the in-flow message out-of-order status, which indicates the out-of-order status of messages that have arrived at the second network device in the data flow to which the data packet belongs;

[0350] Optionally, the remote status acquisition module is also used for:

[0351] Based on the port congestion status and in-flow packet out-of-order degree corresponding to at least one ingress port, the remote path congestion status corresponding to multiple first paths is determined.

[0352] Optionally, the notification message includes a port congestion status field and an out-of-order status field. The port congestion status field is used to carry the port congestion status corresponding to at least one inbound port, and the out-of-order status field is used to carry the out-of-order degree of packets within the flow.

[0353] Optionally, the remote status acquisition module is also used for:

[0354] Receive notification messages, which carry the congestion status of the remote paths corresponding to the first paths.

[0355] Optionally, the notification message includes a path congestion status field, which carries the congestion status of the remote paths corresponding to the multiple first paths.

[0356] Optionally, the path selection module 1502 includes:

[0357] The flow analysis unit is used to determine the out-of-order type of the data stream to which a data packet belongs;

[0358] The strategy determination unit is used to determine the path selection strategy for data packets based on the out-of-order flow type.

[0359] The path selection unit is used to select the target path from multiple first paths based on the path selection strategy.

[0360] Optionally, the out-of-order flow types include Type I out-of-order flow and Type II out-of-order flow; the flow analysis unit is specifically used for:

[0361] If the out-of-order degree of the intra-flow messages in the data stream to which the data packet belongs is greater than the message out-of-order threshold, then the out-of-order type of the data stream to which the data packet belongs is determined to be the first type of out-of-order flow.

[0362] If the out-of-order degree of the intra-flow messages in the data stream to which the data packet belongs is less than or equal to the out-of-order threshold, then the out-of-order type of the data stream to which the data packet belongs is determined to be a second type of out-of-order flow.

[0363] Optionally, the path selection strategy includes a first selection strategy and a second selection strategy. The first selection strategy indicates that a first type of path is used to transmit data packets, which is a path that is congested. The second selection strategy indicates that a second type of path is used to transmit data packets, which is a path that is not congested.

[0364] The strategy determination unit is specifically used for:

[0365] If the out-of-order flow type is the first type of out-of-order flow, then the path selection strategy for the data packet is determined to be the first selection strategy;

[0366] If the out-of-order flow type is the second type of out-of-order flow, then the path selection strategy for the data packets is determined to be the second selection strategy.

[0367] Optionally, the message forwarding device 1500 further includes:

[0368] The port filtering module is used to determine the packet container size of each outgoing port in the first network device. The packet container size indicates the total number of bytes of packets sent by the corresponding outgoing port in the current statistical period. At least one outgoing port is determined from the outgoing ports whose packet container size is less than the container threshold.

[0369] Optionally, the data stream to which the data packet belongs is either a first type of data stream or a second type of data stream; wherein, the five-tuple information of each data packet in the first type of data stream is the same, and the data packets in the second type of data stream correspond to the same source network device and the same destination network device in the communication network.

[0370] In this embodiment, for a data packet to be forwarded, the packet forwarding device first determines the second network device in the communication network that is the last to forward the data packet based on the indication information carried in the data packet, and determines the global path congestion status corresponding to each of the multiple packet transmission paths (i.e., first paths) from the first network device to the second network device. For each first path, since the global path congestion status indicates the overall congestion situation of the entire first path and can reflect the congestion situation of all network devices on the first path when forwarding data packets, the packet forwarding device selects the target path for forwarding the data packet from the multiple first paths based on the global path congestion status corresponding to each of the multiple first paths. Therefore, it can be seen that the packet forwarding device comprehensively considers the congestion situation of the entire packet transmission path during the forwarding process of the data packet, improving the accuracy of target path selection.

[0371] Furthermore, when the packet forwarding device forwards multiple data packets, the packet forwarding method provided in this application embodiment can improve the accuracy of packet-by-packet routing, thereby making the load balancing effect of the first network device better.

[0372] It should be noted that the message forwarding device 1500 provided in the above embodiments is only illustrated by the division of the above functional modules when forwarding data packets. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the message forwarding device provided in the above embodiments and the message forwarding method embodiment shown in FIG7 belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0373] Figure 16 is a schematic diagram of another message forwarding device provided in an embodiment of this application. This message forwarding device can be implemented by software, hardware, or a combination of both as part or all of a target device in a communication network. The target device can be the second network device shown in Figures 3-13 above, or a second computing node connected to the second network device and used to receive data packets. Referring to Figure 16, the message forwarding device 1600 includes: a port congestion analysis module 1600 and an announcement module 1602.

[0374] The port congestion analysis module 1600 is used to determine the port congestion status corresponding to at least one ingress port of the received data packet. The port congestion status indicates the congestion situation of the corresponding ingress port.

[0375] The notification module 1602 is used to send a notification message to a first network device in the communication network based on the port congestion status corresponding to at least one ingress port if a notification message trigger condition is detected. This is to instruct the first network device to determine the global path congestion status corresponding to multiple first paths based on the notification message. The first network device is a network device that forwards data packets before a second network device in the communication network. The multiple first paths refer to the packet transmission paths from the first network device to the second network device. The global path congestion status indicates the global congestion situation of the corresponding first path.

[0376] Optionally, the message forwarding device 1600 further includes:

[0377] The out-of-order analysis module is used to determine the out-of-order status of packets within the data stream to which the data packet belongs. The out-of-order status of packets within the data stream to which the data packet belongs indicates the out-of-order status of packets that have reached the target device.

[0378] The notification module 1602 is also used for:

[0379] Based on the port congestion status and in-flow packet out-of-order degree corresponding to at least one ingress port, an announcement message is sent to the first network device.

[0380] Optionally, the port congestion analysis module 1600 is specifically used for:

[0381] Determine the proportion of congested packets corresponding to at least one ingress port. The proportion of congested packets is the ratio between the total number of data packets received in the corresponding ingress port and the number of data packets carrying congestion flags in the data packets received in the ingress port.

[0382] If the proportion of congested packets is greater than the port congestion threshold, then the corresponding inbound port is determined to be congested.

[0383] If the proportion of congested packets is less than or equal to the port congestion threshold, then it is determined that there is no congestion on the corresponding inbound port.

[0384] Optionally, the notification message triggering conditions include at least one of the following:

[0385] The out-of-order degree of in-stream packets is greater than the out-of-order threshold;

[0386] At least one of the input ports is congested;

[0387] The time interval since the last notification message has reached the notification interval duration.

[0388] Optionally, the notification message carries the port congestion status corresponding to at least one ingress port.

[0389] Optionally, the notification message includes a port congestion status field, which carries the port congestion status corresponding to at least one ingress port.

[0390] Optionally, the notification message carries at least one port congestion status and in-flow packet out-of-order status corresponding to each ingress port.

[0391] Optionally, the notification message includes a port congestion status field and an out-of-order status field. The port congestion status field is used to carry the port congestion status corresponding to at least one inbound port, and the out-of-order status field is used to carry the out-of-order degree of packets within the flow.

[0392] Optionally, the notification message carries the remote path congestion status of multiple first paths corresponding to the data packet, and the remote path congestion status indicates the congestion status of network devices other than the first network device on the corresponding first path.

[0393] The message forwarding device 1600 also includes:

[0394] The path status analysis module is used to determine the remote path congestion status of multiple first paths based on the port congestion status corresponding to at least one ingress port.

[0395] Optionally, the notification message carries the remote path congestion status of multiple first paths corresponding to the data packet, and the remote path congestion status indicates the congestion status of network devices other than the first network device on the corresponding first path.

[0396] The message forwarding device 1600 also includes:

[0397] The path status analysis module is used to determine the remote path congestion status of multiple first paths based on the out-of-order packet rate within the flow and the port congestion status corresponding to at least one ingress port.

[0398] Optionally, the notification message includes a path congestion status field, which carries the congestion status of the remote paths corresponding to the multiple first paths.

[0399] Optionally, the data stream to which the data packet belongs is either a first type of data stream or a second type of data stream; wherein, the five-tuple information of each data packet in the first type of data stream is the same, and the data packets in the second type of data stream correspond to the same source network device and the same destination network device in the communication network.

[0400] In this embodiment, when the target device receives a data packet, the packet forwarding device can determine the port congestion status corresponding to at least one ingress port of the received data packet. Based on the port congestion status corresponding to each of the at least one ingress port, and upon detecting a notification message trigger, the device sends a notification message to the first network device. This instructs the first network device to determine the global path congestion status of the packet transmission path from the first network device to the target device based on the information carried in the notification message. Thus, by sending notification messages to the first network device, the device is assisted in determining the global path congestion status of the packet transmission path, thereby more efficiently determining the target path for forwarding each data packet and improving the accuracy of packet-by-packet routing when forwarding data packets.

[0401] It should be noted that the message forwarding device 1600 provided in the above embodiments, when sending an announcement message to the first network device based on the reception of data packets, is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the message forwarding device provided in the above embodiments and the message forwarding method embodiment shown in FIG11 belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0402] Figure 17 is a schematic diagram of the structure of a network device provided in an embodiment of this application. When the above-described packet forwarding method is executed, the network device can be a first network device or a second network device.

[0403] Referring to Figure 17, the network device 1700 can be a switch, router, access gateway, or other device capable of receiving and forwarding data packets. The network device 1700 includes a main control board 1710, an interface board 1730, and an interface board 1740. In the case of multiple interface boards, a switching network board (not shown in Figure 17) may be included, which is used to complete data exchange between the various interface boards (interface boards are also called line cards or service boards).

[0404] The main control board 1710 performs functions such as system management, equipment maintenance, and protocol processing. Interface boards 1730 and 1740 provide various service interfaces (e.g., POS interface, GE interface, ATM interface, etc.) and implement data stream forwarding. The main control board 1710 primarily has three types of functional units: a system management control unit, a system clock unit, and a system maintenance unit. The main control board 1710, interface board 1730, and interface board 1740 communicate with each other via a system bus connected to the system backplane. Interface board 1730 includes one or more processors 1731. Processors 1731 control and manage the interface board, communicate with the central processing unit on the main control board, and handle data stream forwarding. The memory 1732 on interface board 1730 stores forwarding table entries; processors 1731 forward data streams by searching the forwarding table entries stored in memory 1732.

[0405] The interface board 1730 includes one or more network interfaces 1733 for receiving data streams sent by computing nodes, other network devices, etc., and forwarding data packets in these data streams according to the instructions of the processor 1731. The specific implementation process will not be described in detail here.

[0406] As shown in Figure 17, this embodiment includes multiple interface boards and employs a distributed forwarding mechanism. Under this mechanism, the operations on interface board 1740 are basically similar to those on interface board 1730, and will not be described further for simplicity. Furthermore, it is understood that the processors 1731 in interface board 1730 and / or 1741 in interface board 1740 in Figure 17 can be dedicated hardware or chips, such as network processors or application-specific integrated circuits (ASICs), to implement the above functions. This implementation method is commonly referred to as using dedicated hardware or chips for the forwarding plane. Of course, processors 1731 and / or 1741 can also use general-purpose processors, such as general-purpose CPUs, to implement the functions described above.

[0407] Furthermore, it should be noted that a network device may have one or more main control boards, including a primary and a backup main control board. Similarly, it may have one or more interface boards; the more data processing capabilities the network device possesses, the more interface boards it provides. When a network device includes multiple interface boards, these boards can communicate through one or more switching network boards, enabling load sharing and redundancy backup. In a centralized forwarding architecture, the network device may not require a switching network board; the interface boards handle the entire system's business data processing. In a distributed forwarding architecture, the network device can include multiple interface boards, which can exchange data through a switching network board, providing high-capacity data exchange and processing capabilities. Therefore, the data access and processing capabilities of a distributed architecture network device are greater than those of a centralized architecture network device. The specific architecture adopted depends on the specific network deployment scenario, and no limitations are imposed here.

[0408] In some embodiments, memory 1732 may be read-only memory (ROM), random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), optical discs (including compact disc read-only memory (CD-ROM), compressed optical discs, laser discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having an instruction or data structure form and accessible by a computer, but not limited thereto. Memory 1732 may exist independently and be connected to processor 1731 via a communication bus. Memory 1732 may also be integrated with processor 1731.

[0409] In some embodiments, network interface 1733 can be a transceiver-like device used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. Network interface 1733 includes a wired network interface and may also include a wireless network interface. The wired network interface can be, for example, an Ethernet interface. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. The wireless network interface can be a WLAN interface, a cellular network communication interface, or a combination thereof, etc. When the network device acts as any network device within a domain, network interface 1733 is used to forward data packets to other network devices; when the network device acts as a head node within a domain, network interface 1733 can also be used to communicate with computing nodes, such as receiving data packets sent by computing nodes.

[0410] In some embodiments, a network device may include multiple processors, each of which may be a single-core processor or a multi-core processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0411] In some embodiments, memory 1732 is used to store a computer program that executes the scheme of this application. Processor 1731 can execute the computer program stored in memory 1732, causing 1700 to execute the steps of the first network device or the second network device in the embodiments shown in FIG3 to FIG13 to execute the data packet forwarding method. The specific implementation can be referred to the detailed description in the embodiments shown in FIG3 to FIG13, which will not be repeated here.

[0412] Figure 18 is a schematic diagram of a computer device according to an embodiment of this application. The computer device may be a second computing node connected to a second network device as shown in Figures 3-13. The computer device includes at least one processor 101, a communication bus 102, a memory 103, and at least one communication interface 104.

[0413] Processor 101 can be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, or one or more integrated circuits for implementing the solutions of this application, such as application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0414] The communication bus 102 is used to transmit information between the aforementioned components. The communication bus 102 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used to represent it in Figure 18, but this does not mean that there is only one bus or one type of bus.

[0415] The memory 103 may be a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc (including a compact disc read-only memory (CD-ROM), a compressed optical disc, a laser disc, a digital versatile optical disc, a Blu-ray disc, etc.), a magnetic disk storage medium, or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but not limited thereto. The memory 103 may exist independently and be connected to the processor 101 via the communication bus 102. Alternatively, the memory 103 may be integrated with the processor 101.

[0416] Communication interface 104 uses any transceiver-like device for communicating with other devices or communication networks. Communication interface 104 includes a wired communication interface and may also include a wireless communication interface. The wired communication interface may be, for example, an Ethernet interface. The Ethernet interface may be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface may be a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof.

[0417] As an example, processor 101 may include one or more CPUs, such as CPU0 and CPU1 as shown in FIG18.

[0418] As an example, a computer device may include multiple processors, such as processor 101 and processor 105 as shown in Figure 18. Each of these processors may be a single-core processor or a multi-core processor. Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0419] In some embodiments, the computer device may further include output devices and input devices. The output device communicates with the processor 101 and can display information in various ways. For example, the output device may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device communicates with the processor 101 and can receive user input in various ways. For example, the input device may be a mouse, keyboard, touchscreen device, or sensing device, etc.

[0420] In some embodiments, memory 103 is used to store program code 110 for executing the scheme of this application, and processor 101 can execute the program code 110 stored in memory 103. The program code 110 may include one or more software modules, and the computer device can implement the packet forwarding method provided in the embodiment of FIG11 above through processor 101 and program code 110 in memory 103.

[0421] This application embodiment also provides a communication node, which includes a processor. The processor is used to execute the steps of the message forwarding method shown in FIG7 above, or to execute the steps of the message forwarding method shown in FIG11 above.

[0422] This application also provides a communication system, which includes a first communication node and a second communication node. The first communication node is used to execute the steps of the message forwarding method shown in FIG7 above, and the second communication node is used to execute the steps of the message forwarding method shown in FIG11 above.

[0423] This application also provides a chip system, which includes a processor. The processor is used to support communication nodes in implementing the steps of the message forwarding method shown in FIG7 above, or in implementing the steps of the message forwarding method shown in FIG11 above.

[0424] Optionally, the chip system also includes a memory for storing necessary program instructions and data for the communication node.

[0425] Alternatively, the chip system may consist of chips or may include chips and other discrete components.

[0426] This application also provides a computer-readable storage medium storing a computer program. When the computer program is run on a network device, it causes the network device to perform the steps of the packet forwarding method shown in FIG7 above, or to perform the steps of the packet forwarding method shown in FIG11 above.

[0427] This application also provides a computer-readable storage medium storing a computer program that, when run on a computing node, causes the computing node to execute the steps of the message forwarding method shown in FIG11.

[0428] This application also provides a computer program product, which includes a computer program that, when run on a network device, causes the network device to perform the steps of the packet forwarding method shown in FIG7 above, or to perform the steps of the packet forwarding method shown in FIG11 above.

[0429] This application also provides a computer program product, which includes a computer program that, when run on a computing node, causes the computing node to perform the steps of the message forwarding method shown in FIG11 above.

[0430] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital versatile disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0431] It should be understood that "multiple" as mentioned herein refers to two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and the terms "first," "second," etc., do not necessarily imply that they are different.

[0432] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0433] The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A message forwarding method, characterized in that, A first network device applied in a communication network; the method includes: Based on the indication information carried in the data packet to be forwarded, multiple first paths are determined. The indication information indicates the second network device that is the last to forward the data packet in the communication network. The multiple first paths refer to the packet transmission paths from the first network device to the second network device. Based on the global path congestion status corresponding to the multiple first paths, a target path is selected from the multiple first paths. The global path congestion status indicates the global congestion situation of the corresponding first path. The target path is used to forward the data packet to the second network device.

2. The method as described in claim 1, characterized in that, Before selecting the target path from the plurality of first paths, the method further includes: Obtain the remote path congestion status corresponding to the multiple first paths respectively. The remote path congestion status indicates the congestion status of network devices other than the first network device on the corresponding first path. Based on the remote path congestion status corresponding to the multiple first paths, the global path congestion status corresponding to the multiple first paths is determined.

3. The method as described in claim 2, characterized in that, The step of determining the global path congestion status based on the remote path congestion status of the multiple first paths includes: Determine the port congestion status corresponding to at least one outgoing port in the first network device, wherein the at least one outgoing port is a port connected to the plurality of first paths, and the port congestion status indicates the congestion situation of the corresponding outgoing port; Based on the port congestion status corresponding to the at least one outgoing port and the remote path congestion status corresponding to the multiple first paths, the global path congestion status corresponding to the multiple first paths is determined.

4. The method as described in claim 2 or 3, characterized in that, The step of obtaining the congestion status of the remote paths corresponding to the multiple first paths includes: Receive notification messages, the notification messages carrying the port congestion status corresponding to at least one ingress port in the second network device, the at least one ingress port being a port connected to the plurality of first paths, the port congestion status indicating the congestion status of the corresponding ingress port; Based on the port congestion status corresponding to the at least one ingress port, the remote path congestion status corresponding to the multiple first paths is determined.

5. The method as described in claim 4, characterized in that, The notification message includes a port congestion status field, which carries the port congestion status corresponding to the at least one ingress port.

6. The method as described in claim 4, characterized in that, The notification message also carries an intra-flow packet out-of-order status, which indicates the out-of-order status of packets that have arrived at the second network device in the data stream to which the data packet belongs; The step of determining the remote path congestion status corresponding to the plurality of first paths based on the port congestion status corresponding to the at least one ingress port includes: Based on the port congestion status corresponding to the at least one ingress port and the out-of-order packet order within the flow, the remote path congestion status corresponding to the multiple first paths is determined.

7. The method as described in claim 6, characterized in that, The notification message includes a port congestion status field and an out-of-order status field. The port congestion status field is used to carry the port congestion status corresponding to the at least one inbound port, and the out-of-order status field is used to carry the out-of-order degree of the packets within the flow.

8. The method as described in claim 2 or 3, characterized in that, The step of obtaining the congestion status of the remote paths corresponding to the multiple first paths includes: Receive notification messages, which carry the congestion status of the remote paths corresponding to the multiple first paths respectively.

9. The method as described in claim 8, characterized in that, The notification message includes a path congestion status field, which carries the congestion status of the remote paths corresponding to the multiple first paths.

10. The method according to any one of claims 6-9, characterized in that, The step of selecting a target path from the multiple first paths based on the global path congestion status corresponding to each of the multiple first paths includes: Determine the out-of-order type of the data stream to which the data packet belongs; Based on the out-of-order flow type, determine the path selection strategy for the data packets; Based on the path selection strategy, the target path is selected from the plurality of first paths.

11. The method as described in claim 10, characterized in that, The out-of-order flow types include Type I out-of-order flow and Type II out-of-order flow; Determining the out-of-order type of the data stream to which the data packet belongs includes: If the out-of-order degree of the intra-flow packets in the data stream to which the data packet belongs is greater than the out-of-order threshold, then the out-of-order type of the data stream to which the data packet belongs is determined to be the first type of out-of-order stream. If the out-of-order degree of the intra-flow packets of the data stream to which the data packet belongs is less than or equal to the out-of-order threshold, then the out-of-order type of the data stream to which the data packet belongs is determined to be the second type of out-of-order flow.

12. The method as described in claim 11, characterized in that, The path selection strategy includes a first selection strategy and a second selection strategy. The first selection strategy indicates that the data packet is transmitted using a first type of path, which is a path that is congested. The second selection strategy indicates that the data packet is transmitted using a second type of path, which is a path that is not congested. The step of determining the path selection strategy for the data packets based on the out-of-order flow type includes: If the out-of-order flow type is the first type of out-of-order flow, then the path selection strategy for the data packet is determined to be the first selection strategy. If the out-of-order flow type is the second type of out-of-order flow, then the path selection strategy for the data packet is determined to be the second selection strategy.

13. The method according to any one of claims 3-12, characterized in that, Before determining the global path congestion status corresponding to the multiple first paths, the method includes: Determine the packet container size of each outgoing port in the first network device, wherein the packet container size indicates the total number of bytes of packets sent by the corresponding outgoing port in the current statistical period; The at least one output port is determined from the output ports whose message container size is less than the container threshold.

14. The method according to any one of claims 1-13, characterized in that, The data stream to which the data packet belongs is either a first type of data stream or a second type of data stream; wherein, the five-tuple information of each data packet in the first type of data stream is the same, and each data packet in the second type of data stream corresponds to the same source network device and the same destination network device in the communication network.

15. A message forwarding method, characterized in that, Applied to the target device; the method includes: Determine the port congestion status corresponding to at least one ingress port of the received data packet, wherein the port congestion status indicates the congestion situation of the corresponding ingress port; If an announcement message trigger condition is detected, an announcement message is sent to a first network device in the communication network based on the port congestion status corresponding to the at least one ingress port, so as to instruct the first network device to determine the global path congestion status corresponding to multiple first paths based on the announcement message. The first network device is a network device in the communication network that forwards the data packets before the second network device. The multiple first paths refer to the packet transmission paths from the first network device to the second network device. The global path congestion status indicates the global congestion situation of the corresponding first path.

16. The method as described in claim 15, characterized in that, The method further includes: Determine the in-flow packet out-of-order degree of the data stream to which the data packet belongs, wherein the in-flow packet out-of-order degree indicates the out-of-order status of packets that have reached the target device in the data stream to which the data packet belongs; The step of sending a notification message to the first network device based on the port congestion status corresponding to the at least one ingress port includes: Based on the port congestion status corresponding to the at least one ingress port and the out-of-order packet rate within the flow, a notification message is sent to the first network device.

17. The method as described in claim 15 or 16, characterized in that, Determining the port congestion state corresponding to at least one ingress port includes: Determine the proportion of congested packets corresponding to the at least one ingress port, wherein the proportion of congested packets is the ratio between the total number of data packets received in the corresponding ingress port and the number of data packets carrying congestion flags in the data packets received in the ingress port; If the proportion of congested packets is greater than the port congestion threshold, then it is determined that the corresponding ingress port is congested. If the proportion of congested packets is less than or equal to the port congestion threshold, then it is determined that the corresponding ingress port is not congested.

18. The method as described in claim 16, characterized in that, The notification message triggering conditions include at least one of the following: The out-of-order degree of the intra-stream packets is greater than the out-of-order threshold; Congestion exists in any one of the at least one inlet ports; The time interval since the last transmission of the notification message has reached the notification interval duration.

19. The method according to any one of claims 15-18, characterized in that, The notification message carries the port congestion status corresponding to each of the at least one ingress port.

20. The method as described in claim 19, characterized in that, The notification message includes a port congestion status field, which carries the port congestion status corresponding to the at least one ingress port.

21. The method according to any one of claims 16-18, characterized in that, The notification message carries the port congestion status corresponding to the at least one ingress port and the out-of-order status of the in-flow packets.

22. The method as described in claim 21, characterized in that, The notification message includes a port congestion status field and an out-of-order status field. The port congestion status field is used to carry the port congestion status corresponding to the at least one inbound port, and the out-of-order status field is used to carry the out-of-order degree of the packets within the flow.

23. The method according to any one of claims 15-18, characterized in that, The notification message carries the remote path congestion status of multiple first paths corresponding to the data packet, and the remote path congestion status indicates the congestion status of network devices other than the first network device on the corresponding first path. Before sending the notification message to the first network device, the method further includes: Based on the port congestion status corresponding to the at least one ingress port, the remote path congestion status corresponding to the multiple first paths is determined.

24. The method according to any one of claims 16-18, characterized in that, The notification message carries the remote path congestion status of multiple first paths corresponding to the data packet, and the remote path congestion status indicates the congestion status of network devices other than the first network device on the corresponding first path. Before sending the notification message to the first network device, the method further includes: Based on the out-of-order status of the packets within the flow and the port congestion status corresponding to the at least one ingress port, the remote path congestion status corresponding to the multiple first paths is determined.

25. The method as described in claim 23 or 24, characterized in that, The notification message includes a path congestion status field, which carries the congestion status of the remote paths corresponding to the multiple first paths.

26. The method according to any one of claims 15-25, characterized in that, The data stream to which the data packet belongs is either a first type of data stream or a second type of data stream; wherein, the five-tuple information of each data packet in the first data stream is the same, and each data packet in the second type of data stream corresponds to the same source network device and the same destination network device in the communication network.

27. A communication node, characterized in that, The communication node includes a processor, which is configured to perform the steps of the method according to any one of claims 1-14, or to perform the steps of the method according to any one of claims 15-26.

28. A communication system, characterized in that, The communication system includes a first communication node and a second communication node, wherein the first communication node is used to perform the steps of the method according to any one of claims 1-14, and the second communication node is used to perform the steps of the method according to any one of claims 15-26.

29. A computer-readable storage medium, characterized in that, The storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the method according to any one of claims 1-14, or the steps of the method according to any one of claims 15-26.

30. A computer program product containing instructions, characterized in that, When the instructions are executed on a computer, the computer performs the steps of the method according to any one of claims 1-14, or the steps of the method according to any one of claims 15-26.

Images