In-network aggregation method and system for heterogeneous resources
Through the sliding window and serial number management mechanism, the performance bottlenecks and hash collision problems of the INA protocol in heterogeneous resource environment are solved, and more efficient in-network aggregation performance and resource utilization are achieved.
Patent Information
- Application Number
- CN202411365380.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-09-29
AI Technical Summary
The existing INA protocols lead to a 'barrel effect' in a heterogeneous resource environment, limiting the in-network aggregation performance, and due to the deterministic design, it is prone to hash collisions and mutual wait locks (MWL) problems, resulting in performance degradation.
Adopt sliding window mechanism and message sequence number, counter field and other mechanisms to allow messages to be gradually aggregated on switches with sufficient resources to avoid hash conflicts, and manage aggregator resources through the maximum conflict sequence number to ensure that messages are on available resources. Perform progressive aggregation.
It effectively makes up for the shortcomings between resources, improves the in-network aggregation performance, avoids the occurrence of MWL, and improves INA throughput and resource utilization.
Smart Images

Figure CN118869582B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to computer network technology, and in particular to a network aggregation method and system for heterogeneous resources. Background Art
[0002] In recent years, driven by programmable network devices, in-network aggregation (INA) has gradually emerged as a new communication and computing model.
[0003] As a new transmission and computing mode, INA requires a proprietary protocol to ensure the correctness and robustness of the aggregation process. Among them, SwitchML is an INA protocol designed for distributed machine learning applications, which is used to perform INA operations using a single switch. ATP extends INA to multi-tenant machine learning and uses the ToR switches in the cluster to perform aggregation, achieving "two-layer" INA acceleration. GISA further releases this constraint, allowing INA to be flexibly deployed in a general multi-level topology, and decouples the design of applications and INA, allowing INA to dynamically provide acceleration for different applications in the same cluster.
[0004] The deployment of INA usually needs to consider two different network resources. One is the link bandwidth on the INA path, whose size will limit the transmission rate of the message on the path (for the convenience of description, it is called transmission throughput). Since the size of the message is often much smaller than the size of the data to be sent, the transmission process is carried out in a pipeline manner, so the minimum link bandwidth on the entire path determines the upper limit of the transmission throughput. Exceeding this limit will cause congestion and packet loss, and will not bring more transmission throughput improvement. The other resource is the number of aggregators on the switch, whose size will limit the aggregation throughput of the switch. When the INA path involves multiple aggregation switches, the switch with the least aggregator resources will become the bottleneck that limits the aggregation throughput. For the convenience of description, the maximum throughput supported by the available aggregators on the switch is called the aggregation capacity of the switch.
[0005] The existing INA protocol is a deterministic design, which requires that INA messages must be aggregated on each specified switch in turn. In this way, for each switch that performs the aggregation operation, the number of messages from the child nodes that need to be aggregated is fixed. n For a switch with multiple child nodes, it can clearly know that each aggregator must aggregate nThe protocol must first check the number of associated messages before forwarding the result to the next node. To this end, these protocols need to avoid hash collisions on the switch aggregator, which may lead to erroneous results. For example, SwitchML and GISA limit the number of messages injected into the network in each batch, thereby preventing irrelevant messages from being hashed to the same aggregator and causing errors; ATP needs to compare the message header and the jobIDAndSequenceNumber field cached on the aggregator to prevent irrelevant messages from being incorrectly aggregated on the same aggregator, and messages with hash collisions can only be sent to the destination node for aggregation (degenerating to the traditional aggregation method).
[0006] However, these deterministic INA protocol designs cause the two resources to impose constraints on each other, resulting in a "barrel effect". Figure 1 The example in illustrates this mutual constraint relationship. Figure 1 As shown, Figure 1 (a) is the case where the aggregator resources on the switch are sufficient, which means that the maximum aggregation throughput that the switch can achieve is higher than the transmission throughput. For this reason, the messages arriving at the switch cannot fully utilize the aggregator resources on the switch, resulting in the aggregator being idle during the transmission process. In addition, if the transmission throughputs from different sub-nodes are different, the actual transmission throughput will converge to the minimum. Figure 1 In (a), although the throughput of child node 1 is higher than that of child node 2, too many packets can only be buffered on the switch aggregator to wait for the associated packets from the remaining child nodes. In this example, even if the aggregator resources on the switch are sufficient, the actual INA throughput is determined by the minimum transmission throughput. Figure 1 (b) is the case of insufficient aggregator resources on the switch, where the transmission throughput of the child nodes is higher than the maximum aggregation throughput supported by the switch. As a result, the actual INA throughput is limited by the switch aggregation capability because the switch does not have enough aggregator resources to buffer these messages to perform aggregation operations.
[0007] When network resources are heterogeneous, the above-mentioned “barrel effect” will cause significant performance degradation of INA. Figure 2 Take as an example, where H1-H4 are source nodes, H5 is the destination node, and the transmission bandwidth of the link and the aggregation capacity of the switch are represented by their respective marked denominators. Figure 2 (a) shows the INA throughput that can be achieved by existing methods (GISA and ATP), which is limited by the aggregation capability of switch S1. Therefore, even if the available bandwidth of the link and the aggregator resources on the remaining switches are sufficient, due to the existence of the "barrel effect", these resources cannot be used to improve the performance of INA, but can only degenerate to the bottleneck throughput of 4Gbps. Figure 2(b) shows the throughput that can be achieved by the GRID method, which improves on ATP by using excess link bandwidth to transmit data to the destination node for aggregation. Although this method can make up for the shortcomings of the "barrel effect" to a certain extent, aggregating excess data at the destination node will cause this part of the traffic to suffer the same many-to-one incast transmission as the traditional method, thereby consuming a large amount of link bandwidth resources. As a result, limited by the available bandwidth of the incoming link of the destination node H5, this method can only achieve a throughput of 5Gbps.
[0008] In addition, in these INA protocols, the aggregation operation is deterministic rather than incremental. Specifically, in existing protocols, each INA message must accurately know each switch where it is aggregated, and the source node needs to limit the number of messages being transmitted in the network (i.e., those that have been sent but not yet confirmed to be received) based on the resource bottlenecks of these switches on the INA path. Otherwise, hash conflicts on the switch aggregator will cause unrelated messages to be aggregated together, or cause messages that should have been aggregated to be scattered on different switches. As a result, the switch will either output incorrect aggregation results or fail to aggregate related messages together. For the latter problem, messages on different switches may keep waiting for each other to arrive at their respective switches to complete the aggregation operation, resulting in deadlock and interruption of the aggregation process. This problem is called Mutual Waiting Lock (MWL). Summary of the invention
[0009] The technical problem to be solved by the present invention is as follows: In view of the above-mentioned problems in the prior art, a method and system for in-network aggregation of heterogeneous resources are provided, which can compensate for the shortage of "short-board" resources by using the "long-board" resources in the link bandwidth on the INA path and the number of aggregators on the switch, improve the INA performance through the complementary mechanism between these two resources, and avoid the occurrence of MWL.
[0010] In order to solve the above technical problems, the technical solution adopted by the present invention is:
[0011] A method for in-network aggregation of heterogeneous resources is applied to a distributed system including a source node, a destination node and a switch, and the method comprises the following steps:
[0012] After receiving the in-network aggregation task, the source node sends a message to the destination node through the in-network aggregation path. If the source node receives the confirmation message corresponding to the sent message within the specified time, the source node uses the sliding window to send the next batch of messages to the destination node through the in-network aggregation path; if the source node does not receive the confirmation message corresponding to the sent message within the specified time, the source node retransmits the sent message to the destination node through the in-network aggregation path;
[0013] The switch obtains the message in real time and aggregates the message, including the following steps:
[0014] The switch obtains the messages from the front-end nodes on the aggregation path in the network in real time;
[0015] If the current message is a retransmitted message, the switch forwards the current message to a back-end node on the aggregation path in the network; if the current message is not a retransmitted message, the switch compares the value of the counter of the current message with the number of leaf nodes of the switch;
[0016] If the value of the counter of the current message is consistent with the number of leaf nodes of the switch, the switch forwards the current message to the back-end node on the aggregation path in the network;
[0017] If the value of the counter of the current message is less than the number of leaf nodes of the switch, the switch calculates the position of the corresponding aggregator according to the value of the TaskID and the sequence number SN of the current message, and then sends the current message to the corresponding aggregator for aggregation. If a hash conflict occurs, the switch forwards the current message to the back-end node on the aggregation path in the network. If no hash conflict occurs, the switch updates the value of the counter of the aggregated message.
[0018] If the value of the counter of the aggregated message is equal to the number of leaf nodes of the switch, the switch forwards the aggregated message to the backend node and clears the corresponding aggregator. If the value of the counter of the aggregated message is less than the number of leaf nodes of the switch, the switch retains the aggregated message in the corresponding aggregator.
[0019] After receiving the intra-network aggregation task, the destination node obtains the message of the front-end node on the intra-network aggregation path in real time. If the value of the counter of the current message is consistent with the number of leaf nodes of the destination node, the destination node sends the corresponding confirmation message to the source node through the intra-network aggregation path.
[0020] Furthermore, if the current message is a retransmitted message, when the switch forwards the current message to a back-end node on the aggregation path in the network, the following steps are included:
[0021] The switch calculates the location of the corresponding aggregator based on the TaskID and sequence number SN of the current message;
[0022] If the corresponding aggregator is empty, or the value of the sequence number SN field of the current message is different from the value of the sequence number SN field of the message in the corresponding aggregator, the switch forwards the current message to the back-end node on the aggregation path in the network;
[0023] If the corresponding aggregator is not empty, and the value of the sequence number SN field of the current message is the same as the value of the sequence number SN field of the message in the corresponding aggregator, the switch forwards the current message to the backend node on the aggregation path in the network and clears the corresponding aggregator.
[0024] Furthermore, when the switch calculates the position of the corresponding aggregator according to the TaskID and the sequence number SN of the current message, it includes:
[0025] According to the TaskID of the current message, find the offset position information of the corresponding in-network aggregation task on the aggregator table of the switch Offset , and the allocated aggregator interval size N ;
[0026] Use the value of the sequence number SN of the current message to the aggregator interval size N Perform a modulo calculation and then compare the calculation result with the offset position information Offset Add them together to get the position of the corresponding aggregator.
[0027] Furthermore, when the current message is sent to the corresponding aggregator for aggregation, it includes:
[0028] If the corresponding aggregator is empty and the value of the sequence number SN field of the current message is less than the maximum conflict sequence number MSN of the corresponding aggregator, the switch forwards the current message to the back-end node on the aggregation path within the network;
[0029] If the corresponding aggregator is empty and the value of the sequence number SN field of the current message is greater than the maximum conflict sequence number MSN of the corresponding aggregator, the switch saves the current message in the corresponding aggregator and updates the maximum conflict sequence number MSN of the corresponding aggregator to the sequence number SN of the current message;
[0030] If the corresponding aggregator is not empty, and the value of the sequence number SN field of the current message is consistent with the value of the sequence number SN field of the message in the corresponding aggregator, the switch aggregates the current message with the message in the corresponding aggregator to obtain an aggregated message;
[0031] If the corresponding aggregator is not empty, and the value of the sequence number SN field of the current message is inconsistent with the value of the sequence number SN field of the message in the corresponding aggregator, the switch forwards the current message to the back-end node on the aggregation path within the network, and updates the maximum conflict sequence number MSN of the corresponding aggregator to the sequence number SN of the current message.
[0032] Furthermore, the destination node is provided with a message buffer, and the message buffer is composed of a cache unit Bucket. After the destination node obtains the message of the front-end node on the aggregation path in the network in real time, the step of processing the incompletely aggregated data message includes:
[0033] The destination node calculates the location of the corresponding cache unit Bucket according to the value of the sequence number SN of the current message;
[0034] If the value of the counter of the current message is inconsistent with the number of leaf nodes of the destination node, and the current message is not a retransmitted message, the destination node checks whether the corresponding cache unit Bucket is empty;
[0035] If the corresponding cache unit Bucket is empty, the destination node saves the data of the current message in the corresponding cache unit Bucket;
[0036] If the corresponding cache unit Bucket is not empty, and the message stored in the corresponding cache unit Bucket is a retransmitted message, the destination node discards the current message;
[0037] If the corresponding cache unit Bucket is not empty, and the message stored in the corresponding cache unit Bucket is not a retransmitted message, the destination node sends the current message to the corresponding cache unit Bucket for aggregation, and updates the value of the counter of the aggregated message;
[0038] If the value of the counter of the aggregated message is equal to the number of leaf nodes of the destination node, the destination node sends a corresponding confirmation message to the source node through the intra-network aggregation path.
[0039] Furthermore, after the destination node obtains the message of the front-end node on the aggregation path in the network in real time, the step of processing the retransmitted message includes:
[0040] The destination node calculates the location of the corresponding cache unit Bucket according to the value of the sequence number SN of the current message;
[0041] If the current message is a retransmitted message, the destination node checks whether the value of the message counter in the corresponding cache unit Bucket is equal to the number of leaf nodes of the destination node;
[0042] If the value of the counter of the message in the corresponding cache unit Bucket is equal to the number of leaf nodes of the destination node, the destination node sends a corresponding confirmation message to the source node through the intra-network aggregation path;
[0043] If the value of the counter of the message in the corresponding cache unit Bucket is not equal to the number of leaf nodes of the destination node, and the message stored in the corresponding cache unit Bucket is not a retransmitted message, the destination node initializes the corresponding cache unit Bucket, and then stores the data of the current message in the corresponding cache unit Bucket;
[0044] If the value of the counter of the message in the corresponding cache unit Bucket is not equal to the number of leaf nodes of the destination node, and the message stored in the corresponding cache unit Bucket is a retransmitted message, the destination node sends the current message to the corresponding cache unit Bucket for aggregation, and updates the value of the counter of the aggregated message;
[0045] If the value of the counter of the aggregated message is equal to the number of leaf nodes of the destination node, the destination node sends a corresponding confirmation message to the source node through the intra-network aggregation path.
[0046] Furthermore, before the destination node sends the current message to the corresponding buffer unit Bucket for aggregation, it also includes a step of determining whether the current message arrives repeatedly, specifically including:
[0047] The destination node modulos the source IP address of the current message with the length of the bitmap of the corresponding cache unit Bucket, and hashes it to the corresponding bit on the bitmap;
[0048] If the hashed bit is the specified value, the current message arrives repeatedly and the destination node discards the current message;
[0049] If the hashed bit is not the specified value, the current message is not a duplicate arrival, and the destination node sends the current message to the corresponding cache unit Bucket for aggregation.
[0050] Furthermore, after the destination node sends the corresponding confirmation message to the source node through the intra-network aggregation path, the method further includes:
[0051] Initialize the cache unit Bucket at the previous symmetrical position of the message buffer. The index expression of the cache unit Bucket at the previous symmetrical position is as follows:
[0052] Bucket.ID =( pkt.SN + N max )%2 N max
[0053] in, SN Indicates the sequence number of the current message. N max Indicates the maximum value of the sliding window on the source node. The expression is as follows:
[0054] N max = B out ×R / S
[0055] in, Bout is the minimum value of the outbound port bandwidth of all source nodes, R is the network round-trip time RTT, S Is the message size.
[0056] Further, the message and the corresponding confirmation message both include an IP header field, a HAP protocol field, and a payload data field. The HAP protocol field includes a congestion signal ECN field and an error information ERR field. When the source node sends a message to the destination node through an intra-network aggregation path or the destination node sends a corresponding confirmation message to the source node through an intra-network aggregation path, if the switch port is congested, the congestion signal ECN field is marked. If an error occurs during aggregation of the message, the error information ERR field is marked. After the source node receives the confirmation message corresponding to the sent message, the following steps are also included:
[0057] If the congestion signal ECN field of the confirmation message is marked, reduce the size of the sliding window;
[0058] If the error information ERR field of the confirmation message is marked, the sent message is retransmitted to the destination node through point-to-point transmission.
[0059] The present invention also proposes an intra-network aggregation system for heterogeneous resources, comprising a source node, a destination node and a switch, wherein:
[0060] The source node is used to send a message to the destination node through the in-network aggregation path after receiving the in-network aggregation task. If a confirmation message corresponding to the sent message is received within a specified time, the next batch of messages are sent to the destination node through the in-network aggregation path using a sliding window; if a confirmation message corresponding to the sent message is not received within the specified time, the sent message is retransmitted to the destination node through the in-network aggregation path;
[0061] The switch is used to obtain the message of the front-end node on the aggregation path in the network in real time; if the current message is a retransmitted message, the current message is forwarded to the back-end node on the aggregation path in the network; if the current message is not a retransmitted message, the value of the counter of the current message is compared with the number of leaf nodes of the switch; if the value of the counter of the current message is consistent with the number of leaf nodes of the switch, the current message is forwarded to the back-end node on the aggregation path in the network; if the value of the counter of the current message is less than the number of leaf nodes of the switch, the position of the corresponding aggregator is calculated according to the value of the TaskID and the sequence number SN of the current message, and the current message is sent to the corresponding aggregator for aggregation; if a hash conflict occurs, the current message is forwarded to the back-end node on the aggregation path in the network; if no hash conflict occurs, the value of the counter of the aggregated message is updated; if the value of the counter of the aggregated message is equal to the number of leaf nodes of the switch, the aggregated message is forwarded to the back-end node and the corresponding aggregator is cleared; if the value of the counter of the aggregated message is less than the number of leaf nodes of the switch, the aggregated message is retained in the corresponding aggregator;
[0062] The destination node is used to obtain the message of the front-end node on the in-network aggregation path in real time after receiving the in-network aggregation task. If the value of the counter of the current message is consistent with the leaf node of the destination node, the corresponding confirmation message is sent to the source node through the in-network aggregation path.
[0063] Compared with the prior art, the advantages of the present invention are:
[0064] In the present invention, after the switch obtains the message, it allocates the corresponding aggregator according to the message sequence number to aggregate the message. In the case of a hash conflict, the message is passed to the next switch on the INA path. Therefore, the hash conflict occurring on the switch aggregator is not avoided. It is only necessary to know which switches can aggregate these messages (these switches can perform the aggregation operation or not), and it is not necessary to accurately know which switches each INA message needs to be aggregated on in sequence. These messages will be gradually aggregated by the switches on the path in a progressive manner along the given INA path: when encountering a switch with available resources, they can be aggregated; otherwise, the switch only needs to forward these messages, and they can try to be aggregated again on the downstream switch, which can effectively make up for the deficiency of the "barrel effect". At the same time, the aggregation of redundant data through the switch with available resources instead of the destination node can effectively reduce the consumption of link bandwidth resources.
[0065] In the present invention, in the event of a hash conflict, the maximum conflict sequence number of the corresponding aggregator is updated according to the message sequence number. When the message sequence number is less than the maximum conflict sequence number recorded in the empty aggregator, it is directly passed to the next switch on the INA path instead of being stored in the empty aggregator. By comparing the message sequence number with the maximum conflict sequence number cached on the switch aggregator, related data packets can be guided to the same switch for aggregation, thereby avoiding the occurrence of MWL.
[0066] In the present invention, a counter field is added to the message, and the counter field of the aggregated message is updated. The switch determines whether the acquired message should be forwarded or aggregated by comparing the counter field with the number of leaf nodes of the switch on the INA path, thereby solving the problem of unstable message aggregation caused by hash conflicts on the aggregator. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] Figure 1 Figure 1 is a schematic diagram showing the impact of switch aggregator resources and link bandwidth on INA performance. Figure 1 (a) is the case where the aggregator resources on the switch are sufficient. Figure 1 (b) is the case where the aggregator resources on the switch are insufficient.
[0068] Figure 2 Schematic diagram of the transmission throughput of two existing intra-network aggregation methods, where Figure 2 (a) is the transmission throughput that can be achieved by the GSIA method and the ATP method, Figure 2 (b) is the transmission throughput that can be achieved by the GRID method.
[0069] Figure 3 A schematic diagram of the transmission throughput of an improved intra-network aggregation method.
[0070] Figure 4 Schematic diagram of transmission throughput of the intra-network aggregation method according to an embodiment of the present invention.
[0071] Figure 5 The diagram is a schematic diagram of a message structure according to an embodiment of the present invention.
[0072] Figure 6 Schematic diagram of the aggregator structure of an embodiment of the present invention.
[0073] Figure 7 The figure is a working flow chart of a source node in the intra-network aggregation method according to an embodiment of the present invention.
[0074] Figure 8 The figure is a working flow chart of a switch in the intra-network aggregation method according to an embodiment of the present invention.
[0075] Fig. 9The figure is a working flow diagram of a destination node in the intra-network aggregation method according to an embodiment of the present invention.
[0076] Fig.10 A schematic diagram of a hash conflict on an aggregator causing mutual waiting locks, where Fig.10 (a) indicates forwarding the message to the next node when a hash conflict occurs. Fig.10 (b) indicates that after the aggregation is completed, the result message is forwarded and the corresponding aggregator is cleared. Fig.10 (c) means hashing the newly arrived message into an empty aggregator for storage.
[0077] Fig.11 A schematic diagram of adding a maximum conflict sequence number to aggregators in an embodiment of the present invention to avoid mutual waiting for locks, wherein Fig.11 (a) indicates that when a hash conflict occurs, the maximum conflict sequence number is updated and the message is forwarded to the next node. Fig.11 (b) indicates that after the aggregation is completed, the maximum conflict sequence number is retained when the result message is forwarded and the corresponding aggregator is cleared. Fig.11 (c) indicates that after the newly arrived message is hashed to an empty aggregator, the message sequence number is inconsistent with the maximum conflict sequence number, so the message is forwarded to the subsequent node. DETAILED DESCRIPTION
[0078] The present invention is further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the present invention is not limited thereby.
[0079] Embodiment 1
[0080] In order to improve throughput compared to existing in-network aggregation methods, Figure 3 A possible improvement method is shown, in which switches S1 and S2 use all their aggregator resources to aggregate the data of the child nodes. This method can increase the INA throughput to 6Gbps. For switch S1, link S1 → The data transmitted by S3 includes aggregated traffic and unaggregated traffic. The aggregated traffic occupies 4Gbps of bandwidth, while the unaggregated traffic includes traffic forwarded from source nodes H1 and H2, which are 2Gbps and 2Gbps respectively. Therefore, these traffics occupy 8Gbps of the transmission bandwidth of the link. Afterwards, these traffics are forwarded via switch S3 and aggregated on switch S5. For switch S5, since its aggregation capacity (10Gbps) is greater than the data transmission rate of the source node (6Gbps), it can aggregate all traffic from the child nodes and only send the aggregated data to the downstream nodes. To this end, this task will occupy 6Gbps of aggregation capacity on switch S5, and on link S5 →H5 only needs to transmit aggregated traffic with a throughput of 6 Gbps. On the other sub-path, switch S2 aggregates data from source nodes H3 and H4 with a throughput of 5 Gbps. Therefore, link S2 → S4 needs to transmit aggregated traffic with a throughput of 5 Gbps, as well as unaggregated traffic from source nodes H3 and H4, which are 1 Gbps and 1 Gbps respectively. These traffic flows are forwarded by switch S4 and then aggregated with the traffic from switch S3 on switch S5.
[0081] This example shows a completely different form of aggregation than existing methods, because the aggregate throughput on the two sub-paths is different. However, existing intra-network aggregation methods cannot achieve such performance under this deployment because they cause the aggregate throughput to shrink to the bottleneck value of the switch aggregation capacity (the so-called "barrel effect"), that is, 4Gps on switch S1. Specifically, for the GISA protocol, if the source nodes H1 and H2 send data at 6Gbps, and switch S1 can only aggregate this data at 4Gbps, in order to avoid hash conflicts, switch S1 must discard excess packets. Therefore, the actual throughput of the traffic forwarded from switch S1 is 4Gbps, which in turn causes switch S5 to only aggregate the 4Gbps traffic that arrives, and the final effective throughput is only 4Gbps. For ATP, the hash conflict on switch S1 causes the conflicting packets to be directly forwarded to the destination node for aggregation, which makes switch S5 only aggregate the aggregated traffic with a throughput of 4Gbps from switch S3, while the rest of the traffic can only perform forwarding operations (determined by the logic of ATP). In addition, on another sub-path, even if switch S2 can aggregate traffic with a throughput of 5 Gbps, 1 Gbps of traffic will trigger an aggregator timeout on switch S5 because no associated messages are hashed to the corresponding aggregator (because messages with hash conflicts on switch S1 will be directly forwarded to the destination node and cannot be aggregated on the downstream switch). Eventually, these excess traffic messages will be retransmitted by the source node and aggregated by the destination node.
[0082] exist Figure 3 On the basis of Figure 4 A more high-performance in-network aggregation method is shown, in which switches S1 and S2 use all their aggregator resources to aggregate the data of the subnodes, and use the idle resources on switches S3 and S4 to further aggregate the traffic that has not been aggregated from switches S1 and S2. Figure 3 The performance bottleneck comes from link S3 →If the bandwidth of S5 is insufficient, the idle resources on switch S3 can be used to further aggregate the traffic that is not aggregated from switch S1, thereby reducing the bandwidth of link S3. → Traffic forwarded on S5. This method can further increase the data transmission rate of the source node to 7 Gbps.
[0083] For switch S1, link S1 → The data transmitted by S3 includes aggregated traffic and unaggregated traffic. The aggregated traffic occupies 4Gbps of bandwidth, while the unaggregated traffic includes the traffic forwarded from source nodes H1 and H2 (3Gbps and 3Gbps unaggregated traffic from the two source nodes, respectively). The unaggregated traffic is transmitted via link S1. → S3 is transmitted to switch S3 to perform aggregation again in a relay manner, which consumes 3Gbps of aggregation capacity on switch S3. On another sub-path, switch S2 aggregates data from source nodes H3 and H4 with a throughput of 5Gbps. Therefore, link S2 → S4 needs to transmit aggregated traffic with a throughput of 5 Gbps, as well as unaggregated traffic from source nodes H3 and H4, which are 2 Gbps and 2 Gbps respectively. These unaggregated traffic flows are transmitted via link S2. → S4 is transmitted to switch S4 to perform aggregation again in a relay manner, which consumes 2 Gbps aggregation capacity on switch S4.
[0084] Finally, link S3 → S5 vs S4 → S5 only needs to transmit the aggregated traffic, and then switch S5 aggregates the traffic. This improvement solution enables the idle resources on the path to be fully utilized, realizing the "complementarity" of aggregator resources and link bandwidth resources, thereby alleviating the INA performance degradation caused by the previous bottleneck resources.
[0085] In order to realize the "complementary" advantages of aggregator resources and link bandwidth resources, the intra-network aggregation can still achieve the best possible acceleration performance under heterogeneous network resources. This embodiment proposes a new message format, called HAP message, such as Figure 5 As shown, the message includes an IP header field, a HAP protocol field and a payload data field. The routing and forwarding of the message are achieved by matching the IP address.
[0086] Figure 5 In the HAP protocol field, there are seven parts, namely, the aggregation task ID field TaskID, the counter CN field, the confirmation message ACK field, the retransmission RST field, the congestion signal ECN field, the error information ERR field and the sequence number SN field. Among them:
[0087] The aggregation task ID field TaskID is used to distinguish messages of different intra-network aggregation tasks.
[0088] The counter CN field is used to record how many source nodes' messages the current message has been aggregated with.
[0089] The ACK field of the acknowledgment message is used to distinguish the data message aggregated in the network from the acknowledgment message ACK. For the acknowledgment message ACK and the corresponding data message, the other 6 field information except the acknowledgment message ACK field in the HAP protocol field of both are the same.
[0090] The retransmission RST field is used to distinguish normal messages from retransmitted messages.
[0091] The congestion signal ECN field is used to transmit congestion signals. When the switch port is congested, this field is marked as 1.
[0092] The error information ERR field is used to record error information during the aggregation process. If an error occurs in the aggregation operation, this field is marked as 1. A typical example is the overflow of an addition operation. For example, if the switch performs addition on a 32-bit integer and the number is too large to cause the expression space to overflow, the aggregation result may be wrong. In this case, the switch will mark the ERR field on the aggregator as 1. When the aggregation operation is completed, the information will be placed in the ERR field of the message to inform the destination node and the source node.
[0093] Finally, the SN field is used to record the sequence number of the message.
[0094] This embodiment also modifies the aggregator structure on the switch, such as Figure 6 As shown, multiple aggregators are pre-compiled on each switch, and these aggregators form an aggregator table, where each aggregator can be located by index. Each aggregator consists of 6 fields, namely, the sequence number SN field, the maximum conflict sequence number MSN field, the counter CN field, the congestion signal ECN field, the error information ERR field and the payload Payload. Among them:
[0095] The sequence number SN field is used to record a group of messages of which sequence number the data currently cached by the aggregator belongs.
[0096] The Maximum Collision Sequence Number MSN field is used to track hash collision information on the aggregator.
[0097] The counter CN field is used to record how many source nodes the data in the current aggregator is the aggregation result of.
[0098] The congestion signal ECN field is used to cache the ECN information in the arriving message.
[0099] The error information ERR field is used to record errors during the aggregation process.
[0100] The payload is used to cache the payload data of the message and to temporarily store the aggregation results. To avoid memory overflow, the upper limit of the payload size in the message cannot exceed the size of the payload on the aggregator.
[0101] Based on the above technical concept, this embodiment proposes an intra-network aggregation method for heterogeneous resources, which is applied to a distributed system including a source node, a destination node and a switch. The working process is as follows:
[0102] After the user submits an intra-network aggregation task request to the network controller, the network controller generates an intra-network aggregation path based on the source / destination node information of the task and assigns a unique TaskID to mark the task. After that, the network controller queries the aggregator occupancy of the switch on the intra-network aggregation path and allocates the aggregator resources of the switch for the task.
[0103] If a switch carries traffic from multiple in-network aggregation tasks, these in-network aggregation tasks share the aggregator resources on the switch equally. After completing the aggregator resource allocation, the controller sends the corresponding configuration information to the data plane, informs all source / destination nodes of the TaskID information corresponding to the task, and starts the in-network aggregation task.
[0104] After receiving the aggregation task in the network, the source node sends a message to the destination node, such as Figure 7 As shown, the source node sends a HAP protocol message to the destination node through the in-network aggregation path. If a confirmation message corresponding to the sent message is received within a specified time, the sliding window is used to send the next batch of messages to the destination node through the in-network aggregation path; if the confirmation message corresponding to the sent message is not received within the specified time, the sent message is retransmitted to the destination node through the in-network aggregation path.
[0105] The switch aggregates the obtained messages, such as Figure 8 As shown, the switch obtains the message of the front-end node on the aggregation path in the network in real time. In this embodiment, the front-end node specifically refers to the front-end switch or node on the aggregation path in the network. For example, Figure 4 The switch S1 in the example has the source nodes H1 and H2 as its front-end nodes. Figure 4The switch S5 in the example has the switches S3 and S4 as its front-end nodes; if the current message is a retransmitted message, the current message is forwarded to the back-end node on the aggregation path in the network; if the current message is not a retransmitted message, the value of the counter of the current message is compared with the number of leaf nodes of the switch; if the value of the counter of the current message is consistent with the number of leaf nodes of the switch, the current message is forwarded to the back-end node on the aggregation path in the network; if the value of the counter of the current message is less than the number of leaf nodes of the switch, the position of the corresponding aggregator is calculated according to the values of the TaskID and the sequence number SN of the current message, and the current message is sent to the corresponding aggregator for aggregation; if a hash conflict occurs, the current message is forwarded to the back-end node on the aggregation path in the network. In this embodiment, the back-end node specifically refers to the back-end switch or node on the aggregation path in the network. For example, for Figure 4 The switch S1 in the example has a backend node of switch S3. Figure 4 The switch S5 in the network has a backend node as the destination node H5. If no hash conflict occurs, the value of the counter of the aggregated message is updated; if the value of the counter of the aggregated message is equal to the number of leaf nodes of the switch, the aggregated message is forwarded to the backend node and the corresponding aggregator is cleared; if the value of the counter of the aggregated message is less than the number of leaf nodes of the switch, the aggregated message is retained in the corresponding aggregator.
[0106] After receiving the in-network aggregation task, the destination node waits for and obtains the message from the source node and then sends a confirmation message to the source node, such as Fig. 9 As shown, the destination node is used to obtain the message of the front-end node on the intra-network aggregation path in real time. If the value of the counter of the current message is consistent with the number of leaf nodes of the destination node, the corresponding confirmation message is sent to the source node through the intra-network aggregation path.
[0107] After the in-network aggregation task is completed, the user will inform the network controller, which will then revoke the configuration information on the corresponding switch and initialize the aggregator occupied by the task. This process is completed at runtime without recompiling the switch. These recycled aggregator resources can be used for other in-network aggregation tasks later.
[0108] The relevant contents are described in detail below.
[0109] In this embodiment, at the source node, the data to be sent is in the form of "vectors". These "vectors" are actually composed of a sequence of "multisets". The "multisets" of "vectors" at the same position of different source nodes constitute a group of related data, which need to be aggregated to obtain the final result. Specifically, if you use D i To represent the source nodei The data that needs to be sent, then D i can be expressed as:
[0110]
[0111] in k represents the number of multi-sets on the vector, V i,j ,1≤ j ≤ k Represents a multiset. The final aggregation result D ∗ is also a sequence of multiple sets, which can be represented as
[0112]
[0113] Where ⊎ represents addition applied to a multiset. The specific format of a multiset depends on the application. For example, for distributed training, the vector is the gradient data, and each multiset is a gradient value. Addition operations are performed on the switch to aggregate these data. For failed block repair of erasure codes, the vector is a coding block, and each multiset is a word of the coding block. XOR operations are performed on the switch to aggregate the corresponding data.
[0114] The source node will need to send data D i Split into multiple messages, each of which carries a fixed number of multisets in turn V i,j (The specific number is determined by the size of the message payload and the size of the multiset.) When the source node sends data, the payload of each message is placed in a fixed number of "multisets" in sequence and then sent one by one. These multisets are in sequence. D i The positions on the headers constitute the sequence numbers (SN) of these messages in sequence, which are recorded in the sequence number SN field of the message header. In this embodiment, messages from different source nodes with the same SN field are referred to as a group of related messages because they need to be aggregated to obtain the final result. In addition, the source node also maintains a sliding window to control the number of messages injected into the network each time, and dynamically adjusts the size of the sliding window through the ECN field in the message header and the network packet loss information as signals. When the message with the ECN field marked arrives at the source node, the source node will halve the size of the sliding window, and then gradually increase the size of the sliding window to achieve control of network congestion. This process is similar to "congestion avoidance" in the TCP protocol. In addition, when packet loss of consecutive sequence numbers occurs, the source node will also reduce the sliding window, and the adjustment process is the same as when the ECN field is marked.
[0115] In addition, when the message marked with the ERR field reaches the source node, the source node can use the conventional point-to-point transmission method to re-perform the aggregation operation on the erroneous part. The specific implementation form can be determined by the application.
[0116] In this embodiment, on the switch, the aggregation operation is performed with the message as the basic unit. For the HAP protocol message arriving at the switch, the switch will search for the corresponding rule entry according to the TaskID field of the message to determine whether the currently arriving message is a message that needs to perform the aggregation operation. For the message that needs to perform the aggregation operation, first calculate the position of the corresponding aggregator according to the TaskID and sequence number SN values of the current message, and then decide whether to aggregate the message by comparing the information in the aggregator. These HAP protocol messages are forwarded or aggregated by the switch on the path and finally reach the destination node, and then the destination node returns a confirmation message (called an ACK message) to each source node to promote the sliding window to send a new data message. In order to distinguish between data messages and ACK messages, the ACK field in the HAP protocol field is marked: if the field is 1, it is an ACK message; if it is 0, it is a data message. In particular, for Allreduce type applications, the ACK message will carry the result data after aggregation; while for Reduce applications, the ACK message does not carry the aggregation result.
[0117] In this embodiment, when the switch calculates the position of the corresponding aggregator according to the TaskID and the value of the sequence number SN of the message, the matching rule is queried through the TaskID of the current message to obtain the offset position information of the corresponding intra-network aggregation task on the aggregator table of the switch. Offset , and the allocated aggregator interval size N , and then use the value of the sequence number SN of the current message to adjust the aggregator interval size N Perform modulo calculation and finally compare the calculation result with the offset position information Offset Add together to get the position of the corresponding aggregator, the position of the aggregator ( Agg.ID ) can be expressed as:
[0118]
[0119] in, pkt.SN Indicates the value of the SN field of the data message.
[0120] In a network environment with heterogeneous resources, switches need to be able to aggregate packets in a "relay" manner. When the aggregator resources on a switch are insufficient to aggregate all traffic, the traffic that cannot be aggregated needs to be forwarded to subsequent switches and attempted to be aggregated again. In this case, the rate at which packets arrive will be higher than the maximum aggregation capacity of the switch.
[0121] In order to be able to distinguish messages with different serial numbers on the aggregator, this embodiment adds a serial number SN field on the aggregator. When a message is hashed to an aggregator for the first time, the data in the payload of the message is cached on the hashed aggregator, and the serial number SN field of the message is also recorded in the serial number SN field on the aggregator. In this way, when another message is hashed to the aggregator, the switch checks whether the serial number SN field on the aggregator is equal to the serial number SN field of the message: if they are the same, it means that the data in the message and the data cached on the aggregator are related, then the switch aggregates the data in the message and the data in the aggregator; otherwise, the two are not related. In the latter case, the switch needs to forward the message to the next hop without performing an aggregation operation.
[0122] Ideally, the forwarded message will be rehashed at the downstream switch and then matched to a corresponding empty aggregator for caching. Subsequently, its associated message (message with the same sequence number SN) will also be forwarded to the switch due to hash conflicts, thus completing the aggregation operation. However, since only a fixed number of aggregators are allocated for this task on the switch, these aggregators cannot cache this data for a long time. Otherwise, during the next RTT period, the source node will send a new batch of messages to the network, and these messages will not be aggregated and forwarded to the downstream switch due to hash conflicts on the aggregator. Eventually, the aggregator resources of all switches on the INA path will be exhausted, and subsequent messages can only be aggregated at the destination node. Therefore, once the aggregation operation of the message is completed, the data on the aggregator needs to be cleared in time to be used to aggregate a new batch of messages.
[0123] However, clearing the aggregator will cause the switch to be unable to determine whether the message currently to be cached has had a hash conflict before, resulting in messages with the same sequence number being cached on different switches. Fig.10 As shown in the figure, suppose that a data packet with sequence number SN=21 arrives at the switch. According to the hash function, suppose that the packet is hashed to Agg.ID=1. After that, the switch compares the SN in the aggregator with the SN of the message to decide the next step for the message. Since the SN recorded in the aggregator is 11 and the SN of the message is 21, the switch believes that there is a hash conflict in the aggregator, so it forwards the message to the subsequent node (such as Fig.10 (a)). Next, when another data packet with SN=11 arrives at the switch, the aggregator will perform the aggregation operation because the SN in the aggregator is the same as the SN of the packet. Assuming that the aggregator can complete the aggregation operation after aggregating the packet (it has aggregated all the packets that need to be aggregated), the aggregator will be initialized to achieve recycling and reuse (such as Fig.10 (b)). Afterwards, if a data packet with SN=21 from another source node arrives at the switch (as shown in Fig.10 (c) shows that the switch uses the same hash function and also maps it to Agg.ID =1. Since the aggregator has been cleared by the ACK message in the previous step, the switch will cache the message on this aggregator. However, since the previously arrived message with SN=21 has been forwarded to the subsequent node by the switch, this message will remain in the current switch. Similarly, the previously arrived message with SN=21 will also remain in the downstream switch waiting for the message to complete the aggregation operation, thus waiting for each other and causing a deadlock. As a result, the occupied aggregator cannot be released (because the destination node has not received the aggregation result and cannot send an ACK message), and the source node may also think that these data messages are lost because it has not received the corresponding ACK message, which may eventually cause the aggregation process to be interrupted.
[0124] Considering that when the source node sends data, the sequence number of the message is marked in order from small to large, which means that the sequence number SN field of the data message is monotonically increasing. Based on this, for a single source node, if the switch currently receives its sequence number SN = sn0 The switch can determine which data packets have sequence numbers less than sn0 The message has been sent by the source node and the sequence number is greater than sn0 Furthermore, for multiple source nodes, the largest sequence number of all the messages received by the current switch from these nodes is s max , then the switch can determine the source nodes with sequence numbers greater than s max The data packet has not yet reached the switch, and the sequence number of a single source node is less than or equal to s max The data packet may have been forwarded to the next node via the switch.
[0125] Therefore, in this embodiment, a maximum conflict sequence number MSN field is added to each aggregator to record the value of the maximum sequence number SN of the message hashed to the aggregator. When a message is hashed to the aggregator, if the SN on the aggregator is equal to the SN of the message, the switch aggregates the message according to the previous process; if a hash conflict occurs, the aggregator needs to update the MSN to the largest value between the SN of the message and the MSN on the aggregator, that is:
[0126]
[0127] in Agg.MSN Represents the value of the Maximum Conflicting Sequence Number MSN field on the aggregator, and pkt.SN Indicates the value of the SN field of the data message. Then, the switch forwards the conflicting message to the subsequent node.
[0128] When a data packet is hashed to an empty aggregator, in order to avoid mutual waiting for the WML lock, the switch needs to first determine the relationship between the MSN value on the aggregator and the SN value in the packet:
[0129] if Agg.MSN ≥ pkt.SN , which means that data packets with SN less than or equal to MSN may have arrived at the switch before the message arrives. Since the SN in the aggregator is not equal to the message SN, the message related to the current message may be forwarded to the subsequent node by the switch due to hash conflict. Therefore, if the current switch caches the message, the message cached by the downstream switch will always wait for the message to complete aggregation. In this case, the switch should not cache the message, but forward it to the subsequent node.
[0130] if Agg.MSN <pkt.SN , which means that the message with this sequence number arrives at the switch for the first time. Then, the switch can cache this message because its related messages (with the same SN as this message) have never arrived at the switch. To this end, the switch needs to first cache the payload of the message to the aggregator, then update the SN of the aggregator to the SN of the message, and update the value of the MSN on the aggregator to the value of the SN of the message.
[0131] like Fig.11 As shown in the figure, a maximum conflict sequence number MSN field is added to the aggregator to record the maximum value of the message sequence number SN currently hashed to the aggregator. When a data message with SN=21 arrives at the switch, the switch hashes it to an aggregator according to the message sequence number SN. Fig.11(a)). Since the existing message sequence number SN on the aggregator is not equal to the sequence number SN of the newly hashed message, the switch believes that a hash conflict has occurred. According to the logic described above, the switch needs to update the maximum conflicting sequence number MSN on the aggregator to the sequence number SN of the newly hashed message. In this example, MSN will be updated to 21, and then the switch forwards the conflicting message to the subsequent node. Afterwards, if a new data message with SN=11 arrives, the switch will also match it to the aggregator according to the hash function ( Fig.11 (b)). Since the SN on the aggregator is the same as the SN of the message, the switch aggregates the message, replaces the data carried in the message with the aggregation result on the aggregator, and forwards the message. At the same time, the switch clears the data cached on the aggregator and clears the SN cache information on the aggregator. Next, when another data message with SN=21 arrives, it will be matched to the same aggregator as the previous data message with SN=21 according to the same hash function ( Fig.11 (c)). Since the aggregator was cleared in the previous step, the switch needs to check the MSN field of the aggregator before caching the data packet. In this example, the MSN of the aggregator was updated to 21 when the previous data packet with SN=21 arrived, which does not meet Agg.MSN< pkt.SN , so the switch will forward the message instead of buffering it.
[0132] The downstream switches will also perform aggregation and forwarding operations according to this logic. If a downstream switch has cached the data packet with SN=21, then the data packet with SN=21 will be hashed to the same aggregator and then the aggregation operation will be performed. If the switch also forwards the previous data packet with SN=21 due to a hash conflict, the MSN of the corresponding aggregator will also be updated, and due to a hash conflict or not satisfying Agg.MSN <pkt.SN The data packet with SN=21 is forwarded. Therefore, this mechanism avoids the situation where the packets that should be aggregated are cached in different switches due to hash conflicts, thereby avoiding the generation of mutual waiting lock WML.
[0133] In summary, in the process of aggregating messages in the switch of this implementation, when the current message is sent to the corresponding aggregator for aggregation, it includes:
[0134] If the corresponding aggregator is empty and the value of the sequence number SN field of the current message is less than the maximum conflict sequence number MSN of the corresponding aggregator, the switch forwards the current message to the back-end node on the aggregation path within the network;
[0135] If the corresponding aggregator is empty and the value of the sequence number SN field of the current message is greater than the maximum conflict sequence number MSN of the corresponding aggregator, the switch saves the current message in the corresponding aggregator and updates the maximum conflict sequence number MSN of the corresponding aggregator to the sequence number SN of the current message;
[0136] If the corresponding aggregator is not empty, and the value of the sequence number SN field of the current message is consistent with the value of the sequence number SN field of the message in the corresponding aggregator, the switch aggregates the current message with the message in the corresponding aggregator to obtain an aggregated message;
[0137] If the corresponding aggregator is not empty, and the value of the sequence number SN field of the current message is inconsistent with the value of the sequence number SN field of the message in the corresponding aggregator, the switch forwards the current message to the back-end node on the aggregation path within the network, and updates the maximum conflict sequence number MSN of the corresponding aggregator to the sequence number SN of the current message.
[0138] In the existing design, since the intra-network aggregation protocol is deterministic, it means that the message must perform the aggregation operation on the specified switch. Therefore, the switch can determine when the aggregation operation is completed by the number of its child nodes. Figure 3 For the example in, the number of aggregations will become uncertain. For switch S1, since the data transmission rate of the source node is higher than the aggregation capability of the switch, some messages will be forwarded to the downstream node due to hash conflicts on the aggregator. A similar situation will occur on switch S2. As a result, for switch S5, the messages from its child nodes S3 and S4 may not be fully aggregated: if a group of messages with a certain sequence number are aggregated on both switches S1 and S2, then switch S5 only needs to perform 2 aggregation operations; if the group of messages from source nodes H1 and H2 are not aggregated on S1, and S2 aggregates the group of messages from source nodes H3 and H4, then switch H5 will receive this group of 3 messages that need to be aggregated, that is, 3 aggregation operations are required; if neither switch S1 nor S2 aggregates the group of messages, then these messages will be forwarded to switch S5 for aggregation, and 4 aggregation operations are required.
[0139] Likewise, Figure 4 For switches S3 and S4, the packets from their child nodes S1 or S2 may not be fully aggregated, so they also have a similar problem of insufficient aggregation. In order to deal with the problem of insufficient aggregation, this embodiment considers that after the aggregation path in the network is determined, the number of upstream leaf nodes of each switch on the path is determined. Figure 3For example, the number of leaf nodes of switches S1 and S2 is 2 (source nodes H1 and H2, and source nodes H3 and H4, respectively), the number of leaf nodes of switches S3 and S4 is also 2 (source nodes H1 and H2, and source nodes H3 and H4, respectively), and the number of leaf nodes of switch S5 is 4 (source nodes H1-H4). In this way, for each switch, it is certain which source nodes' messages need to be aggregated. Therefore, the switch can determine whether the aggregation operation has been completed by checking whether the arriving message has been aggregated with the remaining leaf nodes.
[0140] Specifically, in this embodiment, whether the aggregation is completed is not determined by recording the number of times the aggregation operation is performed, but a counter CN field is added to each aggregator to record how many leaf nodes the currently cached message data has been aggregated with the corresponding message data. At the same time, a counter CN field is also added to the message header to record how many source nodes the data in the current message is aggregated with.
[0141] For the source node, the initial value of the CN field in the sent data message is set to 1. After the message arrives at the switch, if the message needs to be cached to the aggregator, the value of CN in the message needs to be recorded to the CN of the aggregator based on the operations described above; if an aggregation operation is performed, the switch needs to update the CN value on the aggregator to the sum of the CN value of the aggregated message and the original CN value on the aggregator. Afterwards, the switch can determine whether the aggregation has been completed based on the relationship between the CN value of the aggregator and the number of its leaf nodes: if the two are the same, it means that the aggregator has aggregated all the messages of the leaf nodes, that is, the aggregation has been completed; if the two are different, it means that there are still leaf nodes whose data has not been aggregated, so it needs to continue waiting for the remaining messages to arrive at the switch.
[0142] by Figure 3 Let's take switch S1 as an example to analyze this decision-making process. For switch S1, if a group of messages with the same sequence number from source nodes H1 and H2 are aggregated, it will only send one aggregated message to the downstream node, and the CN value of the message is 2; otherwise, if the switch cannot aggregate these messages due to hash conflicts, then switch S1 will forward the two messages from source nodes H1 and H2, that is, send two messages to the downstream node, and the CN value of the messages is 1. Switch S2 also experiences two similar situations. On switch S5, the messages it receives will be divided into the following four situations:
[0143] Switches S1 and S2 do not perform aggregation operations on the messages: switch S5 will receive 4 non-aggregated messages, and the CN of these messages is 1. Therefore, switch S5 must perform aggregation operations on the 4 messages until the CN on the aggregator is equal to 4, and then the aggregation is completed.
[0144] Switch S1 aggregates the message, but switch S2 does not aggregate the message: switch S5 will receive 1 aggregated message and 2 non-aggregated messages, and the CN of the aggregated message is equal to 2. Therefore, only after switch S5 aggregates these 3 messages, the CN on the aggregator is equal to 4, and then the aggregation is considered complete.
[0145] Switch S1 does not aggregate packets, but switch S2 aggregates packets: This situation is the same as the previous one.
[0146] Switches S1 and S2 both aggregate the messages: switch S5 will receive two aggregated messages, and the CN of the messages is 2. Therefore, after switch S5 aggregates the two messages, the CN on the aggregator is equal to 4, so it can be determined that the aggregation is completed.
[0147] According to the above analysis, no matter what the situation is, switch S5 can accurately determine the completion of aggregation, because it always needs to aggregate the messages from source nodes H1-H4. In other words, no matter whether the messages from the child nodes are fully aggregated, the switch can determine whether the aggregation operation is completed by whether the arriving message has been aggregated with the relevant messages of the other child nodes, because the number of leaf nodes on the aggregation path of the switch in the network is fixed. In addition, Figure 4 The switches S3 and S4 in FIG. 1 can also use this logic to determine whether the aggregation is completed.
[0148] In order to avoid interruption of the aggregation process due to packet loss, the source node in this embodiment adopts a timeout retransmission mechanism, and sets a timer for each data message it sends. If the confirmation message ACK with the same sequence number CN as the message is not received before the timer times out, the retransmission mechanism is triggered. However, directly retransmitting the timed-out data message to the network may cause potential errors, because the timeout may be caused by packet loss at other source nodes. In this case, the switch may wait because it has not received the corresponding message, but other source nodes may also trigger retransmission because they have not received the ACK message. These retransmitted messages may be mistaken by the switch as normal data messages and perform aggregation operations, and pass the wrong results to subsequent nodes.
[0149] In order to avoid this situation, this embodiment adds a retransmission RST field in the message header to distinguish normal data messages from retransmission messages. Among them, the RST of the normally sent message is set to 0, while the field of the retransmission message is 1. After receiving the HAP protocol message, the switch needs to first check whether the RST of the message is 0: if it is 0, the message is a normal data message, and the switch can perform the next aggregation or forwarding operation on the message according to the logic described above; if it is 1, the message is a retransmission message, then the switch needs to check and clear the corresponding aggregator first, because the aggregator may still retain part of the aggregation results, which will interfere with the subsequent process. For this reason, if the SN of the message is the same as the SN on the hash aggregator, the switch needs to initialize the aggregator first, and then forward the message; if the two are not the same, then the switch can directly forward the message.
[0150] In particular, when message loss occurs, the switch will not be able to perform aggregation operations on the group of messages because all retransmitted messages are forwarded. To this end, this embodiment places the message aggregation operation in the case of packet loss on the destination node for execution. When packet loss occurs, the source node will trigger a timeout, and these timed-out messages will eventually be forwarded to the destination node. Afterwards, the destination node aggregates these messages, and when the sum of the CNs of all aggregated messages is equal to the number of source nodes, the destination node sends an ACK message to the source node. The ACK message is the same as a normal ACK message, which is forwarded along a predetermined path and eventually reaches each source node. It is worth noting that if the ACK message arrives at a fork node in the path, the node needs to copy the message to multiple egress ports in accordance with the routing rules to multicast the message.
[0151] Based on the above content, in this embodiment, when the switch aggregates the message, the following steps are also included:
[0152] The switch obtains the message of the front-end node on the aggregation path in the network in real time;
[0153] If the current message is a retransmitted message, the switch forwards the current message to the back-end node on the aggregation path within the network, including:
[0154] The switch calculates the location of the corresponding aggregator based on the TaskID and sequence number SN of the current message;
[0155] If the corresponding aggregator is empty, or the value of the sequence number SN field of the current message is different from the value of the sequence number SN field of the message in the corresponding aggregator, the switch forwards the current message to the back-end node on the aggregation path in the network;
[0156] If the corresponding aggregator is not empty, and the value of the sequence number SN field of the current message is the same as the value of the sequence number SN field of the message in the corresponding aggregator, the switch forwards the current message to the backend node on the aggregation path in the network and clears the corresponding aggregator.
[0157] In the process of processing retransmitted messages, a special case is that the confirmation message ACK suffers from partial message loss on the way back, which causes some source nodes to receive the confirmation message ACK, but the remaining source nodes trigger the retransmission operation again because they do not receive the confirmation message ACK. In this case, the destination node will only receive retransmitted data messages from some source nodes, and cannot complete the aggregation operation again. To avoid this situation, in this embodiment, a message buffer (Packet Buffer) is set at the destination node to cache a certain number of aggregation results. After the retransmitted message arrives at the destination node, if the message buffer already has an aggregation result, the destination node directly returns an ACK message without performing the aggregation operation again.
[0158] The size of the message buffer should be at least the maximum value of the sliding window on the source node (denoted as N max ) to avoid retransmitted packets failing to obtain cached aggregation results. In particular, the maximum sliding window value N max The minimum value of the outbound port bandwidth of all source nodes (denoted as B out ) to set, because the aggregate throughput within the network will not exceed B out The value of . It can be deduced that:
[0159]
[0160] in R is the network’s round-trip time RTT, S is the message size. The maximum value of the sliding window can also be set to a value greater than N max However, since sending more data to the network will only increase link congestion, congestion control will also control the sliding window size to N max Within.
[0161] The message buffer in this embodiment is composed of a cache unit Bucket, each of which is used to operate a message with a specific sequence number, and is composed of a bitmap Bitmap, a counter CN field, a retransmission RST field, a congestion signal ECN field, an error information ERR field, and a payload Payload field. Among them:
[0162] The bitmap is used to distinguish different source node messages. It consists of multiple bits, and the initial value is all 0. By taking the source IP address of the message modulo the length of the bitmap, it can be hashed to a bit on the bitmap, so that messages from different source nodes can be distinguished by different hashed bits.
[0163] The counter CN field is used to record the number of aggregations (it has the same function as the counter CN field on the switch aggregator). The initial value is empty.
[0164] The retransmission RST field is used to record whether the cached data comes from the retransmission message, and its initial value is 0;
[0165] The congestion signal ECN field is used to record the congestion information in the message, and its initial value is 0;
[0166] The error information ERR field is used to record error information in the message (for example, memory overflow when performing addition aggregation). The initial value is 0.
[0167] The payload is used to cache data. Its size is the same as the payload of the message, and its initial value is empty.
[0168] Specifically, the maximum value of the sliding window is N max , the message buffer size is at least 2 N max A cache unit Bucket. Fig. 9 As shown in the figure, when the data message arrives at the destination node, the destination node calculates the cache unit Bucket position of the message in the message buffer according to its sequence number SN field. The expression is as follows:
[0169] Bucket.ID = pkt.SN %2 N max
[0170] The destination node then checks whether the counter CN value of the message is equal to the number of its leaf nodes, that is, the number of all source nodes. If they are equal, the message has been fully aggregated, and the counter CN, congestion signal ECN field, error information ERR field and payload data in the message are placed in the corresponding cache unit Bucket, and then reply with an acknowledgment message ACK; if they are not equal, check whether the retransmission RST field of the message is 1. This step is divided into the following two cases:
[0171] If it is 0, the message is a normal data message. This may be because the destination node needs to aggregate some of the aggregation results from the upstream node. To this end, the destination node first checks whether the retransmission RST field in the cache unit Bucket is 1: if it is 1, it means that packet loss has occurred (and triggered the re-aggregation of the destination node), then the message must be discarded to avoid errors; otherwise, the aggregation operation can be performed according to the normal process, that is, the destination node first aggregates the payload Payload data in the message and the payload Payload data in the cache unit Bucket using a given function, and then updates the counter CN field of the cache unit Bucket to the sum of the original CN value of the cache unit Bucket and the CN value of the message, and at the same time performs an OR operation on the ECN field and ERR field of the cache unit Bucket and the corresponding fields in the message (if the Bucket is empty, the CN, ECN, ERR and Payload data of the message are directly placed in the Bucket). After that, the destination node checks whether the CN value of the Bucket is equal to the number of source nodes to determine whether to reply an ACK message.
[0172] If it is 1, the message is a retransmitted message. To this end, the destination node needs to first check whether the CN value in the Bucket is equal to the number of source nodes. If they are equal, the confirmation message ACK can be directly constructed and replied (because the complete aggregation result has been retained in the Bucket). Otherwise, the source node needs to check whether the retransmitted RST field on the Bucket is 0: if it is 0, the data in the Bucket may be an incomplete part of the data aggregated by the destination node. The destination node needs to initialize the Bucket first to avoid affecting the reaggregation, and then put the RST, CN, ECN, ERR and Payload data of the message into the Bucket; if it is 1, it means that the data cached in the Bucket is the data being reaggregated, so the aggregation operation needs to be performed on the message. In particular, before performing the aggregation operation, the destination node also needs to determine whether the retransmitted message arrives repeatedly, which is achieved by checking the Bitmap in the Bucket: the destination node first hashes the source IP address of the message to a bit on the Bitmap by taking the modulus of the Bitmap length, and then checks whether the bit is 1. If it is 1, it means that the retransmitted message has been aggregated and needs to be discarded; if the hashed bit is 0, it means that the retransmitted message is hashed to the corresponding cache unit Bucket for the first time, and the aggregation operation can be performed, and the bit position is set to 1. The length of the Bitmap should be greater than or equal to the number of source nodes to avoid hash conflicts. After performing the aggregation operation, the destination node checks whether the CN value of the Bucket is equal to the number of source nodes to determine whether to reply an ACK message.
[0173] Based on the above content, in the process of sending a confirmation message from the destination node to the source node, after the destination node obtains the message from the front-end node on the aggregation path in the network in real time, the step of processing the incompletely aggregated data message also includes:
[0174] The destination node calculates the location of the corresponding cache unit Bucket according to the value of the sequence number SN of the current message;
[0175] If the counter value of the current message is inconsistent with the leaf node of the destination node, and the current message is not a retransmitted message, the destination node checks whether the corresponding cache unit Bucket is empty;
[0176] If the corresponding cache unit Bucket is empty, the destination node saves the data of the current message in the corresponding cache unit Bucket;
[0177] If the corresponding cache unit Bucket is not empty, and the message stored in the corresponding cache unit Bucket is a retransmitted message, the destination node discards the current message;
[0178] If the corresponding cache unit Bucket is not empty, and the message stored in the corresponding cache unit Bucket is not a retransmitted message, the destination node sends the current message to the corresponding cache unit Bucket for aggregation, and updates the value of the counter of the aggregated message;
[0179] If the value of the counter of the aggregated message is equal to the number of leaf nodes of the destination node, the destination node sends a corresponding confirmation message to the source node through the intra-network aggregation path.
[0180] In addition, in the process of sending a confirmation message from the destination node to the source node, after the destination node obtains the message from the front-end node on the aggregation path in the network in real time, the step of processing the retransmitted message includes:
[0181] The destination node calculates the location of the corresponding cache unit Bucket according to the value of the sequence number SN of the current message;
[0182] If the current message is a retransmitted message, the destination node checks whether the value of the message counter in the corresponding cache unit Bucket is equal to the number of leaf nodes of the destination node;
[0183] If the value of the counter of the message in the corresponding cache unit Bucket is equal to the number of leaf nodes of the destination node, the destination node sends a corresponding confirmation message to the source node through the intra-network aggregation path;
[0184] If the value of the counter of the message in the corresponding cache unit Bucket is not equal to the number of leaf nodes of the destination node, and the message stored in the corresponding cache unit Bucket is not a retransmitted message, the destination node initializes the corresponding cache unit Bucket, and then stores the data of the current message in the corresponding cache unit Bucket;
[0185] If the value of the counter of the message in the corresponding cache unit Bucket is not equal to the number of leaf nodes of the destination node, and the message stored in the corresponding cache unit Bucket is a retransmitted message, the destination node determines whether the current message arrives repeatedly, specifically by taking the modulus of the source IP address of the current message and the length of the bitmap of the corresponding cache unit Bucket to hash to the corresponding bit on the bitmap;
[0186] If the hashed bit is the specified value, the current message arrives repeatedly and the destination node discards the current message;
[0187] If the hashed bit is not the specified value, the current message is not a duplicate arrival. The destination node sends the current message to the corresponding bucket for aggregation and updates the counter value of the aggregated message.
[0188] If the value of the counter of the aggregated message is equal to the number of leaf nodes of the destination node, the destination node sends a corresponding confirmation message to the source node through the intra-network aggregation path.
[0189] In this embodiment, when replying the confirmation message ACK, the destination node needs to initialize the cache unit Bucket at a symmetrical position on the message buffer. This is because the sliding window size of the source node does not exceed N max , so once the data in the current cache unit Bucket has been aggregated, it means that in the current message sequence number N max The previous message must be confirmed to be received by the source node (the sliding window cannot move to a message that has not been confirmed to be received). N max In this way, the cache unit Bucket of the buffer can be recycled to avoid significant memory overhead on the destination node.
[0190] Therefore, in the process of sending a confirmation message from the destination node to the source node, after sending the corresponding confirmation message to the source node through the intra-network aggregation path, the following steps are also included:
[0191] Initialize the cache unit Bucket at the previous symmetrical position of the message buffer. The index expression of the cache unit Bucket at the previous symmetrical position is as follows:
[0192] Bucket.ID =( pkt.SN + N max )%2 N max
[0193] in, pkt.SN Indicates the value of the SN field of the current message. N max Indicates the maximum value of the sliding window on the source node.
[0194] Embodiment 2
[0195] This embodiment provides an intra-network aggregation system for heterogeneous resources, including a source node, a destination node, and a switch, wherein:
[0196] The source node is used to send a message to the destination node through the in-network aggregation path after receiving the in-network aggregation task. If a confirmation message corresponding to the sent message is received within a specified time, the next batch of messages are sent to the destination node through the in-network aggregation path using a sliding window; if a confirmation message corresponding to the sent message is not received within the specified time, the sent message is retransmitted to the destination node through the in-network aggregation path;
[0197] The switch is used to obtain the message of the front-end node on the aggregation path in the network in real time; if the current message is a retransmitted message, the current message is forwarded to the back-end node on the aggregation path in the network; if the current message is not a retransmitted message, the value of the counter of the current message is compared with the number of leaf nodes of the switch; if the value of the counter of the current message is consistent with the number of leaf nodes of the switch, the current message is forwarded to the back-end node on the aggregation path in the network; if the value of the counter of the current message is less than the number of leaf nodes of the switch, the position of the corresponding aggregator is calculated according to the value of the TaskID and the sequence number SN of the current message, and the current message is sent to the corresponding aggregator for aggregation; if a hash conflict occurs, the current message is forwarded to the back-end node on the aggregation path in the network; if no hash conflict occurs, the value of the counter of the aggregated message is updated; if the value of the counter of the aggregated message is equal to the number of leaf nodes of the switch, the aggregated message is forwarded to the back-end node and the corresponding aggregator is cleared; if the value of the counter of the aggregated message is less than the number of leaf nodes of the switch, the aggregated message is retained in the corresponding aggregator;
[0198] The destination node is used to obtain the message of the front-end node on the in-network aggregation path in real time after receiving the in-network aggregation task. If the value of the counter of the current message is consistent with the leaf node of the destination node, the corresponding confirmation message is sent to the source node through the in-network aggregation path.
[0199] In summary, the present invention discloses an intra-network aggregation method and system for heterogeneous resources, which can improve the performance of intra-network aggregation in resource heterogeneous networks. The present invention does not avoid hash conflicts occurring on switch aggregators. It only needs to know which switches can aggregate these messages (these switches can perform aggregation operations or not), without accurately knowing which switches each intra-network aggregation message needs to perform aggregation on in sequence. These messages will be gradually aggregated by the switches on the path in a progressive manner along a given intra-network aggregation path: when encountering a switch with available resources, they can be aggregated; otherwise, the switch only needs to forward these messages, and they can try to be aggregated again on the downstream switch. The present invention uses the monotonically increasing feature of the message sequence number (SN) to implement such an aggregation process, and tracks the hash conflicts occurring on the aggregator by recording the current maximum conflict sequence number SN (denoted as MSN) on the switch. By comparing the SN of the data packet with the MSN cached on the switch aggregator, the associated data packets can be guided to the same switch for aggregation, thereby avoiding the occurrence of mutual waiting lock MWL.
[0200] In addition, hash conflicts on the aggregator will also cause the aggregation of messages from the switch subnodes to become uncertain, because the subnodes may have performed aggregation operations on all related messages, or only a part of them may be aggregated. Therefore, it is difficult for the switch to determine whether these messages can be forwarded, and it is also difficult to determine when the aggregation operation is completed. To solve this problem, the present invention adds a counter field to the message header to record how many associated messages each message has currently performed aggregation operations with. After the message arrives at the switch, the switch determines whether it should be forwarded or aggregated by comparing the counter field with the number of leaf nodes on the aggregation path of the switch in the network (rather than the number of subnodes). This method relies on another key fact, that is, once the aggregation path in the network is determined, the number of leaf nodes of each switch on the path is determined. Therefore, the switch can know whether it has completed aggregation with all associated messages on the remaining subnodes through the counter field of the message: if so, the switch can directly forward the message; otherwise, the switch needs to cache the message to wait for the messages of the remaining subnodes to complete the aggregation operation until the total number of aggregations is equal to the number of leaf nodes of the switch. In addition, the present invention can also effectively deal with problems such as network packet loss, and ensure the correctness and reliability of the in-network aggregation process in complex packet loss scenarios.
[0201] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.
Claims
1. A method for in-network aggregation of heterogeneous resources, characterized in that: Applied to a distributed system including a source node, a destination node and a switch, the method comprises the following steps: After receiving the in-network aggregation task, the source node sends a message to the destination node through the in-network aggregation path. If the source node receives the confirmation message corresponding to the sent message within the specified time, the source node uses the sliding window to send the next batch of messages to the destination node through the in-network aggregation path; if the source node does not receive the confirmation message corresponding to the sent message within the specified time, the source node retransmits the sent message to the destination node through the in-network aggregation path; The switch obtains the message in real time and aggregates the message, including the following steps: The switch obtains the messages from the front-end nodes on the aggregation path in the network in real time; If the current message is a retransmitted message, the switch forwards the current message to a back-end node on the aggregation path in the network; if the current message is not a retransmitted message, the switch compares the value of the counter of the current message with the number of leaf nodes of the switch; If the value of the counter of the current message is consistent with the number of leaf nodes of the switch, the switch forwards the current message to the back-end node on the aggregation path in the network; If the value of the counter of the current message is less than the number of leaf nodes of the switch, the switch calculates the position of the corresponding aggregator according to the value of the TaskID and the sequence number SN of the current message, and then sends the current message to the corresponding aggregator for aggregation. If a hash conflict occurs, the switch forwards the current message to the back-end node on the aggregation path in the network. If no hash conflict occurs, the switch updates the value of the counter of the aggregated message. If the value of the counter of the aggregated message is equal to the number of leaf nodes of the switch, the switch forwards the aggregated message to the backend node and clears the corresponding aggregator; if the value of the counter of the aggregated message is less than the number of leaf nodes of the switch, the switch retains the aggregated message in the corresponding aggregator; After receiving the intra-network aggregation task, the destination node obtains the message of the front-end node on the intra-network aggregation path in real time. If the value of the counter of the current message is consistent with the number of leaf nodes of the destination node, the destination node sends the corresponding confirmation message to the source node through the intra-network aggregation path.
2. The method for in-network aggregation of heterogeneous resources according to claim 1, characterized in that: If the current message is a retransmitted message, when the switch forwards the current message to the back-end node on the aggregation path in the network, the following steps are included: The switch calculates the location of the corresponding aggregator based on the TaskID and sequence number SN of the current message; If the corresponding aggregator is empty, or the value of the sequence number SN field of the current message is different from the value of the sequence number SN field of the message in the corresponding aggregator, the switch forwards the current message to the back-end node on the aggregation path in the network; If the corresponding aggregator is not empty, and the value of the sequence number SN field of the current message is the same as the value of the sequence number SN field of the message in the corresponding aggregator, the switch forwards the current message to the backend node on the aggregation path in the network and clears the corresponding aggregator.
3. The method for in-network aggregation of heterogeneous resources according to claim 1, characterized in that: When the switch calculates the position of the corresponding aggregator based on the TaskID and sequence number SN of the current message, it includes: According to the TaskID of the current message, find the offset position information of the corresponding in-network aggregation task on the aggregator table of the switch Offset , and the allocated aggregator interval size N ; The aggregator table is composed of multiple aggregators pre-compiled on the switch; Use the value of the sequence number SN of the current message to the aggregator interval size N Perform a modulo calculation and then compare the calculation result with the offset position information Offset Add them together to get the position of the corresponding aggregator.
4. The method for in-network aggregation of heterogeneous resources according to claim 1, characterized in that: When the current message is sent to the corresponding aggregator for aggregation, it includes: If the corresponding aggregator is empty and the value of the sequence number SN field of the current message is less than the maximum conflict sequence number MSN of the corresponding aggregator, the switch forwards the current message to the back-end node on the aggregation path within the network; If the corresponding aggregator is empty and the value of the sequence number SN field of the current message is greater than the maximum conflict sequence number MSN of the corresponding aggregator, the switch saves the current message in the corresponding aggregator and updates the maximum conflict sequence number MSN of the corresponding aggregator to the sequence number SN of the current message; If the corresponding aggregator is not empty, and the value of the sequence number SN field of the current message is consistent with the value of the sequence number SN field of the message in the corresponding aggregator, the switch aggregates the current message with the message in the corresponding aggregator to obtain an aggregated message; If the corresponding aggregator is not empty, and the value of the sequence number SN field of the current message is inconsistent with the value of the sequence number SN field of the message in the corresponding aggregator, the switch forwards the current message to the back-end node on the aggregation path within the network, and updates the maximum conflict sequence number MSN of the corresponding aggregator to the sequence number SN of the current message.
5. The method for in-network aggregation of heterogeneous resources according to claim 1, characterized in that: The destination node is provided with a message buffer, and the message buffer is composed of a cache unit Bucket. After the destination node obtains the message of the front-end node on the aggregation path in the network in real time, the step of processing the incompletely aggregated data message includes: The destination node calculates the location of the corresponding cache unit Bucket according to the value of the sequence number SN of the current message; If the value of the counter of the current message is inconsistent with the number of leaf nodes of the destination node, and the current message is not a retransmitted message, the destination node checks whether the corresponding cache unit Bucket is empty; If the corresponding cache unit Bucket is empty, the destination node saves the data of the current message in the corresponding cache unit Bucket; If the corresponding cache unit Bucket is not empty, and the message stored in the corresponding cache unit Bucket is a retransmitted message, the destination node discards the current message; If the corresponding cache unit Bucket is not empty, and the message stored in the corresponding cache unit Bucket is not a retransmitted message, the destination node sends the current message to the corresponding cache unit Bucket for aggregation, and updates the value of the counter of the aggregated message; If the value of the counter of the aggregated message is equal to the number of leaf nodes of the destination node, the destination node sends a corresponding confirmation message to the source node through the intra-network aggregation path.
6. The method for in-network aggregation of heterogeneous resources according to claim 5, characterized in that: After the destination node obtains the message from the front-end node on the aggregation path in the network in real time, the step of processing the retransmitted message includes: The destination node calculates the location of the corresponding cache unit Bucket according to the value of the sequence number SN of the current message; If the current message is a retransmitted message, the destination node checks whether the value of the message counter in the corresponding cache unit Bucket is equal to the number of leaf nodes of the destination node; If the value of the counter of the message in the corresponding cache unit Bucket is equal to the number of leaf nodes of the destination node, the destination node sends a corresponding confirmation message to the source node through the intra-network aggregation path; If the value of the counter of the message in the corresponding cache unit Bucket is not equal to the number of leaf nodes of the destination node, and the message stored in the corresponding cache unit Bucket is not a retransmitted message, the destination node initializes the corresponding cache unit Bucket, and then stores the data of the current message in the corresponding cache unit Bucket; If the value of the counter of the message in the corresponding cache unit Bucket is not equal to the number of leaf nodes of the destination node, and the message stored in the corresponding cache unit Bucket is a retransmitted message, the destination node sends the current message to the corresponding cache unit Bucket for aggregation, and updates the value of the counter of the aggregated message; If the value of the counter of the aggregated message is equal to the number of leaf nodes of the destination node, the destination node sends a corresponding confirmation message to the source node through the intra-network aggregation path.
7. The method for in-network aggregation of heterogeneous resources according to claim 6, characterized in that: Before the destination node sends the current message to the corresponding buffer unit Bucket for aggregation, it also includes the step of determining whether the current message arrives repeatedly, specifically including: The destination node modulos the source IP address of the current message with the length of the bitmap of the corresponding cache unit Bucket, and hashes it to the corresponding bit on the bitmap; If the hashed bit is the specified value, the current message arrives repeatedly and the destination node discards the current message; If the hashed bit is not the specified value, the current message is not a duplicate arrival, and the destination node sends the current message to the corresponding cache unit Bucket for aggregation.
8. The method for in-network aggregation of heterogeneous resources according to claim 5 or 6, characterized in that: After the destination node sends the corresponding confirmation message to the source node through the intra-network aggregation path, the following also includes: Initialize the cache unit Bucket at the previous symmetrical position of the message buffer. The index expression of the cache unit Bucket at the previous symmetrical position is as follows: Bucket.ID =( pkt.SN + N max )%2 N max in, SN Indicates the sequence number of the current message. N max Indicates the maximum value of the sliding window on the source node. The expression is as follows: N max = B out ×R / S in, B out is the minimum value of the outbound port bandwidth of all source nodes, R is the network round-trip time RTT, S Is the message size.
9. The method for in-network aggregation of heterogeneous resources according to claim 1, characterized in that: The message and the corresponding confirmation message both include an IP header field, a HAP protocol field, and a payload data field. The HAP protocol field includes a congestion signal ECN field and an error information ERR field. When the source node sends a message to the destination node through an intra-network aggregation path or the destination node sends a corresponding confirmation message to the source node through an intra-network aggregation path, if the switch port is congested, the congestion signal ECN field is marked. If an error occurs during aggregation of the message, the error information ERR field is marked. After the source node receives the confirmation message corresponding to the sent message, the following steps are also included: If the congestion signal ECN field of the confirmation message is marked, reduce the size of the sliding window; If the error information ERR field of the confirmation message is marked, the sent message is retransmitted to the destination node through point-to-point transmission.
10. An intra-network aggregation system for heterogeneous resources, characterized in that: It includes source node, destination node and switch, where: The source node is used to send a message to the destination node through the in-network aggregation path after receiving the in-network aggregation task. If a confirmation message corresponding to the sent message is received within a specified time, the next batch of messages are sent to the destination node through the in-network aggregation path using a sliding window; if a confirmation message corresponding to the sent message is not received within the specified time, the sent message is retransmitted to the destination node through the in-network aggregation path; The switch is used to obtain the message of the front-end node on the aggregation path in the network in real time; if the current message is a retransmitted message, the current message is forwarded to the back-end node on the aggregation path in the network; if the current message is not a retransmitted message, the value of the counter of the current message is compared with the number of leaf nodes of the switch; if the value of the counter of the current message is consistent with the number of leaf nodes of the switch, the current message is forwarded to the back-end node on the aggregation path in the network; if the value of the counter of the current message is less than the number of leaf nodes of the switch, the position of the corresponding aggregator is calculated according to the value of the TaskID and the sequence number SN of the current message, and the current message is sent to the corresponding aggregator for aggregation; if a hash conflict occurs, the current message is forwarded to the back-end node on the aggregation path in the network; if no hash conflict occurs, the value of the counter of the aggregated message is updated; if the value of the counter of the aggregated message is equal to the number of leaf nodes of the switch, the aggregated message is forwarded to the back-end node and the corresponding aggregator is cleared; if the value of the counter of the aggregated message is less than the number of leaf nodes of the switch, the aggregated message is retained in the corresponding aggregator; The destination node is used to obtain the message of the front-end node on the in-network aggregation path in real time after receiving the in-network aggregation task. If the value of the counter of the current message is consistent with the leaf node of the destination node, the corresponding confirmation message is sent to the source node through the in-network aggregation path.
Citation Information
Patent Citations
Message packet loss recovery method and system for aggregation in edge stateless network
CN118041885A
Switch resource management method and system for aggregation in edge stateless network
CN118301114A
