Algorithm for reducing hash jitter caused by number change of LAG members

Abstract

The invention discloses an algorithm for reducing hash jitter caused by change of the number of LAG members, and relates to the technical field of link aggregation, and the algorithm comprises the following steps: step 1, real-time monitoring; step 2, starting early warning; step 3, initial establishment of a copy; step 4, data migration; step 5, node optimization; and step 6, switching verification. According to the algorithm for reducing hash jitter caused by the change of the number of the LAG members, a data distribution mechanism of a consistent hash ring is optimized by dynamically allocating virtual nodes, when the change of the number of the LAG members causes hash mapping adjustment, the problem of data concentration or dispersion caused by simple reallocation cannot occur, migration and rerouting of a large amount of data are avoided, and the algorithm is simple and convenient to operate. The interference of hash jitter on the stability of data transmission is reduced to the greatest extent, the continuity of data transmission is ensured, the packet loss rate is reduced, and reliable support is provided for services with high real-time requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of link aggregation, and in particular to an algorithm for reducing hash jitter caused by a change in the number of LAG members. Background Art

[0002] M-LAG (Multichassis link aggregation) virtualizes two physical devices into a virtual M-LAG system at the aggregation level to access the network, achieving cross-device link aggregation, thereby providing device-level redundancy protection and traffic load sharing.

[0003] In the M-LAG system, the M-LAG protocol technology data traffic between two neighboring M-LAG devices is transmitted through the peer-link (peer link); in addition to the IPL link, there is also a Keepalive link between the two devices, which is used to detect the status of the neighbors, that is, to exchange Keepalive messages to perform MAD (MultiActiveDetection) when the peer-link link fails, to avoid the two M-LAG devices running as the master device role.

[0004] The invention with publication number CN119070972A discloses a method and device for preventing hash polarization of multi-level cross-device link aggregation connections. As shown in the above invention, the existing method includes recording the root node hash algorithm; setting the root node hash algorithm for each effective cross-device link aggregation group; detecting the polarized cross-device link aggregation group; screening a new hash algorithm for the polarized cross-device link aggregation group; and setting a new hash algorithm for the polarized cross-device link aggregation group. However, the existing algorithm often causes hash jitter problems when the number of LAG members changes, which may lead to unstable data transmission, reduced efficiency, and load imbalance. Summary of the invention

[0005] In view of the deficiencies of the prior art, the present invention provides an algorithm for reducing hash jitter caused by changes in the number of LAG members, thereby solving the existing problems.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: an algorithm for reducing hash jitter caused by changes in the number of LAG members, comprising the following steps:

[0007] Step 1: Real-time monitoring: Use link heartbeat and port polling technology to closely monitor the status of LAG members around the clock and accurately capture dynamics such as member joining, leaving or failure;

[0008] Step 2: Start the warning: When the number of members changes, the warning is triggered immediately. At the same time, the administrator is notified, the hash adjustment process is started, and the flow of non-critical data is suspended to reserve time for adjustment;

[0009] Step 3: Initial creation of replicas: Evaluate the carrying capacity of new members based on their bandwidth and latency characteristics, build temporary hash table replicas, and preliminarily plan the data slot layout;

[0010] Step 4: Data migration: Sort the original hash table data according to the preset priority, and first move the hot spots and high-priority business data to the corresponding new member slots of the temporary replica in small batches or multiple batches. Verify the data integrity and consistency batch by batch, and mark the migration complete after confirmation.

[0011] Step 5: Node optimization: According to the new LAG member configuration, recalculate the virtual node distribution of the consistent hash ring, insert new members into nodes as needed, and migrate some nodes near old members; when a member exits, smoothly transfer the corresponding node load to the surviving member node;

[0012] Step 6: Switch verification: After the temporary replica data is migrated and the hash ring is stable, enable the new hash table, resume data transmission, and continuously observe indicators such as throughput, latency, and packet loss rate for comparison and verification. If any abnormality is found, roll back the original hash table in time and re-analyze and adjust.

[0013] Preferably, in step 1, the system monitors the status of LAG members in real time, and promptly discovers the joining, exiting or failure of members through link heartbeat detection, port status polling and other technologies.

[0014] Preferably, once a quantity change is detected in step 2, an early warning mechanism is immediately triggered, a notification is sent to the system administrator, and the hash adjustment process is started, while the transmission of some non-critical data is suspended to gain a time window for the adjustment.

[0015] Preferably, in step three, a temporary hash table copy is constructed, and the carrying capacity of the new member is evaluated according to its characteristics (such as bandwidth, delay, etc.), and data slots are preliminarily allocated.

[0016] Preferably, in step 4, the data in the original hash table is sorted according to a pre-set priority rule, and hot data and high-priority business data are preferentially migrated in small batches and multiple batches to the slots of the new members corresponding to the temporary replica. After each batch is migrated, the integrity and consistency of the data are verified, and after ensuring that there are no errors, it is marked as migrated.

[0017] Preferably, in step five, on the consistent hash ring, the distribution of virtual nodes is recalculated according to the new LAG member configuration. For new members, a corresponding number of virtual nodes are reasonably inserted into the ring according to their performance parameters, and some virtual nodes adjacent to old members are migrated in a load-balanced manner. For exiting members, the data carried by their corresponding virtual nodes are smoothly transferred to the virtual nodes of other surviving members. By dynamically adjusting the mapping relationship between virtual nodes and actual LAG members, the stability of the hash ring and the uniformity of data distribution are maintained.

[0018] Preferably, in step six, when the data migration in the temporary copy is completed and the hash ring reconstruction is stable, the old and new hash tables are switched, the new hash table is officially put into use, and the suspended data transmission is resumed. For a period of time after the switch, the data transmission performance is continuously monitored, including indicators such as throughput, delay, and packet loss rate, and compared with the data before the switch, the algorithm's suppression effect on hash jitter is verified. If an abnormality is found, it is rolled back to the original hash table in time and re-analyzed and adjusted.

[0019] Preferably, in step 3, when allocating the initial number of virtual nodes to LAG members or dynamically adjusting the number of virtual nodes according to member performance, a weighted calculation method is adopted, assuming that the bandwidth of the member is B, the processing capacity is P, and the delay is D, and the corresponding weights are w B 、w P 、w D , the calculation formula of comprehensive score S is as follows:

[0020] S=w B ×B+w P ×P+w D ×D;

[0021] Based on the comprehensive score S, within the preset virtual node number range [N min ,N max ] Determine the number of virtual nodes N, and simply set it to a linear relationship:

[0022] N = k × S + b;

[0023] Where k and b are constants predetermined based on system experience and requirements.

[0024] Preferably, in step 4, when migrating data from the original hash table to the temporary copy, the data flow priority Prio t (e.g. measured by the number of visits and data transfer volume in the past period of time) and recent access frequency Freq r (such as the number of visits in the past hour) to determine the migration order, let the comprehensive migration priority Pri of data item i i The calculation formula is:

[0025]

[0026] Where α and β are pre-set weight coefficients used to balance the importance of traffic priority and recent access frequency;

[0027] According to Pri i Sort the data in descending order by size and migrate Pri first. i Data items with high values.

[0028] Preferably, in step 5, when the number of members changes and the load needs to be redistributed, for example, after a member exits, its virtual node load is transferred to other surviving members, and a weighted round-robin or similar strategy is adopted. Suppose the set of surviving members is M={m 1 ,m 2 ,…,m n}, member m j The current load carrying capacity (determined by bandwidth, remaining processing resources, etc.) is The load to be distributed is L, distributed to member m j Load The calculation formula is as follows:

[0029]

[0030] Beneficial Effects

[0031] The present invention provides an algorithm for reducing hash jitter caused by changes in the number of LAG members. Compared with the prior art, it has the following beneficial effects:

[0032] 1. This algorithm reduces hash jitter caused by changes in the number of LAG members. By dynamically allocating virtual nodes, it optimizes the data distribution mechanism of the consistent hash ring. When the change in the number of LAG members triggers the hash mapping adjustment, there will be no data concentration or dispersion problems caused by simple reallocation, avoiding the migration and rerouting of large amounts of data, minimizing the interference of hash jitter on the stability of data transmission, ensuring the continuity of data transmission, reducing the packet loss rate, and providing reliable support for services with high real-time requirements.

[0033] 2. This algorithm reduces hash jitter caused by changes in the number of LAG members. By introducing an improved consistent hash ring, multiple virtual nodes are allocated to each LAG member on the ring. The number of virtual nodes is dynamically allocated based on the member's bandwidth, processing power and other attributes. Members with large bandwidth and strong processing power correspond to more virtual nodes. When the number of members changes, the load is balanced by adjusting the distribution of virtual nodes rather than simply redistributing hash values. This allows data to be distributed more evenly under the new LAG structure, reducing hash concentration or dispersion problems caused by the increase or decrease of members. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the algorithm flow of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] See Figure 1 The present invention provides an algorithm for reducing hash jitter caused by changes in the number of LAG members, comprising the following steps:

[0037] Step 1: Real-time monitoring: The system monitors the status of LAG members in real time, and promptly detects the joining, exiting, or failure of members through technologies such as link heartbeat detection and port status polling.

[0038] Step 2: Early warning activation: Once a quantity change is detected, the early warning mechanism is immediately triggered, a notification is sent to the system administrator, and the hash adjustment process is initiated. At the same time, the transmission of some non-critical data is suspended to gain a time window for adjustment.

[0039] Step 3: Initial creation of replicas: Build a temporary hash table replica, evaluate the carrying capacity of the new member based on its characteristics (such as bandwidth, latency, etc.), and preliminarily allocate data slots.

[0040] When the number of LAG members changes, the algorithm does not immediately recalculate the hash mapping of all data, but adopts an incremental adjustment strategy. First, a temporary hash table copy is created, and the newly added LAG members are assigned to the free slots in this copy. At the same time, some data is gradually migrated from the original hash table to the new copy. The migration is based on factors such as data traffic priority and recent access frequency. Priority is given to migrating data that has high real-time requirements and is frequently accessed, ensuring that key data can quickly adapt to the new link layout.

[0041] Professional network testing tools are used to accurately measure the available bandwidth of new members, simulated data loading is used to detect the upper limit of their processing capabilities, and high-precision network probes are used to obtain data transmission delay values.

[0042] When allocating the initial number of virtual nodes to LAG members or dynamically adjusting the number of virtual nodes based on member performance, a weighted calculation method is used. Let the member's bandwidth be B, processing power be P, and delay be D. The corresponding weights are w B 、w P 、w D , the calculation formula of comprehensive score S is as follows:

[0043] S=w B ×B+w P ×P+w D ×D;

[0044] Based on the comprehensive score S, within the preset virtual node number range [N min ,N max ] Determine the number of virtual nodes N, and simply set it to a linear relationship:

[0045] N = k × S + b;

[0046] Where k and b are constants predetermined based on system experience and requirements.

[0047] According to the comprehensive score and the current load balancing requirements of the system, a corresponding number of virtual nodes are allocated to the new member on the consistent hash ring, and a suitable insertion position is selected. The selection of the insertion position follows the load distribution principle, giving priority to areas where the virtual nodes on the ring are sparsely distributed and the load is relatively light, to ensure that the integration of the new member will not cause local overload.

[0048] Step 4: Data migration: Sort the data in the original hash table according to the pre-set priority rules, and migrate hot data and high-priority business data in small batches and multiple batches to the slots of the new members corresponding to the temporary replica. After each batch is migrated, verify the integrity and consistency of the data and mark it as migrated after ensuring that it is correct.

[0049] When migrating data from the original hash table to the temporary copy, the data traffic priority Prio t (e.g. measured by the number of visits and data transfer volume in the past period of time) and recent access frequency Freq r (such as the number of visits in the past hour) to determine the migration order, let the comprehensive migration priority Pri of data item i i The calculation formula is:

[0050]

[0051] Where α and β are pre-set weight coefficients used to balance the importance of traffic priority and recent access frequency;

[0052] According to Pri i Sort the data in descending order by size and migrate Pri first. i Data items with high values.

[0053] Create a temporary hash table copy and preliminarily plan the data slots that can be carried by the new member based on its performance characteristics. For the massive data in the original hash table, carefully sort it according to pre-set data traffic priority, recent access frequency and other rules. Prioritize hot data and high-priority business data that have extremely high real-time requirements and are frequently accessed, and gradually migrate them to the slots corresponding to the new members in the temporary copy in a robust manner in small batches and multiple batches. After each batch of migration is completed, use the data verification algorithm to strictly verify the integrity and consistency of the data, ensure that the data is accurate, and then mark it as successfully migrated, to ensure that key data can adapt to the new link layout first, and seamlessly connect the system operation after the new member joins.

[0054] Once a LAG member is detected to have left, the system quickly locks all virtual nodes corresponding to the member on the consistent hash ring. Through the load balancing scheduling algorithm, the data carried by these virtual nodes is smoothly and efficiently transferred to the virtual nodes of other surviving members. The load balancing scheduling algorithm can flexibly adopt polling, weighted polling, or dynamic allocation strategies based on real-time load monitoring. For example, if weighted polling is used, the weights must be recalculated in advance based on the current performance indicators of the surviving members to ensure that the data can be reasonably distributed according to the remaining carrying capacity of each surviving member, avoid uneven data distribution caused by the departure of members, and maintain the overall stable operation of the system.

[0055] Step 5, node optimization: On the consistent hash ring, recalculate the distribution of virtual nodes according to the new LAG member configuration. For new members, insert a corresponding number of virtual nodes on the ring according to their performance parameters, and migrate some virtual nodes near old members in a load-balanced manner. For exiting members, smoothly transfer the data carried by their corresponding virtual nodes to the virtual nodes of other surviving members. By dynamically adjusting the mapping relationship between virtual nodes and actual LAG members, the stability of the hash ring and the uniformity of data distribution are maintained.

[0056] After completing the data transfer, the system promptly cleans up the virtual nodes left by the departing members and optimizes and reconstructs the consistent hash ring. It re-examines the distribution and load of the remaining members, dynamically adjusts the distribution density of the virtual nodes based on the performance of each surviving member, and further optimizes the flow of data on the ring, so that the system can still run efficiently when the number of members decreases, and continuously ensures the stability and balance of data transmission.

[0057] When the number of members changes and the load needs to be redistributed, for example, after a member exits, its virtual node load is transferred to other surviving members. A weighted round-robin or similar strategy is adopted. Suppose the set of surviving members is M = {m 1 ,m 2 ,…,m n}, member m j The current load carrying capacity (determined by bandwidth, remaining processing resources, etc.) is The load to be distributed is L, distributed to member m j Load The calculation formula is as follows:

[0058]

[0059] Step 6: Switching verification: When the data migration in the temporary replica is completed and the hash ring reconstruction is stable, switch the old and new hash tables, put the new hash table into use, and resume the suspended data transmission. For a period of time after the switch, continuously monitor the data transmission performance, including throughput, latency, packet loss rate and other indicators, and compare the data before the switch to verify the algorithm's suppression effect on hash jitter. If an abnormality is found, roll back to the original hash table in time and re-analyze and adjust.

[0060] In the embodiment of the present invention, when a change in the number of LAG members is detected, whether a new member is added or a member withdraws:

[0061] For new members, their performance indicators are also evaluated first, and the comprehensive score is calculated according to the above weighted method. Then, according to the current system load and overall performance distribution, find a suitable position on the consistent hash ring to insert the corresponding number of virtual nodes. The selection of the insertion position should consider balancing the load of each area on the ring as much as possible to avoid excessive local load. For example, if there are already many virtual nodes corresponding to high-load members in a certain area of ​​the ring, the virtual nodes of the new members will be inserted into the relatively free area first.

[0062] For the exiting member, the system quickly locates all the corresponding virtual nodes, and smoothly transfers the data carried by these virtual nodes to the virtual nodes of other surviving members according to the load balancing principle. The load balancing principle can be based on comprehensive considerations of factors such as data traffic and current node load. For example, polling or weighted polling can be used to distribute the load of the virtual nodes of the exiting member to other surviving member virtual nodes with remaining carrying capacity to ensure that data distribution remains even.

[0063] In the embodiment of the present invention, the system will re-evaluate the performance of LAG members regularly (e.g., every (T) time interval) during operation, because the actual performance of members may change with factors such as changes in the network environment and aging of equipment. The method of re-evaluating indicators and calculating comprehensive scores is the same as that in the initial allocation stage.

[0064] In the embodiment of the present invention, according to the new comprehensive score, compared with the current number of virtual nodes, if it is found that the performance of the member has improved or decreased to a certain extent, the number of its virtual nodes needs to be adjusted. If the performance is improved, the number of virtual nodes is appropriately increased on the consistent hash ring, and the positions of the newly added virtual nodes are reasonably allocated to ensure the stability of the ring; if the performance is reduced, the number of virtual nodes is reduced, and the data carried by the reduced virtual nodes is also smoothly migrated to other suitable virtual nodes to maintain efficient and stable data transmission.

[0065] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art, and the model parameters of each electrical appliance are not specifically limited, and conventional equipment can be used.

[0066] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0067] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An algorithm for reducing hash jitter caused by changes in the number of LAG members, characterized in that: The following steps are involved: Step 1: Real-time monitoring: Use link heartbeat and port polling technology to closely monitor the status of LAG members around the clock and accurately capture the dynamics of member joining, leaving or failure; Step 2: Start the warning: When the number of members changes, the warning is triggered immediately. At the same time, the administrator is notified, the hash adjustment process is started, and the flow of non-critical data is suspended to reserve time for adjustment; Step 3: Initial creation of replicas: Evaluate the carrying capacity of new members based on their bandwidth and latency characteristics, build temporary hash table replicas, and preliminarily plan the data slot layout; Step 4: Data migration: Sort the original hash table data according to the preset priority, and first move the hot spots and high-priority business data to the corresponding new member slots of the temporary replica in small batches or multiple batches. Verify the data integrity and consistency batch by batch, and mark the migration complete after confirmation. Step 5: Node optimization: According to the new LAG member configuration, recalculate the virtual node distribution of the consistent hash ring, insert new members into nodes as needed, and migrate some nodes near old members; when a member exits, smoothly transfer the corresponding node load to the surviving member node; Step 6: Switch verification: After the temporary replica data is migrated and the hash ring is stable, enable the new hash table, resume data transmission, and continuously observe the throughput, latency, and packet loss rate indicators, compare and verify, and roll back the original hash table in time to re-analyze and adjust in case of abnormalities.

2. The algorithm for reducing hash jitter caused by changes in the number of LAG members according to claim 1, characterized in that: In the step 1, the system monitors the status of LAG members in real time, and promptly discovers the joining, exiting or failure of members through link heartbeat detection and port status polling technology.

3. The algorithm for reducing hash jitter caused by changes in the number of LAG members according to claim 1, characterized in that: Once a quantity change is detected in step 2, the early warning mechanism is immediately triggered, a notification is sent to the system administrator, and the hash adjustment process is started. At the same time, the transmission of some non-critical data is suspended to gain a time window for adjustment.

4. The algorithm for reducing hash jitter caused by changes in the number of LAG members according to claim 1, characterized in that: In the step three, a temporary hash table copy is constructed, the carrying capacity of the new member is evaluated according to its characteristics, and data slots are preliminarily allocated.

5. The algorithm for reducing hash jitter caused by changes in the number of LAG members according to claim 1, characterized in that: In step 4, the data in the original hash table is sorted according to a pre-set priority rule, and hot data and high-priority business data are preferentially migrated in small batches and multiple batches to the slots of the new members corresponding to the temporary replica. After each batch is migrated, the integrity and consistency of the data are verified, and after ensuring that there are no errors, it is marked as migrated.

6. The algorithm for reducing hash jitter caused by changes in the number of LAG members according to claim 1, characterized in that: In the step five, on the consistent hash ring, the distribution of virtual nodes is recalculated according to the new LAG member configuration. For new members, a corresponding number of virtual nodes are reasonably inserted into the ring according to their performance parameters, and some virtual nodes adjacent to old members are migrated in a load-balanced manner. For exiting members, the data carried by their corresponding virtual nodes are smoothly transferred to the virtual nodes of other surviving members. By dynamically adjusting the mapping relationship between virtual nodes and actual LAG members, the stability of the hash ring and the uniformity of data distribution are maintained.

7. The algorithm for reducing hash jitter caused by changes in the number of LAG members according to claim 1, characterized in that: In step six, when the data migration in the temporary copy is completed and the hash ring reconstruction is stable, the old and new hash tables are switched, the new hash table is officially put into use, and the suspended data transmission is resumed. For a period of time after the switch, the data transmission performance is continuously monitored, including indicators of throughput, delay and packet loss rate, and compared with the data before the switch, the algorithm's suppression effect on hash jitter is verified. If an abnormality is found, it is rolled back to the original hash table in time and re-analyzed and adjusted.

8. The algorithm for reducing hash jitter caused by changes in the number of LAG members according to claim 1, characterized in that: In step 3, when allocating the initial number of virtual nodes to LAG members or dynamically adjusting the number of virtual nodes according to member performance, a weighted calculation method is used. Let the bandwidth of the member be B, the processing capacity be P, and the delay be D. The corresponding weights are w B 、w P 、w D , the calculation formula of comprehensive score S is as follows: S=w B ×B+w P ×P+w D ×D; Based on the comprehensive score S, within the preset virtual node number range [N min ,N max ] Determine the number of virtual nodes N, and simply set it to a linear relationship: N = k × S + b; Where k and b are constants predetermined based on system experience and requirements.

9. The algorithm for reducing hash jitter caused by changes in the number of LAG members according to claim 1, characterized in that: In step 4, when migrating data from the original hash table to the temporary copy, the data traffic priority Prio t and recent access frequency Freq r To determine the migration order, let the comprehensive migration priority Pri of data item i be i The calculation formula is: Where α and β are pre-set weight coefficients used to balance the importance of traffic priority and recent access frequency; According to Pri i Sort the data in descending order by size and migrate Pri first. i Data items with high values.

10. The algorithm for reducing hash jitter caused by changes in the number of LAG members according to claim 1, characterized in that: In step 5, when the number of members changes and the load needs to be redistributed, after the member exits, its virtual node load is transferred to other surviving members, and a weighted round-robin or similar strategy is adopted. Suppose the set of surviving members is M = {m1, m2, ..., m n }, member m j The current load carrying capacity is The load to be distributed is L, distributed to member m j Load The calculation formula is as follows:

Images