Flow monitoring method, device and equipment based on sketch optimization

Abstract

The invention discloses a traffic supervision method, device and equipment based on sketch optimization. The method comprises the following steps: initializing a counter and a network mapper using a skecth data structure; mapping a network flow to a corresponding counter based on the network mapper, and checking the value of the corresponding counter; wherein the value of the corresponding counter represents the number of tokens currently owned by the corresponding counter; executing corresponding operation on the network flow according to the value of the corresponding counter, and updating the value of the corresponding counter; and obtaining the current state and rate of each network flow based on the current values of all counters. According to the invention, each flow can be managed fairly and accurately, and a large amount of storage space and computing resources can be saved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of traffic regulation, and in particular to a traffic regulation method, device and equipment based on sketch optimization. Background Art

[0002] In order to achieve higher network quality, some traffic policing methods need to be used. Traffic policing is a mechanism that measures network traffic and compares it with predetermined policies or rate standards, taking certain measures such as dropping, marking or delaying traffic that exceeds the specified rate to ensure that the traffic complies with the contract or service level agreement (SLA). This helps avoid overuse of network resources and ensure fairness and service quality.

[0003] In the prior art, the Token Bucket Algorithm is a strategy widely used in network traffic management and control. Its core idea is to control the data transmission rate by simulating the generation and consumption of "tokens". Specifically, Figure 1 As shown in the figure, in the token bucket algorithm, there is a virtual "bucket" that fills tokens at a certain fixed rate. When a data packet or network flow needs to be transmitted, it must obtain the corresponding number of tokens from the bucket. If there are enough tokens in the bucket, the data packet can be transmitted immediately, consuming the corresponding tokens. If there are no tokens or insufficient tokens in the bucket, the data packet may be temporarily delayed, discarded, or marked as low priority until enough tokens are available. In this way, it can allow short bursts of data while limiting the average data transmission rate over a long period of time. In addition, to prevent unlimited accumulation of tokens, the capacity of the token bucket is upper bounded, and tokens exceeding this upper limit will be discarded.

[0004] Although existing technologies have certain effects in traffic regulation, they may not be efficient enough in large-scale, high-concurrency network environments and have the following disadvantages:

[0005] ●Scalability issues: Traditional token bucket algorithms are designed for single or global traffic. When a large number of independent traffic flows need to be tracked and controlled, it becomes impractical to configure and manage an independent token bucket for each flow, resulting in a large waste of resources.

[0006] ● Storage and computational efficiency: For large-scale networks or high-concurrency traffic, maintaining a token bucket or related state for each active network flow may lead to storage and computational bottlenecks.

[0007] ● Traffic conflicts: In scenarios with high concurrent traffic, multiple flows may compete for tokens in the same token bucket at the same time, which may cause some flows to get more bandwidth while other flows are restricted.

[0008] These shortcomings indicate that there is still room for improvement in the prior art, especially in reducing storage overhead and improving computer scalability. New technical solutions may need to address these issues to provide more effective data compression and similarity comparison methods. Summary of the invention

[0009] In order to solve the above problems, the present invention discloses a traffic regulation method, device and equipment based on sketch optimization. The method provides a compact way to track a large number of independent flows based on the sketch data structure, and maps each flow to a corresponding counter through a hash function. This design allows each flow to be managed fairly and accurately, while also saving a lot of storage space and computing resources. Among them, Sketch is an approximate algorithm that processes large data streams and provides them with a compact data structure to approximately represent certain characteristics of the original data. The purpose of the Sketch algorithm is to process large-scale data streams with limited memory and computing resources, and to quickly and approximately answer queries about the data.

[0010] The technical solution of the present invention includes the following contents.

[0011] A traffic monitoring method based on skecth optimization, the method comprising:

[0012] Initialize the counter and network mapper using skecth data structure;

[0013] Based on the network mapper, a network flow is mapped to a corresponding counter, and a value of the corresponding counter is checked; wherein the value of the corresponding counter represents the number of tokens currently owned by the corresponding counter;

[0014] According to the value of the corresponding counter, perform a corresponding operation on the network flow and update the value of the corresponding counter;

[0015] Get the current status and rate of each network flow based on the current values ​​of all counters.

[0016] Further, the counter and network mapper initialization using the skecth data structure include:

[0017] Set the maximum value of each counter;

[0018] Initialize the values ​​of all counters to a set value;

[0019] A hash function is defined, wherein the hash function is used to map different network flows to different counters.

[0020] Further, performing corresponding operations on the network flow according to the value of the corresponding counter includes:

[0021] When the value of the corresponding counter is greater than zero, continue the transmission of the network flow and reduce the value of the corresponding counter by 1;

[0022] When the value of the corresponding counter is zero, a corresponding blocking operation is performed according to a set policy.

[0023] Furthermore, the blocking operation includes: limiting the network flow, discarding the network flow or delaying the transmission of the network flow.

[0024] Furthermore, the method further comprises:

[0025] At fixed time intervals or when triggered by specific events, the values ​​of all counters increase at a fixed rate until the maximum value of the counter is reached;

[0026] and,

[0027] When the value of a counter reaches the maximum value of the counter, the added tokens are discarded to ensure that the value of the counter does not overflow.

[0028] Furthermore, the method further comprises:

[0029] If the network flow continues to consume a large number of tokens or often causes the token bucket to be empty, adjust the implementation strategy of the counter and / or network mapper; wherein the implementation strategy includes: changing the hash function corresponding to the network mapper, increasing the maximum value of the counter, or increasing the token replenishment rate.

[0030] A traffic monitoring device based on skecth optimization, the device comprising:

[0031] Initialization module, used to initialize the counter and network mapper using skecth data structure;

[0032] A traffic mapping module, used for mapping a network flow to a corresponding counter based on the network mapper, and checking the value of the corresponding counter; wherein the value of the corresponding counter represents the number of tokens currently owned by the corresponding counter;

[0033] A flow execution module, used for executing corresponding operations on the network flow according to the value of the corresponding counter, and updating the value of the corresponding counter;

[0034] The traffic policing module is used to obtain the current status and rate of each network flow based on the current values ​​of all counters.

[0035] A computer device comprises: a processor, and a memory storing computer program instructions; when the processor executes the computer program instructions, any of the above-mentioned traffic regulation methods based on skecth optimization is implemented.

[0036] A computer-readable storage medium having computer program instructions stored thereon, characterized in that when the computer program instructions are executed by a processor, any of the above-mentioned skecth-optimized traffic regulation methods is implemented.

[0037] The technical solution provided by the embodiments of the present disclosure includes at least the following beneficial effects:

[0038] 1) Large-scale concurrent traffic management: For highly concurrent network environments, traditional data structures may have difficulty effectively tracking each flow. Using Sketch, a large number of concurrent flows can be handled in a limited space, ensuring that each flow is subject to appropriate rate limits.

[0039] 2) Dynamic rate control: Since each flow is mapped to an independent counter, independent rate limits can be implemented for each flow. This provides network administrators with greater flexibility to dynamically adjust based on the nature or priority of each flow.

[0040] 3) Reduce resource consumption: By using the sketch data structure, storage space is saved, thereby reducing memory consumption.

[0041] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0043] Figure 1 Schematic diagram of the existing token bucket algorithm.

[0044] Figure 2 Schematic diagram of the flow regulation method based on sketch optimization of the present invention.

[0045] Figure 3 Token bucket schematic diagram of the traffic regulation method based on sketch optimization of the present invention.

[0046] Figure 4 A schematic diagram of a network mapper of a traffic regulation method based on sketch optimization of the present invention. DETAILED DESCRIPTION

[0047] Exemplary embodiments will be described in detail below with reference to the accompanying drawings.

[0048] High-speed network devices (such as routers, switches, or load balancers) need to process a large number of concurrent data flows in a short period of time. Figure 2 As shown, the present invention can help high-speed network devices to manage and control traffic more efficiently and ensure the stability and smoothness of the network by using the skecth counter as the storage container of the token bucket and determining the counter position where the network flow should be stored based on the network mapper.

[0049] 1) Use skecth's counter as the storage container of the token bucket.

[0050] a) The value of each counter actually represents the number of tokens at that location. Figure 3 As shown in Figure 1, when traffic enters and requests a token, the corresponding counter value decreases. When the counter value decreases to 0, it means that the token bucket is empty and the traffic may be restricted or dropped.

[0051] b) Similar to the traditional token bucket algorithm, the counter here also obtains new tokens at a fixed rate, that is, the value of the counter increases at a constant rate. If the traffic is small or there is no traffic consuming tokens, the value of the counter will continue to increase until it reaches the upper limit.

[0052] 2) Network mapper.

[0053] like Figure 4 As shown in Figure 1, the network flow passes through the hash function to determine the counter position where it should be stored. This ensures that the same flow information is stored in the same counter, and different flows have a high probability of having different counter positions.

[0054] Specifically, the flow control method based on sketch optimization of the present invention is implemented through the following steps.

[0055] 1. Initialization.

[0056] a) Set the maximum value of each counter (representing the token bucket).

[0057] b) Initialize all counters to 0 or other initial values.

[0058] c) Define a hash function to map network flows to specific counters.

[0059] 2. Network inflow.

[0060] a) Apply a hash function to the incoming network flow to determine the counter location to which it should be mapped.

[0061] b) Check the value of the corresponding counter (representing the number of tokens).

[0062] 3. Token consumption.

[0063] a) If the counter value is greater than 0, traffic can continue and the counter value is decremented by 1 (consuming one token).

[0064] b) If the counter value is 0, perform corresponding actions according to the policy (for example, limit, drop or delay the traffic).

[0065] 4. Token replenishment.

[0066] a) At fixed time intervals or when triggered by specific events, the values ​​of all counters increase at a fixed rate, indicating that tokens are added to the token bucket.

[0067] b) If the value of a counter has reached the preset maximum value, any additional tokens added will be discarded to ensure that the counter value does not overflow.

[0068] 5. Traffic monitoring and management.

[0069] a) By checking the value of each counter, the network administrator can understand the current status and rate of each flow.

[0070] b) If a flow continues to consume a large number of tokens or often causes the token bucket to be empty, consider adjusting the counter and / or

[0071] Or the implementation strategy of the network mapper, or take other measures. Among them, the implementation strategy includes: changing the hash function corresponding to the network mapper, increasing the maximum value of the counter, or increasing the token replenishment rate.

[0072] 6. Dynamic adjustment.

[0073] According to the characteristics of the network flow or the changes in the network environment, the hash function, the maximum value of the counter or the replenishment rate of the token can be dynamically adjusted.

[0074] In summary, the traditional token bucket algorithm mainly focuses on a single flow or global flow. The present invention uses a sketch structure to track and limit a large number of independent flows, each of which is mapped to an independent counter through a hash function. In addition, since the sketch is a compact data structure, the present invention can track a large amount of flow information in a limited storage space.

[0075] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the present disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof.

Claims

1. A traffic monitoring method based on skecth optimization, characterized in that: The method comprises: Initialize the counter and network mapper using skecth data structure; Based on the network mapper, a network flow is mapped to a corresponding counter, and a value of the corresponding counter is checked; wherein the value of the corresponding counter represents the number of tokens currently owned by the corresponding counter; According to the value of the corresponding counter, perform a corresponding operation on the network flow and update the value of the corresponding counter; Get the current status and rate of each network flow based on the current values ​​of all counters.

2. The method according to claim 1, characterized in that The counter and network mapper initialization using the skecth data structure includes: Set the maximum value of each counter; Initialize the values ​​of all counters to a set value; A hash function is defined, wherein the hash function is used to map different network flows to different counters.

3. The method according to claim 1, characterized in that The performing corresponding operations on the network flow according to the value of the corresponding counter includes: When the value of the corresponding counter is greater than zero, continue the transmission of the network flow and reduce the value of the corresponding counter by 1; When the value of the corresponding counter is zero, a corresponding blocking operation is performed according to a set policy.

4. The method according to claim 3, characterized in that The blocking operation includes: limiting the network flow, discarding the network flow, or delaying the transmission of the network flow.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: At fixed time intervals or when triggered by specific events, the values ​​of all counters increase at a fixed rate until the maximum value of the counter is reached; and, When the value of a counter reaches the maximum value of the counter, the added tokens are discarded to ensure that the value of the counter does not overflow.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: If the network flow continues to consume a large number of tokens or often causes the token bucket to be empty, adjust the implementation strategy of the counter and / or network mapper; wherein the implementation strategy includes: changing the hash function corresponding to the network mapper, increasing the maximum value of the counter, or increasing the token replenishment rate.

7. A traffic monitoring device based on skecth optimization, characterized in that: The device comprises: Initialization module, used to initialize the counter and network mapper using skecth data structure; A traffic mapping module, used for mapping a network flow to a corresponding counter based on the network mapper, and checking the value of the corresponding counter; wherein the value of the corresponding counter represents the number of tokens currently owned by the corresponding counter; A flow execution module, used for executing corresponding operations on the network flow according to the value of the corresponding counter, and updating the value of the corresponding counter; The traffic policing module is used to obtain the current status and rate of each network flow based on the current values ​​of all counters.

8. A computer device, characterized in that: include: a processor, and a memory storing computer program instructions; When the processor executes the computer program instructions, the traffic regulation method based on skecth optimization described in any one of claims 1-6 is implemented.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by the processor, the traffic regulation method based on skecth optimization described in any one of claims 1-6 is implemented.

Images