Traffic policing method, device and equipment based on sketch optimization

Abstract

This invention discloses a traffic policing method, apparatus, and device based on sketch optimization. The method includes: initializing a counter and a network mapper using the sketch data structure; mapping a network flow to a corresponding counter based on the network mapper, and checking the value of the corresponding counter; wherein the value of the corresponding counter represents the number of tokens currently held by the corresponding counter; performing corresponding operations on the network flow according to the value of the corresponding counter, and updating the value of the corresponding counter; and obtaining the current state and rate of each network flow based on the current values ​​of all counters. This invention enables fair and accurate management of each flow while saving significant storage space and computing resources.

Description

Technical Field

[0001] This disclosure relates to the field of traffic monitoring technology, and in particular to a traffic monitoring method, apparatus and device based on sketch optimization. Background Technology

[0002] To achieve higher network quality, traffic policing methods are needed. Traffic policing is a mechanism that measures network traffic and compares it to predetermined policies or rate standards. Traffic exceeding the specified rate is subject to actions such as dropping, tagging, or delaying to ensure that traffic complies with contracts or Service Level Agreements (SLAs). This helps prevent the overuse of network resources and ensures fairness and service quality.

[0003] In existing technologies, the token bucket algorithm is a widely used strategy in network traffic management and control. Its core idea is to control the data transmission rate by simulating the generation and consumption of "tokens." Specifically, such as... Figure 1 As shown, in the token bucket algorithm, there exists a virtual "bucket" that is filled with tokens at a fixed rate. When a data packet or network flow needs to be transmitted, it must obtain the corresponding number of tokens from the bucket. If there are enough tokens in the bucket, the data packet can be transmitted immediately, consuming the corresponding tokens. If there are no tokens in the bucket or insufficient tokens, the data packet may be temporarily delayed, dropped, or marked as low priority until enough tokens are available. In this way, it allows for short bursts of data while limiting the average data transmission rate over long periods. Furthermore, to prevent the unlimited accumulation of tokens, the capacity of the token bucket has an upper limit; tokens exceeding this limit are discarded.

[0004] While existing technologies have some effectiveness in traffic management, they may not be efficient enough in large-scale, high-concurrency network environments and have the following drawbacks:

[0005] ● Scalability issues: Traditional token bucket algorithms are designed for single or global traffic. When it is necessary to track and control a large number of independent flows, configuring and managing a separate token bucket for each flow becomes impractical, leading to a significant waste of resources.

[0006] ● Storage and computation efficiency: For large-scale networks or high-concurrency traffic, maintaining a token bucket or associated state for each active network flow can lead to storage and computation bottlenecks.

[0007] ● Traffic conflicts: In high-concurrency traffic scenarios, multiple flows may compete for tokens in the same token bucket at the same time, which may result in some flows getting more bandwidth while other flows are limited.

[0008] These shortcomings indicate room for improvement in the existing technology, particularly in reducing storage overhead and enhancing scalability. New technological solutions may need to address these issues to provide more efficient data compression and similarity comparison methods. Summary of the Invention

[0009] To address the aforementioned issues, this invention discloses a traffic monitoring method, apparatus, and device based on sketch optimization. This method provides a compact way to track large amounts of independent traffic based on the sketch data structure, mapping each traffic flow to a corresponding counter using a hash function. This design ensures fair and accurate management of each traffic flow while saving significant storage space and computing resources. Sketch is an approximation algorithm that processes large data streams and provides a compact data structure to approximate certain characteristics of the original data. The goal of the Sketch algorithm is to process large-scale data streams with limited memory and computing resources and to quickly and approximately answer queries about the data.

[0010] The technical solution of the present invention includes the following:

[0011] A traffic policing method based on skecth optimization, the method comprising:

[0012] Initialize the counter and network mapper that use the skecth data structure;

[0013] Based on the network mapper, a network flow is mapped to a corresponding counter, and the value of the corresponding counter is checked; wherein, the value of the corresponding counter represents the number of tokens currently held by the corresponding counter;

[0014] Based on the value of the corresponding counter, perform the corresponding operation on the network flow and update the value of the corresponding counter;

[0015] Based on the current values ​​of all counters, obtain the current state and rate of each network flow.

[0016] Furthermore, the initialization of the counter and network mapper using the skecth data structure includes:

[0017] Set the maximum value for each counter;

[0018] Initialize all counter values ​​to a set value;

[0019] Define a hash function that maps different network flows to different counters.

[0020] Further, performing corresponding operations on the network flow based on the value of the corresponding counter includes:

[0021] If the value of the corresponding counter is greater than zero, the transmission of the network stream continues, and the value of the corresponding counter is decremented by 1.

[0022] If the value of the corresponding counter is zero, the corresponding blocking operation is executed according to the set strategy.

[0023] Furthermore, the blocking operation includes: restricting the network flow, discarding the network flow, or delaying the transmission of the network flow.

[0024] Furthermore, the method also includes:

[0025] At fixed time intervals or when triggered by a specific event, the values ​​of all counters increase at a fixed rate until they reach the maximum value of the counter.

[0026] and,

[0027] If the value of a counter reaches its maximum value, any newly added tokens are discarded to ensure that the counter does not overflow.

[0028] Furthermore, the method also includes:

[0029] If the network flow continuously consumes a large number of tokens or frequently leads to token bucket shortages, the implementation strategy of the counter and / or network mapper is adjusted; wherein, the implementation strategy includes: changing the hash function corresponding to the network mapper, increasing the maximum value of the counter, or increasing the token replenishment rate.

[0030] A traffic policing device based on skecth optimization, the device comprising:

[0031] An initialization module is used to initialize counters and network mappers that use the skecth data structure;

[0032] A traffic mapping module is used to map a network flow to a corresponding counter based on the network mapper, and check the value of the corresponding counter; wherein the value of the corresponding counter represents the number of tokens currently held by the corresponding counter;

[0033] The traffic execution module is used to perform corresponding operations on the network flow according to the value of the corresponding counter, and update the value of the corresponding counter.

[0034] The traffic policing module is used to obtain the current status and rate of each network flow based on the current values ​​of all counters.

[0035] A computer device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements any of the above-described skecth-optimized traffic policing methods.

[0036] A computer-readable storage medium storing computer program instructions thereon, characterized in that, when the computer program instructions are executed by a processor, they implement any of the above-described flow policing methods based on skecth optimization.

[0037] The technical solutions provided in this disclosure have at least the following beneficial effects:

[0038] 1) Large-scale concurrent traffic management: In high-concurrency network environments, traditional data structures may struggle to effectively track each flow. Using sketch, a large number of concurrent flows can be processed within a limited space, ensuring that each flow is appropriately rate-limited.

[0039] 2) Dynamic rate control: Because each flow is mapped to an independent counter, independent rate limits can be implemented for each flow. This provides network administrators with greater flexibility to dynamically adjust based on the nature or priority of each flow.

[0040] 3) Reduced resource consumption: By using the sketch data structure, storage space is saved, thereby reducing memory consumption.

[0041] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0043] Figure 1 A schematic diagram of the existing token bucket algorithm.

[0044] Figure 2 A schematic diagram of the traffic monitoring method based on sketch optimization of this invention.

[0045] Figure 3 A schematic diagram of the token bucket in the traffic policing method based on sketch optimization of this invention.

[0046] Figure 4 A schematic diagram of the network mapper for the sketch-optimized traffic policing method of this invention. Detailed Implementation

[0047] The exemplary embodiments will now be described in detail with reference to the accompanying drawings.

[0048] For high-speed network devices (such as routers, switches, or load balancers) that need to process large amounts of concurrent data streams in a short period of time, such as Figure 2 As shown, this invention helps high-speed network devices manage and control traffic more efficiently by using the token bucket's counter as a storage container and determining the counter location where network flows should be stored based on the network mapper, thus ensuring network stability and smoothness.

[0049] 1) Use the skecth counter as the storage container for the token bucket.

[0050] a) The value of each counter actually represents the number of tokens at that position. For example... Figure 3 As shown, when traffic comes in and requests a token, the corresponding counter value decreases. When the counter value decreases to 0, it means that the token bucket is empty, and the traffic may be throttled or dropped.

[0051] b) Similar to the traditional token bucket algorithm, the counter here also receives new tokens at a fixed rate, meaning the counter value increases at a uniform pace. If the traffic is low or no traffic consumes tokens, the counter value will continue to increase until it reaches its limit.

[0052] 2) Network mapper.

[0053] like Figure 4 As shown, network flows are processed by a hash function to determine the counter position where they should be stored. This ensures that identical traffic information is stored in the same counter, and that different traffic flows have a high probability of having different counter positions.

[0054] Specifically, the sketch-optimized traffic monitoring method of this invention is implemented through the following steps.

[0055] 1. Initialization.

[0056] a) Set the maximum value for each counter (representing the token bucket).

[0057] b) Initialize all counters to 0 or other initial values.

[0058] c) Define a hash function to map network flows to a specific counter.

[0059] 2. Network inflow.

[0060] a) Use a hash function on the incoming network flow to determine the counter position it should be mapped to.

[0061] b) Check the value of the corresponding counter (representing the number of tokens).

[0062] 3. Token consumption.

[0063] a) If the counter value is greater than 0, the flow can continue and the counter value will be decremented by 1 (consuming one token).

[0064] b) If the counter value is 0, perform the corresponding action according to the policy (e.g., limit, drop, or delay traffic).

[0065] 4. Token replenishment.

[0066] a) At fixed time intervals or when triggered by a specific event, the values ​​of all counters increase at a fixed rate, indicating that tokens are added to the token bucket.

[0067] b) If the value of a counter has reached the preset maximum value, any newly added tokens will be discarded to ensure that the counter value does not overflow.

[0068] 5. Traffic monitoring and management.

[0069] a) By checking the values ​​of each counter, network administrators can understand the current status and rate of each flow.

[0070] b) If a flow consistently consumes a large number of tokens or frequently results in an empty token bucket, consider adjusting the counter and / or...

[0071] This could involve implementing a different strategy for the network mapper, or taking other measures. Such implementation strategies include: changing the hash function corresponding to the network mapper, increasing the maximum value of the counter, or increasing the token replenishment rate.

[0072] 6. Dynamic adjustment.

[0073] Based on the characteristics of network flow or changes in the network environment, the hash function, the maximum value of the counter, or the token replenishment rate can be dynamically adjusted.

[0074] In summary, while traditional token bucket algorithms primarily focus on single or global traffic flows, this invention uses a sketch structure to track and limit a large number of independent traffic flows, each mapped to an independent counter via a hash function. Furthermore, because sketch is a compact data structure, this invention can track a large amount of traffic information within limited storage space.

[0075] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for traffic policing based on skecth optimization, characterized in that, The method comprises: initializing the counter and the network mapper using the sketch data structure; wherein the network mapper initialization comprises defining a hash function for mapping different network flows to different counters; mapping a network flow to a corresponding counter based on the network mapper and checking the value of the corresponding counter; wherein the value of the corresponding counter represents the number of tokens currently owned by the corresponding counter; performing a corresponding operation on the network flow according to the value of the corresponding counter and updating the value of the corresponding counter; obtaining the current state and rate of each network flow based on the current values of all counters; wherein performing a corresponding operation on the network flow according to the value of the corresponding counter comprises: if the value of the corresponding counter is greater than zero, continuing the transmission of the network flow and reducing the value of the corresponding counter by 1; and if the value of the corresponding counter is zero, performing a corresponding blocking operation according to a set policy, the blocking operation comprising limiting the network flow, discarding the network flow or delaying the transmission of the network flow.

2. The method of claim 1, wherein, The initializing the counter and the network mapper using the sketch data structure comprises: setting the maximum value of each counter; initializing the values of all counters to a set value.

3. The method of any one of claims 1 to 2, wherein, The method further comprises: increasing the values of all counters at a fixed rate until reaching the maximum value of the counter under a fixed time interval or a specific event trigger; and discarding the added tokens to ensure that the value of the counter does not overflow when the value of a counter reaches the maximum value of the counter. The method further comprises:

4. The method according to any one of claims 1 to 2, characterized in that, adjusting the implementation strategy of the counter and / or the network mapper if the network flow continuously consumes a large number of tokens or frequently causes the token bucket to be empty; wherein the implementation strategy comprises changing the hash function corresponding to the network mapper, increasing the maximum value of the counter or increasing the token replenishment rate. The apparatus comprises:

5. A traffic policing device based on skecth optimization, characterized by, an initialization module for initializing the counter and the network mapper using the sketch data structure; wherein the network mapper initialization comprises defining a hash function for mapping different network flows to different counters; a flow mapping module for mapping a network flow to a corresponding counter based on the network mapper and checking the value of the corresponding counter; wherein the value of the corresponding counter represents the number of tokens currently owned by the corresponding counter; a flow execution module for performing a corresponding operation on the network flow according to the value of the corresponding counter and updating the value of the corresponding counter; a flow monitoring module for obtaining the current state and rate of each network flow based on the current values of all counters; wherein performing a corresponding operation on the network flow according to the value of the corresponding counter comprises: if the value of the corresponding counter is greater than zero, continuing the transmission of the network flow and reducing the value of the corresponding counter by 1; and if the value of the corresponding counter is zero, performing a corresponding blocking operation according to a set policy, the blocking operation comprising limiting the network flow, discarding the network flow or delaying the transmission of the network flow. In a case where the value of the corresponding counter is zero, a corresponding blocking operation is performed according to a set policy, the blocking operation including limiting the network flow, discarding the network flow, or delaying transmission of the network flow.

6. A computer device, comprising: The method comprises: a processor, and a memory storing computer program instructions; the processor implements the computer program instructions to realize the method for flow management based on sketch optimization according to any one of claims 1-4.

7. A computer-readable storage medium having stored thereon computer program instructions, wherein, the computer program instructions are executed by the processor to realize the method for flow management based on sketch optimization according to any one of claims 1-4.

Images