Differential flow measurement methods, devices, and media based on Sketch data structures

Abstract

This invention discloses a differentiated traffic measurement method, apparatus, and medium based on a Sketch data structure. The method includes: establishing a two-layer Sketch data structure, where the upper-layer storage unit records the stream number, priority, and number of data packets, and the lower-layer storage unit records the number of data packets that failed to be saved in the upper layer; allocating different numbers of hash functions to streams of different priorities in the upper layer according to requirements, while using a fixed number of hash functions in the lower layer; during an update operation, attempting to insert the incoming stream into the upper-layer storage unit, and if the attempt fails, inserting it into the lower-layer storage unit. This invention can improve the measurement accuracy of high-priority streams while consuming less memory resources, reduce the hash overhead when saving low-priority streams, and improve processing speed.

Description

Technical Field

[0001] This invention relates to the technical fields of network data traffic analysis, big data processing, and distributed datasets, and in particular to a differentiated traffic measurement method, device, and medium based on the Sketch data structure. Background Technology

[0002] Network measurement plays a crucial role in many applications, including traffic scheduling, congestion control, load balancing, and anomaly detection. However, with the continuous increase in link capacity and network traffic, it is difficult to record traffic information accurately and in a timely manner using on-chip memory with minimal memory. Therefore, achieving fast and accurate traffic estimation with limited memory is challenging.

[0003] The Sketch data structure has become a typical solution for approximate traffic estimation because it achieves a good trade-off between accuracy, speed, and memory usage. By using a probabilistic data structure consisting of a shared array of counters, Sketch-based solutions significantly reduce memory footprint and have bounded error. However, most Sketch-based solutions tend to focus on designing innovative update strategies or special architectures to reduce detection errors and achieve approximate preservation of traffic information, while neglecting the fact that prioritization in traffic measurement is crucial in some practical scenarios. For example, firewalls need to track flows from suspicious hosts with higher priority to obtain more detailed information, while tracking flows from trusted hosts with lower priority and coarser granularity. Since high-priority flows are fewer in number but carry more important information, they should be monitored and measured more carefully.

[0004] To address this issue, a new type of priority-aware sketch has recently emerged. Its core idea is to divide the sketch into two parts, storing high-priority and low-priority streams separately to achieve differentiated measurement accuracy. While existing priority-aware sketches offer different structures and operations for low-priority and high-priority streams, they face a dilemma in choosing between high accuracy and speed. Specifically, the first part of the MC-Sketch consists of sequential priority tables, with each storage unit recording the stream number, priority, and stream size. Low-priority streams arriving at the switch are repeatedly evicted from the first part of the MC-Sketch because their priority is lower than existing streams in the hash bucket, until they finally reach the second part. Cuckoo Sketch uses a cuckoo hash to store high-priority streams in the first part; however, for a low-priority stream, even if it is initially recorded in the first part, subsequent high-priority streams will repeatedly kick it out in an attempt to find an empty storage unit in the first part to save it. When the number of kickouts reaches a preset kickout threshold, the low-priority stream is finally saved to the second part. In this type of priority-aware Sketch, a large number of low-priority streams need to pass through the first part and are finally stored in the second part. Therefore, the huge computational overhead of hash operations is not negligible, which also leads to lower throughput.

[0005] Therefore, under the uneven distribution of network traffic, how to provide more protection for high-priority flows with limited memory to achieve high accuracy, while maintaining low hash overhead for a large number of low-priority flows to achieve high throughput, in order to achieve differentiated traffic measurement, is a problem worth studying. Summary of the Invention

[0006] To address the problem that existing priority-aware Sketch data structures struggle to achieve a good balance between accuracy and speed, the present invention aims to provide a differentiated traffic measurement method, apparatus, and medium based on the Sketch data structure. This method can assign different numbers of hash functions to incoming flows with different priorities, thereby providing more protection for high-priority flows while reducing the hash overhead required to store low-priority flows and improving processing speed.

[0007] Firstly, a differentiated traffic measurement method based on the Sketch data structure is provided, including:

[0008] Step 1: Establish a two-level Sketch data structure. The higher-level storage unit records the stream number, priority, and number of data packets, while the lower-level storage unit records the number of data packets that failed to be saved in the higher level.

[0009] Step 2: Receive data packets and obtain the flow number and priority of the incoming flow;

[0010] Step 3: At higher levels, dynamically allocate different numbers of hash functions to incoming streams of different priorities. Streams with higher priorities are allocated a larger number of hash functions. At lower levels, use a fixed number of hash functions.

[0011] Step 4: When performing an update operation, determine whether there is a matching flow number in the mapping unit of the incoming flow in the higher layer. If there is, increment the counter of the matching unit by 1, and then go to step 2; otherwise, go to step 5.

[0012] Step 5: Determine whether there is an empty cell in the high-level mapping unit of the incoming flow. If there is at least one empty cell, store the incoming flow in one of the empty cells in the high-level unit. That is, set the flow number of the empty cell to the number of the incoming flow, set the priority to the priority of the incoming flow, set the counter to 1, and then go to step 2; otherwise, go to step 6.

[0013] Step 6: Determine if the priority of the incoming flow is greater than the lowest priority of the incoming flow in the higher-level mapping unit. If so, insert the flow recorded in the lowest-priority mapping unit into the lower level, and then store the incoming flow in that mapping unit. That is, set the flow number of that mapping unit to the number of the incoming flow, set the priority to the priority of the incoming flow, set the counter to 1, and then go to step 2; otherwise, insert the incoming flow into the lower level, and then go to step 2.

[0014] According to the first aspect, in one possible implementation, an increasing linear function is used at a higher level to dynamically allocate the number of hash functions to incoming streams of different priorities. That is, more hash functions are allocated to streams with higher priorities, and fewer hash functions are allocated to streams with lower priorities. This provides more protection for high-priority streams while reducing the hash overhead of low-priority streams.

[0015] According to the first aspect, in one possible implementation, the linear function is expressed as y = kx + b; where x represents the priority of the incoming flow; y represents the number of hash functions allocated to the incoming flow at higher levels; k is a scaling factor, the value of which satisfies the condition that the higher the priority x of the incoming flow, the larger the number of hash functions y allocated to the incoming flow; and b is a constant.

[0016] According to the first aspect, in one possible implementation, when a stream is inserted into a lower layer, a fixed number of hash functions are assigned to it, and the counters of all mapping units of the stream in the lower layer are updated.

[0017] According to the first aspect, in one possible implementation, if an incoming flow is inserted into a lower layer, the counters of all mapping units of the incoming flow in the lower layer are incremented by 1; if the flow recorded by the mapping unit with the lowest priority in the higher layer is inserted into the lower layer, the counters of all mapping units of that flow in the lower layer are incremented by the counter value in the mapping unit with the lowest priority in the higher layer.

[0018] According to the first aspect, in one possible implementation, an initialization operation is included before storing network flow information in the established two-layer Sketch data structure. The initialization operation includes clearing the storage units of the high-level and low-level layers, specifically: setting the counters of all storage units in the low-level layer to 0, setting the flow numbers of all storage units in the high-level layer to null, setting the priority to 0, and setting the counters to 0.

[0019] According to the first aspect, in one possible implementation, a query operation is also included, wherein the query operation is as follows: given the stream number and priority to be queried, the number of hash functions of the stream at the higher level is obtained according to the priority of the stream to be queried, and it is determined whether there is a stream number that matches the incoming stream in the mapping unit at the higher level; if there is, the counter value in the matching unit is reported as the query value; otherwise, the minimum counter value of the stream to be queried in all mapping units at the lower level is reported as the query value.

[0020] Secondly, a differentiated flow measurement device based on the Sketch data structure is provided, including:

[0021] A memory on which computer programs are stored;

[0022] A processor, configured to load and execute the computer program, implement the differentiated traffic measurement method based on the Sketch data structure as described in any of the first aspects above.

[0023] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the differentiated traffic measurement method based on the Sketch data structure described in any of the first aspects above.

[0024] Fourthly, a computer program product is provided that, when the computer program product is run on a computer, causes the computer to execute the differentiated traffic measurement method based on the Sketch data structure described in any of the first aspects above.

[0025] Beneficial effects

[0026] This invention proposes a differentiated traffic measurement method, device, and medium based on the Sketch data structure. It employs a two-layer Sketch structure, assigning different numbers of hash functions to incoming flows with varying priorities in the higher layer. The higher the priority of the incoming flow, the more hash functions are assigned in the higher layer, increasing the chance of retaining high-priority flows and thus improving their measurement accuracy. Simultaneously, a fixed number of hash functions are used in the lower layer, keeping the error of low-priority flows within a reasonable range. Furthermore, since fewer hash functions are assigned to a large number of low-priority flows, memory access and hash overhead are reduced, contributing to increased throughput. This invention can dynamically provide different levels of protection for flows of different priorities using priority information while consuming relatively little memory resources, achieving differentiated measurement accuracy. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of the Sketch data structure provided in an embodiment of the present invention.

[0029] Figure 2 This is a flowchart of the differentiated traffic measurement method based on the Sketch data structure provided in this embodiment of the invention.

[0030] Figure 3 These are schematic diagrams illustrating four different update scenarios of the incoming flow provided in this embodiment of the invention; wherein, (a) is a schematic diagram of the incoming flow being inserted into an empty cell in a higher layer; (b) is a schematic diagram of the incoming flow already existing in a higher layer and its counter being updated; (c) is a schematic diagram of the incoming flow failing to be saved in a higher layer and being inserted into a lower layer; and (d) is a schematic diagram of the incoming flow replacing the flow in the higher layer mapping unit and inserting the flow in the higher layer mapping unit into the lower layer.

[0031] Figure 4 This is a performance comparison chart of the differentiated traffic measurement methods with adaptive allocation and fixed allocation of hash function numbers provided in the embodiments of the present invention; wherein (a) and (b) are respectively schematic diagrams showing the change of the average relative error of high-priority flow with memory growth and the change of the average hash overhead of all flows with memory growth in the differentiated traffic measurement methods with different numbers of hash functions allocated to different priorities and with a fixed number of hash functions allocated to different priorities.

[0032] Figure 5 This is a performance comparison chart of various traffic measurement methods under a random priority allocation strategy using the CAIDA 2018 dataset, provided by an embodiment of the present invention. Among them, (a), (b), (c), and (d) are schematic diagrams showing the changes in the average relative error of high-priority flows with memory growth, the changes in the average relative error of low-priority flows with memory growth, the changes in the average relative error of all flows with memory growth, and the changes in the F1 score of high-priority flows with memory growth, respectively, for various traffic measurement methods that randomly allocate priorities to different flows.

[0033] Figure 6 This is a performance comparison chart of various traffic measurement methods under the priority allocation strategy based on flow size, provided by an embodiment of the present invention, using the CAIDA 2018 dataset; where (a), (b), (c), and (d) are respectively schematic diagrams of the change of average relative error of high-priority flow with memory growth, the change of average relative error of low-priority flow with memory growth, the change of average relative error of all flows with memory growth, and the change of F1 score of high-priority flow with memory growth.

[0034] Figure 7 This is a schematic diagram illustrating the changes in throughput of various traffic measurement methods with memory growth when using the CAIDA 2018 dataset and randomly assigning priorities to different flows, as provided in an embodiment of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0036] This invention provides a differentiated traffic measurement method based on the Sketch data structure. It first establishes a two-layer Sketch data structure, such as... Figure 1 As shown, in this two-layer Sketch data structure, the storage unit at the higher level records the stream number, priority, and number of data packets, while the storage unit at the lower level records the number of data packets that failed to be saved in the higher level.

[0037] At higher levels, different numbers of hash functions are dynamically allocated to incoming streams of different priorities, with higher-priority streams being allocated a larger number of hash functions; at lower levels, a fixed number of hash functions are used.

[0038] When dynamically allocating different numbers of hash functions to incoming streams of different priorities at a higher level, the number of hash functions corresponding to incoming streams of different priorities can be preset. However, when setting the number of hash functions for incoming streams of different priorities, it is necessary to ensure that the higher the priority of the incoming stream, the larger the number of hash functions is allocated. This enables the dynamic allocation of different numbers of hash functions to incoming streams of different priorities at a higher level.

[0039] Of course, in some embodiments, when dynamically allocating different numbers of hash functions to incoming flows of different priorities at higher levels, it is preferable to use an increasing linear function at higher levels to dynamically allocate the number of hash functions to incoming flows of different priorities. That is, more hash functions are allocated to flows with higher priorities, and fewer hash functions are allocated to flows with lower priorities. This provides more protection for high-priority flows while reducing the hash overhead of low-priority flows. Specifically, the linear function can be expressed as y = kx + b; where x represents the priority of the incoming flow; y represents the number of hash functions allocated to the incoming flow at higher levels; k is a proportionality coefficient, the value of which satisfies the condition that the higher the priority x of the incoming flow, the larger the number of hash functions y allocated to the incoming flow. k can take the value of positive integers such as 1, 2, and 3; b is a constant, and can take the value of integers such as 0, 1, and 2; the values ​​of k and b must also satisfy the condition that the minimum value of y is not less than the fixed number of hash functions used at lower levels.

[0040] When performing an update operation, the workflow diagram of the Differentiated Traffic Measurement Method (PA-Sketch) based on the Sketch data structure is as follows: Figure 2 As shown, the specific steps include the following:

[0041] S1: Receive data packets and obtain the flow number f and priority p of the incoming flow;

[0042] S2: In the higher layer, based on the priority p of the incoming flow and the number of hash functions for the incoming flow f, the incoming flow is mapped to the higher layer storage unit;

[0043] S3: Determine if a matching flow number exists in the mapping unit of the incoming flow at the higher level. If it does, increment the counter of that matching unit by 1. Figure 3 If shown in (b), then proceed to step S1; otherwise, proceed to step S4.

[0044] S4: Determine if there is an empty cell in the high-level mapping unit of the incoming flow. If at least one empty cell exists, store the incoming flow in one of the empty cells in the high-level unit. Specifically, set the flow number of the empty cell to the flow number f of the incoming flow, set its priority to the priority p of the incoming flow, and set the counter to 1. Figure 3 If shown in (a), then proceed to step S1; otherwise, proceed to step S5.

[0045] S5: Determine if the priority of the incoming flow is greater than the lowest priority of the incoming flow in the higher-level mapping unit. If so, insert the flow recorded in the lowest-priority mapping unit into the lower level, and then store the incoming flow in that mapping unit. Specifically, set the flow number of that mapping unit to the flow number f of the incoming flow, set its priority to the priority p of the incoming flow, and set the counter to 1. Figure 3 As shown in (d), then proceed to step S1; otherwise, insert the incoming stream into the lower layer, as shown in (d). Figure 3 As shown in (c), then proceed to step S1.

[0046] It should be understood that when a stream is inserted into a lower layer, a fixed number of hash functions are assigned to it, and the counters of all mapping units of that stream in the lower layer are updated. Specifically, if an incoming stream is inserted into a lower layer, the counters of all mapping units of the incoming stream in the lower layer are incremented by 1; if the stream recorded by the lowest priority mapping unit in the higher layer is inserted into the lower layer, the counters of all mapping units of that stream in the lower layer are incremented by the counter value of the lowest priority mapping unit in the higher layer.

[0047] It should also be noted that the method provided in this embodiment includes an initialization operation before storing network flow information in the established two-layer Sketch data structure. The initialization operation includes clearing the storage units of the high-level and low-level layers. Specifically, it involves setting the counters of all storage units in the low-level layer to 0, setting the flow numbers of all storage units in the high-level layer to null, setting the priority to 0, and setting the counters to 0.

[0048] In some embodiments, a query operation is further included, wherein the query operation is as follows: given the flow number and priority to be queried, the number of hash functions of the flow at the higher level is obtained according to the priority of the flow to be queried, and it is determined whether there is a flow number that matches the incoming flow in the mapping unit at the higher level; if there is, the counter value in the matching unit is reported as the query value; otherwise, the minimum counter value of the flow to be queried in all mapping units at the lower level is reported as the query value.

[0049] The differentiated traffic measurement method based on the Sketch data structure provided in the above embodiments designs a two-layer Sketch structure. In the higher layer, different numbers of hash functions are assigned to incoming flows with different priorities. The higher the priority of the incoming flow, the more hash functions are assigned in the higher layer, thereby increasing the chance of retaining high-priority flows and improving the measurement accuracy of high-priority flows. Simultaneously, a fixed number of hash functions are used in the lower layer, keeping the error of low-priority flows within a reasonable range. On the other hand, since fewer hash functions are assigned to a large number of low-priority flows, memory access and hash overhead are reduced, contributing to increased throughput. This invention can dynamically provide different levels of protection for flows of different priorities using priority information while consuming relatively few memory resources, achieving differentiated measurement accuracy.

[0050] This invention also provides a differentiated traffic measurement device based on the Sketch data structure, comprising:

[0051] A memory on which computer programs are stored;

[0052] The processor, when loading and executing the computer program, implements the differentiated traffic measurement method based on the Sketch data structure described in the above embodiments.

[0053] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the differentiated traffic measurement method based on the Sketch data structure described in the above embodiments.

[0054] This invention also provides a computer program product that, when run on a computer, causes the computer to execute the differentiated traffic measurement method based on the Sketch data structure described in the above embodiments.

[0055] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0056] The performance of the present invention will be tested using an experimental simulation to verify its feasibility and advantages.

[0057] Experimental Simulation: Performance testing of this invention was conducted on a server equipped with a six-core CPU, running Ubuntu 18.04.3. Performance testing of this invention was performed on the anonymous internet dataset CAIDA2018, ensuring that the number of flows for each priority follows a Zipf distribution. Where x represents the number of flows, a represents the Zipf distribution skewness (set to 2), and γ represents the number of priorities (set to 8), with priorities ranging from 1 to 8 from low to high. Due to space constraints, the 8 priorities are divided into high-priority and low-priority categories. The 7th and 8th priorities belong to the high-priority category, while the others belong to the low-priority category. In this experiment, the higher layers use a linear function y = x, where x represents the priority of the incoming flow; y represents the number of hash functions assigned to the incoming flow at the higher layers; the lower layers use a fixed number of hash functions, set to 2.

[0058] like Figure 4As shown in the figure, (a) and (b) are schematic diagrams illustrating the change of the average relative error of the high-priority stream with memory growth and the change of the average hash overhead of all streams with memory growth, respectively, for PA-Sketch with different numbers of hash functions assigned to different priorities and PA-Sketch with a fixed number of hash functions assigned to different priorities. Here, PA represents PA-Sketch with different numbers of hash functions assigned to different priorities, and PA-1, PA-4, and PA-8 represent PA-Sketch with fixed numbers of 1, 4, and 8 hash functions assigned to different priorities, respectively. As can be seen from the figure, PA-Sketch (PA) with different numbers of hash functions assigned to different priorities and PA-Sketch (PA-8) with a fixed number of 8 hash functions assigned to different priorities achieve relatively close AREs for the high-priority stream, but PA reduces the hash overhead by an average of 64.4% compared to PA-8.

[0059] like Figure 5 As shown, (a), (b), (c), and (d) are schematic diagrams illustrating the changes in the average relative error of high-priority flows with memory growth, the average relative error of low-priority flows with memory growth, the average relative error of all flows with memory growth, and the F1 score of high-priority flows with memory growth, respectively, when different flows are randomly assigned priorities. Figure 5 As shown in (a), when memory is less than or equal to 150KB, the average relative error of high-priority streams in Cuckoo Sketch and MC-Sketch is significantly higher than that in PA-Sketch. This indicates that when memory is limited, CuckooSketch and MC-Sketch cannot fully utilize available memory space to store information from high-priority streams. For ElasticSketch, the average relative error of high-priority streams is the highest, and there is almost no difference in the average relative error between high-priority and low-priority streams. This is because ElasticSketch does not consider the priority differences of streams, resulting in the same estimation error for high-priority and low-priority streams. Compared to state-of-the-art methods, PA-Sketch reduces the average relative error of high-priority streams by an average of 4.83 times. Figure 5 As shown in (b) and (c), PA-Sketch's average relative error for low-priority streams and the average relative error for all streams are slightly lower than those of Elastic Sketch and MC-Sketch. This indicates that even when low-priority streams dominate, PA-Sketch still successfully reduces the average relative error of high-priority streams without affecting overall performance. Therefore, PA-Sketch maintains the average relative errors of low-priority streams and all streams at similar levels. Figure 5Figure (d) presents the F1 score for the high-priority stream. With 100KB of memory, PA-Sketch achieved an F1 score of 0.88, which is 1.83 times higher than Cuckoo Sketch and MC-Sketch. Note that because Elastic Sketch is a non-priority-based approach and does not record priority information, its F1 score is not measured under a random priority strategy.

[0060] like Figure 6 As shown in the figure, (a), (b), (c), and (d) are respectively schematic diagrams illustrating the changes in the average relative error of high-priority flows with memory growth, the average relative error of low-priority flows with memory growth, the average relative error of all flows with memory growth, and the F1 score of high-priority flows with memory growth, when different flows are prioritized according to their flow size. Note that Elastic Sketch supports the detection of large flows. Therefore, when different flows are prioritized according to their flow size, their F1 scores can be calculated statistically. As can be seen from the figure, PA-Sketch effectively reduces the average relative error of high-priority flows while keeping the average relative error of low-priority flows within an acceptable range.

[0061] like Figure 7 The diagram illustrates how PA-Sketch's throughput changes with memory growth when different stream priorities are randomly assigned. PA-Sketch's throughput is significantly higher than Cuckoo Sketch and MC-Sketch, but slightly lower than Elastic Sketch. Specifically, PA-Sketch achieves a throughput of 9.52 Mpps, which is 1.03 times and 4.35 times higher than MC-Sketch and CuckooSketch, respectively. These results demonstrate that PA-Sketch can effectively reduce hash overhead while providing differentiated services, thereby improving throughput.

[0062] In summary, the differentiated flow measurement method based on the Sketch data structure provided by this invention has superior performance, can provide different levels of protection for flows of different priorities, and achieve differentiated measurement accuracy.

[0063] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0064] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0065] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0066] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0067] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A differentiated traffic measurement method based on the Sketch data structure, characterized in that, include: Step 1: Establish a two-level Sketch data structure. The higher-level storage unit records the stream number, priority, and number of data packets, while the lower-level storage unit records the number of data packets that failed to be saved in the higher level. Step 2: Receive data packets and obtain the flow number and priority of the incoming flow; Step 3: At higher levels, dynamically allocate different numbers of hash functions to incoming streams of different priorities, with higher priority streams being allocated more hash functions; at lower levels, use a fixed number of hash functions. The minimum number of hash functions allocated by the higher layers for incoming streams is not less than the fixed number of hash functions used by the lower layers; Step 4: When performing an update operation, determine whether there is a flow number matching the incoming flow in the high-level mapping unit. If there is, increment the counter of the mapping unit that matches the flow number by 1, and then go to step 2. Otherwise, proceed to step 5; Step 5: Determine whether there is an empty cell in the mapping unit of the incoming flow in the higher layer. If there is at least one empty cell, store the incoming flow in one of the empty cells in the higher layer, and then go to step 2. Otherwise, proceed to step 6; Step 6: Determine whether the priority of the incoming flow is greater than the lowest priority of the incoming flow in the higher-level mapping unit. If so, insert the flow recorded in the mapping unit with the lowest priority into the lower level, then store the incoming flow in that mapping unit, and then go to step 2. Otherwise, insert the incoming stream into the lower layer, and then proceed to step 2; If an incoming flow is inserted into a lower layer, the counters of all mapping units of the incoming flow in the lower layer are incremented by 1; if the flow recorded by the lowest priority mapping unit in the higher layer is inserted into the lower layer, the counters of all mapping units of that flow in the lower layer are incremented by the counter value of the lowest priority mapping unit in the higher layer.

2. The differentiated traffic measurement method based on the Sketch data structure according to claim 1, characterized in that, At higher levels, an incremental linear function is used to dynamically allocate the number of hash functions to incoming streams of different priorities. This means that more hash functions are allocated to streams with higher priorities and fewer hash functions are allocated to streams with lower priorities. This provides more protection for high-priority streams while reducing the hash overhead of low-priority streams.

3. The differentiated traffic measurement method based on the Sketch data structure according to claim 2, characterized in that, The linear function is expressed as: ;in, Indicates the priority of the incoming flow; This indicates the number of hash functions allocated at higher levels for incoming streams; k This is a scaling factor whose value satisfies the priority of the incoming flow. The higher the number of hash functions assigned to the incoming stream, the better. The larger; b It is a constant.

4. The differentiated traffic measurement method based on the Sketch data structure according to claim 1, characterized in that, When a stream is inserted into a lower layer, a fixed number of hash functions are assigned to it, and the counters of all mapping units of that stream in the lower layer are updated.

5. The differentiated traffic measurement method based on the Sketch data structure according to claim 1, characterized in that, Before establishing a two-layer Sketch data structure to store network flow information, an initialization operation is also included, which includes clearing the storage cells of the higher and lower layers.

6. The differentiated traffic measurement method based on the Sketch data structure according to claim 1, characterized in that, It also includes a query operation, which is as follows: given the flow number and priority to be queried, the number of hash functions of the flow at the higher level is obtained according to the priority of the flow to be queried, and it is determined whether there is a flow number that matches the incoming flow in the mapping unit at the higher level; if there is, the counter value in the mapping unit that matches the flow number is reported as the query value; otherwise, the minimum counter value of the flow to be queried in all mapping units at the lower level is reported as the query value.

7. A differentiated flow measurement device based on the Sketch data structure, characterized in that, include: A memory on which computer programs are stored; A processor, configured to load and execute the computer program, implement the differentiated traffic measurement method based on the Sketch data structure as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the differentiated traffic measurement method based on the Sketch data structure as described in any one of claims 1 to 6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the differentiated traffic measurement method based on the Sketch data structure as described in any one of claims 1 to 6.

Images