A full-flow microsecond-level measurement method and system based on data plane microburst measurement

Abstract

The application discloses a kind of full flow microsecond level measurement method and system based on data plane microburst measurement, the method specifically includes: in data plane, time slot traffic statistics is carried out to data stream, obtains current time slot traffic observation value and last time slot traffic observation value;Based on current time slot traffic observation value and last time slot traffic observation value, calculate first-order difference, obtain flow rate variation;Based on flow rate variation and historical cumulative variation, by logarithmic projection technology executes lightweight chi-square test, calculates burst score;The burst score is compared with preset threshold value, if exceed preset threshold value then generate burst report and report to control surface;In control surface, receive burst report, fill the data between adjacent burst report, reconstruct the continuous flow rate curve of data stream.The application solves the contradiction between communication overhead, hardware adaptability and measurement granularity of existing method, thereby realizing high fidelity, low overhead microsecond level telemetry.

Description

Technical Field

[0001] This invention relates to the field of computer network management and monitoring technology, specifically to a full-flow microsecond-level measurement method and system based on data plane micro-burst measurement. Background Technology

[0002] With the widespread adoption of cloud computing, big data, and high-performance computing, link speeds in modern data center networks have reached 100Gbps or even higher. In high-speed networks, traffic exhibits a high degree of burstiness; microbursts often occur and disappear within microseconds (μs). Although these brief traffic fluctuations are extremely short-lived, they are sufficient to cause packet loss, increase latency, and severely impact the performance of latency-sensitive applications such as RDMA. Therefore, obtaining microsecond-level network visibility is crucial for fault diagnosis, congestion control, and resource scheduling.

[0003] Traditional network monitoring methods are mainly divided into two categories: one is CPU-based periodic polling (such as SNMP) or sampling (such as sFlow / NetFlow). These methods usually operate at millisecond or even second-level granularity, which cannot capture microsecond-level transient events, and sampling will lose a lot of details; the other is sketch-based telemetry methods. Although measurements can be performed on the data plane, in order to obtain fine-grained data, the control plane needs to pull the entire sketch data from the switch at a high frequency (such as every few microseconds).

[0004] However, existing technologies suffer from the following fundamental contradictions:

[0005] 1. Communication bandwidth bottleneck: To achieve microsecond-level monitoring of the entire network, reporting all measurement data to the control plane in real time would consume a huge amount of bandwidth (e.g., up to 100Gbps), which is unacceptable in a production environment.

[0006] 2. Limited Hardware Resources: While programmable switches (such as Intel Tofino) offer high data plane processing speeds, their resources are extremely limited. They do not support floating-point operations or loop operations, and their on-chip memory (SRAM, TCAM) capacity is limited. This makes it difficult to run complex statistical detection algorithms at line speeds.

[0007] 3. Difficulty in maintaining state: In microsecond-level continuous measurement, how to efficiently clear or reset the counter of the previous moment without blocking the pipeline is a tricky engineering problem.

[0008] Existing solutions, such as Umon, attempt to utilize compression techniques like wavelet transform, but often require buffering data for long windows, resulting in detection delays of several milliseconds, which cannot meet the needs of real-time control. Therefore, there is an urgent need for a telemetry method that can capture microsecond-level transient changes, keep communication overhead to an extremely low level, and adapt to the hardware constraints of programmable switches. Summary of the Invention

[0009] The purpose of this invention is to provide a full-flow microsecond-level measurement method and system based on data plane microburst measurement. The microsecond-level full-flow measurement method based on the "Report-on-Burst" concept (hereinafter referred to as the BurstMon method) solves the contradiction between communication overhead, hardware adaptability and measurement granularity in existing methods, thereby achieving high-fidelity and low-overhead microsecond-level telemetry, and solving at least one of the above-mentioned problems in the prior art.

[0010] In a first aspect, the present invention provides a full-flow microsecond-level measurement method based on data plane micro-burst measurement, the method specifically comprising:

[0011] In the data plane of the programmable switch, the time-slotted traffic statistics of the data stream are performed using a time-stamped time sketch data structure to obtain the current time-slot traffic observation value and the previous time-slot traffic observation value.

[0012] The first-order difference between the current time-slot flow observation and the previous time-slot flow observation is calculated on the data plane to obtain the flow rate change.

[0013] Based on the changes in flow rate and historical cumulative changes, a lightweight chi-square test is performed on the data surface using logarithmic projection techniques to calculate the burst score.

[0014] The burst score is compared with a preset threshold. If it exceeds the preset threshold, a burst report containing flow identifier, timestamp and flow observation value is generated and reported to the control plane.

[0015] The control plane receives burst reports and uses a linear interpolation algorithm to fill in the data between adjacent burst reports, reconstructing the continuous flow rate curve of the data stream.

[0016] Secondly, the present invention provides a full-flow microsecond-level measurement system based on data plane micro-burst measurement, the system specifically comprising:

[0017] The time sketch measurement unit is used to perform time-slot traffic statistics on the data stream in the data plane of the programmable switch using a time sketch data structure with timestamps, and to obtain the current time slot traffic observation value and the previous time slot traffic observation value.

[0018] The flow rate differential unit is used to calculate the first-order difference between the current time slot flow observation value and the previous time slot flow observation value on the data plane to obtain the flow rate change.

[0019] The arithmetic approximation calculation unit is used to perform a lightweight chi-square test on the data surface using logarithmic projection techniques based on the change in flow rate and the historical cumulative change, and to calculate the burst score.

[0020] The burst filtering unit is used to compare the burst score with a preset threshold. If the burst score exceeds the preset threshold, a burst report containing flow identifier, timestamp and flow observation value is generated and reported to the control plane.

[0021] The data filling and reconstruction unit is used to receive burst reports at the control plane, fill the data between adjacent burst reports using a linear interpolation algorithm, and reconstruct the continuous flow rate curve of the data stream.

[0022] Thirdly, the present invention provides a network programmable device, comprising: a memory and a processor, and a computer program stored in the memory, wherein when the computer program is executed on the processor, it implements the full-flow microsecond-level measurement method based on data plane microburst measurement as described in any of the above methods.

[0023] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the full-flow microsecond-level measurement method based on data plane micro-burst measurement as described in any of the above methods.

[0024] Compared with the prior art, the present invention has at least one of the following technical effects:

[0025] 1. This invention successfully resolves the contradiction between communication bandwidth, computing resources and measurement accuracy in microsecond-level network telemetry by introducing Time-Sketch, three-sketch rotation mechanism, logarithmic projection-based approximate calculation and sparse reporting strategy into the data plane, providing an efficient, practical and high-fidelity solution for the refined management of modern high-speed networks.

[0026] 2. This invention solves the problem of excessive communication overhead or insufficient accuracy of existing telemetry technologies at the microsecond level, and achieves traffic monitoring with microsecond-level resolution with extremely low communication bandwidth overhead.

[0027] 3. This invention solves the problem that programmable switches cannot support complex floating-point operations and loop operations.

[0028] 4. This invention can accurately capture transient micro-burst events and is suitable for real-time monitoring and fault diagnosis of high-speed data center networks.

[0029] 5. This invention resolves the contradiction between communication overhead, hardware adaptability, and measurement granularity in existing methods, thereby achieving high-fidelity, low-overhead microsecond-level telemetry.

[0030] 6. This invention does not report data at every moment. Instead, it uses statistical methods to intelligently identify the "burst" moments of traffic within the data plane of the switch and only reports these sparse key points. The control plane then reconstructs the complete traffic curve through interpolation. By only reporting statistically significant burst points, the communication bandwidth requirement is reduced by several orders of magnitude (e.g., controlled below 0.1Gbps) while maintaining microsecond-level visibility.

[0031] 7. This invention utilizes the linear processing capability of the data plane to achieve finer-grained flow monitoring at 10µs. The reconstructed curve has a cosine similarity of over 95% with the actual flow, enabling accurate capture of micro-bursts and transient congestion.

[0032] 8. The innovative logarithmic projection technology of this invention enables complex statistical testing algorithms to run on switch chips that only support integer operations for the first time. The Time-Sketch design cleverly avoids the limitation that the hardware does not support cyclic clearing.

[0033] 9. The flow curve reconstructed by the present invention based on sparse points has a cosine similarity of more than 95% with the actual flow, and can accurately restore the occurrence time and intensity of micro-bursts.

[0034] 10. Compared to compression methods that require data caching (such as the millisecond-level delay caused by wavelet transform), this invention employs a packet-by-packet detection mechanism, achieving a detection and reporting latency of only a few hundred nanoseconds for sudden events, supporting real-time closed-loop network control. All detection logic is integrated into the data plane pipeline, processing each data packet at line speed without affecting normal network forwarding performance. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating a full-flow microsecond-level measurement method based on data plane micro-burst measurement, provided by an embodiment of the present invention.

[0037] Figure 2 This is a schematic diagram illustrating the principle of the Time-Sketch data structure and the three-sketch rotation mechanism provided in an embodiment of the present invention;

[0038] Figure 3This is a schematic diagram of an approximate arithmetic pipeline based on logarithmic projection provided in an embodiment of the present invention;

[0039] Figure 4 This is a comparison chart of the errors between logarithmic projection scheme data surface approximation calculation and CPU floating-point arithmetic under different scaling factors provided in an embodiment of the present invention, as well as a schematic diagram of the table overhead of the method of the present invention.

[0040] Figure 5 This is a comparison chart showing the effect of reconstructing flow curves based on sparse reporting points using a control surface according to an embodiment of the present invention;

[0041] Figure 6 This is a schematic diagram comparing the communication overhead of the traditional network telemetry method omniwindow, umon and the method of the present invention, according to an embodiment of the present invention.

[0042] Figure 7 This is a comparison diagram of the flow detection delay between the traditional network telemetry method umon and the method of the present invention, provided in an embodiment of the present invention;

[0043] Figure 8 This is a comparison chart of throughput changes before and after deployment of the method of the present invention on a programmable switch according to an embodiment of the present invention;

[0044] Figure 9 This is a schematic diagram of the structure of a full-flow microsecond-level measurement system based on data plane microburst measurement according to an embodiment of the present invention;

[0045] Figure 10 This is a schematic diagram of a network programmable device provided in an embodiment of the present invention. Detailed Implementation

[0046] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0047] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0048] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0049] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0050] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0051] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0052] In this application embodiment, the entity executing the process includes a terminal device. This terminal device includes, but is not limited to, devices capable of executing the methods disclosed in this application, such as servers, computers, smartphones, and tablets. Figure 1 A flowchart illustrating a full-flow microsecond-level measurement method based on data plane micro-burst measurement according to an embodiment of the present invention is shown below in detail:

[0053] S101, in the data plane of the programmable switch, uses a time-stamped time sketch data structure to perform time-slot traffic statistics on the data flow, and obtains the current time-slot traffic observation value and the previous time-slot traffic observation value.

[0054] In this embodiment, on the data plane of the programmable switch, multiple rotating time-sketch data structures are used to perform time-slot-based traffic statistics on each data stream flowing through the switch, and to obtain the traffic observation value of the current time slot and the traffic observation value of the previous time slot.

[0055] The time sketch is created by adding a timestamp field to each counter cell in the CM-sketch (Count-Min Sketch);

[0056] The update logic of the time sketch is as follows: when a data packet arrives, read the timestamp of the counter unit mapped by the hash; if the unit timestamp is not equal to the current global timeslot index, it is determined that the data of the unit has expired, the counter is reset to the current data packet length and the timestamp is updated; if the unit timestamp is equal to the current global timeslot index, the data packet length is incremented.

[0057] The multiple rotated time sketches comprise three independent time sketch instances: an active sketch, a stable sketch, and an old sketch, managed using a three-sketching rotation mechanism.

[0058] 1) The active sketch is used to count the current time slot. Traffic data;

[0059] 2) The stable sketch is used to save and provide the previous time slot. Complete traffic statistics;

[0060] 3) Old sketches are used to preserve time slots. The data, and upon entering the time slot It is then converted into a new active sketch;

[0061] Through the aforementioned rotation mechanism, the accumulated value of the current time slot can be read simultaneously at any given time. The final value of the previous time slot To avoid write pollution.

[0062] S102, calculate the first-order difference between the current time slot flow observation value and the previous time slot flow observation value on the data plane to obtain the flow rate change.

[0063] In this embodiment, the first-order difference between the current time-slot flow observation and the previous time-slot flow observation is calculated on the data plane to obtain the flow rate change.

[0064] S103, based on the change in flow rate and the historical cumulative change, performs a lightweight chi-square test on the data surface using logarithmic projection technique to calculate the burst score.

[0065] In this embodiment, a lightweight chi-square test is performed on the data plane based on the change in flow rate and the historical cumulative change to calculate the burst score.

[0066] The lightweight chi-square test performed on the data plane is achieved through logarithmic projection techniques, specifically including:

[0067] 1. Construct a logarithmic lookup table (Log Table) and an exponential lookup table (Exp Table) to pre-store the logarithmic and exponential values ​​of integer field values;

[0068] 2. Convert the multiplication, division, and square root operations required for calculating the emergency score into addition, subtraction, and shift operations in the logarithmic field;

[0069] 3. The aforementioned emergency scoring The approximate calculation formula is:

[0070] ;

[0071] in, This represents the current rate of change. This represents the cumulative historical changes. For the current time, This indicates taking the absolute value.

[0072] The hardware implementation process of the logarithmic projection technology is distributed across multiple pipeline stages of the switch:

[0073] Phase 1: Use the Match-Action Table to find the logarithm of the operand in parallel;

[0074] The second stage involves using the Arithmetic Logic Unit (ALU) to perform integer addition and subtraction operations on the logarithmic values ​​obtained from the lookup table.

[0075] The third stage involves using the matching-action table to find the index value of the calculation result and obtain an approximate burst score.

[0076] S104 compares the burst score with a preset threshold. If the burst score exceeds the preset threshold, a burst report containing flow identifier, timestamp, and flow observation value is generated and reported to the control plane.

[0077] In this embodiment, it is determined whether the burst score exceeds a preset threshold. If it does, it is determined that a traffic burst has been detected, and a burst report containing a flow identifier, timestamp, and traffic observation value is generated and sent to the control plane. If it does not exceed the threshold, no report is sent.

[0078] The preset threshold is configurable and is used to balance the sensitivity of burst detection with the communication overhead of uploading to the control plane; the method also includes dynamically adjusting the preset threshold according to the instructions fed back from the control plane.

[0079] S105 receives burst reports in the control plane, uses a linear interpolation algorithm to fill in the data between adjacent burst reports, and reconstructs the continuous flow rate curve of the data stream.

[0080] In this embodiment, the control plane receives the burst report, uses a linear interpolation algorithm to fill the data between adjacent burst reports, and reconstructs the continuous flow rate curve of the data stream.

[0081] The linear interpolation algorithm is as follows:

[0082] Given two consecutive emergency reporting points and ,in For time, For flow rate, and These are the times of the first and second emergency reporting points, respectively. In time flow rate, In time The flow rate; for any point in time Its reconstruction rate The calculation is as follows: .

[0083] As can be seen from steps S101 to S105 above, the system architecture of the full-flow microsecond-level measurement method based on data plane micro-burst measurement disclosed in an embodiment of the present invention mainly consists of two closely cooperating parts: a data plane component deployed on a programmable switch (such as a switch based on an Intel Tofino chip) and a control plane component deployed on a controller. The data plane component is responsible for performing high-frequency, line-speed flow measurement and burst detection tasks, and specifically includes the following four functional modules:

[0084] Flow-level Measurement Module: This module is responsible for performing precise byte-by-byte statistics on each data stream passing through the switch within discrete and continuous microsecond-level time slots (e.g., every 10μs). To meet the processing requirements of high-speed pipelines, this module adopts an innovative Time-Sketch data structure and a three-sketch rotation mechanism, solving the read-write conflicts and state reset problems existing in traditional measurement methods.

[0085] Burst Detection Module: This module receives the current flow rate value output by the measurement module. and the flow rate at the previous moment Calculate the first-order difference of the flow rate ( Subsequently, the module uses a lightweight chi-square test algorithm to assess the degree of deviation of the current rate change from the historical average, i.e., statistical significance.

[0086] Approximate Arithmetic via Lookup Tables: For complex operations such as division and square root involved in the burst detection algorithm that are not supported by the switch hardware, this module uses logarithmic projection technology and pre-built lookup tables to transform the above operations into integer addition and subtraction operations in the logarithmic field, thereby completing the calculation of the burst score under line speed conditions.

[0087] Reporting Logic: This module compares the calculated burst score with a preset threshold. If the score exceeds the threshold, the system determines that a "burst event" has occurred in the flow and generates a digest containing the flow ID, timestamp, and original flow value, which is then reported to the control plane. If the score does not exceed the threshold, the detection result is discarded directly, and reporting is suppressed to save bandwidth.

[0088] The control plane component is responsible for processing sparse reported data and restoring the full picture, and it specifically includes the following two functional modules:

[0089] Data receiving and parsing module: responsible for listening to the summary message channel from the switch, parsing and storing sparse burst data.

[0090] The Reconstruction and Analysis module utilizes a linear interpolation algorithm to reconstruct continuous, high-fidelity microsecond-level flow rate curves from the received discrete burst points. Based on the reconstructed curves, this module further supports advanced network diagnostic tasks such as micro-burst localization and transient congestion root cause analysis.

[0091] Through the above architecture, this invention offloads the heavy traffic filtering task to the data plane and transmits only key information to the control plane, thereby reducing communication overhead by several orders of magnitude while ensuring microsecond-level visibility.

[0092] In some embodiments, step S101 above, which involves using a timestamped time sketch data structure to perform time-slot-based traffic statistics on the data stream and obtain the current time-slot traffic observation value and the previous time-slot traffic observation value, specifically includes:

[0093] Multiple parallel sketch instances with timestamp fields are maintained on the data plane. Each sketch instance's counter unit is associated with a timestamp that records the latest update slot.

[0094] When a data packet arrives, the packet's flow identifier hash is mapped to the corresponding counter cell of each sketch instance, and the timestamp of the counter cell is read and compared with the current global slot index.

[0095] If the timestamp of the counter unit is inconsistent with the current global timeslot index, the data of the counter unit is determined to be expired, the count value of the counter unit is reset to the current data packet length and the timestamp is updated to the current global timeslot index;

[0096] If the timestamp of the counter unit is consistent with the current global timeslot index, then the current data packet length is added to the count value of the counter unit;

[0097] The role of multiple sketch instances is managed through a rotation mechanism, and the current time slot flow observation value and the previous time slot flow observation value are output.

[0098] In this embodiment, a timestamped sketch structure is designed on the switch data plane. Each counting unit contains a traffic counter and a timestamp. When a data packet arrives, the timestamp is used to determine whether the unit belongs to the current microsecond-level time slot. If the timestamp is outdated, the unit is logically cleared (overwritten) and the timestamp is updated; if the timestamp matches, the traffic is accumulated. By introducing a timestamp field into the sketch unit, cross-time slot measurement without full table reset is achieved, solving the problem that P4 language does not support cyclic register reset and ensuring the purity of continuous microsecond-level measurements.

[0099] Precise flow rate measurement at the microsecond level is fundamental to the entire system. Existing technologies using a dual Count-Min Sketch (CMS) alternation scheme struggle to achieve efficient full zeroing under the P4 programming model. Without zeroing, old data contaminates the measurement of new time slots; cyclic zeroing, on the other hand, blocks the packet processing pipeline, leading to packet loss. To address this technical challenge, this embodiment designs a Time-Sketch data structure and a 3-SketchRotation mechanism.

[0100] The Time-Sketch data structure is designed as follows:

[0101] like Figure 2 As shown, this embodiment improves upon the standard Count-Min Sketch. In each hash bucket cell of the Sketch array, in addition to the counter storing traffic statistics, a timestamp field is added. Specifically, each cell in the Time-Sketch... It contains two fields:

[0102] 1. Value Used to accumulate the number of bytes of traffic;

[0103] 2. Timestamp The timestamp records the time slot index where this unit was last updated.

[0104] By introducing timestamps, this invention implements a "logical zeroing" mechanism, which eliminates the need for physical batch reset of the register array and instead determines the timeliness of the data by comparing timestamps.

[0105] The specific measurement steps are as follows:

[0106] Step S201: When a data packet arrives at the switch, its five-tuple information (source IP, destination IP, source port, destination port, protocol number) is parsed as a key, and its index position in the Sketch is calculated using multiple independent hash functions. At the same time, obtain the current global time slot index. and data packet length ;

[0107] Step S202: For each hash-mapped unit The data plane pipeline reads the timestamp currently stored in this unit. and count value ;

[0108] Step S203: Read the cell timestamp Compared with the current global time slot index A comparison is made, and different update logic is executed based on the comparison result:

[0109] Scenario 1 (Data is outdated, overwrite is performed): If This indicates that the cell stores data from the previous or earlier time slot, which is considered "dirty data." In this case, the system performs an overwrite operation, resetting the counter to the current data packet length. and update the timestamp to .Right now: , This is equivalent to clearing and initializing the cell the instant it is accessed, thus avoiding a full table scan;

[0110] Scenario 2 (Fresh data, perform accumulation): If This indicates that the cell has been updated by other data packets within the current time slot, and the data is "fresh." At this point, the system performs an increment operation, updating the counter value to... The timestamp remains unchanged (or is updated to) ).Right now: .

[0111] This mechanism ensures that measurements within any time slot start from a "clean" state, and the operational complexity is O(n log n). It is fully compatible with line-speed processing requirements.

[0112] Furthermore, the sketch instances include active sketches, stable sketches, and stale sketches; the management of multiple sketch instances' roles through the three-sket rotation mechanism, outputting the current time-slot flow observation value and the previous time-slot flow observation value, specifically includes:

[0113] Assign the sketch instance corresponding to the modulo result of the current global time slot index as an active sketch, and accumulate and count the traffic data of the current time slot through the active sketch;

[0114] Assign the sketch instance corresponding to the modulo result of the current global slot index minus one as a stable sketch, and save and provide the traffic data of the previous slot through the stable sketch;

[0115] Assign the sketch instance corresponding to the modulo result of the current global time slot index minus two as an old sketch, and save the traffic data of the time slot after the current global time slot index minus two through the old sketch;

[0116] Within any of the aforementioned time slots, the current cumulative count value of the data stream is read from the active sketch as the current time slot flow observation value, and the historical count value of the data stream is read from the stable sketch as the previous time slot flow observation value.

[0117] When the current global time slot index increments, the roles of sketch instances are reassigned according to the modulo rule, so that the original stale sketches are cleared and converted into new active sketches, the original active sketches are converted into stable sketches, and the original stable sketches are converted into stale sketches.

[0118] In this embodiment, in order to calculate the rate change of traffic, it is necessary to simultaneously access the traffic at the current moment ( ) and the flow rate at the previous time step ( This invention introduces a three-sketch rotation mechanism (active, stable, and stale), which rotates with the time slot. The current time slot is filled with the active sketch, while the stable sketch is read as a historical reference, enabling continuous and uninterrupted measurement.

[0119] Specifically, in order to calculate changes in traffic rate, the burst detection algorithm needs to simultaneously access the "cumulative traffic of the current time slot" ( ")" and "Final flow of the previous time slot ()" A single Sketch cannot simultaneously meet the needs of writing current data and reading historical static data. Therefore, this embodiment introduces a three-sketch rotation mechanism.

[0120] like Figure 2 As shown, the system maintains three independent Time-Sketch instances on the data plane, denoted as follows: The system uses the current time slot index. Perform a modulo operation on 3 to dynamically assign roles to the three Sketch objects:

[0121] Active Sketch ): In write mode. Current time slot. All data packets arriving within the sketch will be written to this sketch. Its internal values ​​represent... Real-time cumulative results.

[0122] Stable Sketch ): In read-only mode. This Sketch saves the previous time slot. Complete statistics. (In the time slot) During this period, no more data is written to this Sketch, and its value is used as... The authoritative measurement values ​​are provided for the detection module to read.

[0123] Stale Sketch ): In read-only mode. This Sketch saves data from two time slots ago. It will be available in the next time slot. When it arrives, it transforms into a new activity sketch. At that time, using the aforementioned Time-Sketch timestamp mechanism, its outdated data will be logically cleared.

[0124] The specific processing flow is as follows: In the time slot Within, for an arriving data packet, if the current timestamp matches the activity sketch... If the timestamps recorded in the data do not match, a burst detection is performed, and the following sequence of atomic operations is executed on the data plane:

[0125] Query Based on the hash index, from the activity sketch Read the current traffic estimate for this stream.

[0126] Query Based on the same hash index, from the stable sketch Read the final flow value of the stream in the previous time slot.

[0127] Pass to pipeline: The read data and It is then passed to the subsequent ALU (Arithmetic Logic Unit) for differential calculation.

[0128] Update event sketches: Update Following step S203 of the Time-Sketch above, update the length of the current data packet to the new active sketch. middle.

[0129] The advantages of this rotation mechanism are: it completely avoids the poorly supported batch register reset operation in the P4 language; and it achieves strict time isolation, ensuring... The measurements will not be contaminated by new writes; continuous and uninterrupted monitoring is achieved without pausing measurements between time slots.

[0130] In some embodiments, step S102 above, which involves calculating a first-order difference on the data plane based on the current time-slot flow observation and the previous time-slot flow observation to obtain the flow rate change, specifically includes:

[0131] In the arithmetic logic unit of the data plane, read the flow observation value of the previous time slot and the flow observation value of the current time slot;

[0132] The difference between the current time slot flow observation and the previous time slot flow observation is calculated in the arithmetic logic unit, and the absolute value of the difference is taken to obtain the change in flow rate.

[0133] The change in flow rate is updated in the historical cumulative change register corresponding to the data flow. The historical cumulative change register is used to provide historical cumulative change when calculating burst scores.

[0134] In this embodiment, for each arriving data packet, the data plane queries the current slot traffic in the time sketch based on the flow identifier (such as a 5-tuple hash). and the flow rate of the previous time slot And calculate the first-order difference. This difference value reflects the instantaneous change in flow rate.

[0135] In some embodiments, step S103 above, which involves performing a lightweight chi-square test on the data surface using logarithmic projection techniques based on the change in flow rate and historical cumulative change, to calculate the burst score, specifically includes:

[0136] Multiply the change in flow rate by the current time slot number to obtain the first product;

[0137] Subtract the first product from the historical cumulative change and take the absolute value to obtain the numerator;

[0138] Multiply the historical cumulative change by the result of subtracting one from the current time slot number to obtain the second product;

[0139] Calculate the square root of the second product to obtain the denominator term;

[0140] Divide the numerator by the denominator to obtain the normalized burst score;

[0141] The normalized burst score is statistically tested using logarithmic projection on the data surface to obtain an approximate burst score under hardware constraints.

[0142] In this embodiment, based on first-order difference and historical cumulative change A lightweight chi-square test is performed on the data plane to calculate burst scores. To overcome the limitation that switches do not support complex operations, this invention introduces logarithmic projection technology. Through pre-computed logarithm and exponent tables, the multiplication, division, and square root operations in the chi-square test formula are converted into integer addition, subtraction, and shift operations, making them compatible with the arithmetic logic unit (ALU) of the switch.

[0143] After obtaining accurate and Subsequently, this embodiment performs lightweight statistical detection within the data plane to identify abnormal changes in traffic rate.

[0144] 1. Detection algorithm principle:

[0145] This invention employs a statistical method based on the Chi-Square Test. Compared to methods such as the KS test or the AD test, the Chi-Square Test is nonparametric, has relatively low computational complexity, and is suitable for operation with limited computational resources and data availability. This embodiment utilizes this test to determine the latest rate change. Is it a statistically significant outlier relative to the historical average rate of change?

[0146] definition To the end of the time slot The cumulative sum of absolute differences:

[0147] ;

[0148] Where k represents the index variable for summation, used to iterate from 1 to... All points in time, Indicates a point in time The rate on, Indicates a point in time The rate on.

[0149] The chi-square statistic is used to quantify the current observation value. relative to historical average The degree of deviation. After simplification, the formula for the normalized burst score used in data surface calculation is as follows:

[0150] ;

[0151] in:

[0152] Normalized Burst Score: This quantifies the degree of deviation of the current observation from the historical cumulative change.

[0153] Chi-square statistic, used to quantify the current observation value. relative to historical average The degree of deviation;

[0154] : The rate of change of the current time slot (absolute value of the first-order difference);

[0155] : The current time slot number (indicating the duration of the stream);

[0156] The sum of historical cumulative changes;

[0157] When calculated The value exceeds the preset threshold. At that time, the system determined that an emergency had occurred.

[0158] 2. Status maintenance:

[0159] To calculate the above formula, the data plane needs to maintain a small amount of state information for each flow. In addition to the flow values ​​stored in Sketch, the flow information also needs to be maintained in the register array. (Historical cumulative difference sum). Since this invention focuses on "change," it utilizes the properties of first-order differences to make state maintenance very lightweight, eliminating the need to store lengthy historical sequences; only... The space complexity is [not specified]. Specifically, for each flow, the data plane maintains a CM-sketch and updates [the data]. , Indicates the previous time slot The sum of historical cumulative changes.

[0160] Furthermore, the step of statistically testing the normalized burst score on the data surface using logarithmic projection techniques to obtain an approximate burst score under hardware constraints specifically includes:

[0161] In the data plane, a logarithmic lookup table and an exponential lookup table are pre-stored. The logarithmic lookup table is used to store the logarithmic values ​​corresponding to the integer field values, and the exponential lookup table is used to store the approximate exponential results corresponding to the logarithmic values.

[0162] Using a logarithmic lookup table, the numerator containing the change in flow rate, the current time slot number, and the historical cumulative change is decomposed and searched to obtain the corresponding first logarithmic value;

[0163] Using a logarithmic lookup table, the denominator of the result containing the historical cumulative change and the current time slot number minus one is parametrically split and searched to obtain the corresponding second logarithmic value;

[0164] In the arithmetic logic unit of the data plane, a subtraction operation is performed based on the first logarithm and the second logarithm to obtain the result logarithm;

[0165] Using the logarithmic result as an index, the index lookup table is queried to obtain an approximate burst score.

[0166] In this embodiment, the above-mentioned burst score formula ( It includes multiplication, division, and square root operations. However, existing programmable switches (such as Intel Tofino) only support simple integer addition and subtraction, bitwise operations, and a limited number of multiplications, and do not support division and square root operations. In order to implement the above statistical tests under hardware constraints, this embodiment proposes a logarithmic projection technique.

[0167] 1. Core Principle:

[0168] This technique utilizes the logarithmic property to transform complex multiplication and division operations into addition and subtraction operations, and square root operations into shift operations. x and y can represent the operands in the specific calculation process of the burst score formula. By simplifying the calculation through logarithmic projection, the efficiency of burst score calculation is improved, enabling the system to make burst judgments faster.

[0169] Multiplication Conversion: ;

[0170] Division conversion: ;

[0171] 2. Hardware pipeline implementation steps:

[0172] like Figure 3 As shown, this embodiment decomposes the calculation of burst score into a three-stage pipeline operation, and uses preset lookup tables to realize nonlinear transformation.

[0173] Step S301: Preprocessing and Entry Distribution:

[0174] Pre-calculation of the control surface Function table and (Exponential) function tables are generated and distributed as Match-Action Tables to the switch's SRAM. To balance accuracy and storage resources, the table size selected in this embodiment is [size to be specified]. This (corresponding to 16-bit precision) is entirely feasible within the memory budget of a Tofino switch. Meanwhile, a scaling factor can be used to improve precision. The numerical values ​​are amplified.

[0175] Step S302: Pipeline Stage 1 - Log Lookup:

[0176] In the arithmetic logic unit of the data plane, prepare the operands to participate in the operation, such as the numerator in the formula. and the terms in the denominator .because It may exceed the lookup table range; it can be split into... and Process them separately. Use the operand as the key to search. The table retrieves the corresponding logarithmic value.

[0177] For example: , , Representing the numerator of the formula The value after logarithmic transformation express The value after logarithmic transformation express The value after logarithmic transformation This represents a logarithmic index table.

[0178] Step S303: Pipeline Stage 2 - Integer Arithmetic performs addition and subtraction operations in the logarithmic field.

[0179] For the denominator in the formula We perform exponentiation within the exponent table to obtain... After approximate calculation, the result is further square rooted to obtain the final result.

[0180] The division of the entire formula is calculated as follows: , This represents the value of the denominator in the formula after logarithmic transformation.

[0181] This stage only involves integer addition, subtraction, and bit shift operations of the ALU, which is fully in line with the hardware capabilities of the switch.

[0182] Step S304: Pipeline Stage 3 – Exp Lookup:

[0183] The logarithmic field results obtained in stage 2 Use the key to look up the index table: The result is an approximate burst score. This represents the index table.

[0184] In some embodiments, in step S103 above, the method further includes:

[0185] The data burst score is obtained by acquiring a preset number of samples on the control plane and calculating it on the data plane, and the CPU burst score is obtained by acquiring the same samples on the CPU based on the chi-square test formula.

[0186] Based on the burst score of the data surface and the burst score of the CPU, the mean square error of all samples is calculated. The mean square error is used to characterize the overall numerical deviation of the approximate calculation.

[0187] Based on data surface burst scoring and CPU burst scoring, the accuracy score of all samples is calculated by comparing the relative error between the two and applying an exponential decay function. The accuracy score is used to characterize the sensitivity and fidelity of approximate calculation to relative error.

[0188] Based on the calculation results of mean squared error and accuracy scores, evaluate and adjust the configuration parameters of logarithmic lookup table and exponential lookup table.

[0189] In this embodiment, mean squared error (MSE) and an accuracy score sensitive to relative error are used to quantify the error. Given There are 3 samples, and their definitions are as follows:

[0190] ;

[0191] ;

[0192] in, and These are scores from the CPU and data aspects, respectively. This represents the mean square error. This represents the accuracy, and N represents the given number of samples.

[0193] Experimental results are as follows Figure 4 This indicates that by setting an appropriate scaling factor (e.g., scaling factor) The numerical accuracy of this approximation method exceeds 99.4%, and the mean square error (MSE) is... The result is less than 0.01. This proves that this method is a high-fidelity alternative to native floating-point operations, making complex statistical calculations possible on the data surface.

[0194] In some embodiments, step S104 above, which compares the burst score with a preset threshold, and if the score exceeds the preset threshold, generates a burst report containing a flow identifier, timestamp, and flow observations and reports it to the control plane, specifically includes:

[0195] In the data plane, the burst score is compared with the preset detection threshold to obtain the threshold comparison result;

[0196] If the threshold comparison result shows that the burst score exceeds the detection threshold, it is determined that a statistically significant burst has occurred in the current data stream, triggering the report generation process;

[0197] The trigger report generation process includes:

[0198] Extract the current time slot flow observation from the active sketch, and extract the previous time slot flow observation from the stable sketch;

[0199] By combining the flow identifier of the data stream, the timestamp corresponding to the current time slot, the current time slot traffic observation value, and the previous time slot traffic observation value, a burst report is generated.

[0200] The burst report is uploaded to the control plane through the standard summary mechanism or mirroring mechanism built into the programmable switch.

[0201] In this embodiment, the calculated burst score is compared with a preset threshold. Only when the score exceeds the threshold does the switch generate a compact summary report containing the flow ID, timestamp, and traffic value and send it to the control plane; otherwise, the data point is ignored. This mechanism leverages the sparsity of network traffic in the time domain (i.e., it is stationary most of the time), greatly reducing the transmission of redundant data.

[0202] After the data plane completes the calculation of the burst score, the reporting decision module executes the following logic:

[0203] The calculated With the preset detection threshold Compare;

[0204] like The flow has been determined to have experienced a statistically significant burst in the current time slot. The data plane immediately generates a digest containing the following fields: Flow ID (e.g., a 5-tuple hash), current timestamp, and current flow value. ) and the flow rate value at the previous moment ( The summary is pushed to the control plane;

[0205] like The current traffic fluctuation is determined to be within the normal range and not statistically significant. The data flow will discard this detection result without generating any reported data.

[0206] This "burst-triggered reporting" mechanism forms the core of BurstMon's efficiency. This is because network traffic is stable most of the time (i.e.,...). Burst scores are typically very low and do not require reporting most of the time. The system only consumes bandwidth to send summaries at "critical points" where traffic fluctuates drastically. Compared to periodically polling the entire Sketch, this strategy reduces communication overhead by several orders of magnitude.

[0207] In some embodiments, step S105 above, which involves receiving burst reports in the control plane, filling the data between adjacent burst reports using a linear interpolation algorithm, and reconstructing the continuous flow rate curve of the data stream, specifically includes:

[0208] The control plane receives burst reports from the data plane through a summary message channel, parses and extracts the flow identifier, timestamp and corresponding flow observation value contained therein;

[0209] The burst reports belonging to the same data stream are sorted according to their timestamps to form a time series that includes multiple discrete burst report points;

[0210] For any two consecutive burst reporting points in time, calculate the slope of the average rate change between the two points based on their respective timestamps and flow observations;

[0211] Using the slope of the average rate change, the estimated flow rate values ​​at each time point between two burst reporting points are calculated and filled in using a linear interpolation formula;

[0212] Iterate through all adjacent burst report points in the data stream and perform a fill operation, then connect the points and the fill points to generate a continuous flow rate curve for the data stream.

[0213] In this embodiment, after the control plane receives the sparse reported sequence, it uses a linear interpolation algorithm to connect each burst point and reconstruct a high-fidelity flow rate curve for subsequent microburst analysis or root cause localization.

[0214] The control plane receives a sparse digest stream from the data plane. The digest contains a series of discrete bursty event points, such as... , indicating time A sudden event occurred, and the rate was... ,Right now This represents the time point when the k-th emergency occurred. Indicates time The rate at which a sudden event occurs.

[0215] This embodiment uses a linear interpolation algorithm on the control plane to reconstruct the complete flow curve. For any two consecutive reporting points... , and These are the times corresponding to any two consecutive reporting points. and They are in time and The reported rate, at any intermediate time rate The estimate is:

[0216] ;

[0217] like Figure 5 This demonstration showcases the reconstruction results of this method on a single flow in Hadoop 15 and WebSearch 25. Despite its simplicity, the method captures all key inflection points (turning points) of traffic changes, resulting in a reconstructed curve that highly matches the actual traffic curve. At a resolution of [resolution value], the cosine similarity between the reconstructed curve and the real curve exceeds 0.95.

[0218] Based on the reconstructed curves, the control surface can be used to perform further advanced analyses, such as:

[0219] Micro-burst localization: Identifying which flows simultaneously experience rate spikes within the same microsecond-level time window, leading to backlog in the switch queue;

[0220] Congestion root cause analysis: Combining topology information, we analyze the path of burst flows to pinpoint the source of network congestion.

[0221] Performance evaluation and analysis of beneficial effects:

[0222] To verify the effectiveness of the invention, a BurstMon prototype was deployed on a hardware testbed based on an Intel Tofino switch and evaluated using real data center workloads (Hadoop 15 and WebSearch 25).

[0223] 1. Reconstruction accuracy assessment:

[0224] With existing Compared to state-of-the-art systems such as those using wavelet compression and OmniWindow (using sliding window aggregation), BurstMon achieves the highest reconstruction accuracy with the same memory budget (256 KB). Figure 5 As shown, BurstMon can faithfully reproduce the shape and amplitude of microsecond-level bursts, while baseline methods often smooth out bursts or introduce distortion.

[0225] 2. Communication overhead and latency assessment:

[0226] like Figure 6 and Figure 7As shown, thanks to the sparse reporting mechanism, BurstMon's uplink communication overhead is extremely low and stable, controlled between 0.05 and 0.07 Gbps, which accounts for only a tiny fraction (less than 0.1%) of the 100Gbps link bandwidth. In terms of latency, BurstMon's burst detection is instantaneous, with a response latency of only about 340 ns.

[0227] 3. Hardware resource consumption assessment:

[0228] This invention is highly hardware-friendly. Deployment data on a Tofino switch (as shown in Table 1) shows that BurstMon only consumes approximately 26% of SRAM resources and does not consume any scarce TCAM resources. The entire logic only occupies 12 pipeline stages (out of a total of 24 stages), meaning it can coexist with other network functions.

[0229]

[0230] In addition, such as Figure 8 As shown, enabling BurstMon has no measurable negative impact on the switch's line-rate forwarding throughput, demonstrating its feasibility for deployment in a production environment.

[0231] Reference Figure 9 An embodiment of the present invention provides a full-flow microsecond-level measurement system 9 based on data plane microburst measurement. The full-flow microsecond-level measurement system 9 based on data plane microburst measurement specifically includes:

[0232] The time sketch measurement unit 901 is used to perform time-slot traffic statistics on the data stream in the data plane of the programmable switch using a time sketch data structure with timestamps, and to obtain the current time slot traffic observation value and the previous time slot traffic observation value.

[0233] The flow rate differential unit 902 is used to calculate the first-order difference in the data plane based on the current time slot flow observation value and the previous time slot flow observation value to obtain the flow rate change.

[0234] Arithmetic approximation calculation unit 903 is used to perform a lightweight chi-square test on the data surface using logarithmic projection techniques based on the change in flow rate and the historical cumulative change to calculate the burst score.

[0235] The burst filtering unit 904 is used to compare the burst score with a preset threshold. If the burst score exceeds the preset threshold, a burst report containing flow identifier, timestamp and flow observation value is generated and reported to the control plane.

[0236] The data filling and reconstruction unit 905 is used to receive burst reports in the control plane, fill the data between adjacent burst reports using a linear interpolation algorithm, and reconstruct the continuous flow rate curve of the data stream.

[0237] It is understandable that, such as Figure 1 The content of the embodiments of the full-flow microsecond-level measurement method based on data plane microburst measurement shown herein is applicable to the embodiments of the full-flow microsecond-level measurement system based on data plane microburst measurement. The specific functions implemented by the embodiments of the full-flow microsecond-level measurement system based on data plane microburst measurement are as follows: Figure 1 The embodiment of the full-flow microsecond-level measurement method based on data plane microburst measurement shown is the same, and the beneficial effects achieved are the same as those described above. Figure 1 The beneficial effects achieved by the embodiment of the full-flow microsecond-level measurement method based on data plane microburst measurement shown are also the same.

[0238] It should be noted that the information interaction and execution process between the above systems are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0239] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0240] Reference Figure 10 The present invention also provides a network programmable device 10, including a memory 1002 and a processor 1001, and a computer program 1003 stored in the memory 1002. When the computer program 1003 is executed on the processor 1001, it implements the full-flow microsecond-level measurement method based on data plane micro-burst measurement as described in any of the above methods.

[0241] The network programmable device 10 can be a smart network interface card (NIC), a programmable switch, an FPGA NIC, or other network devices. The network programmable device 10 may include, but is not limited to, a processor 1001 and a memory 1002. Those skilled in the art will understand that... Figure 10 This is merely an example of a network programmable device and does not constitute a limitation on the network programmable device 10. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0242] The processor 1001 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0243] In some embodiments, the memory 1002 may be an internal storage unit of the network programmable device 10, such as a hard disk or memory of the network programmable device 10. In other embodiments, the memory 1002 may be an external storage device of the network programmable device 10, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the network programmable device 10. Furthermore, the memory 1002 may include both internal and external storage units of the network programmable device 10. The memory 1002 is used to store operating systems, applications, bootloaders, data, and other programs, such as the program code of computer programs. The memory 1002 can also be used to temporarily store data that has been output or will be output.

[0244] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the full-flow microsecond-level measurement method based on data plane micro-burst measurement as described in any of the above methods.

[0245] In this embodiment, if the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0246] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0247] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0248] In the embodiments disclosed in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0249] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

Claims

1. A full-flow microsecond-level measurement method based on data plane micro-burst measurement, characterized in that, The method specifically includes: In the data plane of the programmable switch, the time-slotted traffic statistics of the data stream are performed using a time-stamped time sketch data structure to obtain the current time-slot traffic observation value and the previous time-slot traffic observation value. The first-order difference is calculated on the data plane based on the current time slot flow observation value and the previous time slot flow observation value to obtain the flow rate change. Based on the changes in flow rate and historical cumulative changes, a lightweight chi-square test is performed on the data surface using logarithmic projection techniques to calculate the burst score. The burst score is compared with a preset threshold. If it exceeds the preset threshold, a burst report containing flow identifier, timestamp and flow observation value is generated and reported to the control plane. The control plane receives burst reports and uses a linear interpolation algorithm to fill in the data between adjacent burst reports, reconstructing the continuous flow rate curve of the data stream. Specifically, the step of using a timestamped time sketch data structure to perform time-slot-based traffic statistics on the data stream and obtain the current time-slot traffic observation value and the previous time-slot traffic observation value includes: Multiple parallel sketch instances with timestamp fields are maintained on the data plane. Each sketch instance's counter unit is associated with a timestamp that records the latest update slot. When a data packet arrives, the packet's flow identifier hash is mapped to the corresponding counter cell of each sketch instance, and the timestamp of the counter cell is read and compared with the current global slot index. If the timestamp of the counter unit is inconsistent with the current global timeslot index, the data of the counter unit is determined to be expired, the count value of the counter unit is reset to the current data packet length and the timestamp is updated to the current global timeslot index; If the timestamp of the counter unit is consistent with the current global timeslot index, then the current data packet length is added to the count value of the counter unit; The role of multiple sketch instances is managed through a three-sket rotation mechanism, and the current time slot flow observation value and the previous time slot flow observation value are output. The sketch instances include active sketches, stable sketches, and stale sketches; the management of multiple sketch instances' roles through a three-sket rotation mechanism, outputting the current time-slot flow observation value and the previous time-slot flow observation value, specifically includes: Assign the sketch instance corresponding to the modulo result of the current global time slot index as an active sketch, and accumulate and count the traffic data of the current time slot through the active sketch; Assign the sketch instance corresponding to the modulo result of the current global slot index minus one as a stable sketch, and save and provide the traffic data of the previous slot through the stable sketch; Assign the sketch instance corresponding to the modulo result of the current global time slot index minus two as an old sketch, and save the traffic data of the current global time slot index minus two through the old sketch; Within any of the aforementioned time slots, the current cumulative count value of the data stream is read from the active sketch as the current time slot flow observation value, and the historical count value of the data stream is read from the stable sketch as the previous time slot flow observation value. When the current global time slot index increments, the roles of sketch instances are reassigned according to the modulo rule, so that the original stale sketches are cleared and converted into new active sketches, the original active sketches are converted into stable sketches, and the original stable sketches are converted into stale sketches.

2. The method according to claim 1, characterized in that, The step of calculating the first-order difference between the current time-slot flow observation and the previous time-slot flow observation on the data plane to obtain the flow rate change specifically includes: In the arithmetic logic unit of the data plane, read the flow observation value of the previous time slot and the flow observation value of the current time slot; The difference between the current time slot flow observation and the previous time slot flow observation is calculated in the arithmetic logic unit, and the absolute value of the difference is taken to obtain the change in flow rate. The change in flow rate is updated in the historical cumulative change register corresponding to the data flow. The historical cumulative change register is used to provide historical cumulative change when calculating burst scores.

3. The method according to claim 2, characterized in that, Based on the changes in flow rate and historical cumulative changes, a lightweight chi-square test is performed on the data surface using logarithmic projection techniques to calculate the burst score, specifically including: Multiply the change in flow rate by the current time slot number to obtain the first product; Subtract the first product from the historical cumulative change and take the absolute value to obtain the numerator; Multiply the historical cumulative change by the result of subtracting one from the current time slot number to obtain the second product; Calculate the square root of the second product to obtain the denominator term; Divide the numerator by the denominator to obtain the normalized burst score; The normalized burst score is statistically tested using logarithmic projection techniques on the data surface to obtain an approximate burst score under hardware constraints.

4. The method according to claim 3, characterized in that, The step of statistically testing the normalized burst score on the data surface using logarithmic projection techniques to obtain an approximate burst score under hardware constraints specifically includes: In the data plane, a logarithmic lookup table and an exponential lookup table are pre-stored. The logarithmic lookup table is used to store the logarithmic values ​​corresponding to the integer field values, and the exponential lookup table is used to store the approximate exponential results corresponding to the logarithmic values. Using a logarithmic lookup table, the numerator containing the change in flow rate, the current time slot number, and the historical cumulative change is decomposed and searched to obtain the corresponding first logarithmic value; Using a logarithmic lookup table, the denominator of the result containing the historical cumulative change and the current time slot number minus one is parametrically split and searched to obtain the corresponding second logarithmic value; In the arithmetic logic unit of the data plane, a subtraction operation is performed based on the first logarithm and the second logarithm to obtain the result logarithm; Using the logarithmic result as an index, the index lookup table is queried to obtain an approximate burst score.

5. The method according to claim 4, characterized in that, The method further includes: The data burst score is obtained by acquiring a preset number of samples on the control plane and calculating it on the data plane, and the CPU burst score is obtained by acquiring the same samples on the CPU based on the chi-square test formula. Based on the burst score of the data surface and the burst score of the CPU, the mean square error of all samples is calculated. The mean square error is used to characterize the overall numerical deviation of the approximate calculation. Based on data surface burst scoring and CPU burst scoring, the accuracy score of all samples is calculated by comparing the relative error between the two and applying an exponential decay function. The accuracy score is used to characterize the sensitivity and fidelity of approximate calculation to relative error. Based on the calculation results of mean squared error and accuracy scores, evaluate and adjust the configuration parameters of logarithmic lookup table and exponential lookup table.

6. The method according to claim 1, characterized in that, The process of comparing the burst score with a preset threshold, and generating a burst report containing a flow identifier, timestamp, and traffic observations if the score exceeds the preset threshold, and then reporting it to the control plane, specifically includes: In the data plane, the burst score is compared with the preset detection threshold to obtain the threshold comparison result; If the threshold comparison result shows that the burst score exceeds the detection threshold, it is determined that a statistically significant burst has occurred in the current data stream, triggering the report generation process; The trigger report generation process includes: Extract the current time slot flow observation from the active sketch, and extract the previous time slot flow observation from the stable sketch; By combining the flow identifier of the data stream, the timestamp corresponding to the current time slot, the current time slot traffic observation value, and the previous time slot traffic observation value, a burst report is generated. The burst report is uploaded to the control plane through the standard summary mechanism or mirroring mechanism built into the programmable switch.

7. The method according to claim 1, characterized in that, The step of receiving burst reports in the control plane and using a linear interpolation algorithm to fill in the data between adjacent burst reports to reconstruct the continuous flow rate curve of the data stream specifically includes: The control plane receives burst reports from the data plane through a summary message channel, parses and extracts the flow identifier, timestamp and corresponding flow observation value contained therein; The burst reports belonging to the same data stream are sorted according to their timestamps to form a time series that includes multiple discrete burst report points; For any two consecutive burst reporting points in time, calculate the slope of the average rate change between the two points based on their respective timestamps and flow observations; Using the slope of the average rate change, the estimated flow rate values ​​at each time point between two burst reporting points are calculated and filled in using a linear interpolation formula; Iterate through all adjacent burst report points in the data stream and perform a fill operation, then connect the points and the fill points to generate a continuous flow rate curve for the data stream.

8. A full-flow microsecond-level measurement system based on data plane micro-burst measurement, characterized in that, The system specifically includes: The time sketch measurement unit is used to perform time-slot traffic statistics on the data stream in the data plane of the programmable switch using a time sketch data structure with timestamps, and to obtain the current time slot traffic observation value and the previous time slot traffic observation value. The flow rate differential unit is used to calculate the first-order difference between the current time slot flow observation value and the previous time slot flow observation value on the data plane to obtain the flow rate change. The arithmetic approximation calculation unit is used to perform a lightweight chi-square test on the data surface using logarithmic projection techniques based on the change in flow rate and the historical cumulative change, and to calculate the burst score. The burst filtering unit is used to compare the burst score with a preset threshold. If the burst score exceeds the preset threshold, a burst report containing flow identifier, timestamp and flow observation value is generated and reported to the control plane. The data filling and reconstruction unit is used to receive burst reports at the control plane, fill the data between adjacent burst reports using a linear interpolation algorithm, and reconstruct the continuous flow rate curve of the data stream. Specifically, the step of using a timestamped time sketch data structure to perform time-slot-based traffic statistics on the data stream and obtain the current time-slot traffic observation value and the previous time-slot traffic observation value includes: Multiple parallel sketch instances with timestamp fields are maintained on the data plane. Each sketch instance's counter unit is associated with a timestamp that records the latest update slot. When a data packet arrives, the packet's flow identifier hash is mapped to the corresponding counter cell of each sketch instance, and the timestamp of the counter cell is read and compared with the current global slot index. If the timestamp of the counter unit is inconsistent with the current global timeslot index, the data of the counter unit is determined to be expired, the count value of the counter unit is reset to the current data packet length and the timestamp is updated to the current global timeslot index; If the timestamp of the counter unit is consistent with the current global timeslot index, then the current data packet length is added to the count value of the counter unit; The role of multiple sketch instances is managed through a three-sket rotation mechanism, and the current time slot flow observation value and the previous time slot flow observation value are output. The sketch instances include active sketches, stable sketches, and stale sketches; the management of multiple sketch instances' roles through a three-sket rotation mechanism, outputting the current time-slot flow observation value and the previous time-slot flow observation value, specifically includes: Assign the sketch instance corresponding to the modulo result of the current global time slot index as an active sketch, and accumulate and count the traffic data of the current time slot through the active sketch; Assign the sketch instance corresponding to the modulo result of the current global slot index minus one as a stable sketch, and save and provide the traffic data of the previous slot through the stable sketch; Assign the sketch instance corresponding to the modulo result of the current global time slot index minus two as an old sketch, and save the traffic data of the current global time slot index minus two through the old sketch; Within any of the aforementioned time slots, the current cumulative count value of the data stream is read from the active sketch as the current time slot flow observation value, and the historical count value of the data stream is read from the stable sketch as the previous time slot flow observation value. When the current global time slot index increments, the roles of sketch instances are reassigned according to the modulo rule, so that the original stale sketches are cleared and converted into new active sketches, the original active sketches are converted into stable sketches, and the original stable sketches are converted into stale sketches.

Images