Microcosmic time series data measurement and detection method and system based on programmable switch

Abstract

The invention discloses a microcosmic time sequence data measurement and detection method and system based on a programmable switch. The method specifically comprises the following steps: judging whether a data packet arrival time and an in-bucket truncation timestamp belong to a new time period or not based on the data packet arrival time and the in-bucket truncation timestamp; when the time period is judged to be a new time period, an asynchronous updating operation is triggered, a statistical value of a previous period is extracted, an in-bucket truncation timestamp is updated, and a counter is reset; in the storage module, the statistical value is stored in a sliding window slot position of a two-dimensional circular queue based on a modular operation result of stream key hash and a current timestamp, and a time sequence data sequence of each stream is constructed; obtaining a time sequence data sequence from the storage module, and calculating time sequence statistical characteristics in a data plane; and inputting the time sequence statistical features and the packet level features into a machine learning model deployed by a data plane for reasoning, and outputting an anomaly detection result. According to the method, hardware constraints can be broken through, time sequence data generation, storage and analysis are completed in a closed loop in a data plane, and the real-time performance and accuracy of network anomaly detection are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer network and network security technology, and particularly relates to a micro timing data measurement and detection method and system based on a programmable switch. BACKGROUND

[0002] With the rapid development of cloud computing and data center networks, network traffic presents the characteristics of high throughput, low latency and dynamic changes. In order to guarantee the reliability and security of the network, real-time network telemetry and anomaly detection become crucial. In recent years, programmable data plane technology represented by P4 language and reconfigurable matching action table (RMT) architecture enables researchers to sink part of the measurement and detection tasks from the control plane to the data plane, thereby achieving nanosecond-level packet processing speed and higher detection throughput. Existing detection schemes based on the data plane usually adopt the architecture of "data plane measurement + data plane detection", and even attempt to deploy machine learning models directly on the switch chip to cope with the increasingly complex network threats.

[0003] However, although programmable data plane technology has made significant progress, the existing data plane anomaly detection methods still have essential limitations in feature expression ability and detection accuracy. The current detection schemes mainly rely on simple packet-level features (such as packet length, port number, protocol type, TTL, etc.) or coarse-grained flow-level statistics (such as total byte number of the flow). These simple features are difficult to capture the dynamic change rules of network traffic in the time dimension, such as periodic fluctuations, sudden traffic surges (micro-bursts) or sudden shifts in traffic patterns. Due to the lack of timing context information, the existing methods often have low detection accuracy and high false positive rate when facing complex attacks or abnormal behaviors with strong time correlation.

[0004] The fundamental reason for the above limitations is that the existing data plane measurement mechanism cannot provide accurate and real-time per-flow timing data for downstream detection tasks under the condition of limited hardware resources. Specifically, the existing technology faces the following main challenges: Firstly, the lack of data accessibility. Mainstream traffic measurement algorithms (such as Count-Min Sketch and its variants) usually use compact hash structures to store statistical information. In order to obtain the statistical data of a specific flow, the traditional method relies on the control plane to read the entire data structure periodically and traverse all flow keys for recovery. However, modern programmable switch architectures (such as Intel Tofino) strictly limit the instruction set in order to guarantee line-speed forwarding, do not support loop operations, and have strict restrictions on the access of stateful registers (such as only one access per stage). This makes it impossible for the data plane logic to traverse the data structure to extract flow statistical information as the control plane does, resulting in the problem of "writing in but not reading out" of measurement data, which cannot be directly used for real-time detection in the data plane.

[0005] Secondly, the limitation of flow coverage. Network anomalies can occur on any flow, and the occurrence time is unpredictable. Existing partial measurement methods usually use hash tables or register arrays to obtain flow-level features, but this often requires pre-defining the target flow to be monitored, which cannot meet the real-time monitoring needs of a large number of concurrent flows. Although some full-flow measurement schemes (such as FlowLiDAR, etc.) attempt to cover all flows, they still rely on the control plane to extract valid information, which cannot meet the millisecond-level or even microsecond-level real-time response needs of the data plane.

[0006] In addition, hardware constraints limit the implementation of complex timing logic. To construct timing data in the data plane, the intuitive method is to create multiple Sketch instances for each time period to record polling, but this will consume huge SRAM resources, exceeding the hardware budget of the switch. At the same time, the switch arithmetic logic unit (ALU) only supports simple integer operations, not complex floating-point calculations, which also brings challenges to the extraction of timing features and the deployment of machine learning models.

[0007] In summary, the existing technology is difficult to simultaneously realize full-flow coverage, microsecond-level granularity of timing data measurement, and real-time anomaly detection based on the timing data in the data plane of a programmable switch. SUMMARY

[0008] The purpose of the present application is to provide a micro timing data measurement and detection method and system based on a programmable switch, which can break through the hardware constraints and complete the generation, storage and analysis of timing data in a closed loop in the data plane, improving the real-time and accuracy of network anomaly detection, to solve at least one of the above existing technical problems.

[0009] In a first aspect, the present application provides a micro timing data measurement and detection method based on a programmable switch, which specifically comprises: The measurement module is deployed in the data plane, and whether it belongs to a new time period is judged based on the time when the data packet arrives and the truncated timestamp in the bucket; When it is determined that it is a new time period, an asynchronous update operation is triggered, the statistical value of the last period is extracted, the truncated timestamp in the bucket is updated and the counter is reset, and the statistical value is sent to the storage module; In the storage module, the statistical value is stored in the sliding window slot of the two-dimensional circular queue based on the hash of the flow key and the modulo operation result of the current timestamp, and the time sequence data sequence of each flow is constructed; Based on the detection module, the time sequence data sequence is obtained from the storage module, and the time sequence statistical characteristics are calculated in the data plane; Based on the detection module, the time sequence statistical characteristics and the packet-level characteristics are input into the machine learning model deployed in the data plane for inference, and the abnormal detection result is output.

[0010] In a second aspect, the present application provides a micro time sequence data measurement and detection system based on a programmable switch, which specifically comprises: The measurement deployment unit is used to deploy the measurement module in the data plane, and whether it belongs to a new time period is judged based on the time when the data packet arrives and the truncated timestamp in the bucket; The asynchronous update unit is used to trigger an asynchronous update operation when it is determined that it is a new time period, extract the statistical value of the last period, update the truncated timestamp in the bucket and reset the counter, and send the statistical value to the storage module; The sequence construction unit is used to store the statistical value in the sliding window slot of the two-dimensional circular queue based on the hash of the flow key and the modulo operation result of the current timestamp in the storage module, and construct the time sequence data sequence of each flow; The time sequence statistical unit is used to obtain the time sequence data sequence from the storage module based on the detection module, and calculate the time sequence statistical characteristics in the data plane; The anomaly detection unit is used to input the time sequence statistical characteristics and the packet-level characteristics into the machine learning model deployed in the data plane for inference based on the detection module, and output the abnormal detection result.

[0011] In a third aspect, the present application provides a computer device, which comprises a memory and a processor and a computer program stored in the memory, and when the computer program is executed on the processor, the micro time sequence data measurement and detection method based on the programmable switch is realized.

[0012] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is run by a processor, the micro time sequence data measurement and detection method based on the programmable switch is realized.

[0013] Compared with the prior art, the present application has at least one of the following technical effects: 1、The application can break through the hardware constraints, complete the timing data generation, storage and analysis in the data plane, and improve the real-time and accuracy of network anomaly detection; 2、The TsSketch proposed in the application realizes asynchronous updating at the flow level by introducing a timestamp mechanism in the bucket, breaks the limitation of global synchronous reset required by the traditional Sketch, can continuously measure the traffic of all active flows in the network at a microsecond level granularity without storing the flow state, and solves the problem that the prior art cannot balance full-flow coverage and fine-grained measurement; 3、The application uses a resubmission mechanism to skillfully avoid the hardware constraint of single-state access of a programmable switch, realizes direct extraction and reset of the counter in the data plane, so that the measurement result can flow to the subsequent detection module in real time without the intervention of the control plane, and greatly reduces the detection delay; 4、The TsQueues proposed in the application adopt a two-dimensional circular queue structure based on hash and timestamp indexing, can efficiently maintain the sliding window data of all flows under the condition of line speed, avoids the huge memory overhead caused by creating an independent Sketch for each period, and successfully builds a complete timing data view in the resource-limited data plane; 5、The application realizes the calculation of high-order timing characteristics such as mean, variance and range in the data plane, significantly improves the feature expression ability of the machine learning model compared with the traditional method which only relies on packet-level characteristics. Experimental results show that after introducing the timing characteristics, the F1 score of anomaly detection is improved by up to 15%, and can effectively identify complex traffic anomalies such as microburst; 6、All modules of the application are optimized for the pipeline architecture of a programmable switch, avoiding complex operations and loop operations. Experiments show that the system has almost no impact on the switch forwarding throughput while ensuring full-flow measurement and real-time detection, and has low resource occupancy, good engineering practicability and scalability. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0015] Figure 1 is a flow diagram of a micro timing data measurement and detection method based on a programmable switch provided by an embodiment of the application; Figure 2 is a whole architecture diagram of a micro timing data measurement and detection method based on a programmable switch provided by an embodiment of the application; Figure 3 is a schematic diagram of an asynchronous update mechanism and a bucket structure of a measurement module TsSketch provided by an embodiment of the present application; Figure 4 is a schematic diagram of a comparison between a zero-clearing reset strategy and a minimum subtraction reset strategy adopted by a measurement module TsSketch provided by an embodiment of the present application; Figure 5 is a schematic diagram of a two-dimensional circular queue structure of a storage module TsQueues provided by an embodiment of the present application; Figure 6 is a schematic diagram of a time sequence feature extraction and detection workflow of a detection module TsAnomaly provided by an embodiment of the present application; Figure 7 is a comparison diagram of measurement accuracy of a measurement module TsSketch and prior art under different memory overheads provided by an embodiment of the present application; Figure 8 is a schematic diagram of a throughput test result of a programmable switch provided by an embodiment of the present application; Figure 9 is a schematic diagram of a micro time sequence data measurement and detection system based on a programmable switch provided by an embodiment of the present application; Figure 10 is a schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0016] In the following description, specific details are set forth in order to provide a thorough understanding of embodiments of the application. However, persons of ordinary skill in the art will readily recognize that embodiments of the application can be practiced without

[0017] In embodiments of the present application, the execution subject of the flow includes a terminal device. The terminal device includes but is not limited to a server, a computer, a smart phone, a tablet computer and other devices capable of executing the method disclosed in the present application. Figure 1 A flowchart of a micro time sequence data measurement and detection method based on a programmable switch provided by an embodiment of the present application is shown, and is described in detail as follows: S101, deploying a measurement module at a data plane, and judging whether it belongs to a new time period based on a packet arrival time and a truncated timestamp in a bucket.

[0018] In this embodiment, a measurement module TsSketch is deployed on the data plane. When a data packet arrives, a hash function is used to calculate the bucket index corresponding to the stream key, and the truncation timestamp maintained in the bucket is used to determine whether the current data packet belongs to a new time period.

[0019] The measurement module TsSketch includes Bucket array, It represents the number of hash functions (i.e., the number of rows). It represents the number of buckets per row, with each bucket maintaining a truncation timestamp. and a counter For arrival times of Size is If the data packet ( If the period length is specified, then it is determined to be the current period, and execution is performed. Otherwise, it will be considered a new cycle. Updated to the start time of the current period.

[0020] S102, when a new time period is determined, an asynchronous update operation is triggered to extract the statistical value of the previous period, update the truncated timestamp in the bucket and reset the counter, and at the same time send the statistical value to the storage module.

[0021] In this embodiment, if the current time period is determined, the measurement module directly accumulates the traffic statistics value in the bucket; if the new time period is determined, the measurement module performs an asynchronous update operation: extracts the statistics value of the previous period, updates the timestamp in the bucket, resets the counter in the bucket through the resubmit mechanism, and sends the extracted statistics value to the storage module.

[0022] The asynchronous update operation utilizes a recommit mechanism to address the single-time state access limitation, specifically comprising two phases: Phase 1: When a periodic switch is detected, extract... Minimum value among hash buckets As a flow estimate, Write the packet header metadata and set the resubmit flag to resend the packet into the ingress pipeline; Phase Two: After the resubmitted data packet re-enters the pipeline, the header information is parsed. and will The counter value in each hash bucket minus This eliminates the contribution from the previous cycle, thus achieving a logical reset of the counter.

[0023] S103, in the storage module, the statistical value based on the modulo operation result of the stream key hash and the current timestamp is stored in the sliding window slot of the two-dimensional circular queue to construct the time-series data sequence of each stream.

[0024] In the embodiment, the storage module TsQueues receives the statistics, and stores the statistics in a sliding window slot corresponding to a two-dimensional cyclic queue according to a modulo operation result of the flow key hash and the current timestamp, to construct a time series of the flow.

[0025] The storage module TsQueues adopts a two-dimensional structure, where is the number of rows, is the length of the sliding window; each row contains a fingerprint field for flow key verification and a data queue with a length of When the statistics are written, if the fingerprint corresponding to the row index matches the current flow fingerprint, the value is written into the queue according to the column index calculated based on the low bits of the timestamp ; if the fingerprints do not match, it is determined as a hash collision or a new flow, and a row eviction strategy is performed, the fingerprint is reset and the queue is emptied before writing a new value.

[0026] S104, based on the detection module, obtaining the time series from the storage module, and calculating the time series statistics in the data plane.

[0027] In the embodiment, the detection module TsAnomaly obtains the time series of the flow from the storage module, and calculates the time series statistics in the data plane.

[0028] The calculation method of the time series statistics includes: calculating the sum and mean of the time series data in a limited pipeline stage by using a divide-and-conquer addition strategy; mapping the values to their square values by using a preset lookup table (Lookup Table), and then calculating the variance; the feature calculation process is completely based on integer operations and lookup table operations, and does not involve floating point operations.

[0029] S105, based on the detection module, inputting the time series statistics and the packet-level features into a machine learning model deployed in the data plane for inference, and outputting an anomaly detection result.

[0030] In the embodiment, the detection module combines the calculated time series statistics and packet-level features, and inputs them into a machine learning model deployed in the data plane pipeline for inference, and outputs a detection result of whether the flow is abnormal.

[0031] ​​The machine learning model is a decision tree, random forest, or Naive Bayes classifier. Its deployment on the data surface is as follows: the logic after model training is converted into match-action tables; first, continuous feature values ​​are discretized into Boolean codes or interval codes through a feature encoding table; then, the encoded feature vectors are used as keys to match in the classification table to obtain the final classification decision.

[0032] As a brief summary of steps S101 to S105 above, the basic idea of ​​this invention is to address the problems of existing programmable data planes lacking temporal context information and being limited by hardware resources in anomaly detection, by designing a measurement and detection closed-loop system that runs entirely within the data plane. Specifically, firstly, an innovative Sketch structure called TsSketch is used to achieve full-stream asynchronous measurement with microsecond-level granularity through an in-bucket timestamp mechanism, and a resubmission mechanism is used to circumvent hardware state access restrictions, ensuring continuous output of periodic statistical data; secondly, a storage module TsQueues based on a two-dimensional circular queue is designed, which uses stream key hashing and timestamp modulo indexing to efficiently maintain the sliding window temporal data of all streams; finally, high-order temporal features (such as mean, variance, etc.) are extracted from the data plane and combined with packet-level features and input into a lightweight machine learning model (such as decision tree, random forest, etc.) to achieve real-time, high-precision detection of complex anomalies such as micro-bursts. This method has advantages such as full-flow coverage, microsecond-level accuracy, low resource consumption, and high detection accuracy. It is applicable to various scenarios such as data center network monitoring, intrusion detection, and traffic analysis, and has significant engineering application value.

[0033] This invention proposes a novel data plane full-flow timing measurement and anomaly detection framework, called TsMon. This framework takes network traffic data packets as input, requires no control plane intervention, and directly outputs anomaly detection results at the data plane. The overall framework is as follows: Figure 2 As shown, it consists of the following three main functional modules, and all modules are deployed in the programmable data plane pipeline: First, the full-flow asynchronous measurement module (TsSketch). This module is responsible for measuring flow-level statistics for each time period. To achieve full flow coverage, TsSketch does not store or predefine flow keys, but instead triggers updates using the data packets themselves: Statistical data of the period in the first Settlement is completed when the next data packet arrives in the same period. To address data accessibility issues, TsSketch introduces an in-bucket timestamp mechanism, transforming the unified global period into independent periods at the stream level. Each bucket independently determines whether a new period has started and resets or updates accordingly. This decentralized asynchronous update logic eliminates dependence on loops or external coordination, ensures compatibility with switch pipeline constraints, and enables measurement results to be directly used for real-time anomaly detection in a streaming manner.

[0034] Second, the sliding window time-series storage module (TsQueues). While TsSketch provides single-period statistics, anomaly detection requires a series of statistical sequences that change over time. TsQueues bridges this gap by maintaining a compact and efficient sliding window to construct time-series data for each stream. TsQueues is implemented as a two-dimensional circular buffer based on stream key hashes and the current period's modulo index. Each row corresponds to a stream's time series, and each column represents a period slot. When new statistics arrive from TsSketch, TsQueues inserts them into the corresponding slot in the buffer. Once new data is inserted or a hash collision is detected, TsQueues forwards the corresponding time-series data row to the anomaly detection module. This allows TsMon to make detection decisions quickly without waiting for control plane intervention. Importantly, TsQueues maintains only a single instance on the data plane, avoiding excessive memory overhead caused by simply instantiating a Sketch for each period, thus meeting the need for efficient maintenance of time-series data for each stream in resource-constrained environments.

[0035] Third, the temporal anomaly detection module (TsAnomaly). This module extends existing data surface detection methods by incorporating temporal dynamics into the feature set. Unlike existing works that rely solely on packet-level or single-period features, TsAnomaly processes rich temporal vectors in real time. After receiving the time series of the stream from TsQueues, TsAnomaly first extracts raw values ​​(such as the number of bytes or packets per period) and then calculates statistical features, including mean, maximum, minimum, variance, and jitter—all calculated using optimized arithmetic operations and table lookup operations. These derived features significantly improve the model's expressive power and detection accuracy. TsAnomaly supports various lightweight machine learning models, including decision trees, Naive Bayes, and random forests. These models are implemented using P4 tables and actions, and can be executed entirely within the match-action pipeline.

[0036] In summary, this invention achieves a complete closed loop within the data plane, from microscopic measurement to time series construction and then to intelligent detection, through the collaborative work of three modules: TsSketch, TsQueues, and TsAnomaly.

[0037] In some embodiments, step S101 above, which involves determining whether a new time period exists based on the arrival time of the data packet and the truncation timestamp within the bucket, specifically includes: The arrival time of the data packet and the corresponding truncation timestamp maintained in the bucket are extracted by the measurement module. Calculate the difference between the arrival time and the truncation timestamp, and compare the difference with the preset period length; If the difference is less than the period length, the data packet is determined to belong to the current time period; otherwise, the data packet is determined to belong to the new time period.

[0038] This embodiment demonstrates the specific implementation of the full-stream asynchronous measurement module TsSketch, as shown in the attached figure. Figure 3 As shown, to achieve microsecond-level granular measurement of network traffic and overcome hardware limitations, TsSketch employs a Sketch structure that supports asynchronous updates. Traditional Sketch relies on control plane polling to read data, which cannot meet the real-time detection requirements of the data plane. TsSketch solves this problem through the following mechanism: Basic structural design: TsSketch consists of one Composed of bucket arrays, in which It represents the number of hash functions (i.e., the number of rows). This is the number of buckets per row. Each bucket maintains two core fields: Truncation timestamp : Indicates the start time of the current period for this bucket. To save memory, a truncated version of the global timestamp is used.

[0039] counter Stores the aggregated value for the current period (such as the number of packets or bytes).

[0040] Processing flow: Assume a size of The data packets at time Arrival, and a fixed period length of (In this embodiment, it is set to 10) TsSketch performs the following steps: Hash mapping: Extract the stream key k and compute Each hash function maps to a bucket index. , ..., .

[0041] Periodic checks: For each bucket, compare and .like This indicates that the data packet belongs to the current period, so it can be directly... Increase .

[0042] In some embodiments, step S102 above, which involves triggering an asynchronous update operation when a new time period is determined, extracting the statistical value of the previous period, updating the truncated timestamp in the bucket and resetting the counter, and simultaneously sending the statistical value to the storage module, specifically includes: When it is determined that the data packet belongs to a new time period, the asynchronous update operation is triggered to extract the minimum value of the counter from multiple hash buckets corresponding to the current stream key, and use it as the statistical value of the previous period. Update the truncation timestamp in the bucket to the start time of the current time period; Write the statistics to the header metadata of the data packet and set the resubmit flag to make the data packet re-enter the processing pipeline to reset the counter; When a packet carrying a resubmit flag is processed again, the statistics are parsed from the packet header metadata and subtracted from the counters of multiple hash buckets to complete the logical reset of the counters. The statistical values ​​are sent to the storage module.

[0043] In this embodiment, the steps of period switching and asynchronous update include: if the data packet belongs to a new period (i.e. If ), then the update logic is triggered: (a) Timestamp update: Set This marks the beginning of a new cycle; (b) Statistical Extraction: Obtaining Minimum value among all related counters This is used as an estimated statistic for the previous cycle of the flow and forwarded to TsQueues; (c) Reset and Re-aggregate: From each counter Subtract To eliminate the contribution from the previous cycle, and then add The accumulation of a new cycle begins.

[0044] Bypassing hardware constraints using a resubmit mechanism: The aforementioned reset and re-aggregation steps involve multiple state memory accesses, violating the "single state memory access" limitation of programmable switches (such as Tofino). To address this issue, this embodiment utilizes the Resubmit feature provided by Tofino to split the update logic into two pipeline stages: Phase 1: Update when a period switch is detected. Extract the minimum value .Will Write a custom header field and set the Resubmit flag, while also... Send to TsQueues. Only packets with the Resubmit flag will re-enter the ingress pipeline.

[0045] Phase Two: After data packet re-entry, parse the custom packet header to obtain... and from Subtract from the counter Complete the clearing of contributions from the previous cycle.

[0046] This two-phase approach enables safe multi-register operations under hardware constraints. Crucially, updates for each stream are independent, enabling full-stream measurement without requiring explicit maintenance of each stream's state.

[0047] As attached Figure 4 As shown, this embodiment does not simply zero out the bucket. Since multiple flows may share a single bucket (hash collision), directly zeroing it out would incorrectly erase the contributions of other flows. TsSketch employs a "minimum subtraction" strategy: subtracting the estimated value of the flow from the relevant bucket. This preserves the statistics of other flows while effectively resetting the count of the current flow, thus maintaining high measurement accuracy even in the presence of hash collisions.

[0048] In some embodiments, in step S103 above, the statistical value of the modulo operation result based on the stream key hash and the current timestamp is stored in the sliding window slot of the two-dimensional circular queue to construct the time-series data sequence of each stream, specifically including: The row index is obtained based on the stream key hash calculation, and the corresponding row is located in the two-dimensional circular queue. Read the fingerprint field stored in this row and compare the fingerprint field with the fingerprint of the current stream; If the fingerprint matches, the column index is calculated based on the low-order bits of the current timestamp, and the statistical value is written to the corresponding slot in the data queue of that row. If the fingerprint does not match, perform a row initialization operation, update the fingerprint field to the fingerprint of the current stream, clear the data queue of the row, and write the statistical value to the initialized queue slot according to the column index; By writing to multiple slots in a sliding window of a two-dimensional circular queue, a continuous sequence of time-series data is constructed and maintained for each stream.

[0049] This embodiment illustrates the specific implementation of the sliding window timing storage module TsQueues, as shown in the attached figure. Figure 5 As shown, based on TsSketch's implementation of single-cycle measurements, TsQueues is responsible for constructing the timing data for each stream. In this embodiment, TsQueues is designed as a lightweight data plane sliding window buffer, capable of storing the most recent l cycles (the default in this embodiment). =8) Statistical data.

[0050] Basic structure: TsQueues is implemented as a A bucket array, in which For the number of rows, This represents the length of the sliding window. Each line represents the storage space for a stream, containing: fingerprint The truncated hash value of the stream key is used to verify row ownership and resolve hash collision issues.

[0051] Data queue : Length is A circular buffer that stores the most recently accessed stream. Statistical values ​​for each period. To reduce hardware costs, the low-order bits of the timestamp are used as an index for direct addressing. The slot in the middle. For example, when =8= At that time, the lowest 3 bits of the timestamp determine the position where the new value is inserted.

[0052] Workflow: When TsSketch outputs periodic statistics for a certain stream... When this happens, perform the following steps: Index location: Hash the stream key to obtain the row index. Read the fingerprint of that row And compare it with the truncation key of the current stream; Data writing: If the fingerprint matches, it will be written according to the current timestamp. Write to data queue The corresponding slot in the [database name]. At this point, the row stores the latest time-series data sequence of the stream. If the fingerprint does not match, it indicates a hash collision or that the position is occupied by an older stream. At this point, row initialization is performed: clear existing data and [database name] is added. Update the fingerprint to the new stream, and Initialize to include A new queue; Data Output: After each write or eviction operation, TsQueues can output the complete time-series data of the current row (e.g., ... , ..., This indicates that the complete time-series data has been divided into data blocks and then forwarded to the downstream TsAnomaly module. Extended Functionality: In addition to the basic flow curve, TsQueues independently collects features across other dimensions by adding extra buckets to each row. For example, to detect micro-bursts, packet arrival intervals are crucial. TsQueues uses these extra buckets to record the arrival time of the previous packet and calculates the time difference between the current packet and the previous packet, thus obtaining the packet interval. This multi-dimensional feature collection capability further enhances the input information for subsequent detection models.

[0053] In some embodiments, step S104 above, which involves obtaining the time-series data sequence from the storage module based on the detection module and calculating the time-series statistical features within the data plane, specifically includes: The detection module receives a time-series data sequence from the storage module, the time-series data sequence including the statistical values ​​of the current stream for each time period within the sliding window; Based on time series data sequences, the sum and mean of the time series data sequences are calculated using a divide-and-conquer addition strategy within the data plane; The squared values ​​of each statistical value in the time series are mapped to their squared values ​​using a pre-set squared lookup table, and the variance of the time series is calculated by combining the mean. The maximum and minimum values ​​are extracted from the time-series data sequence through parallel comparison operations, and the range is calculated based on the maximum and minimum values; The sum, mean, variance, maximum, minimum, and range are output as time series statistical features.

[0054] Furthermore, the step of calculating the sum and mean of the time-series data sequence using a divide-and-conquer addition strategy within the data plane, based on the time-series data sequence, specifically includes: Extract the periodic statistical values ​​from the time-series data sequence; In the first pipeline stage, the statistical values ​​of each cycle are grouped and calculated in parallel to obtain multiple partial sums of the first stage; In at least one subsequent pipeline stage, the partial sum of the output of the previous stage is used as input to continue grouping and parallel addition operations to obtain the partial sum of the next stage, until the sum of the time-series data sequence is obtained after a preset number of stages. The mean of the time series data sequence is calculated using arithmetic shift operations based on the sum and the sliding window length.

[0055] This embodiment demonstrates the specific implementation of the timing anomaly detection module TsAnomaly, as shown in the attached figure. Figure 6 As shown, TsAnomaly is the decision-making core of the TsMon system, responsible for performing real-time inference using time-series data provided by TsQueues. This embodiment demonstrates how to deploy feature engineering and machine learning models within the data plane.

[0056] Time-series feature engineering: TsQueues outputs the original flow curve (e.g., , , ..., (This represents the various parts of the original traffic curve after data segmentation). To improve the model's detection capability, TsAnomaly first calculates higher-order statistical features within the data plane. Since the switch hardware does not support complex loops and floating-point operations, this embodiment employs the following optimization strategy: Divide and Conquer: When calculating the sum and mean, a divide-and-conquer approach is used to avoid excessively long series (exceeding hardware limitations) caused by linear accumulation. For example, in the first stage of parallel computation... = , = ...in the next stage of calculation ...and so on. This only requires... The summation can be completed in one stage. This represents the partial sum calculated in the i-th stage. , and These represent the partial sums calculated in stages 1, 2, and 12, respectively. This indicates the number of stages required to complete all stages. Indicates the length of the sliding window.

[0057] Calculating squares and variances using a lookup table: Variance calculation requires squaring, but switches do not support multiplication. TsAnomaly provides a built-in square lookup table, using numerical... The key is used to directly look up the table. Combining the aforementioned mean, using the formula... Calculate the variance. Where the numerical values ​​are... Used as a key (index) in a square lookup table. Representing numerical values The square of, The variance of a random variable X is used to measure the dispersion of a data set. Represents random variables The expected value is the average of the squares of the values ​​in the dataset. Let x represent the expected value of the random variable x, and let y represent the average value of the data set.

[0058] Extreme values ​​and range: The maximum value (Max) and minimum value (Min) within the window are obtained in parallel through a multi-level comparison tree, and the range (Delta = Max - Min) is calculated, where Delta represents the range.

[0059] In some embodiments, in step S105 above, the step of inputting temporal statistical features and packet-level features into a machine learning model deployed on the data plane for inference based on the detection module, and outputting anomaly detection results, specifically includes: The temporal statistical features are combined with the packet-level features extracted from the packet header to form a joint feature vector; The continuous eigenvalues ​​in the joint eigenvector are discretized into corresponding interval codes or Boolean codes using the feature encoding table in the data plane pipeline. The discretized interval codes or Boolean codes are concatenated into feature keys, and the feature keys are used as query inputs to the matching-action classification table corresponding to the machine learning model; Based on the match-action classification table, precise matching is performed according to the feature key, and the corresponding classification action is output. The classification action is used to indicate whether the current data packet belongs to an abnormal flow.

[0060] In this embodiment, the feature vector constructed by TsAnomaly not only includes packet-level features (such as TTL, TCPFlags), but also rich stream-level temporal features (such as Sum, Mean, Variance, Min, Max, Jitter, etc.), as shown in Table 1.

[0061] Table 1

[0062] Data deployment for machine learning models: This embodiment supports lightweight models such as decision trees, random forests, and Naive Bayes. These models are trained offline and then compiled into a matching-action table supported by the P4 language.

[0063] Feature encoding: First, the continuous feature values ​​are converted into discrete codes through a range matching table (e.g., "feature A belongs to interval 3" corresponds to code "10").

[0064] Decision execution: The encoding of all features is concatenated into a long bit string, which is used as a key for precise matching in the classification table. The matching result is the classification decision (such as "normal" or "abnormal").

[0065] Furthermore, the pre-training step of the machine learning model includes: Collect a training dataset containing both normal and abnormal traffic, and extract training features from the training dataset that correspond to the joint feature vector; The selected classification model is trained offline using training features to obtain the trained model parameters and decision logic. The classification model is a decision tree model, a random forest model, or a Naive Bayes model. The trained model parameters and decision logic are converted into matching-action table logic supported by the data plane pipeline; The matching-action table logic is compiled and loaded into the data plane of the programmable switch to form a feature encoding table and a matching-action classification table.

[0066] In this embodiment, a training dataset containing normal traffic and abnormal traffic is constructed. The data source may include traffic samples collected from the actual network environment or simulated network attack traffic, wherein the abnormal traffic needs to cover a variety of typical attack types (such as DDoS, port scanning, traffic surge, etc.). Training features corresponding to the joint feature vector are extracted from the training dataset, specifically including two types of features: (1) Temporal statistical features: For the statistical values ​​of each time period within the sliding window of each flow, the sum, mean, variance, range, maximum and minimum values ​​are calculated; (2) Packet-level features: Fields directly extracted from the packet header, such as packet length, port number, protocol type, TTL value, etc. The above two types of features are combined into a training feature vector according to a preset format to ensure that its structure is consistent with the joint feature vector input in the subsequent inference stage.

[0067] Based on the extracted training features, a classification model is selected for offline training. Available models include decision tree models, random forest models, or Naive Bayes models; the specific choice depends on the size of the training data and the complexity of the features. For example, if the amount of training data is small and the feature dimensionality is low, a decision tree model can be preferred; if high-dimensional features or non-linear relationships need to be handled, a random forest model is used to improve generalization ability.

[0068] The training process iteratively optimizes model parameters (such as the splitting threshold of the decision tree and the number of trees in the random forest) to achieve a preset threshold (such as above 95%) in classifying normal and abnormal traffic. After training, the model parameters (such as weights and splitting node values) and decision logic (such as the classification rule tree structure or probability calculation logic) are saved as the basis for subsequent logical transformations.

[0069] The model parameters and decision logic obtained from offline training are converted into matching-action table logic supported by the data surface pipeline. Specifically, if the classification model is a decision tree model or a random forest model, the splitting rules of each decision tree or random forest are decomposed into multi-level matching-action tables. For example, if a splitting node branches based on "variance > threshold", it is converted into a first-level matching table, with the rule key being "variance interval" and the action value being the index of the next-level matching table; the classification result (normal / abnormal) of the terminal leaf node is converted into the final action, such as setting the action flag to 0 or 1.

[0070] If the classification model is a Naive Bayes model, the prior and conditional probabilities of each feature are converted into lookup tables. For example, for the "mean" feature, the anomaly probabilities corresponding to its different value ranges are pre-calculated and stored as a probability lookup table; during inference, the posterior probabilities are calculated by querying each feature table and jointly calculating them, and finally, the classification action is output by comparing thresholds.

[0071] The converted match-action table logic is compiled into a configuration file recognizable by the programmable switch data plane. The compilation process needs to check whether the logic meets hardware constraints (such as SRAM capacity, ALU operation type). If it exceeds the constraints, the table structure needs to be optimized (such as merging multi-level tables, reducing the number of rules). After compilation, the configuration file is loaded into the programmable switch data plane to form two types of key tables: (1) Feature encoding table: used to discretize continuous feature values ​​into interval encoding or Boolean encoding. (2) Match-action classification table: stores the correspondence between classification rules and actions.

[0072] In this embodiment, the machine learning model is successfully deployed to the data plane of the programmable switch. Subsequently, it can be combined with the time-series data sequence and packet-level features output by the measurement module and storage module to achieve real-time anomaly detection.

[0073] Furthermore, the step of performing precise matching based on the feature key according to the matching-action classification table and outputting the corresponding classification action specifically includes: Perform precise matching between the feature keys and the rule keys pre-stored in the matching-action classification table; When a match is successful, a classification action associated with the corresponding rule key is executed. This classification action is used to indicate whether the current data packet belongs to an abnormal flow. The results of the classification action are written to the metadata of the current data packet, and the corresponding control plane alarm is triggered to complete the indication of the abnormal status of the flow to which the current data packet belongs.

[0074] In this embodiment, the programmable switch has completed the deployment and configuration of related modules, including measurement modules, storage modules, detection modules, etc., and the matching-action classification table corresponding to the machine learning model has been pre-stored in the data plane.

[0075] During the data packet processing in the data plane, the detection module combines temporal statistical features with packet-level features extracted from the packet header to form a joint feature vector. Then, through the feature encoding table in the data plane pipeline, the continuous feature values ​​in the joint feature vector are discretized into corresponding interval codes or Boolean codes, and these discretized interval codes or Boolean codes are concatenated to form the feature key.

[0076] The data plane performs an exact match between the concatenated feature keys and the pre-stored rule keys in the match-action classification table. The match-action classification table pre-stores various rule keys, each associated with a specific classification action. These actions indicate whether the current data packet belongs to an abnormal flow. For example, a rule key might be an encoded form of a specific combination of traffic characteristics. A match is considered successful when the feature key exactly matches a rule key.

[0077] When a feature key matches exactly with a rule key in the match-action classification table, the data plane executes the classification action associated with that rule key. For example, if the classification action associated with a successfully matched rule key is "abnormal," it indicates that the current data packet belongs to a flow with an abnormal condition; if the associated classification action is "normal," it indicates that the current data packet belongs to a flow in a normal state.

[0078] After performing the classification action, the data plane writes the classification result into the metadata of the current data packet. Metadata is the part of the data packet used to store additional information. By writing the classification result into the metadata, the abnormal status information of the data packet can be easily obtained in subsequent processing.

[0079] Simultaneously, the data plane will trigger corresponding control plane alarms. The control plane is responsible for the overall management and control of the network. When the data plane detects abnormal traffic, it can promptly transmit the abnormal information to network administrators or relevant control programs by triggering control plane alarms. After receiving the alarm, the control plane can take further processing measures according to preset policies, such as isolating abnormal flows and logging abnormalities, thereby indicating the abnormal status of the current data packet's flow and ensuring the secure and stable operation of the network.

[0080] This embodiment implements a matching-action classification table, performs precise matching based on feature keys, and outputs the corresponding classification action, thereby accurately indicating the abnormal state of the flow to which the data packet belongs.

[0081] To verify the effectiveness, accuracy, and hardware feasibility of the micro-time data measurement and detection system (TsMon) based on a programmable switch proposed in this invention, this embodiment underwent extensive experimental evaluation in a real hardware testbed and simulation environment.

[0082] 1. Experimental Environment and Dataset Setup In one embodiment of the invention, experimental evaluation is conducted using a real hardware testbed. The testbed consists of two high-performance servers (serving as traffic senders and receivers) and a programmable switch based on the Intel Tofino 1 chip, interconnected via a 100 Gbps high-speed link. The servers are configured to run Ubuntu 20.04, equipped with Intel Xeon Gold processors and 64GB of memory. The core logic of the TsMon system (including the TsSketch, TsQueues, and TsAnomaly modules) is written in P4 and compiled and deployed in the data plane pipeline of the Tofino switch.

[0083] To comprehensively evaluate system performance, this invention uses three representative datasets: (1) WIDE MAWI dataset: This dataset is derived from real backbone network traffic from the WIDE project in Japan. It includes a 15-minute packet tracking record from October 5, 2024, containing approximately 3 million IPv4 packets. This dataset is primarily used to evaluate the measurement accuracy of TsSketch in real-world traffic environments. (2) UNSW-NB15 dataset: A comprehensive cybersecurity benchmark dataset constructed by the University of New South Wales, containing real benign traffic (such as Web, Email, VoIP) and nine types of synthetic attack traffic (such as DoS, Backdoor, Exploits, etc.). This experiment extracted approximately 3 million IPv4 packets, covering eight attack types, to evaluate the detection accuracy of TsAnomaly before and after the introduction of time-series features; (3) Burst_1GE and Burst_2GE datasets: These are real microburst traffic data collected from large enterprise network environments. We sampled data packets for one hour on gateway links with port rates of 1 Gbps and 2 Gbps, respectively, obtaining approximately 4K and 15K data packets. The characteristic of this dataset is that all data packets have the same length, making it difficult to distinguish anomalies based solely on packet-level features. It is mainly used to verify the coverage capability of time-series features in detecting hidden anomalies such as microbursts.

[0084] 2. Accuracy evaluation of the TsSketch measurement module To verify the performance of TsSketch in full-stream asynchronous measurement, this embodiment compares it with existing mainstream measurement algorithms, including the classic Count-Min Sketch (CMS), Persist-CMS, and the latest OmniWindow and WaveSketch algorithms. All algorithms have a time period of 10 microseconds.

[0085] To quantify measurement accuracy, three evaluation metrics were introduced: Cosine Similarity: Measures the degree of alignment between the estimated time series vector and the true time series vector in terms of direction. The closer the value is to 1, the higher the waveform reproduction accuracy.

[0086] Energy Similarity: Measures the recovery rate of total flow energy (sum of squares), reflecting the ability to capture large flow fluctuations.

[0087] Mean Relative Error (ARE): Measures the average level of the estimation error per flow across all cycles.

[0088] like Figure 7 As shown, the experimental results indicate that: (1) In terms of cosine similarity: TsSketch consistently outperforms all baseline methods. Especially in memory-constrained scenarios (e.g., 5KB to 50KB), TsSketch achieves approximately 20% higher cosine similarity than the state-of-the-art WaveSketch algorithm; (2) In terms of energy similarity: TsSketch showed a significant advantage, nearly 50% higher than the baseline method. This indicates that TsSketch can capture the dramatic fluctuations in flow and energy distribution more accurately; (3) In terms of mean relative error (ARE): TsSketch maintained an extremely low error level.

[0089] The above advantages are mainly due to TsSketch's unique asynchronous update mechanism, which avoids the data staleness and discontinuity problems caused by the periodic global reset of traditional Sketch, and ensures that high-fidelity time-series data can still be obtained with low memory overhead.

[0090] 3. Accuracy assessment of the TsAnomaly detection module To demonstrate the enhancing effect of time-series data on anomaly detection, this embodiment deployed three machine learning models—Decision Tree (DT), Random Forest (RF), and Naive Bayes (NB)—on a Tofino switch and conducted comparative tests under two feature configurations: Configuration P: Use only packet-level features (such as packet length, TTL, TCP flags); Configure TS+P: Combine packet-level features with the time-series statistical features extracted by this invention (such as mean, variance, range, etc.).

[0091] The experiment used accuracy, recall, and F1 score as evaluation metrics.

[0092] Table 2 ; Table 3

[0093] (1) Performance on the UNSW-NB15 dataset As shown in Table 2, in multi-class anomaly detection tasks, the model incorporating temporal features (TS+P) significantly outperforms the model using only bag-level features (P) across all metrics. Specifically, the performance difference after incorporating temporal features is typically between 10% and 15%. The Random Forest model RF (TS+P) performs best, achieving an accuracy of 0.91 and an F1 score of 0.79 in the six-class anomaly classification task, demonstrating the crucial role of temporal features in capturing complex attack patterns.

[0094] (2) Performance on the Burst microburst dataset As shown in Table 3, the advantages of temporal features are particularly evident in the specific scenario of micro-burst detection: the model (P) relying solely on packet-level features is completely unable to detect micro-burst anomalies, achieving an F1 score of 0 for the Burst category. This is because all packets in this dataset have the same length, lacking discriminative power. The model combining temporal features (TS+P) can effectively identify micro-bursts, with the decision tree DT (TS+P) and random forest RF (TS+P) achieving an F1 score as high as 0.96. This fully demonstrates that the temporal feature extraction method proposed in this invention significantly expands the coverage of data surface anomaly detection, enabling it to identify hidden anomalies that traditional methods cannot detect.

[0095] 4. Hardware resource overhead and throughput assessment To evaluate the feasibility of TsMon in a real production environment, this embodiment statistically analyzes its resource consumption and impact on forwarding performance on an Intel Tofino 1 switch.

[0096] (1) Throughput test This embodiment uses DPDK to generate a 100 Gbps line-rate flow rate for a stress test lasting up to 15 seconds. For example... Figure 8 As shown, a switch with TsMon deployed is compared to a baseline program that only performs MAC forwarding (DMAC). The results show that the throughput curve of TsMon almost overlaps with that of the baseline program, without causing any significant degradation in forwarding performance. This indicates that although TsMon introduces a resubmit mechanism, its impact on overall forwarding efficiency is negligible because it is only triggered at the moment of periodic switching, fully meeting the line-rate processing requirements.

[0097] Table 4

[0098] (2) Resource occupancy analysis As shown in Table 4, TsMon's utilization rate of various hardware resources on the switch is within a reasonable range: SRAM: Occupies approximately 12.68%, mainly used to store TsSketch bucket arrays and TsQueues sliding windows; TCAM: occupies approximately 17.86%, mainly used for stream classification and wildcard matching; VLIW instructions: account for approximately 15.62%, indicating moderate logical complexity; Logical table ID: Consumes the most resources at 50.00%, but this still leaves ample room for other network functions (such as routing tables and ACLs). Furthermore, experiments revealed that the overall system resource consumption is not simply the sum of the individual submodules; thanks to resource reuse between modules, the overall overhead is further reduced.

[0099] In summary, the experimental results strongly demonstrate the superiority of the TsMon system in terms of measurement accuracy, detection accuracy, and hardware efficiency, and verify that the technical approach of realizing full-flow micro-time series analysis and real-time anomaly detection on a programmable data plane is feasible.

[0100] Reference Figure 9 An embodiment of the present invention provides a micro-time series data measurement and detection system 9 based on a programmable switch. The micro-time series data measurement and detection system 9 based on a programmable switch specifically includes: Measurement deployment unit 901 is used to deploy measurement modules on the data plane and determine whether it belongs to a new time period based on the arrival time of the data packet and the truncation timestamp in the bucket. The asynchronous update unit 902 is used to trigger an asynchronous update operation when a new time period is determined, extract the statistical value of the previous period, update the truncated timestamp in the bucket and reset the counter, and send the statistical value to the storage module at the same time. The sequence construction unit 903 is used in the storage module to store statistical values ​​into the sliding window slot of a two-dimensional circular queue based on the modulo operation result of the stream key hash and the current timestamp, thereby constructing the time-series data sequence of each stream. The time series statistics unit 904 is used to obtain time series data sequences from the storage module based on the detection module and calculate time series statistical features in the data plane; The anomaly detection unit 905 is used to input time-series statistical features and packet-level features into the machine learning model deployed on the data plane for inference based on the detection module, and output anomaly detection results.

[0101] It is understandable that, such as Figure 1 The content of the micro-time series data measurement and detection method embodiments based on programmable switches shown herein is applicable to the micro-time series data measurement and detection system embodiments based on programmable switches. The specific functions implemented in the micro-time series data measurement and detection system embodiments based on programmable switches are as follows: Figure 1 The embodiment of the micro-time series data measurement and detection method based on a programmable switch shown is the same, and the beneficial effects achieved are the same as those described above. Figure 1 The beneficial effects achieved by the embodiment of the micro-time data measurement and detection method based on programmable switches shown are also the same.

[0102] It should be noted that the information interaction and execution process between the above systems are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0103] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0104] Reference Figure 10 The present invention also provides a computer device 10, including a memory 1002 and a processor 1001, and a computer program 1003 stored in the memory 1002. When the computer program 1003 is executed on the processor 1001, it implements the micro-time data measurement and detection method based on a programmable switch as described in any of the above methods.

[0105] The computer device 10 may be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device 10 may include, but is not limited to, a processor 1001 and a memory 1002. Those skilled in the art will understand that... Figure 10 The computer device 10 is merely an example and does not constitute a limitation on the computer device 10. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0106] The processor 1001 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0107] In some embodiments, the memory 1002 may be an internal storage unit of the computer device 10, such as a hard disk or memory of the computer device 10. In other embodiments, the memory 1002 may be an external storage device of the computer device 10, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device 10. Further, the memory 1002 may include both internal and external storage units of the computer device 10. The memory 1002 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 1002 can also be used to temporarily store data that has been output or will be output.

[0108] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the micro-time data measurement and detection method based on a programmable switch as described in any of the above methods.

[0109] In this embodiment, if the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0110] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0111] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0112] In the embodiments disclosed in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

Claims

1. A programmable switch-based micro-chronological data measurement and detection method, characterized in that, The method specifically comprises: Deploying a measurement module in the data plane, judging whether it belongs to a new time period based on the data packet arrival time and the truncated timestamp in the bucket; Triggering an asynchronous update operation when it is judged to be a new time period, extracting the statistical value of the last period, updating the truncated timestamp in the bucket and resetting the counter, and sending the statistical value to the storage module; In the storage module, based on the flow key hash and the modulo operation result of the current timestamp, the statistical value is stored in the sliding window slot of the two-dimensional circular queue, and the time sequence data sequence of each flow is constructed; Based on the detection module, the time sequence data sequence is obtained from the storage module, and the time sequence statistical characteristics are calculated in the data plane; Based on the detection module, the time sequence statistical characteristics and the packet level characteristics are input into the machine learning model deployed in the data plane for inference, and the abnormal detection result is output.

2. The method of claim 1, wherein, The method specifically comprises: Extracting the arrival time of the data packet and the truncated timestamp maintained in the corresponding bucket through the measurement module; Calculate the difference between the arrival time and the truncated timestamp, and compare the difference with the preset period length; If the difference is less than the period length, it is judged that the data packet belongs to the current time period, otherwise it is judged that the data packet belongs to a new time period.

3. The method of claim 1, wherein, The method specifically comprises: When it is judged that the data packet belongs to a new time period, trigger the asynchronous update operation, extract the minimum value of the counter from the multiple hash buckets corresponding to the current flow key as the statistical value of the last period; Update the truncated timestamp in the bucket to the start time of the current time period; Write the statistical value to the packet header metadata of the data packet and set the recommit flag, so that the data packet reenters the processing pipeline to reset the counter; When the data packet carrying the recommit flag is processed again, parse the statistical value from the packet header metadata, and subtract the statistical value from the counters of the multiple hash buckets respectively to complete the logical reset of the counter; Send the statistical value to the storage module.

4. The method of claim 1, wherein, The method specifically comprises: Based on the flow key hash, the row index is calculated, and the corresponding row in the two-dimensional circular queue is located; Read the fingerprint field stored in the row, and compare the fingerprint field with the fingerprint of the current flow; If the fingerprints match, calculate the column index according to the low bit of the current timestamp, and write the statistical value to the corresponding slot in the data queue of the row; If the fingerprints do not match, perform row initialization operation, update the fingerprint field to the fingerprint of the current flow, empty the data queue of the row, and write the statistical value to the initialized queue slot according to the column index; Through the circular writing of multiple slots of the sliding window of the two-dimensional circular queue, the continuous time sequence data sequence of each flow is constructed and maintained.

5. The method of claim 1, wherein, The method specifically comprises: The detection module receives a time series data sequence from the storage module, the time series data sequence including statistical values of each time period within a sliding window of a current flow; Based on the time series data sequence, the sum and mean of the time series data sequence are calculated in the data plane using a divide-and-conquer addition strategy; Each statistical value in the time series data sequence is mapped to its square value through a preset square lookup table, and the variance of the time series data sequence is calculated in combination with the mean; The maximum value and the minimum value are extracted from the time series data sequence through a parallel comparison operation, and the range is calculated based on the maximum value and the minimum value; The sum, mean, variance, maximum value, minimum value and range are output as time series statistical features.

6. The method of claim 5, wherein, The detection module receives a time series data sequence from the storage module, the time series data sequence including statistical values of each time period within a sliding window of a current flow; Based on the time series data sequence, the sum and mean of the time series data sequence are calculated in the data plane using a divide-and-conquer addition strategy, specifically including: Extracting statistical values of each period in the time series data sequence; In the first pipeline stage, the statistical values of each period are grouped and the partial sums of the first stage are calculated in parallel; In at least one subsequent pipeline stage, the partial sum output by the previous stage is taken as input, and grouping and parallel addition operations are continued to obtain the partial sum of the next stage, until the sum of the time series data sequence is obtained after a predetermined number of stages; 7. The method of claim 1, wherein, Based on the sum and the length of the sliding window, the mean of the time series data sequence is calculated through an arithmetic shift operation. The detection module receives a time series data sequence from the storage module, the time series data sequence including statistical values of each time period within a sliding window of a current flow; The time series statistical features and the packet-level features are combined to form a joint feature vector; The continuous feature values in the joint feature vector are discretized into corresponding interval encoding or Boolean encoding through the feature encoding table in the data plane pipeline; The discretized interval encoding or Boolean encoding is spliced into a feature key, and the feature key is input into the matching-action classification table of the machine learning model as a query; 8. The method of claim 7, wherein, Based on the matching-action classification table, the corresponding classification action is output according to the feature key, which is used to indicate whether the flow to which the current data packet belongs is abnormal. The pre-training step of the machine learning model includes: Collecting a training data set containing normal traffic and abnormal traffic, and extracting training features corresponding to the joint feature vector from the training data set; Using the training features to train the selected classification model offline to obtain trained model parameters and decision logic, the classification model being a decision tree model, a random forest model or a naive Bayes model; Convert the trained model parameters and decision logic into the matching-action table logic supported by the data plane pipeline; 9. The method of claim 7, wherein, Compile and load the matching-action table logic into the data plane of the programmable switch to form the feature encoding table and the matching-action classification table. The detection module receives a time series data sequence from the storage module, the time series data sequence including statistical values of each time period within a sliding window of a current flow; The time series statistical features and the packet-level features are combined to form a joint feature vector; The continuous feature values in the joint feature vector are discretized into corresponding interval encoding or Boolean encoding through the feature encoding table in the data plane pipeline; The discretized interval encoding or Boolean encoding is spliced into a feature key, and the feature key is input into the matching-action classification table of the machine learning model as a query; Based on the matching-action classification table, the corresponding classification action is output according to the feature key, which is used to indicate whether the flow to which the current data packet belongs is abnormal. The result of the classification action is written into metadata of the current data packet, and a corresponding control plane alarm is triggered to complete the abnormal state indication of the flow to which the current data packet belongs.

10. A programmable switch based micro-chronograph data measurement and detection system, characterized by, The system specifically comprises: A measurement deployment unit is configured to deploy a measurement module in a data plane, and determine whether a data packet belongs to a new time period based on a time of arrival of the data packet and a truncated timestamp in a bucket; An asynchronous updating unit is configured to trigger an asynchronous updating operation when it is determined that the data packet belongs to the new time period, extract a statistical value of a previous period, update the truncated timestamp in the bucket and reset a counter, and send the statistical value to a storage module; A sequence construction unit is configured to store the statistical value into a sliding window slot of a two-dimensional circular queue in the storage module based on a hash of a flow key and a modulo operation result of a current timestamp, and construct a time sequence data sequence of each flow; A time sequence statistical unit is configured to obtain the time sequence data sequence from the storage module based on a detection module, and calculate time sequence statistical features in the data plane; An anomaly detection unit is configured to input the time sequence statistical features and packet-level features into a machine learning model deployed in the data plane based on the detection module, perform inference, and output an anomaly detection result.

Images