A method and system for predicting network traffic based on flow-aware sketches
By using a flow-aware sketch structure and time-weighted feature selection method, the problems of flow level structure loss and high storage cost in network traffic prediction are solved, and efficient traffic prediction under fixed storage is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2026-06-05
- Publication Date
- 2026-07-03
AI Technical Summary
Existing network traffic prediction methods lose flow-level structure when aggregating traffic prediction inputs. Traditional sketches are difficult to retain key flow features under fixed storage, and maintaining complete flow-by-flow states is costly and cannot meet the requirements of online deployment.
A flow-aware sketch structure is adopted, and information on candidate recurrent flows and non-resident flows is stored through resident key-value items and small flow counters, respectively. Combined with time weights and feature filtering, a window-level multivariate feature input time series prediction model is generated.
By prioritizing the retention of recurrent flow information under fixed storage, the collision loss between recurrent and small flows is reduced, and the prediction model is enhanced to utilize flow level composition and tail change information, making it suitable for online traffic prediction in high-speed networks.
Smart Images
Figure CN122339985A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of network measurement, network traffic prediction, and data flow summarization, specifically to a network traffic prediction method and system based on flow-aware sketches. Background Technology
[0002] Network traffic prediction plays a crucial role in resource scheduling, congestion control, link planning, anomaly early warning, and quality of service assurance. As networks grow larger, numerous heterogeneous flows coexist, and the arrival, disappearance, bursts, and scale changes of these flows directly impact short-term aggregated traffic. Existing traffic prediction methods typically use aggregated packet counts, aggregated byte counts, or other coarse-grained time series data as input, focusing on improving downstream prediction models, such as recurrent neural networks, convolutional time series models, Transformer-like models, or multilayer perceptron-like models.
[0003] However, using only the aggregated sequence as input can obscure the flow-level structure. Bursts, decays, or switches of multiple different flows may exhibit similar total changes after aggregation, making it difficult for predictive models to distinguish whether future traffic changes are driven by a few heavy flows, several medium flows, or a large number of small flows. For high-speed networks, directly maintaining the complete flow-by-flow state requires significant storage and processing resources, making it difficult to meet online deployment requirements.
[0004] Traditional sketching structures, such as Count-Min Sketch, can approximate the counting of data flows with fixed storage, but their design goals are primarily flow counting and reflow estimation, rather than prediction-oriented feature representation. Hash collisions in traditional sketches mix reflows and background small flows in the same counter, reducing the ability to retain useful reflow activity, tail variations, and short-term burst structures. On the other hand, simply averaging or summing the observations at different second levels within a window ignores the varying contributions of different second-level snapshots to future flow predictions.
[0005] Therefore, there is a need for a network traffic prediction technology that can retain flow-level prediction signals under limited storage and online processing conditions and can be connected to various downstream prediction models. Summary of the Invention
[0006] The purpose of this invention is to provide a network traffic prediction method and system based on flow-aware sketches, in order to solve the problems of lost flow level structure in existing aggregated traffic prediction inputs, high maintenance cost of complete flow-by-flow state, and difficulty in preserving key flow features for prediction tasks using traditional sketches.
[0007] To achieve the above objectives, the present invention provides the following technical solution: A network traffic prediction method based on flow-aware sketches includes: Step 1: Obtain the incoming network data packet stream online, and extract the flow identifier and corresponding traffic count increment from each data packet; Step 2: Write the network data packet stream into the flow-aware sketch structure according to the preset time granularity to obtain a time-segmented sketch snapshot. The flow-aware sketch structure includes multiple buckets, each bucket including at least one resident key value item and at least one small flow counter. The resident key value item is used to store the flow identifier of the candidate reflow and its estimated flow value, and the small flow counter is used to store the aggregated flow value of the non-resident flow. Step 3: During the training phase, for each sketch snapshot, candidate statistical features are extracted from the resident key-value items and the small stream counter. Predictive features are selected based on the correlation between the candidate statistical features and the prediction target, and time weights are learned. Step 4, Online Prediction Stage: Within a time window containing multiple sketch snapshots, the prediction features at each time granularity are weighted and aggregated based on the time weights learned during the training stage and the selected prediction features to obtain window-level multivariate features. Step 5: Standardize the window-level multivariate features and input the standardized window-level multivariate feature sequence into the time series prediction model to obtain the predicted value of future network traffic.
[0008] Further, in step 2, writing the network data packet stream into the flow-aware sketch structure includes: S101, For the arriving data packet, locate the corresponding bucket using a hash function based on its flow identifier; S102, If the flow identifier already exists in the resident key value field of the corresponding bucket, then increase the estimated flow value in the resident key value field by the flow count increment; If the flow identifier does not exist in the resident key value field of the corresponding bucket and the corresponding bucket has an idle resident key value field, then insert the flow identifier and its initial estimated flow value into the idle resident key value field. If the flow identifier does not exist in the resident key value field of the corresponding bucket and there is no idle resident key value field in the corresponding bucket, the flow count increment is first added to the small flow counter, and the current sketch estimate of the flow identifier is compared with the minimum resident estimated flow value in the corresponding bucket. When the current sketch estimate is greater than the minimum resident estimated flow value, the corresponding minimum resident key value field is replaced with the flow identifier, and the replaced resident estimated flow value is replaced in the small flow counter.
[0009] Furthermore, each bucket includes three resident key-value items and a small stream counter.
[0010] Furthermore, the flow identifier includes at least one of source address, destination address, source port, destination port, and protocol type, and the traffic count increment includes a packet count increment or a byte count increment; The candidate statistical features include at least one of the following: number of packets, mean, standard deviation, maximum value, variance, quantile features, and reflow statistical features.
[0011] Furthermore, in step 3, the selection of predictive features based on the correlation between candidate statistical features and the prediction target includes: S201, calculate the Pearson correlation coefficient between each candidate statistical feature and the predicted target in the next time window when lagging by one time window on the training data; S202, calculate the cross-correlation between candidate statistical features; S203. Determine the set of features for prediction based on the correlation strength between candidate statistical features and the prediction target, cross-dataset stability, and complementarity among candidate statistical features.
[0012] Furthermore, the time weights satisfy non-negativity and normalization constraints, and are obtained by maximizing the average correlation between each predicted feature dimension after weighted aggregation and the predicted target. The time weights are learned only on the training set and are used consistently during the online phase. The optimization objective of the time weights is: , , , Where w is the weight coefficient to be optimized. This represents maximizing the solution by optimizing w to maximize the index within the parentheses, where m is the dimension of the predicted feature, and Corr(·) represents the Pearson correlation coefficient. This represents the training window set, where each time window contains T sketch snapshots, w i Let z be the time weight of the i-th sketch snapshot, and let z be the weighted aggregate feature value of the j-th feature within the t-th time window under the weighting coefficient of w. t,j (w), y t Let be the actual predicted target value in the t-th time window, and st represent the constraint.
[0013] Furthermore, given that the time window contains T sketch snapshots and the predicted feature dimension is m, let the predicted feature vector of the i-th sketch snapshot within the t-th time window be x. t,i The time weight is w i Then the window-level multivariate feature z t satisfy: ,in, And w i ≥0.
[0014] Furthermore, the time series prediction model can be any one of the following: recurrent neural network model, convolutional sequence model, Transformer-type model, or multilayer perceptron-type model; The time window contains a preset number of consecutive second-level sketch snapshots.
[0015] The present invention also provides a network traffic prediction system based on flow-aware sketches, for implementing the network traffic prediction method described above, comprising: The data acquisition module is used to acquire the network data packet streams arriving online and extract the flow identifier and traffic count increment; The sketch update module is used to write network data packet streams into the flow-aware sketch structure at a preset time granularity, and generate sketch snapshots segmented by time. The training configuration module is used to extract candidate statistical features from sketch snapshots during the training phase, filter prediction features based on the correlation between candidate statistical features and prediction targets, and learn time weights that satisfy non-negativity constraints and normalization constraints. The online feature generation module is used to perform weighted aggregation of the predicted features corresponding to multiple sketch snapshots within a time window according to the predicted features and time weights determined by the training configuration module during the online prediction stage, and generate window-level multivariate features. The prediction module is used to standardize window-level multivariate features and input the standardized multivariate feature sequence into the time series prediction model to output future network traffic prediction values.
[0016] Furthermore, the sketch update module includes a resident flow update unit, a free item insertion unit, and a candidate reflow replacement unit; The resident flow update unit is used to update the corresponding counter when an existing resident flow enters the bucket; The idle item insertion unit is used to insert newly entering traffic into the idle item when there is an idle resident item; The candidate reflow replacement unit is used to accumulate the remaining non-resident flows, and when the sketch estimate of a non-resident flow is greater than the minimum resident estimated flow value in the bucket, the non-resident flow is promoted to a resident flow, and the estimated flow value of the replaced resident flow is replaced in the small flow counter.
[0017] Compared with the prior art, the beneficial effects of the present invention are:
[0018] 1. Under a fixed storage budget, candidate reflow information is retained first, while the total background flow is retained through a small flow counter, reducing the loss of prediction information caused by the collision between reflow and small flow in traditional sketches;
[0019] 2. The sketch is extended from a simple counting structure to a prediction-oriented flow representation layer, enabling downstream prediction models to utilize additional information such as flow stage composition, tail changes, and reflow activity;
[0020] 3. Reduce redundant features through relevance screening, thereby lowering the input dimensionality and the burden on the prediction model;
[0021] 4. Emphasize second-level snapshots with greater predictive value through an in-window time weighting mechanism, avoiding the dilution of time information caused by simple averaging or summation;
[0022] 5. Decoupled from specific prediction models, it can be adapted to recurrent neural networks, convolutional models, Transformer-type models and MLP-type models;
[0023] 6. It does not require maintaining a complete flow state, making it suitable for online traffic measurement and short-term forecasting in high-speed networks. Attached Figure Description
[0024] Figure 1 This is a flowchart of a network traffic prediction method based on flow-aware sketches according to the present invention.
[0025] Figure 2 This is a schematic diagram of the HeavySketch bucket structure in one embodiment of the present invention.
[0026] Figure 3 This is a schematic diagram of feature correlation screening in one embodiment of the present invention. Detailed Implementation
[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0028] Please see Figure 1 A network traffic prediction method based on flow-aware sketches includes:
[0029] Step 1: Obtain the online incoming network data packet stream, and extract the flow identifier and corresponding traffic count increment from each data packet.
[0030] The flow identifier includes at least one of the following: source address, destination address, source port, destination port, and protocol type. The flow count increment includes either the packet count increment or the byte count increment.
[0031] Step 2: Write the network data packet stream into the flow-aware sketch structure according to the preset time granularity to obtain a time-segmented sketch snapshot. The flow-aware sketch structure includes multiple buckets, each bucket including at least one resident key value item and at least one small flow counter. The resident key value item is used to store the flow identifier of the candidate reflow and its estimated flow value, and the small flow counter is used to store the aggregated flow value of the non-resident flow.
[0032] Preferably, each bucket includes three resident key-value items and a small flow counter.
[0033] Writing a network packet stream into a stream-aware sketch structure includes:
[0034] S101, For the arriving data packet, locate the corresponding bucket using a hash function based on its flow identifier;
[0035] S102, if the flow identifier already exists in the resident key value field of the corresponding bucket, then increase the estimated flow value in the resident key value field by the flow count increment.
[0036] If the flow identifier does not exist in the resident key value field of the corresponding bucket and the corresponding bucket has an idle resident key value field, then insert the flow identifier and its initial estimated flow value into the idle resident key value field.
[0037] If the flow identifier does not exist in the resident key value field of the corresponding bucket and there is no idle resident key value field in the corresponding bucket, then the flow count increment is first added to the small flow counter, and the current sketch estimate of the flow identifier is compared with the minimum resident estimated flow value in the corresponding bucket. When the current sketch estimate is greater than the minimum resident estimated flow value, the corresponding minimum resident key value field is replaced with the flow identifier, and the replaced resident estimated flow value is added to the small flow counter. When the current sketch estimate is not greater than the minimum resident estimated flow value, no further operation is performed after adding the flow count increment to the small flow counter.
[0038] Step 3: During the training phase, for each sketch snapshot, candidate statistical features are extracted from the resident key-value items and the small stream counter. Predictive features are selected based on the correlation between the candidate statistical features and the prediction target, and time weights are learned.
[0039] Predictive features are selected based on the correlation between candidate statistical features and the prediction target, including:
[0040] S201, calculate the Pearson correlation coefficient between each candidate statistical feature and the predicted target in the next time window when lagging by one time window on the training data. The time window contains a preset number of consecutive second-level sketch snapshots.
[0041] S202, calculate the cross-correlation between candidate statistical features;
[0042] S203. Determine the set of features for prediction based on the correlation strength between candidate statistical features and the prediction target, cross-dataset stability, and complementarity among candidate statistical features.
[0043] The time weights satisfy nonnegativity and normalization constraints, and are obtained by maximizing the average correlation between each predicted feature dimension after weighted aggregation and the prediction target. The time weights are learned only on the training set and are used consistently during the online phase. The optimization objective of the time weights is:
[0044]
[0045]
[0046]
[0047] Where w is the weight coefficient to be optimized. This represents maximizing the solution by optimizing w to maximize the index within the parentheses, where m is the dimension of the predicted feature, and Corr(·) represents the Pearson correlation coefficient. This represents the training window set, where each time window contains T sketch snapshots, w i Let z be the time weight of the i-th sketch snapshot, and let z be the weighted aggregate feature value of the j-th feature within the t-th time window under the weighting coefficient of w. t,j (w), y t Let be the actual predicted target value in the t-th time window, and st represent the constraint.
[0048] Given a time window containing T sketch snapshots and a predicted feature dimension of m, let the predicted feature vector of the i-th sketch snapshot within the t-th time window be x. t,i The time weight is w i Then the window-level multivariate feature z t satisfy: ,in, And w i ≥0.
[0049] Step 4, Online Prediction Stage: Within a time window containing multiple sketch snapshots, the prediction features at each time granularity are weighted and aggregated based on the time weights learned during the training stage and the selected prediction features to obtain window-level multivariate features.
[0050] Step 5: Standardize the window-level multivariate features and input the standardized window-level multivariate feature sequence into the time series prediction model to obtain the predicted value of future network traffic.
[0051] The time series prediction model can be any one of the following: recurrent neural network model, convolutional sequence model, Transformer-type model, or multilayer perceptron-type model.
[0052] Continue reading Figure 1 The present invention also provides a network traffic prediction system based on flow-aware sketches to implement the network traffic prediction method described above, including a data acquisition module, a sketch update module, a training configuration module, an online feature generation module, and a prediction module, all of which are computer programs.
[0053] The data acquisition module is used to acquire the network data packet streams arriving online and extract the flow identifier and traffic count increment. The specific implementation of the data acquisition module refers to step 1 of the network traffic prediction method described above.
[0054] The sketch update module is used to write network data packet streams into a flow-aware sketch structure at a preset time granularity, generating time-segmented sketch snapshots. The buckets in the flow-aware sketch structure include resident key-value entries and small flow counters. The sketch update module includes a resident flow update unit, an idle entry insertion unit, and a candidate reflow replacement unit. Specifically, the resident flow update unit updates the corresponding counter when an existing resident flow enters a bucket; the idle entry insertion unit inserts newly entering traffic into an idle entry when an idle resident entry exists; and the candidate reflow replacement unit accumulates the remaining non-resident flows, and when the sketch estimate of a non-resident flow is greater than the minimum resident estimated flow value in the bucket, it promotes the non-resident flow to a resident flow and replaces the estimated flow value of the replaced resident flow in the small flow counter. The specific implementation of the sketch update module refers to step 2 of the network traffic prediction method described above.
[0055] The training configuration module is used to extract candidate statistical features from the sketch snapshot during the training phase, filter prediction features based on the correlation between the candidate statistical features and the prediction target, and learn time weights that satisfy non-negativity constraints and normalization constraints. The specific implementation of the training configuration module refers to step 3 of the network traffic prediction method described above.
[0056] The online feature generation module is used in the online prediction phase to perform weighted aggregation of the predicted features corresponding to multiple sketch snapshots within a time window, based on the predicted features and time weights determined by the training configuration module, to generate window-level multivariate features. The specific implementation of the online feature generation module refers to step 4 of the network traffic prediction method described above.
[0057] The prediction module is used to standardize window-level multivariate features and input the standardized multivariate feature sequence into the time series prediction model to output future network traffic prediction values. The specific implementation of the prediction module refers to step 5 of the network traffic prediction method described above.
[0058] Example 1: Online Traffic Feature Extraction and Prediction
[0059] In this embodiment, network devices, traffic acquisition servers, or gateway nodes receive data packets from mirror ports, sampling modules, or traffic acquisition programs. For each data packet, the system extracts a 5-tuple as a flow identifier, which includes the source IP address, destination IP address, source port, destination port, and protocol type.
[0060] The system maintains a flow-aware sketch, HeavySketch, for each time granularity, such as... Figure 2 As shown in the diagram, this sketch consists of d rows, each with w buckets. Each bucket contains k resident key-value items and a small flow counter. The resident key-value items include a flow identifier field and an estimated flow value field. The small flow counter does not distinguish between specific flow identifiers and is used to accumulate non-resident flow.
[0061] When a data packet arrives, the system calculates a hash value based on the flow identifier and locates the corresponding bucket. If the flow identifier exists in the bucket, the corresponding estimated flow value is directly incremented. If the flow identifier does not exist in the bucket but there is an idle resident key-value item, the flow is inserted into the idle item. If the flow identifier does not exist in the bucket and all resident items are occupied, the data packet increment is added to the small flow counter, and the value of the small flow counter is used as an approximate estimate of the current cumulative flow of the flow in the sketch. If this estimate is greater than the minimum resident estimated flow value in the bucket, the flow is considered more likely to become a candidate reflow useful for future predictions and is promoted to a resident flow; at the same time, the estimated flow value of the replaced resident item is replaced with the value of the small flow counter to avoid completely discarding background flow contributions.
[0062] The above process does not require maintaining an accurate global flow table, and the update complexity is controlled by the number of hash rows and the number of entries per bucket. When the number of entries per bucket is a small constant, the single-packet processing overhead is suitable for online deployment.
[0063] Secondly, the system generates sketch snapshots at a second-level granularity. Each sketch snapshot contains the resident entries and small flow counter states after the incoming data packets within that second have been written to the sketch. The system extracts candidate statistical features from each snapshot.
[0064] Candidate statistical features may include:
[0065] 1. The number of aggregated packets or bytes, used to reflect the overall traffic volume;
[0066] 2. The average of bucket counts or dwell times, used to reflect the overall load;
[0067] 3. Standard deviation and variance are used to reflect the degree of dispersion;
[0068] 4. Maximum value, used to reflect local extreme flow rates;
[0069] 5. Quantile characteristics such as p90 and p99 are used to reflect tail flow and suddenness;
[0070] 6. Top-k reflow statistical features, used to reflect the intensity of candidate reflow activity.
[0071] To avoid using future information, all feature selection in this embodiment is performed only on the training data. For each candidate feature, the system calculates its statistic within time window t-1 and the prediction target within time window t. The prediction target can be the number of packets in the next window.
[0072] Simultaneously, the system calculates the cross-correlation between candidate features. When some candidate features are highly correlated with the prediction target but highly redundant, the system prioritizes feature combinations with lower dimensionality, stronger interpretability, or greater stability across datasets. In this way, the system obtains compact prediction input, reducing the cost of model training and online inference.
[0073] In one implementation, according to Figure 3 The feature correlation coefficients shown indicate that three features—mean, p99, and top10—were selected as prediction features. Mean represents the overall load level, p99 represents extreme or tail flow, and top10 represents the average or cumulative activity intensity of the top ten candidate reflows. These features can be determined based on correlation analysis of the training data, or adjusted to other numbers or types of features depending on the network scenario.
[0074] A prediction window contains T consecutive second-level sketch snapshots. Instead of simply averaging these T snapshots, the system learns a shared temporal weight vector w = [w1, w2, ..., w...]. T Each weight represents the contribution of the corresponding second-level snapshot within the window to the future prediction target.
[0075] For each window t, the system represents the predicted features extracted from the i-th second-level snapshot as x. t,i The weighted window-level features are:
[0076]
[0077] The weights satisfy nonnegativity and normalization constraints. The system learns this weight vector on the training set to maximize the average correlation between the weighted features and the prediction target. After learning, the weights are fixed and used in validation, testing, and online phases. Since the weights are derived solely from the statistical relationships of the training set, future labels are not required during testing or online phases.
[0078] This time-weighting mechanism can highlight second-level observations that are more predictive within a window. For example, when candidate reflows are concentrated or tail flow increases significantly within a second, the sketch features of that second may be more indicative of the flow in the next window, and the time-weighting mechanism can enhance their contribution.
[0079] The system standardizes the weighted aggregated window-level multivariate features according to the training set mean and standard deviation, forming an input sequence of shape "lookback length × feature dimension". The downstream prediction model receives this sequence and outputs traffic prediction values for one or more future time steps.
[0080] This invention does not limit the type of downstream prediction model. Optional models include LSTM, GRU, Autoformer, TimesNet, iTransformer, TimeMixer, TSMixer, or other prediction models capable of handling multivariate time series inputs. Because the upstream output interface is unified, there is no need to modify the sketch update and feature extraction logic when replacing the prediction model.
[0081] Example 2: Deployment Method
[0082] This invention can be deployed on network edge devices, traffic acquisition servers, data center gateways, software-defined network controllers, or offline analysis platforms. For high-speed online scenarios, the sketch update module can be deployed close to the data packet acquisition end to reduce the pressure of retaining and transmitting original data; the feature extraction and prediction modules can be deployed on the same node or on a centralized analysis server.
[0083] The system can perform stream identifier hashing and sketching locally, outputting only aggregated statistical features or standardized feature sequences, thereby reducing the risk of exposing the original stream identifier.
[0084] Optional variations of the present invention:
[0085] 1. The number of resident key-value items in each bucket is not limited to three; it can be set to one, two, or more depending on the storage budget.
[0086] 2. The time granularity is not limited to the second level; it can be set to the millisecond, second, or minute level depending on the network sampling frequency.
[0087] 3. The length of the time window is not limited to five time granularities and can be adjusted according to the prediction target and network control cycle;
[0088] 4. The prediction target is not limited to the number of packets, but can also be the number of bytes, link utilization, queue length, or congestion risk indicators;
[0089] 5. Time weights can be retrained periodically to adapt to the shift in network traffic concepts;
[0090] 6. Standardization methods are not limited to mean-variance standardization; minimum-maximum normalization, robust standardization, or quantile normalization can also be used.
[0091] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for network traffic prediction based on flow-aware sketching, the method comprising: include: Step 1: Obtain the incoming network data packet stream online, and extract the flow identifier and corresponding traffic count increment from each data packet; Step 2: Write the network data packet stream into the flow-aware sketch structure according to the preset time granularity to obtain a time-segmented sketch snapshot. The flow-aware sketch structure includes multiple buckets, each bucket including at least one resident key value item and at least one small flow counter. The resident key value item is used to store the flow identifier of the candidate reflow and its estimated flow value, and the small flow counter is used to store the aggregated flow value of the non-resident flow. Step 3: During the training phase, for each sketch snapshot, candidate statistical features are extracted from the resident key-value items and the small stream counter. Predictive features are selected based on the correlation between the candidate statistical features and the prediction target, and time weights are learned. Step 4, Online Prediction Stage: Within a time window containing multiple sketch snapshots, the prediction features at each time granularity are weighted and aggregated based on the time weights learned during the training stage and the selected prediction features to obtain window-level multivariate features. Step 5: Standardize the window-level multivariate features and input the standardized window-level multivariate feature sequence into the time series prediction model to obtain the predicted value of future network traffic.
2. The method of claim 1, wherein, In step 2, writing the network data packet stream into the flow-aware sketch structure includes: S101, For the arriving data packet, locate the corresponding bucket using a hash function based on its flow identifier; S102, If the flow identifier already exists in the resident key value field of the corresponding bucket, then increase the estimated flow value in the resident key value field by the flow count increment; If the flow identifier does not exist in the resident key value field of the corresponding bucket and the corresponding bucket has an idle resident key value field, then insert the flow identifier and its initial estimated flow value into the idle resident key value field. If the flow identifier does not exist in the resident key value field of the corresponding bucket and there is no idle resident key value field in the corresponding bucket, the flow count increment is first added to the small flow counter, and the current sketch estimate of the flow identifier is compared with the minimum resident estimated flow value in the corresponding bucket. When the current sketch estimate is greater than the minimum resident estimated flow value, the corresponding minimum resident key value field is replaced with the flow identifier, and the replaced resident estimated flow value is replaced in the small flow counter.
3. The method of claim 1, wherein, Each bucket includes three resident key-value items and a small stream counter.
4. The method of claim 1, wherein, The flow identifier includes at least one of source address, destination address, source port, destination port, and protocol type; the traffic count increment includes packet count increment or byte count increment. The candidate statistical features include at least one of the following: number of packets, mean, standard deviation, maximum value, variance, quantile features, and reflow statistical features.
5. The network traffic prediction method based on flow-aware sketches according to claim 1, characterized in that, In step 3, the selection of prediction features based on the correlation between candidate statistical features and the prediction target includes: S201, calculate the Pearson correlation coefficient between each candidate statistical feature and the predicted target in the next time window when lagging by one time window on the training data; S202, calculate the cross-correlation between candidate statistical features; S203. Determine the set of features for prediction based on the correlation strength between candidate statistical features and the prediction target, cross-dataset stability, and complementarity among candidate statistical features.
6. The network traffic prediction method based on flow-aware sketches according to claim 1, characterized in that, The time weights satisfy nonnegativity and normalization constraints, and are obtained by maximizing the average correlation between each predicted feature dimension after weighted aggregation and the predicted target. The time weights are learned only on the training set and are used consistently during the online phase. The optimization objective of the time weights is: ， ， ， Where w is the weight coefficient to be optimized. This represents maximizing the solution by optimizing w to maximize the index within the parentheses, where m is the dimension of the predicted feature, and Corr(·) represents the Pearson correlation coefficient. This represents the training window set, where each time window contains T sketch snapshots, w i Let z be the time weight of the i-th sketch snapshot, and let z be the weighted aggregate feature value of the j-th feature within the t-th time window under the weighting coefficient of w. t,j (w), y t Let be the actual predicted target value in the t-th time window, and st represent the constraint.
7. The network traffic prediction method based on flow-aware sketches according to claim 6, characterized in that, Given a time window containing T sketch snapshots and a predicted feature dimension of m, let the predicted feature vector of the i-th sketch snapshot within the t-th time window be x. t,i The time weight is w i Then the window-level multivariate feature z t satisfy: ,in, And w i ≥0.
8. The network traffic prediction method based on flow-aware sketches according to claim 1, characterized in that, The time series prediction model can be any one of the following: recurrent neural network model, convolutional sequence model, Transformer-type model, or multilayer perceptron-type model. The time window contains a preset number of consecutive second-level sketch snapshots.
9. A network traffic prediction system based on flow-aware sketches, used to implement the network traffic prediction method as described in any one of claims 1-8, characterized in that, include: The data acquisition module is used to acquire the network data packet streams arriving online and extract the flow identifier and traffic count increment; The sketch update module is used to write network data packet streams into the flow-aware sketch structure at a preset time granularity, and generate sketch snapshots segmented by time. The training configuration module is used to extract candidate statistical features from sketch snapshots during the training phase, filter prediction features based on the correlation between candidate statistical features and prediction targets, and learn time weights that satisfy non-negativity constraints and normalization constraints. The online feature generation module is used to perform weighted aggregation of the predicted features corresponding to multiple sketch snapshots within a time window according to the predicted features and time weights determined by the training configuration module during the online prediction stage, and generate window-level multivariate features. The prediction module is used to standardize window-level multivariate features and input the standardized multivariate feature sequence into the time series prediction model to output future network traffic prediction values.
10. A network traffic prediction system based on flow-aware sketches according to claim 9, characterized in that, The sketch update module includes a resident flow update unit, a free item insertion unit, and a candidate reflow replacement unit; The resident flow update unit is used to update the corresponding counter when an existing resident flow enters the bucket; The idle item insertion unit is used to insert newly entering traffic into the idle item when there is an idle resident item; The candidate reflow replacement unit is used to accumulate the remaining non-resident flows, and when the sketch estimate of a non-resident flow is greater than the minimum resident estimated flow value in the bucket, the non-resident flow is promoted to a resident flow, and the estimated flow value of the replaced resident flow is replaced in the small flow counter.