Network traffic dataset generation method and system based on small sample and diffusion model
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Chinese People's Liberation Army Cyberspace Force Information Engineering University
- Filing Date
- 2026-06-03
- Publication Date
- 2026-08-07
AI Technical Summary
[0004]针对现有有技术存在的在小样本场景下生成数据质量不稳定、易引入噪声的技术问题，本发明提出了一种基于小样本和扩散模型的网络流量数据集生成方法及系统，实现了对离散特征和连续特征的差异化处理，并通过筛选机制剔除低质量样本，从而提升了数据的可用性与下游任务的准确性
[0034]本发明提供的方法仅需少量真实流量数据即可生成高质量的网络流量数据集，有效缓解高质量标注数据稀缺及特殊领域数据难获取的问题。通过将类别特征（源端口、目的端口、协议）与连续特征分别处理，并利用嵌入层学习类别特征间的潜在关联，在完整保留流量数据的语义信息的同时，避免了传统独热编码导致的高维稀疏问题。通过引入条件机制，控制生成数据的类别分布，提高了生成数据的实用性和可控性。并且还结合数据集蒸馏技术从大规模生成数据中提炼出高质量的样本，进一步提高了生成数据的纯度和多样性，使生成的流量数据更贴近真实网络环境。
Smart Images
Figure CN122533975A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a method and system for generating network traffic datasets based on small sample size and diffusion models. Background Technology
[0002] Detecting attacks and other dangerous behaviors in network traffic has always been a hot topic in the field of cybersecurity. However, the limitations of datasets pose some challenges to deep learning research. Current methods for detecting intrusions and dangerous behaviors based on network traffic face two major challenges: first, the scarcity of high-quality labeled data, with some datasets often difficult to share due to privacy restrictions, or failing to reflect real user behavior and new attack patterns due to outdated scenarios; second, the difficulty in obtaining large quantities of traffic datasets for specific domains, which limits the training of supervised classification algorithms and lacks evaluation benchmarks for unsupervised anomaly detection methods.
[0003] To address the problem of data scarcity, generative data augmentation methods have been extensively studied. Existing techniques typically employ generative models such as Generative Adversarial Networks (GANs) to simulate large-scale real-world network traffic. However, GAN models often face issues like pattern collapse and unstable generation quality in small-sample scenarios, resulting in insufficient diversity of generated traffic data and an inability to cover the complex distribution of real-world traffic. Furthermore, existing generative methods often directly process mixed features, ignoring the fundamental differences between discrete features (such as port numbers and protocol types) and continuous features (such as flow duration) in network traffic data, leading to the loss of semantic information in the generated data. Simultaneously, existing techniques lack effective mechanisms for screening the quality of generated data; low-quality samples may introduce noise, thereby reducing the accuracy of downstream detection tasks. Therefore, how to generate high-quality, high-fidelity network traffic data in data-scarce scenarios has become an urgent technical problem to be solved. Summary of the Invention
[0004] To address the technical problems of unstable data quality and easy introduction of noise in existing technologies in small sample scenarios, this invention proposes a method and system for generating network traffic datasets based on small sample and diffusion models. It realizes differentiated processing of discrete and continuous features, and removes low-quality samples through a screening mechanism, thereby improving the usability of data and the accuracy of downstream tasks.
[0005] In a first aspect, the present invention provides a method for generating network traffic datasets based on small sample size and a diffusion model, comprising:
[0006] Collect real network traffic data and label it to obtain small sample network traffic data, and preprocess the small sample network traffic data;
[0007] The preprocessed small sample network traffic data is input into a preset MLP-based conditional diffusion model to obtain synthetic network traffic data; wherein, the MLP-based conditional diffusion model includes an embedding layer, a time coding layer and an MLP network;
[0008] The quality of the synthetic network traffic data is evaluated using a pre-trained classification model, and synthetic traffic data that meets preset conditions is selected as the target network traffic dataset based on the quality evaluation results.
[0009] Furthermore, the preprocessing includes: labeling the category data in the real network traffic data and standardizing the continuous data in the real network traffic data.
[0010] Furthermore, the preprocessed small-sample network traffic data is input into a pre-defined MLP-based conditional diffusion model to obtain synthetic network traffic data, specifically including:
[0011] The preprocessed small sample network traffic data is mapped through the embedding layer to obtain a true representation of traffic features;
[0012] The sampling diffusion time step is used to forward noise the real flow feature representation according to the time step to obtain a noisy flow feature representation;
[0013] The time step is mapped to a time vector through the time coding layer, and then concatenated with the preprocessed small sample network traffic data, the noisy traffic feature representation, and the label of the real network traffic data to obtain the conditional feature representation;
[0014] The conditional feature representation is passed through the MLP network for noise prediction, and the noisy traffic feature representation is denoised based on the prediction results to obtain synthetic network traffic data.
[0015] Furthermore, the MLP-based conditional diffusion model is pre-trained using a classifier-free guidance strategy to discard the labels of the real network traffic data with a preset probability.
[0016] Furthermore, the loss function for training the MLP-based conditional diffusion model is:
[0017]
[0018] In the formula,
[0019]
[0020] in, Represents the loss function. A conditional label indicating that the condition is empty. Expressing expectations, Indicates Gaussian noise. Indicates input At time step and conditions Below noise The predicted value, Represents the original data. The cumulative coefficient representing the diffusion process. This represents the sample at time t.
[0021] Furthermore, the pre-trained classification model uses a teacher model trained on a real traffic dataset;
[0022] The preset conditions include: the prediction confidence level is greater than a preset threshold, and the predicted label is consistent with the true label.
[0023] Furthermore, the step of using a pre-trained classification model to perform quality assessment on the synthetic network traffic data, and selecting synthetic traffic data that meets preset conditions as the target network traffic dataset based on the quality assessment results, includes:
[0024] Calculate the predicted probability distribution of the teacher model for the synthetic traffic data;
[0025] The predicted label and prediction confidence level are determined based on the predicted probability distribution;
[0026] Filter the synthetic traffic data whose prediction confidence is greater than a preset threshold and whose prediction label is consistent with the generated label.
[0027] Secondly, the present invention provides a network traffic dataset generation system based on a small sample size and a diffusion model, comprising:
[0028] The real traffic data collection module is used to collect real network traffic data and label it to obtain small sample network traffic data, and to preprocess the small sample network traffic data.
[0029] The synthetic traffic data generation module is used to input preprocessed small sample network traffic data into a preset MLP-based conditional diffusion model to obtain synthetic network traffic data; wherein, the MLP-based conditional diffusion model includes an embedding layer, a time coding layer, and an MLP network;
[0030] The synthetic traffic data filtering module is used to evaluate the quality of the synthetic network traffic data using a pre-trained classification model, and select synthetic traffic data that meets preset conditions as the target network traffic dataset based on the quality evaluation results.
[0031] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as described in the first aspect.
[0032] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.
[0033] The beneficial effects of this invention are as follows:
[0034] The method provided by this invention can generate high-quality network traffic datasets with only a small amount of real traffic data, effectively alleviating the problems of scarcity of high-quality labeled data and difficulty in obtaining data in special domains. By processing categorical features (source port, destination port, protocol) and continuous features separately, and utilizing embedding layers to learn the potential correlations between categorical features, the semantic information of the traffic data is fully preserved while avoiding the high-dimensional sparsity problem caused by traditional one-hot encoding. By introducing a conditional mechanism to control the category distribution of the generated data, the practicality and controllability of the generated data are improved. Furthermore, by combining dataset distillation technology to extract high-quality samples from large-scale generated data, the purity and diversity of the generated data are further improved, making the generated traffic data closer to the real network environment. Attached Figure Description
[0035] Figure 1 A flowchart illustrating a method for generating network traffic datasets based on a small sample size and diffusion model, provided in an embodiment of the present invention;
[0036] Figure 2 This is a schematic diagram of the data generation process provided in an embodiment of the present invention;
[0037] Figure 3 This is a schematic diagram of the noise processing flow provided in an embodiment of the present invention;
[0038] Figure 4 A schematic diagram illustrating the DDPM generation process provided in an embodiment of the present invention;
[0039] Figure 5 This is a schematic diagram of a data diversification processing flow provided in an embodiment of the present invention;
[0040] Figure 6 The accuracy variation graph of the method of the present invention during training with a ratio of [0.5, 0.5] (0 / 1) is provided for embodiments of the present invention.
[0041] Figure 7 The accuracy variation graph of the CGAN method provided in this embodiment of the invention during training with a ratio of [0.5, 0.5] (0 / 1);
[0042] Figure 8 The accuracy variation graph of the method of the present invention during training with a ratio of [0.6, 0.4] (0 / 1) is provided for embodiments of the present invention.
[0043] Figure 9 The accuracy variation graph of the CGAN method provided in this embodiment of the invention during training with a ratio of [0.6, 0.4] (0 / 1);
[0044] Figure 10 The accuracy variation graph of the method of the present invention during training with a ratio of [0.7, 0.3] (0 / 1) is provided for embodiments of the present invention.
[0045] Figure 11 The accuracy variation graph of the CGAN method provided in this embodiment of the invention during training with a ratio of [0.7, 0.3] (0 / 1);
[0046] Figure 12 The accuracy variation graph of the method of the present invention during training with a ratio of [0.8, 0.2] (0 / 1) is provided for embodiments of the present invention.
[0047] Figure 13 The accuracy variation graph of the CGAN method provided in this embodiment of the invention during training with a ratio of [0.8, 0.2] (0 / 1);
[0048] Figure 14 The accuracy variation graph of the method of the present invention during training with a ratio of [0.9, 0.1] (0 / 1) is provided for embodiments of the present invention.
[0049] Figure 15 The accuracy variation graph of the CGAN method provided in this embodiment of the invention during training with a ratio of [0.9, 0.1] (0 / 1);
[0050] Figure 16 A structural diagram of a network traffic dataset generation system based on a small sample size and diffusion model provided in an embodiment of the present invention;
[0051] Figure 17 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0053] like Figure 1 As shown, this embodiment of the invention provides a method for generating network traffic datasets based on small sample size and diffusion models, including:
[0054] S101: Collect real network traffic data and label it to obtain small sample network traffic data, and preprocess the small sample network traffic data; wherein, the preprocessing includes: labeling and encoding the categorical data in the real network traffic data, and standardizing the continuous data in the real network traffic data.
[0055] Specifically, such as Figure 2 and Figure 5 As shown, a separate label encoder is used to process category features such as source port, destination port, and transport protocol. Let the original feature dimensions be... , , After being tagged, an integer index is obtained. ,in The sample size is M. For M continuous features... z-score standardization is used: ,in, and These are the mean and standard deviation of the training set, respectively.
[0056] Understandably, standardization is a crucial step in data preprocessing, aiming to transform data of different scales, units, and distributions into a common benchmark. This ensures that features are treated fairly during analysis or modeling, improving algorithm performance and stability. This transformation eliminates technical noise and systematic biases introduced by measurement tools, environment, or inherent statistical characteristics in the raw data, thereby revealing the true underlying patterns. Data standardization is not about blindly homogenizing data, but a deliberate process designed to achieve comparisons while preserving essential information.
[0057] Z-score standardization (also known as standardization or z-score transformation) is the most commonly used standardization method. It transforms data into a distribution with a mean of 0 and a standard deviation of 1 by representing each data point as the number of standard deviations from the mean. Z-score standardization is crucial for many distance-based machine learning algorithms, such as K-Nearest Neighbors (k-NN), Support Vector Machines (SVM), and Principal Component Analysis (PCA). Unstandardized data can lead to features with larger numerical values dominating the learning process, resulting in biased and inaccurate models.
[0058] S102: Input the preprocessed small sample network traffic data into a preset MLP-based conditional diffusion model to obtain synthetic network traffic data; wherein, the MLP-based conditional diffusion model includes an embedding layer, a time coding layer and an MLP network.
[0059] The preprocessed small sample network traffic data is input into a pre-defined MLP-based conditional diffusion model, such as... Figure 2 and Figure 3 As shown, its processing flow includes:
[0060] S102.1: The preprocessed small sample network traffic data is mapped through the embedding layer to obtain a true representation of traffic features.
[0061] Specifically, embedding matrices are constructed for the features of the input small sample network traffic data:
[0062]
[0063] in, Represents the embedding dimension, and the embedding process is as follows: And concatenate the embedding matrices to obtain .
[0064] It is understandable that key attributes extracted from real network traffic data (such as IP address, destination port, and transmission protocol) constitute the features of the model's input data. These discrete categorical features are mapped to integer indices during preprocessing, collectively forming the model's input vocabulary. Traditional methods typically represent each feature value as a high-dimensional, sparse, one-hot vector, leading to a sharp expansion of the input dimensionality, low computational efficiency, and an inability to capture potential correlations between features. Therefore, this invention employs an embedding layer to encode the input features.
[0065] Each feature value of the input data (such as a specific IP address or port number) after passing through the embedding layer is transformed into a low-dimensional, dense, continuous vector representation. This embedding layer is essentially a trainable lookup table with the number of rows equal to the vocabulary size of the corresponding feature, and the number of columns (i.e., the embedding dimension) is a hyperparameter. This significantly reduces the input dimensionality compared to the large number of multidimensional sparse vectors required by traditional one-hot encoding. In this architecture, the embedding layer serves as the first layer of the model, and its weights are optimized through backpropagation during training. This allows the model to automatically learn semantically informative feature representations: in the embedding space, semantically or functionally similar feature values (e.g., ports 80 and 443 commonly used in web services) have geometrically similar vector representations. This distributed representation, adaptively learned by the model, provides more informative feature inputs for downstream classification tasks, ultimately outputting the probability of each vocabulary value appearing in a given context through a softmax classifier.
[0066] S102.2: Sampling diffusion time step, which adds noise to the real flow feature representation according to the time step to obtain a noisy flow feature representation.
[0067] S102.3: The time step is mapped to a time vector through a time coding layer, and then concatenated with the preprocessed small sample network traffic data, the noisy traffic feature representation, and the labels of the real network traffic data to obtain the conditional feature representation.
[0068] Specifically, the time step Mapped to a 256-dimensional vector:
[0069]
[0070] in, The total number of diffusion steps, in the embodiments of the present invention .
[0071] Tags are also embedded into real network traffic data to generate... For unlabeled real network traffic data, use learnable empty embeddings. .
[0072] like Figure 5 As shown, the preprocessed small sample network traffic data, noisy traffic feature representation, time vector, and label embedding of real network traffic data are concatenated to obtain the conditional feature representation.
[0073] S102.4: The conditional feature representation is passed through an MLP network for noise prediction, and the noisy traffic feature representation is denoised based on the prediction results to obtain synthetic network traffic data.
[0074] Specifically, the conditional features represent the predicted noise with the same dimension as the input after passing through the MLP network.
[0075] Diffusion models are understandably a type of generative model inspired by nonequilibrium thermodynamics. These models progressively add noise to the data to disrupt its structure, then learn the inverse process—that is, progressively recovering the data structure from the noise to generate new samples. DDPM (Denoising Diffusion Probability Model) demonstrates superior performance in high-quality image synthesis and provides a natural, progressive lossy decompression scheme. Figure 4This describes the generation process of DDPM. The diffusion model takes random Gaussian noise as input and gradually produces meaningful real data. Each step in this process is stochastic, resulting in different data outputs each time, leading to excellent data diversity. However, this also has a drawback: the generated data is uncontrollable, making it impossible to control the generation process and produce the desired data content. Additional information can be introduced into the diffusion process to guide the entire model. Let's assume this additional information is... It can be a category label, introduced The subsequent model became a... The conditional probability distribution is given by the condition.
[0076] S103: Use a pre-trained classification model to evaluate the quality of synthetic network traffic data, and select synthetic traffic data that meets preset conditions as the target network traffic dataset based on the quality evaluation results. The pre-trained classification model uses a teacher model trained on a real traffic dataset; the preset conditions include: prediction confidence greater than a preset threshold, and prediction labels consistent with the true labels.
[0077] Specifically, the pre-trained classification model trains multiple classifiers on real traffic datasets, including random forests, gradient boosting trees, and K-nearest neighbors, and selects the model with the best performance on the validation set as the teacher model. The selection process includes:
[0078] Calculate the predicted probability distribution of the teacher model for synthetic traffic data;
[0079] The predicted label and prediction confidence level are determined based on the predicted probability distribution;
[0080] Screening prediction confidence greater than preset threshold And the predicted labels are consistent with the generated labels. Synthetic traffic data. Among them, Indicates the confidence level. Indicates the predicted label, This indicates the generated tag, i.e., the actual tag.
[0081] Understandably, this step involves the final selection of the generated dataset through dataset distillation. Dataset distillation, as a dataset reduction method, addresses the problem of excessively large and imbalanced datasets by synthesizing a small, representative dataset from a large amount of data. Existing dataset distillation methods can be categorized into meta-learning and data matching frameworks based on whether they explicitly mimic the performance of the target data. Although dataset distillation has demonstrated impressive performance in compressing datasets, it still has some limitations, such as distilling high-resolution data or data with complex label spaces. It can be seen that traditional data distillation primarily aims to address the high cost and complexity issues arising from excessively large datasets. This invention, however, filters the generated original dataset through dataset distillation, synthesizing an ideal dataset with excessively noisy data removed.
[0082] As one possible implementation method, such as Figure 5 As shown, the MLP-based conditional diffusion model is pre-trained using a classifier-free strategy, discarding labels from real network traffic data with a preset probability. This embodiment uses probability... Discard conditional labels randomly.
[0083] The loss function for training the conditional diffusion model based on MLP is:
[0084]
[0085] In the formula,
[0086]
[0087] in, Represents the loss function. A conditional label indicating that the condition is empty. Expressing expectations, Indicates Gaussian noise. Indicates input At time step and conditions Below noise The predicted value, Represents the original data. The cumulative coefficient representing the diffusion process. This represents the sample at time t.
[0088] To verify the effectiveness of the method provided by this invention, this embodiment is verified through experiments.
[0089] 1. Abnormal network traffic feature dataset
[0090] The dataset contains network traffic characteristics generated by abnormal network behavior. Table 1 shows the basic characteristics of the dataset.
[0091] Table 1. Some typical features of the dataset
[0092]
[0093] To verify the effectiveness of the method provided by this invention, this embodiment uses conditional Wasserstein distance as an evaluation metric and verifies it in a downstream classification task.
[0094] The similarity between the generated data in the target network traffic dataset and the actual traffic-based network dataset distribution is evaluated using the conditional Wasserstein distance.
[0095] The conditional Wasserstein distance, also known as the bulldozer distance, is a powerful tool for measuring the difference between two probability distributions. It originates from optimal transport theory, and its core idea is to calculate the minimum "work" required to "transport" one probability distribution (imagine a pile of soil) into another.
[0096] Its mathematical definition is as follows: the goal is to find an optimal "transportation plan":
[0097]
[0098] in, and Let represent two probability distributions to be compared. Let represent the set of all possible joint distributions (also known as couplings or transportation plans), whose marginal distributions are exactly . and . It can be understood as from Points in Transport to Points in What is its mass? Represents the midpoint of the metric space and points The distance between them (such as Euclidean distance). It is an order parameter. The most commonly used are the first-order Wasserstein distance (p=1) and the second-order Wasserstein distance (p=2).
[0099] Unlike information-theoretic divergences such as KL divergence and JS divergence, which only consider the proportion of probability values, Wasserstein distance takes into account the geometric structure of the space in which the distribution resides. For example, even if two distributions have similar shapes, Wasserstein distance can accurately capture any spatial offset between them. The most crucial advantage of Wasserstein distance is its ability to handle non-overlapping distributions. When the support sets (regions with non-zero probability density) of two distributions do not overlap, KL divergence and JS divergence become ineffective (KL divergence is infinite, and JS divergence is a constant log2), leading to vanishing gradients and preventing the model from learning. However, Wasserstein distance, even in such cases, smoothly reflects the proximity between distributions, providing an effective gradient signal.
[0100] The core objective of conditional Wasserstein distance is to compare two distributions under specific conditions. From a computational perspective, conditional Wasserstein distance is typically achieved by comparing conditional distributions. Taking the discrete label case as an example, the calculation of two distributions... and Under given conditions The conditional Wasserstein distance is calculated separately for each category. Below, the conditional distributions corresponding to the two distributions. and The Wasserstein distance between them is calculated, and then a weighted average is taken according to the class probabilities. Its mathematical form can be expressed as:
[0101]
[0102] in, express and The standard Wasserstein distance calculation, based on conditional distributions, decomposes the global distribution comparison into comparisons within each domain. First, the data is divided into groups according to conditional variables (such as class labels). Then, for each group, the Wasserstein distance between their conditional distributions is calculated. Finally, these conditional distances are weighted and summed according to their prior probabilities to obtain the global conditional Wasserstein distance.
[0103] The conditional Wasserstein distance also has a dual form. This is very useful in generative adversarial networks, such as in Conditional WGAN (CWGAN). Its objective function is typically of the form:
[0104]
[0105] Among them, the discriminator It is a function that satisfies Lipschitz constraints and receives not only data samples It also receives condition information. The training objective is to enable the discriminator to distinguish data from conditional distributions. and The first evaluation compares the similarity between the generated data and the actual traffic-based network dataset distribution when different label ratios are used. Therefore, we calculate the conditional Wasserstein distance between the probability distribution of the generated data and the real dataset. The conditional Wasserstein distance was chosen to avoid issues such as JS divergence caused by non-overlapping distributions. The results are shown in Table 2.
[0106] As shown in Table 2, under different label ratios (e.g., [0.5, 0.5], [0.9, 0.1]), the conditional Wasserstein distance of the data generated by this invention is significantly lower than that of the traditional CGAN method. For example, when the label ratio is [0.5, 0.5], the conditional Wasserstein distance of this invention is 0.0874, while that of CGAN is 0.4438. This indicates that the data distribution generated by this invention is closer to the real data distribution, effectively avoiding the evaluation failure problem caused by non-overlapping distributions.
[0107] Table 2 Wassertein distance for different label ratios
[0108]
[0109] Table 3. Accuracy for a ratio of [0.5, 0.5] (0 / 1)
[0110]
[0111] Table 4. Accuracy for ratios of [0.6, 0.4] (0 / 1)
[0112]
[0113] Table 5. Accuracy for ratios of [0.7, 0.3] (0 / 1)
[0114]
[0115] Table 6. Accuracy for a ratio of [0.8, 0.2] (0 / 1)
[0116]
[0117] Table 7. Accuracy for a ratio of [0.9, 0.1] (0 / 1).
[0118]
[0119] Augmented data was used to train various downstream classifiers (such as AdaBoost, ExtraTrees, GradientBoosting, KNN, and RandomForest), and tested on the real test set. Experimental results are shown in Tables 3 to 7. Figures 6 to 15 As shown, the augmented data generated by this invention achieved extremely high accuracy in downstream tasks. Particularly in the extremely imbalanced scenario with a label ratio of [0.9, 0.1], the RandomForest classifier trained using the augmented data of this invention achieved an accuracy of 1.0, and the GradientBoosting classifier achieved an accuracy of 0.9953. In contrast, the data generated by the traditional CGAN method performed extremely poorly in downstream tasks, with an accuracy generally around 0.35, making it difficult to effectively support model training. This result strongly demonstrates that the combined strategy of "diffusion model generation + teacher model selection" of this invention effectively solves the problem of unstable data quality in small sample scenarios, and the generated augmented data can significantly improve the performance of downstream network traffic analysis tasks.
[0120] like Figure 16 As shown, this embodiment of the invention provides a network traffic dataset generation system based on a small sample size and a diffusion model, comprising:
[0121] The real traffic data collection module is used to collect real network traffic data and label it to obtain small sample network traffic data, and to preprocess the small sample network traffic data.
[0122] The synthetic traffic data generation module is used to input preprocessed small sample network traffic data into a preset MLP-based conditional diffusion model to obtain synthetic network traffic data; wherein, the MLP-based conditional diffusion model includes an embedding layer, a time coding layer and an MLP network;
[0123] The synthetic traffic data filtering module is used to evaluate the quality of synthetic network traffic data using a pre-trained classification model, and select synthetic traffic data that meets preset conditions as the target network traffic dataset based on the quality evaluation results.
[0124] It should be noted that the network traffic dataset generation system provided in this embodiment of the invention is for implementing the above method, and its specific functions can be referred to the above method embodiments, which will not be repeated here.
[0125] Figure 17 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 17As shown, the electronic device may include a processor 1701, a communications interface 1702, a memory 1703, and a communication bus 1704. The processor 1701, communications interface 1702, and memory 1703 communicate with each other via the communication bus 1704. The processor 1701 can call logical instructions stored in the memory 1703 to execute a network traffic dataset generation method, which includes steps S101 to S103 of the above embodiments.
[0126] Furthermore, when the logical instructions in the aforementioned memory 1703 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0127] This invention also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the network traffic dataset generation method provided in the above-described method embodiments, including S101 to S103 of the above embodiments.
[0128] This invention also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the network traffic dataset generation method provided in the above-described method embodiments, including S101 to S103 of the above embodiments.
[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for generating network traffic datasets based on small sample size and diffusion models, characterized in that, include: Collect real network traffic data and label it to obtain small sample network traffic data, and preprocess the small sample network traffic data; The preprocessed small sample network traffic data is input into a preset MLP-based conditional diffusion model to obtain synthetic network traffic data; wherein, the MLP-based conditional diffusion model includes an embedding layer, a time coding layer and an MLP network; The quality of the synthetic network traffic data is evaluated using a pre-trained classification model, and synthetic traffic data that meets preset conditions is selected as the target network traffic dataset based on the quality evaluation results.
2. The method for generating network traffic datasets based on small sample size and diffusion model according to claim 1, characterized in that, The preprocessing includes: labeling the category data in the real network traffic data and standardizing the continuous data in the real network traffic data.
3. The method for generating network traffic datasets based on small sample size and diffusion model according to claim 1, characterized in that, The preprocessed small sample network traffic data is input into a pre-defined MLP-based conditional diffusion model to obtain synthetic network traffic data, specifically including: The preprocessed small sample network traffic data is mapped through the embedding layer to obtain a true representation of traffic features; The sampling diffusion time step is used to forward noise the real flow feature representation according to the time step to obtain a noisy flow feature representation; The time step is mapped to a time vector through the time coding layer, and then concatenated with the preprocessed small sample network traffic data, the noisy traffic feature representation, and the label of the real network traffic data to obtain the conditional feature representation; The conditional feature representation is passed through the MLP network for noise prediction, and the noisy traffic feature representation is denoised based on the prediction results to obtain synthetic network traffic data.
4. The method for generating network traffic datasets based on small sample size and diffusion model according to claim 1 or 3, characterized in that, The MLP-based conditional diffusion model is pre-trained using a classifier-free guidance strategy, and the labels of the real network traffic data are discarded with a preset probability.
5. The method for generating network traffic datasets based on small sample size and diffusion model according to claim 4, characterized in that, The loss function for training the MLP-based conditional diffusion model is: In the formula, in, Represents the loss function. A conditional label indicating that the condition is empty. Expressing expectations, Indicates Gaussian noise. Indicates input At time step and conditions Below noise The predicted value, Represents the original data. The cumulative coefficient representing the diffusion process. This represents the sample at time t.
6. The method for generating network traffic datasets based on small sample size and diffusion model according to claim 1, characterized in that, The pre-trained classification model uses a teacher model trained on a real traffic dataset; The preset conditions include: the prediction confidence level is greater than a preset threshold, and the predicted label is consistent with the true label.
7. The method for generating network traffic datasets based on small sample size and diffusion model according to claim 1, characterized in that, The process of using a pre-trained classification model to perform quality assessment on the synthetic network traffic data, and selecting synthetic traffic data that meets preset conditions as the target network traffic dataset based on the quality assessment results, includes: Calculate the predicted probability distribution of the teacher model for the synthetic traffic data; The predicted label and prediction confidence level are determined based on the predicted probability distribution; Filter the synthetic traffic data whose prediction confidence is greater than a preset threshold and whose predicted label is consistent with the generated label.
8. A network traffic dataset generation system based on small sample size and diffusion model, characterized in that, include: The real traffic data collection module is used to collect real network traffic data and label it to obtain small sample network traffic data, and to preprocess the small sample network traffic data. The synthetic traffic data generation module is used to input preprocessed small sample network traffic data into a preset MLP-based conditional diffusion model to obtain synthetic network traffic data; wherein, the MLP-based conditional diffusion model includes an embedding layer, a time coding layer, and an MLP network; The synthetic traffic data filtering module is used to evaluate the quality of the synthetic network traffic data using a pre-trained classification model, and select synthetic traffic data that meets preset conditions as the target network traffic dataset based on the quality evaluation results.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.