Network flow monitoring and identifying method based on unbalanced data flow

Abstract

The invention provides a network flow monitoring and identifying method based on unbalanced data flow, which comprises the following steps of: acquiring network flow data of each network node, storing the network flow data in a structured format, and marking the acquired network flow data; generating a training sample according to the collected network flow data, and generating spatial distribution of the sample by adopting a sample distribution adaptive correction method and through dynamic adjustment; training a network traffic data expansion model, and increasing the number of samples according to the network traffic data expansion model; and inputting the network traffic data after feature extraction into a classifier for classifier model training, and performing network traffic monitoring identification by using the trained classifier model. According to the method, the problem of insufficient data is solved, and the generalization ability and precision of a subsequent model are effectively improved by enhancing the diversity and authenticity of the generated samples.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a network traffic monitoring and identification method based on unbalanced data flow. BACKGROUND

[0002] With the wide popularity of the Internet and the continuous advancement of informatization process, the scale and complexity of network traffic continue to increase. Network traffic data not only contains regular normal communication, but also may contain abnormal behavior, such as distributed denial of service attack (DDoS), scanning attack and other potential threats. Accurate identification of these abnormal traffic is an important part of network security. However, the existing network traffic analysis methods face many challenges in data collection, processing and model training.

[0003] The sample expansion method based on random sampling or simple generation algorithm in the prior art generates data samples that are difficult to truly reflect the complex distribution characteristics of network traffic, resulting in that the expanded data cannot effectively improve the training effect of the model, in addition, there is lack of a method for dynamically adjusting the distribution of generated samples, resulting in significant deviation between the generated samples and the real data. The traditional support vector machine has a heavy computing burden when processing large-scale network traffic data, and lacks effective management of abnormal values and noise data, resulting in poor performance of the model on unbalanced data, in addition, the traditional classification method is difficult to dynamically optimize for different types of traffic characteristics, affecting the accuracy and robustness of the classification result. SUMMARY

[0004] The present application provides a network traffic monitoring and identification method based on unbalanced data flow, which aims to

[0005] In order to achieve the above purpose, the present application provides a network traffic monitoring and identification method based on unbalanced data flow, comprising:

[0006] Step 1, collecting network traffic data of each network node, storing in a structured format, and labeling the collected network traffic data;

[0007] Step 2, generating training samples according to the collected network traffic data, and using a sample distribution self-adaptive correction method, and dynamically adjusting the spatial distribution of the generated samples;

[0008] Step 3, training the network traffic data expansion model, and increasing the number of samples according to the network traffic data expansion model;

[0009] Step 4, inputting the feature extracted network traffic data into the classifier for classifier model training, and using the trained classifier model for network traffic monitoring and identification.

[0010] Further, the collection method of the network traffic data comprises an automatic script and a network monitoring tool.

[0011] Further, the storage format of the network traffic data is JSON format.

[0012] Further, the step 2 specifically comprises:

[0013] initializing the generator and the discriminator of the generative adversarial network, providing initial parameters for the generator and the discriminator, ensuring that the generator can generate samples based on random noise, and the discriminator can distinguish the authenticity of the input samples, the input of the generator is a random noise vector , the generated sample is , the input of the discriminator is a network traffic data sample , and the parameters of the generator and the discriminator are and , respectively;

[0014] initializing the sample distribution adaptive correction module, the sample distribution adaptive correction module dynamically adjusts the distribution of the generated sample by calculating the distribution difference between the generated network traffic data and the real network traffic data, so that it is more in line with the characteristics of the actual network traffic data, and the correction process of the sample is as follows:

[0015]

[0016] wherein, is a correction term, , represents the difference between the mean of the generated sample and the real sample, is the corrected generated sample, is a regulation coefficient, and are the mean of the real and generated samples, respectively;

[0017] repeat the above steps until the preset stopping iteration condition is met, and then the model training is completed.

[0018] Further, the step 2 further comprises:

[0019] The correction term can also be calculated by integrating the distribution difference, which is expressed as:

[0020]

[0021] wherein, and are the probability density functions of the real sample and the generated sample, respectively; is a regulation coefficient, used to quantify the difference between the distributions; represents the differential of the generated sample.

[0022] Furthermore, the step 2 further includes:

[0023] The amendment The distribution difference between the generated samples and the real samples can also be quantified by the Wasserstein distance, and dynamic correction can be performed in combination with the feature manifold gradient alignment to guide the samples to move toward the real data manifold and enhance the geometric structure consistency, which can be expressed as:

[0024]

[0025] in, is the manifold alignment strength coefficient, The true distribution and generate distribution 1-Wasserstein distance; Characteristic manifold The gradient direction on .

[0026] Furthermore, the step 4 specifically includes:

[0027] It is a sample enhanced by perturbation. It constructs perturbation samples near the decision boundary in the feature space, forcing the model to learn a more robust classification interface. The acquisition method is expressed as:

[0028]

[0029] in, Indicates the The feature vector of the network traffic data point after feature extraction; is the disturbance intensity coefficient, the default value is 0.05, which controls the disturbance amplitude; It is the gradient of the loss function of the support vector machine with respect to the feature, identifying the classification sensitive direction; is the L2 norm, used for gradient normalization;

[0030] During the training process, a dynamic optimization strategy is adopted to adjust the model parameters according to the results of each round of training, so that the model can adapt to the imbalance and characteristics of network traffic data after feature extraction;

[0031] The learning rate of the support vector machine is calculated through an adaptive adjustment mechanism and is expressed as:

[0032]

[0033] in, For the The learning rate of the support vector machine for the iteration; is the basic learning rate of the support vector machine; is a small constant, For the L2 norm of the gradient of the loss function of the network traffic data points after feature extraction;

[0034] The manifold consistency regularization term is calculated to constrain the consistency of the original sample and the adversarial sample in the decision space, and is expressed as:

[0035]

[0036] wherein, The manifold regularization term forces the decision function to maintain Lipschitz continuity on the local manifold and harden the classification boundary;

[0037] is the decision function output of the support vector machine when the input is The calculation method is represented as ;

[0038] is the decision function output of the support vector machine when the input is The calculation method is represented as ;

[0039] wherein, represents the square operation, which strengthens the local manifold smoothness; is the weight vector of the support vector machine; is the bias term of the support vector machine;

[0040] To enhance the model's ability to identify abnormal values and unknown patterns, the model is retrained using labeled abnormal network traffic data, and the final model parameters are optimized in the following manner:

[0041]

[0042] wherein, is the regularization parameter of the support vector machine, used to control the model complexity; is the norm of the weight vector of the support vector machine, used to reduce overfitting; is the boundary strengthening coefficient, which balances the classification loss and boundary robustness; is the loss function of the support vector machine; is the optimal parameter of the support vector machine;

[0043] The calculation method of the loss function of the adjusted support vector machine is represented as:

[0044]

[0045] wherein, is the hinge loss function; is the classification boundary uncertainty regularization term function; to dynamically adjust the coefficient;

[0046] The calculation mode of the classification boundary uncertainty regularization term function is represented as:

[0047]

[0048] The dynamic adjustment coefficient is calculated according to the proportion of positive class samples in the network traffic data set after feature extraction.

[0049] Further, the step 4 further comprises:

[0050] After all the training and optimization steps are completed, the final model parameters are locked and deployed to cope with actual application scenarios, as the optimized final model parameters.

[0051] Further, the step 4 further comprises:

[0052] The collected original network traffic data is input into the trained feature extraction model for feature processing, and the processed features are input into the classifier model for classification, thereby obtaining a classification result.

[0053] The above scheme of the present application has the following advantages:

[0054] The present application adopts a generative adversarial network based on sample distribution adaptive correction for data augmentation, and generates samples consistent with the distribution of real traffic data through the generative adversarial network to dynamically adjust the distribution of generated samples and improve the diversity and authenticity of generated samples. In addition, by introducing a sample distribution correction module, the difference between generated samples and real sample distribution is calculated, and the mean difference and distribution difference integral are used to adjust the generation target of the generator, solving the problem of mismatch between generated samples and real data. The support vector machine algorithm based on robust reinforcement is used for classification, and the support vector pre-screening mechanism is used to identify key samples of the classification decision boundary to reduce the processing of redundant data. In the classification stage, the abnormal value detection and uncertainty management are combined, the support vector machine model is optimized by adjusting the loss function and dynamic learning rate, and the adaptability to non-balanced data and noise interference is improved. The generative adversarial network based on sample distribution adaptive correction can better reflect the distribution characteristics of real network traffic, enrich the training data set, and solve the problem of insufficient data. By enhancing the diversity and authenticity of generated samples, the generalization ability and precision of the subsequent model are effectively improved. In addition, the support vector machine reduces the interference of irrelevant samples through the pre-screening mechanism, significantly reduces the computational burden, and the optimized classification model can more accurately distinguish normal traffic, abnormal traffic and other types of abnormal traffic, and performs stably under non-balanced data and noise conditions.

[0055] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a schematic diagram of the process of the present invention;

[0057] Figure 2 Schematic diagram of the impact of different data augmentation techniques on network traffic classification performance;

[0058] Figure 3 Schematic diagram of the robust performance of different algorithms in unbalanced data scenarios;

[0059] Figure 4 Schematic diagram of the detection capabilities of different algorithms for various types of network attacks. DETAILED DESCRIPTION

[0060] To make the technical problems, technical solutions, and advantages to be solved by the present invention more clear, the following is a detailed description with reference to the accompanying drawings and specific embodiments. It is obvious that the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0061] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0062] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they may refer to a locking connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0063] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0064] like Figure 1 As shown, an embodiment of the present invention provides a network traffic monitoring and identification method based on unbalanced data flow, including: collecting network traffic data of each network node, storing it in a structured format, and annotating the collected network traffic data; generating training samples based on the collected network traffic data, and adopting a method for adaptive correction of sample distribution, and dynamically adjusting the spatial distribution of generated samples; training a network traffic data expansion model, and increasing the number of samples based on the network traffic data expansion model; inputting the network traffic data after feature extraction into a classifier to train the classifier model, and using the trained classifier model to perform network traffic monitoring and identification.

[0065] The main steps are as follows:

[0066] Step 1: Data collection and annotation

[0067] The network traffic data collected in the present invention originates from multiple network nodes, including but not limited to enterprise-level routers, firewalls and personal devices. The collection method relies on automated scripts and network monitoring tools, specifically including tools such as Wireshark and Tcpdump to capture data packets flowing through the network in real time. All captured network traffic data is stored in a structured format, specifically in JSON format.

[0068] In one embodiment, the network traffic data attributes are represented as follows: Indicates the size of network traffic packets. Indicates the network traffic data packet interval time, Indicates the protocol type. Indicates the source IP address. Indicates the target IP address, Indicates the source port number, Indicates the target port number, Represents the transport layer payload, Indicates the packet sequence number in the same communication session, Indicates the network status flag.

[0069] It should be noted that this embodiment only illustrates a network traffic data format and type of the present invention. In actual applications, the attributes of network traffic data are usually more than 10 attributes, and the number of attributes of network traffic data may reach dozens or even hundreds.

[0070] Furthermore, the collected network traffic data is labeled. The labeling method of the present invention is manual labeling. In one embodiment, the labeled categories include normal traffic, abnormal traffic (such as DDoS attacks, scanning attacks, etc.), and other types of abnormal traffic.

[0071] S2. Network traffic data expansion

[0072] It is understandable that in the task of the present invention, the collection, acquisition, labeling and preprocessing of network traffic training data are time-consuming and labor-intensive, and insufficient training samples can easily lead to poor model generalization ability and affect the accuracy of the model.

[0073] The present invention adopts a generative adversarial network algorithm based on adaptive correction of sample distribution to generate samples, thereby realizing network traffic data expansion. The adaptive correction of sample distribution dynamically adjusts the spatial distribution of generated samples to better simulate the real distribution of network traffic data and enhance the diversity and authenticity of generated samples.

[0074] Specifically, the training process of the generative adversarial network algorithm based on adaptive correction of sample distribution is as follows:

[0075] S201, initialize the generator and discriminator of the generative adversarial network, provide initial parameters for the generator and discriminator, ensure that the generator can generate samples based on random noise, and the discriminator can judge the authenticity of the input samples. The input of the generator is a random noise vector , the generated samples are , the input of the discriminator is the network traffic data sample , the parameters of the generator and discriminator are and .

[0076] S202: Initialize the sample distribution adaptive correction module. The sample distribution adaptive correction module dynamically adjusts the distribution of generated samples by calculating the distribution difference between the generated network traffic data and the actual network traffic data to make it more consistent with the characteristics of the actual network traffic data. The process of correcting the sample is as follows:

[0077]

[0078] in, is the correction term, , represents the difference between the mean of the generated sample and the true sample;

[0079] Generate samples after correction;

[0080] is the adjustment coefficient;

[0081] and are the means of the real and generated samples respectively.

[0082] Preferably, Set to 0.2.

[0083] In addition, the amendment The distribution difference integral can also be calculated, represented as:

[0084]

[0085] wherein, and are the probability density functions of the real samples and the generated samples respectively;

[0086] is an adjustment coefficient, used to quantify the difference between the distributions;

[0087] denotes the differential of the generated sample.

[0088] Preferably, is set to 0.1.

[0089] In addition, the correction term The distribution difference between the generated sample and the real sample can also be quantified by the Wasserstein distance, combined with the feature manifold gradient alignment for dynamic correction. The traditional mean correction only translates the sample space, and the present application combines the manifold gradient to guide the sample to move towards the real data manifold, enhancing the geometric structure consistency, represented as:

[0090]

[0091] wherein, is the manifold alignment intensity coefficient. Preferably, is set to 0.15;

[0092] is the 1-Wasserstein distance between the real distribution and the generated distribution .

[0093] is the gradient direction on the feature manifold .

[0094] S203, the generator receives random noise , generates a preliminary sample through its neural network, represented as:

[0095]

[0096] wherein, is the generator function, is the parameter of the generator.

[0097] After the sample is generated, the generated sample is adjusted by the adaptive correction module to enhance the diversity and authenticity of the sample.

[0098] S204, the discriminator evaluates the authenticity of the sample, and the discriminator evaluates the corrected generated sample and real samples For true and false classification, the output probability is:

[0099]

[0100] in, is the discriminator function;

[0101] is the Sigmoid activation function, is the weight of the discriminator, is the bias term.

[0102] Furthermore, the loss function of the discriminator is expressed as:

[0103]

[0104] in, Express expectations, is the loss function of the discriminator;

[0105] is the first adaptive weighting coefficient;

[0106] is the second adaptive weighting coefficient.

[0107] Preferably, Set to 0.1, Set to 0.3.

[0108] S205. Calculate the loss function and update the parameters. The optimization goal of the generator's loss function is to make it more difficult for the discriminator to identify the generated samples as generated samples. The calculation method is expressed as:

[0109]

[0110] in, is the manifold regularization coefficient, preferably, Set to 0.3;

[0111] represents the matrix Frobenius norm;

[0112] is the projection matrix of the sample on the manifold.

[0113] Furthermore, the synthetic loss function of the generative adversarial network is expressed as:

[0114]

[0115] in, is the synthetic loss function for generating adversarial networks, is the manifold learning loss term.

[0116] Furthermore, the manifold learning loss term is used to constrain the distribution of generated samples on the low-dimensional manifold, and the calculation method is expressed as:

[0117]

[0118] in, is the number of batch samples input to the discriminator.

[0119] Furthermore, the parameters of the neural network are updated using the gradient descent method, which can be expressed as:

[0120]

[0121]

[0122] in, is the learning rate of the generative adversarial network;

[0123] Indicates parameter update operation;

[0124] represents the gradient of the synthetic loss function of the generative adversarial network with respect to the generator parameters;

[0125] Represents the gradient of the synthetic loss function of the generative adversarial network with respect to the discriminator parameters.

[0126] Preferably, Set to 0.01.

[0127] S206, repeat the above steps until the preset stop iteration condition is met, which means that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations. Preferably, the preset maximum number of iterations is set to 1000 times.

[0128] After the network traffic data expansion model is trained, the trained network traffic data expansion model is used to increase the number of samples. In one embodiment, assuming that the original collected samples are 800 and the network traffic data expansion model generates 200 samples through expansion, the expanded network traffic data set contains 1000 samples.

[0129] S3, classifier classification

[0130] The network traffic data after feature extraction is input into a classifier for classifier model training, and the application adopts a support vector machine algorithm based on robust reinforcement as a classification algorithm, aiming at the problem that a traditional support vector machine model performs poorly when processing large-scale and unbalanced network traffic data, and through a pre-screening mechanism, key network traffic data points affecting the classification decision boundary are identified to reduce the computational burden, and an abnormal value detection and uncertainty management are combined to optimize the loss function to enhance the robustness to noise and abnormal values.

[0131] The training process of the support vector machine algorithm based on robust reinforcement is as follows:

[0132] S301, in the initial stage of model training, the parameters of the support vector machine model are initialized to ensure that the model can start training from a reasonable starting point, and the initialized model parameters are represented as:

[0133]

[0134] Among them, is the weight vector of the support vector machine, which determines the importance of the features;

[0135] is the bias term of the support vector machine, which is used to adjust the classification threshold;

[0136] is the parameter of the support vector machine.

[0137] S302, in order to reduce the amount of network traffic data processing in the training process, a support vector pre-screening mechanism is adopted to identify key network traffic data points that may have a significant impact on the classification decision boundary, specifically, based on the hinge loss function, the following optimization function is realized:

[0138]

[0139]

[0140] Among them, is the total number of network traffic data points;

[0141] represents the disturbance sample generated by the i-th network traffic data point;

[0142] represents the label of the i-th feature extracted network traffic data point;

[0143] is the hinge loss function, which is used to measure the classification error.

[0144] It should be noted that,​​ It is a sample enhanced by perturbation. It constructs perturbation samples near the decision boundary in the feature space, forcing the model to learn a more robust classification interface. The acquisition method is expressed as:

[0145]

[0146] in, Indicates the The feature vector of the network traffic data point after feature extraction;

[0147] is the disturbance intensity coefficient, the default value is 0.05, which controls the disturbance amplitude;

[0148] It is the gradient of the loss function of the support vector machine with respect to the feature, identifying the classification sensitive direction;

[0149] is the L2 norm, used for gradient normalization.

[0150] S303. During the training process, a dynamic optimization strategy is adopted to adjust the model parameters according to the results of each round of training so that the model can adapt to the imbalance and characteristics of the network traffic data after feature extraction. The optimization method of the model parameters is expressed as follows:

[0151]

[0152] in, For the The parameters of the support vector machine for the iteration;

[0153] For the The parameters of the support vector machine for the iteration;

[0154] is the learning rate of the support vector machine;

[0155] represents the loss function of the support vector machine, is the gradient of the loss function of the support vector machine.

[0156] Furthermore, the learning rate of the support vector machine is calculated through an adaptive adjustment mechanism and is expressed as:

[0157]

[0158] in, For the The learning rate of the support vector machine for the iteration;

[0159] is the basic learning rate of the support vector machine;

[0160] For small constants, preferably, take ;

[0161] For the L2 norm of the gradient of the loss function of the network traffic data points after feature extraction.

[0162] S304, calculate the manifold consistency regularization term

[0163] Calculate the manifold consistency regularization term, constrain the consistency of the original sample and the adversarial sample in the decision space, expressed as:

[0164]

[0165] Where, The manifold regularization term forces the decision function to maintain Lipschitz continuity on the local manifold, hardening the classification boundary.

[0166] The decision function output of the support vector machine when the input is , the calculation method is expressed as ;

[0167] Similarly, The decision function output of the support vector machine when the input is , the calculation method is expressed as ;

[0168] Indicates the square operation, which strengthens the local manifold smoothness;

[0169] is the weight vector of the support vector machine;

[0170] is the bias term of the support vector machine.

[0171] S305, in order to enhance the recognition ability of the model to abnormal values and unknown patterns, the labeled abnormal network traffic data is used to retrain the model, and the final model parameter optimization method is expressed as:

[0172]

[0173] Where, is the regularization parameter of the support vector machine, which is used to control the model complexity;

[0174] is the norm of the weight vector of the support vector machine, which is used to reduce overfitting;

[0175] is a boundary robustness coefficient, and is set to 0.1 by default, balancing the classification loss and the boundary robustness;

[0176] is a loss function of the support vector machine;

[0177] is an optimal parameter of the support vector machine.

[0178] Further, the calculation manner of the adjusted loss function of the support vector machine is represented as:

[0179]

[0180] wherein, is a hinge loss function;

[0181] is a classification boundary uncertainty regularization term function;

[0182] is a dynamic adjustment coefficient.

[0183] Further, the calculation manner of the classification boundary uncertainty regularization term function is represented as:

[0184]

[0185] Further, the dynamic adjustment coefficient is calculated according to the proportion of positive class samples in the network traffic data set after feature extraction, and is represented as:

[0186]

[0187] wherein, is an indicator function.

[0188] S305, after completing all the training and optimization steps, the final model parameters are locked and deployed to cope with actual application scenarios, that is, As the final model parameters after optimization, it ensures that the model has high-efficiency classification and recognition ability in various environments.

[0189] S4, network traffic monitoring and identification

[0190] The trained model is used for network traffic monitoring and identification. In an embodiment, the collected original network traffic data is input into the trained feature extraction model for feature processing, and further, the processed features are input into the classifier model for classification, and then the classification result is obtained. In this embodiment, the classification categories include normal traffic, abnormal traffic (such as DDoS attack, scanning attack, etc.), and other types of non-normal traffic.

[0191] As Figure 2As shown, the influence of different data augmentation techniques on network traffic classification performance is evaluated. The classification accuracy of four methods, namely no data augmentation, classic oversampling technique SMOTE, traditional generative adversarial network, and the generative adversarial network based on sample distribution self-adaptive correction proposed in the present application, is compared intuitively through a column chart. The experimental results show that the method of the present application is significantly higher than other methods in the column chart, and the error bar range is smaller, indicating that it not only improves the classification accuracy, but also enhances the model stability, embodying the core advantage of the data augmentation module of the present application: by dynamically adjusting the spatial distribution of generated samples, the complex feature distribution of real network traffic is better simulated, and the augmented samples have higher diversity and authenticity, thereby effectively solving the problem of poor model generalization ability caused by insufficient training samples.

[0192] As shown in Figure 3 The robust performance of different algorithms under the non-balanced data scenario is also analyzed. The F1-score trend of traditional support vector machine, random forest, XGBoost and the method of the present application is shown in the line chart as the proportion of abnormal traffic increases. As shown in the figure, when the proportion of abnormal traffic is low, the performance of all algorithms decreases, but the decrease of the method of the present application is the smallest; as the proportion increases, the curve of the method of the present application is always located at the top and fluctuates most smoothly. The technical breakthrough of the classifier module of the present application is verified, which reduces the computational burden by pre-screening key data points, combines anomaly detection and manifold consistency regularization, significantly enhances the adaptability of the model to non-balanced data, and makes the classification decision boundary stable in complex and variable network environment.

[0193] As shown in Figure 4 The detection ability of different algorithms on various network attacks is analyzed. The detection accuracy of traditional support vector machine, random forest and the method of the present application under five common attack types is compared in the grouped column chart. As can be seen from the figure, the method of the present application reaches the highest position in the column of all attack types, especially in the detection of complex threats such as distributed denial of service attack and structured query language injection attack. The experimental results prove the effectiveness of the overall technical framework of the present application, from data collection and labeling, adaptive augmentation to robust classification, making the model able to fully capture the subtle feature differences of various attacks and realize accurate identification of diversified network security threats.

[0194] The application adopts a generative adversarial network based on sample distribution adaptive correction for data augmentation, and generates samples consistent with the distribution of real traffic data through the generative adversarial network to dynamically adjust the distribution of generated samples and improve the diversity and authenticity of the generated samples.

[0195] In addition, a support vector machine algorithm based on robust reinforcement is used for classification, and the support vector machine algorithm is used for classification, and the support vector machine algorithm is used for classification.

[0196] The generative adversarial network based on sample distribution adaptive correction of the application can better reflect the distribution characteristics of real network traffic, enrich the training data set, solve the problem of insufficient data, and effectively improve the generalization ability and precision of the subsequent model by enhancing the diversity and authenticity of the generated samples.

[0197] In addition, the support vector machine reduces the interference of irrelevant samples through the pre-screening mechanism, significantly reduces the computational burden, and optimizes the classification model to more accurately distinguish normal traffic, abnormal traffic and other types of abnormal traffic, and performs stably under non-balanced data and noise conditions.

[0198] The above is the preferred embodiment of the application, and it should be noted that for those skilled in the art, without departing from the principles of the application, several improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the application.

Claims

1. A network traffic monitoring and identification method based on unbalanced data flow, characterized in that: include: Step 1: Collect network traffic data of each network node, store it in a structured format, and annotate the collected network traffic data; Step 2: generating training samples based on the collected network traffic data, and using a method of adaptively correcting sample distribution to dynamically adjust the spatial distribution of generated samples; Step 3: training a network traffic data expansion model and increasing the number of samples according to the network traffic data expansion model; Step 4: Input the network traffic data after feature extraction into the classifier to train the classifier model, and use the trained classifier model to perform network traffic monitoring and identification.

2. The network traffic monitoring and identification method based on unbalanced data flow according to claim 1 is characterized in that: The network traffic data is collected by automated scripts and network monitoring tools.

3. The network traffic monitoring and identification method based on unbalanced data flow according to claim 1 is characterized in that: The storage format of the network traffic data is JSON format.

4. The network traffic monitoring and identification method based on unbalanced data flow according to claim 1 is characterized in that: The step 2 specifically includes: Initialize the generator and discriminator of the generative adversarial network, provide initial parameters for the generator and discriminator, ensure that the generator can generate samples based on random noise, and the discriminator can judge the authenticity of the input samples. The input of the generator is a random noise vector , the generated samples are , the input of the discriminator is the network traffic data sample , the parameters of the generator and discriminator are and ; Initialize the sample distribution adaptive correction module. The sample distribution adaptive correction module dynamically adjusts the distribution of generated samples by calculating the distribution difference between the generated network traffic data and the actual network traffic data to make it more consistent with the characteristics of the actual network traffic data. The process of correcting the sample is as follows: ； in, is the correction term, , represents the difference between the mean of the generated sample and the true sample, To generate samples after correction, is the adjustment coefficient, and are the means of the real and generated samples respectively; Repeat the above steps until the preset stop iteration condition is met, and the model training is completed.

5. The network traffic monitoring and identification method based on unbalanced data flow according to claim 4 is characterized in that: The step 2 further comprises: The amendment It can also be calculated using the distribution difference integral, expressed as: ； in, and are the probability density functions of real samples and generated samples respectively; is the adjustment coefficient, which is used to quantify the difference between distributions; represents the differentiation of the generated samples.

6. The network traffic monitoring and identification method based on unbalanced data flow according to claim 4 is characterized in that: The step 2 further comprises: The amendment The distribution difference between the generated samples and the real samples can also be quantified by the Wasserstein distance, and dynamic correction can be performed in combination with the feature manifold gradient alignment to guide the samples to move toward the real data manifold and enhance the geometric structure consistency, which can be expressed as: ； in, is the manifold alignment strength coefficient, The true distribution and generate distribution 1-Wasserstein distance; Characteristic manifold The gradient direction on .

7. The network traffic monitoring and identification method based on unbalanced data flow according to claim 1 is characterized in that: The step 4 specifically includes: It is a sample enhanced by perturbation. It constructs perturbation samples near the decision boundary in the feature space, forcing the model to learn a more robust classification interface. The acquisition method is expressed as: ； in, Indicates the The feature vector of the network traffic data point after feature extraction; is the disturbance intensity coefficient, the default value is 0.05, which controls the disturbance amplitude; It is the gradient of the loss function of the support vector machine with respect to the feature, identifying the classification sensitive direction; is the L2 norm, used for gradient normalization; During the training process, a dynamic optimization strategy is adopted to adjust the model parameters according to the results of each round of training, so that the model can adapt to the imbalance and characteristics of network traffic data after feature extraction; The learning rate of the support vector machine is calculated through an adaptive adjustment mechanism and is expressed as: ； in, For the The learning rate of the support vector machine for the iteration; is the basic learning rate of the support vector machine; is a small constant, For the The L2 norm of the gradient of the loss function of the network traffic data point after feature extraction; Calculate the manifold consistency regularization term to constrain the consistency between the original sample and the adversarial sample in the decision space, expressed as: ； in, represents the manifold regularization term, which forces the decision function to maintain Lipschitz continuity on the local manifold and harden the classification boundary; For input The decision function output of the support vector machine when is expressed as ; For input The decision function output of the support vector machine when is expressed as ; in, Represents the square operation, which strengthens the smoothness of the local manifold; is the weight vector of the support vector machine; is the bias term of the support vector machine; In order to enhance the model's ability to identify outliers and unknown patterns, the model is retrained using labeled abnormal network traffic data. The method for optimizing the final model parameters is expressed as follows: ； in, is the regularization parameter of the support vector machine, which is used to control the complexity of the model; is the norm of the weight vector of the support vector machine, which is used to reduce overfitting; is the boundary reinforcement coefficient, balancing the classification loss and boundary robustness; is the loss function of the support vector machine; is the optimal parameter of the support vector machine; The loss function of the adjusted support vector machine is calculated as: ； in, is the hinge loss function; is the classification boundary uncertainty regularization function; is the dynamic adjustment coefficient; The calculation method of the classification boundary uncertainty regularization function is expressed as: ； The dynamic adjustment coefficient is calculated based on the proportion of positive samples in the network traffic dataset after feature extraction.

8. The network traffic monitoring and identification method based on unbalanced data flow according to claim 7 is characterized in that: The step 4 further comprises: After all training and optimization steps are completed, the final model parameters are locked and deployed to cope with actual application scenarios. As the final model parameters after optimization.

9. The network traffic monitoring and identification method based on unbalanced data flow according to claim 8 is characterized in that: The step 4 further comprises: The collected raw network traffic data is input into the trained feature extraction model for feature processing, and the processed features are input into the classifier model for classification to obtain the classification results.

Images