A method and device for detecting abnormal network traffic

Abstract

The present invention provides a method and device for detecting anomalies in network traffic, wherein the method comprises: constructing a width learning model, obtaining sample data, training and learning the width learning model using the sample data, determining a weight penalty factor for the sample data, and adjusting the loss function of the width learning model according to the weight penalty factor; pruning each enhanced node in the width learning model based on its contribution, and updating the loss function of the width learning model; and dynamically retraining the width learning model offline using the sample data with the goal of minimizing the model loss of the width learning model to obtain a target width learning model. Through the present invention, the real-time performance and accuracy of the width learning model are improved, solving the problem of poor detection real-time performance and accuracy in existing related technologies.

Description

Technical Field

[0001] The present invention relates to the field of communication network technology, and in particular to a method and device for detecting network traffic anomaly. Background Art

[0002] New power systems leverage digital technology to achieve intelligent interaction and coordinated optimization across all aspects of power generation, grid operation, load storage, and storage, effectively aggregating vast amounts of adjustable resources to support real-time dynamic response. Compared to traditional power systems, new power systems manage a large number of unsecured distributed terminals, rely heavily on information and communications technology, and coordinate the management of large amounts of confidential data, making them a high-risk target for cyberattacks. With the development of the social environment and technological advancements, cyberattack methods targeting new power systems are constantly evolving, posing a significant threat to the security and stability of power systems. Therefore, timely and accurate detection of dynamically evolving cyberattacks in new power systems is particularly important.

[0003] The network traffic of new power systems is characterized by distribution, complexity, and real-time nature, requiring anomaly detection methods to have good real-time processing, online detection, and accurate prediction capabilities. However, most existing methods cannot balance the accuracy and real-time performance of the detection model and lack reliable detection methods that can dynamically adapt to new attacks. Width learning models can quickly adapt to new data inputs and flexibly update models. Compared with deep learning models, they are more suitable for detection scenarios with high real-time requirements and rapidly changing network environments. However, as training time increases, the structure of the width learning model will continue to expand horizontally, adding enhanced nodes to improve performance. This will lead to redundant enhanced nodes, complex model structure, and face the problems of forgetting learned old data and being unable to effectively adapt to changes in data distribution in incremental learning.

[0004] There is currently no good solution to the problems of poor detection real-time performance and accuracy in existing related technologies. Summary of the Invention

[0005] The present invention provides a method and device for detecting anomalies in network traffic, which are used to solve the defects of poor detection real-time performance and accuracy in the prior art.

[0006] In a first aspect, the present invention provides a method for detecting network traffic anomalies, comprising:

[0007] Constructing a width learning model, obtaining sample data, training the width learning model using the sample data, determining a weight penalty factor for the sample data, and adjusting a loss function of the width learning model according to the weight penalty factor;

[0008] Based on the contribution of each enhanced node in the width learning model, pruning the enhanced nodes and updating the loss function of the width learning model;

[0009] With the goal of minimizing the model loss of the width learning model, dynamically retraining the width learning model offline using the sample data to obtain a target width learning model;

[0010] Flow data is acquired, and anomaly detection is performed on the flow data using the target width learning model to obtain an anomaly detection result of the flow data.

[0011] According to a network traffic anomaly detection method provided by the present invention, the width learning model is trained using the sample data, and the loss function of the width learning model is adjusted, including:

[0012] Inputting the sample data into the width learning model to determine the intra-class distance and inter-class distance of the sample features of the sample data;

[0013] In batch learning and incremental learning scenarios, assigning sample weights to the sample data based on the intra-class distance and inter-class distance of the sample features;

[0014] The weight penalty factor is determined according to the sample weight of the sample data, and the weight penalty factor is used to improve the loss function of the width learning model.

[0015] According to a network traffic anomaly detection method provided by the present invention, determining the intra-class distance and inter-class distance of the sample features of the sample data includes:

[0016] Determine the characteristic mean of the category to which each sample data belongs;

[0017] Determining the intra-class distance of the sample feature based on the sample feature and the feature mean of the corresponding class;

[0018] The inter-class distance of the sample feature is determined according to the mean of the sample feature and the features of other classes.

[0019] According to a network traffic anomaly detection method provided by the present invention, a sample weight is assigned to the sample data according to the intra-class distance and the inter-class distance of the sample features, comprising:

[0020] Determining the intra-class distance and inter-class distance of the sample data according to the intra-class distance and inter-class distance of the sample features;

[0021] Determining a distance factor of the sample data according to the intra-class distance and the inter-class distance of the sample data;

[0022] The sample weight of the sample data is determined according to the distance factor of the sample data.

[0023] According to a network traffic anomaly detection method provided by the present invention, pruning the enhanced nodes based on the contribution of each enhanced node in the width learning model includes:

[0024] For each of the enhancement nodes, determining a contribution of the enhancement node according to a difference in output of the width learning model before and after removing the enhancement node;

[0025] In batch learning and incremental learning scenarios, the sample data are divided into categories, and the contribution of the enhancement nodes of the width learning model is ranked within each category;

[0026] Select a number of enhancement nodes with the lowest contribution corresponding to each category, take the intersection of the enhancement nodes selected for each category, and perform pruning on the enhancement nodes in the intersection.

[0027] According to a network traffic anomaly detection method provided by the present invention, in batch learning and incremental learning scenarios, the sample data is classified into categories, including:

[0028] In a batch learning scenario, the sample data is divided according to the label category;

[0029] In the incremental learning scenario, the sample data is divided according to how new it is.

[0030] According to a network traffic anomaly detection method provided by the present invention, pruning the enhanced nodes in the intersection includes:

[0031] The output weight of the enhancement node is modified to 0.

[0032] According to a network traffic anomaly detection method provided by the present invention, updating the loss function of the width learning model includes:

[0033] For the width learning model after the pruning process, incrementally learning new representations of new class sample data and old class sample data from the sample data;

[0034] Based on the adjusted loss function of the width learning model, the model loss of the width learning model is updated.

[0035] According to a network traffic anomaly detection method provided by the present invention, the width learning model is dynamically retrained offline using the sample data, comprising:

[0036] Determining an example set from the sample data through data windowing and flock selection, and training the width learning model through the example set;

[0037] The model loss of the trained width learning model is evaluated through a dynamic threshold-based model performance evaluation mechanism.

[0038] In a second aspect, the present invention further provides a network traffic anomaly detection device, comprising:

[0039] A construction module is used to construct a width learning model, obtain sample data, train the width learning model with the sample data, determine a weight penalty factor for the sample data, and adjust the loss function of the width learning model according to the weight penalty factor;

[0040] An updating module, configured to prune the enhanced nodes based on the contribution of each enhanced node in the width learning model and update the loss function of the width learning model;

[0041] an evaluation module, configured to dynamically retrain the width learning model offline using the sample data with the goal of minimizing the model loss of the width learning model to obtain a target width learning model;

[0042] The detection module is used to obtain flow data, perform anomaly detection on the flow data through the target width learning model, and obtain an anomaly detection result of the flow data.

[0043] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the network traffic anomaly detection method as described in the first aspect above is implemented.

[0044] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the network traffic anomaly detection method as described in the first aspect above.

[0045] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the network traffic anomaly detection method as described in the first aspect above.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The network traffic anomaly detection method provided by the present invention improves the width learning model by training it and adjusting its loss function. Furthermore, by pruning enhanced nodes, the method prioritizes the output results of enhanced nodes with greater reference value. This reduces the computational complexity of the width learning model, improves its training efficiency and effectiveness, ensures its real-time performance and accuracy, and addresses the issues of poor real-time detection and accuracy found in existing related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 It is a flow chart of the network traffic anomaly detection method proposed by the present invention;

[0050] Figure 2 This is a flow chart of training a width learning model in an embodiment of the present invention;

[0051] Figure 3 is a flow chart of pruning processing in an embodiment of the present invention;

[0052] Figure 4 is a flow chart of dynamic offline retraining of a width learning model in an embodiment of the present invention;

[0053] Figure 5 This is a structural block diagram of the network traffic anomaly detection device proposed by the present invention;

[0054] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0055] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0056] The present invention provides a method for detecting network traffic anomaly. Figure 1 This is a flow chart of the network traffic anomaly detection method proposed by the present invention. Figure 1As shown, the method includes the following steps:

[0057] Step S101: construct a width learning model, obtain sample data, train the width learning model through the sample data, determine the weight penalty factor of the sample data, and adjust the loss function of the width learning model according to the weight penalty factor.

[0058] Step S102: based on the contribution of each enhanced node in the width learning model, prune the enhanced nodes and update the loss function of the width learning model.

[0059] Step S103 , with the goal of minimizing the model loss of the width learning model, dynamically retrain the width learning model offline using sample data to obtain a target width learning model.

[0060] Step S104: Acquire traffic data, perform anomaly detection on the traffic data using a target width learning model, and obtain an anomaly detection result of the traffic data.

[0061] In this method, a broad learning system (BLS) is first constructed. The BLS can be a traditional network traffic anomaly detection model. To improve the detection accuracy of the BLS, sample data of network traffic with known anomalies is obtained. The BLS is trained using the sample data, a weight penalty factor for the sample data is determined, and the loss function of the BLS is adjusted based on the weight penalty factor to improve the BLS, thereby alleviating the problems of catastrophic forgetting and concept drift. During the training process, the BLS nodes are pruned based on their contribution. The contribution indicates the contribution of the BLS node to the BLS model's correct prediction of network traffic anomalies. The lower the contribution, the less valuable the output of the BLS node is and the less needed it is. Therefore, in this method, the BLS nodes are pruned based on their contribution, giving priority to the output of nodes with greater reference value. This also reduces the computational complexity of the BLS model, improves the training efficiency and effectiveness of the BLS model, and ensures the real-time and accuracy of the BLS model. After the pruned width learning model, the width learning model is dynamically retrained offline using sample data, with the goal of minimizing the model loss of the width learning model. The enhanced nodes of the width learning model can also be pruned repeatedly until the number of pruning rounds reaches a pre-set number. Ultimately, the target width learning model is obtained. Finally, the traffic data requiring anomaly detection is obtained, and anomaly detection is performed on the traffic data using the target width learning model to obtain anomaly detection results for the traffic data. This method not only improves the loss function of the width learning model, but also optimizes the enhanced nodes of the width learning model, resulting in a more efficient and accurate target width learning model. This solves the problems of poor detection real-time performance and accuracy in existing related technologies.

[0062] In some of these embodiments, Figure 2 This is a flowchart of training the width learning model in an embodiment of the present invention. For ease of description, this process is recorded as the basic model learning process. Figure 2As shown, step S101 trains the width learning model using sample data and adjusts the loss function of the width learning model, including: inputting the sample data into the width learning model to determine the intra-class distance and inter-class distance of the sample features of the sample data; in batch learning and incremental learning scenarios, assigning sample weights to the sample data based on the intra-class distance and inter-class distance of the sample features; determining a weight penalty factor based on the sample weight of the sample data, and using the weight penalty factor to improve the loss function of the width learning model. In this embodiment, the sample feature refers to the mapping feature vector output by the mapping node layer of the width learning model. The advantage is that it can more intuitively measure the similarity and difference of the sample data in the feature space.

[0063] In this embodiment, the intra-class distance and inter-class distance are calculated using the Euclidean distance calculation method. Determining the intra-class distance and inter-class distance of the sample features of the sample data includes: determining the feature mean of the category to which each sample data belongs; determining the intra-class distance of the sample features based on the feature mean of the sample features and the corresponding category; and determining the inter-class distance of the sample features based on the feature mean of the sample features and the feature mean of other categories.

[0064] Specifically, calculate the characteristic mean of the category to which the sample data belongs. Assume that the category to which the sample belongs is c and the sample characteristic value is m i , there are N c samples, then the characteristic mean of this category is:

[0065]

[0066] in, represents the feature mean of category c, i represents the sample data in category c; for any category, the feature mean of other categories is recorded as

[0067] The formula for calculating the Euclidean distance between the sample feature and the feature mean of the category is as follows:

[0068]

[0069] Among them, k represents the vector dimension, that is, the number of mapping feature nodes. Correspondingly, the Euclidean distance between the sample feature and the feature mean of other categories is recorded as

[0070] In this embodiment, sample weights are assigned to sample data based on the intra-class distance and inter-class distance of sample features, including: determining the intra-class distance and inter-class distance of the sample data based on the intra-class distance and inter-class distance of the sample features; determining the distance factor of the sample data based on the intra-class distance and inter-class distance of the sample data; and determining the sample weight of the sample data based on the distance factor of the sample data.

[0071] Specifically, compare the sample features with the feature mean of the same category distance As the intra-class distance of the sample data, the sample feature and the mean of other category features are distance As the inter-class distance of the sample data.

[0072] The distance factor is determined by both the intra-class and inter-class distances, indicating the difference between the sample and the sample data of the same class and the sample data of different classes. If the training scenario is a batch learning scenario, the distance factor of the sample data is the ratio of the inter-class distance to the intra-class distance. The specific formula is as follows:

[0073]

[0074] Among them, D i It represents the distance factor of the sample data. The larger the inter-class distance of the sample (the greater the difference from the different classes) and the smaller the intra-class distance (the smaller the difference from the same class), the larger the distance factor is, which can reflect that the sample data has higher discrimination and better typicality.

[0075] If the training scenario is an incremental learning scenario, the new and old class distinction parameter ω is introduced to divide the sample data into new and old categories. The distance factor of the sample data is calculated as follows:

[0076]

[0077] Among them, ω is a parameter for distinguishing between new and old classes. When the sample data is new class data, ω=0, which means that only the inter-class distance of the sample is calculated. The larger the inter-class distance of the sample (the greater the difference with the other class), the better the typicality. When the sample is old class data, ω∈(0.5,1), which means that both intra-class distance and inter-class distance are considered at the same time, and the intra-class distance is more inclined to reflect the typicality of the sample. At this time, the larger the intra-class distance (the greater the difference with the same class), the sample data can be regarded as a new representation of the old class data, and its typicality for the training and learning of the width learning model is better.

[0078] From the above process, we can see that the larger the distance factor of the sample data, the more typical the sample data is. Therefore, the sample weight of the sample data matches the size of the distance factor. The calculation formula of the sample weight is as follows:

[0079] W i =D i

[0080] Among them, W i Represents the sample weight of the sample data.

[0081] Then, in both batch and incremental learning scenarios, the sample weights are converted into weight penalty factors, which are used to improve the loss function of the wide learning model. Specifically, a positive adjustment parameter α is introduced to adjust the size of the weight penalty factor. In batch learning, α = 1, and the positive adjustment parameter is the standard value. In incremental learning, the size of α varies depending on the new and old categories of the sample data, indicating the importance of learning new sample data and strengthening the memory of old sample data.

[0082] More specifically, the weight penalty factor C w It is an n×n diagonal matrix, and the values ​​on the diagonal are C ii From the sample data x i The sample weight W i The calculation formula of the weight penalty factor is as follows:

[0083] C ii =α×W i

[0084] Here, α is a positive adjustment parameter, indicating that the weight penalty factor is proportional to the sample weight. In batch learning scenarios, α = 1, which is the standard value. In incremental learning scenarios, to better distinguish the importance of new and old class sample data, when the sample data is new, the value of α is slightly larger than the standard value, indicating that learning new class data is more important for the wide learning model. When the sample data is old class data, the value of α is slightly larger than the α value of the new class data, indicating that strengthening the memory of old class data is more important for the wide learning model.

[0085] The weight penalty factor is added to the loss function of the width learning model, and the loss function of the width learning model is adjusted to a weighted loss function. The specific formula is as follows:

[0086]

[0087] Among them, C W Denotes the weighting matrix, A denotes the concatenation matrix of the augmentation and mapping nodes, W denotes the output weights, Y denotes the predicted output, and λ is the regularization coefficient, which approaches 0 in practice. The width learning model adjusts its parameters by minimizing the weighted loss function. Because high-weight samples have a greater impact on the loss function, the width learning model prioritizes adjusting itself to better fit these sample data. Consequently, the width learning model can more effectively identify and focus on representative samples.

[0088] In some of these embodiments, Figure 3 Flowchart of pruning process in the embodiment of the present invention is as follows: Figure 3As shown, for the convenience of description, the process is recorded as the online learning process of the width learning model. In step S102, based on the contribution of each enhancement node in the width learning model, the enhancement nodes are pruned, including: for each enhancement node, the contribution of the enhancement node is determined according to the output difference of the width learning model before and after removing the enhancement node; in batch learning and incremental learning scenarios, the sample data are classified respectively, and the contribution of the enhancement nodes of the width learning model is sorted within each category; several enhancement nodes with the lowest contribution corresponding to each category are selected, the intersection of the enhancement nodes selected for each category is taken, and the enhancement nodes in the intersection are pruned.

[0089] Specifically, in batch learning and incremental learning scenarios, the sample data are classified respectively, including: in batch learning scenarios, classification according to the label category of the sample data; in incremental learning scenarios, classification according to the newness of the sample data. Generally speaking, in order to make the width learning model pay more attention to the enhancement nodes with higher contribution, it is only necessary to reduce the output weight of the enhancement nodes with lower contribution. As a more preferred solution, the enhancement nodes in the intersection are pruned, including: modifying the output weight of the enhancement node to 0, so that the width learning model only pays attention to the enhancement nodes with higher contribution.

[0090] In this embodiment, first, for the kth enhanced node, the model output including this node is y, and the model output after removing this node is y'. The calculation formula of the output difference is as follows:

[0091] ||y-y'||=||H k W k ||=||H k ||||W k ||

[0092] Among them, H k represents the output of the enhancement node, W k Represents the output weight corresponding to the enhancement node. Then, the contribution of the enhancement node is calculated based on the above output difference. For N sample data, the calculation formula for the contribution of the kth enhancement node is as follows:

[0093]

[0094] Among them, S k represents the contribution of the kth enhancement node, i represents the number of the sample data, H ikRepresents the output of the k-th enhancement node for sample data i. The sample data is divided into categories in both batch learning and incremental learning scenarios, and the contribution of the enhancement nodes is sorted according to the category. Specifically, for the batch learning scenario, the category division of the sample data is divided according to the label category; for the incremental learning scenario, the category division of the sample data is divided according to the degree of newness, and then, within each category, the contribution value of the enhancement nodes is sorted from large to small. For each category, the last q nodes in the sorted list are taken as candidate pruning nodes, and the intersection of each candidate pruning node list is taken; the enhancement nodes in the intersection are pruned, and the output weight corresponding to the enhancement node is modified to 0.

[0095] Updating the loss function of the width learning model includes: incrementally learning new representations of new class sample data and old class sample data from sample data for the pruned width learning model; and updating the model loss of the width learning model based on the adjusted loss function of the width learning model. Specifically, after adjusting the output weights of the enhancement nodes of the width learning model, the loss function of the width learning model is also updated to the following weighted form:

[0096] L'=C W||Y'-Y|| =C W ||AW-Y||

[0097] Among them, L' represents the loss function value, C w is a weighted matrix, Y represents the predicted output, Y' represents the actual output, A represents the concatenation matrix of the enhancement node and the mapping node, and W represents the output weight.

[0098] In some of these embodiments, Figure 4 This is a flow chart of the dynamic offline retraining of the width learning model in an embodiment of the present invention. Figure 4 As shown, step S103, the width learning model is dynamically retrained offline through sample data, including: determining a paradigm set from the sample data through data window and flock selection, and training the width learning model through the paradigm set; and evaluating the model loss of the trained width learning model through a model performance evaluation mechanism based on a dynamic threshold.

[0099] In this embodiment, a fixed-size data window is first set. A weighted sampling strategy is used to select sample data within the window, assigning different weights to sample data from different time periods. When the sample data is updated, a herd selection strategy is used to select typical old-class sample data as the example set, and other old-class sample data is eliminated. Then, a dynamic threshold-based model performance evaluation mechanism is used to determine whether the width learning model loss exceeds the threshold. The threshold is expressed as a function related to the model loss, and the threshold θ is expressed as follows:

[0100] θ=L-β×e α×x

[0101] Here, x represents the difference in loss between the previous and next learning phases of the wide learning model, α and β are positive scaling factors used to adjust the threshold's sensitivity to model loss, and L is an empirically determined baseline loss threshold. This setting allows the threshold θ to dynamically adjust as the model loss changes. When the model loss difference x increases slightly, the threshold θ decreases more slowly; however, when the loss difference x increases significantly, the threshold θ decreases more rapidly, making it more sensitive to sharp declines in model performance and triggering more frequent dynamic offline retraining. If the model loss exceeds the threshold θ, the example set is reselected and the model loss of the wide learning model is evaluated. If the model loss does not exceed the threshold θ, the pruning of the augmented nodes in the wide learning model is repeated, i.e., the online learning process is repeated, and the model loss of the wide learning model is re-evaluated. This process continues until the preset number of online learning rounds is reached, resulting in the target wide learning model.

[0102] In summary, this method adopts a weighted width learning method based on class correlation. By comprehensively calculating the intra-class and inter-class distances of sample data to evaluate its typicality, it adaptively assigns sample weights to sample data in both batch learning and incremental learning scenarios. Furthermore, it introduces a weight penalty factor to improve the loss function of the width learning model. This weighted width learning method can enhance the width learning model's learning of the associations between new and old class sample data, allowing the width learning model to effectively learn new sample data while retaining the memory of old sample data, thus avoiding the problems of concept drift and catastrophic forgetting. Furthermore, this method adopts a pruning method for the width learning model. In both batch learning and incremental learning scenarios, the method compares the contribution of enhanced nodes to sample data of different categories and prunes the enhanced nodes. This effectively optimizes the network structure of the width learning model, further enhances the width learning model's adaptability to samples of different categories, and improves the detection efficiency and performance of the width learning model. Specifically, the present invention adopts an adaptive online learning framework for width learning, which is divided into three stages: a basic model learning stage, an online learning stage, and a dynamic offline retraining stage. A model performance evaluation mechanism based on dynamic thresholds is introduced, and offline retraining is adaptively activated according to the monitoring results, avoiding the problems of catastrophic forgetting and concept drift. By effectively combining offline learning and online learning of the width learning model, the real-time performance of the anomaly detection method is improved.

[0103] The present invention also provides a network traffic anomaly detection device. The network traffic anomaly detection device provided by the present invention is described below. The network traffic anomaly detection device described below and the network traffic anomaly detection method described above can be referred to in correspondence with each other. Figure 5 This is a structural block diagram of the network traffic anomaly detection device proposed by the present invention. Figure 5 As shown, the device includes:

[0104] Construction module 501 is used to build a width learning model, obtain sample data, train the width learning model with the sample data, determine a weight penalty factor of the sample data, and adjust the loss function of the width learning model according to the weight penalty factor;

[0105] An updating module 502 is configured to prune the enhanced nodes based on the contribution of each enhanced node in the width learning model and update the loss function of the width learning model;

[0106] An evaluation module 503 is configured to dynamically retrain the width learning model offline using sample data with the goal of minimizing the model loss of the width learning model to obtain a target width learning model;

[0107] The detection module 504 is used to obtain traffic data, perform anomaly detection on the traffic data through the target width learning model, and obtain anomaly detection results of the traffic data.

[0108] When the present device is in use, first, the construction module 501 constructs a width learning model, which can be a traditional network traffic anomaly detection model. In order to improve the detection accuracy of the width learning model, a number of sample data of network traffic with known anomalies or not are obtained, and the width learning model is trained and learned through the sample data. The loss function of the width learning model is adjusted to improve the width learning model. Then, during the training process, the update module 502 prunes the enhancement nodes based on the contribution of each enhancement node in the width learning model. The contribution indicates the contribution of the enhancement node to the width learning model's correct prediction of the abnormal conditions of the network traffic. The lower the contribution, the less reference value the output result of the enhancement node has and the less it is needed. Therefore, when the present device is in use, the enhancement nodes are pruned based on the contribution, and the output results of the enhancement nodes with more reference value are considered as much as possible. At the same time, the amount of calculation of the width learning model is reduced, the training efficiency and efficiency of the width learning model are improved, and the real-time and accuracy of the width learning model are ensured. For the width learning model after pruning, the evaluation module 503 takes minimizing the model loss of the width learning model as the goal, and dynamically retrains the width learning model offline through sample data. It can also repeatedly prune the enhanced nodes of the width learning model until the number of pruning processes reaches a pre-set number of rounds, and finally obtain the target width learning model. Finally, the detection module 504 obtains the flow data that needs to be detected for anomalies, and performs anomaly detection on the flow data through the target width learning model to obtain the anomaly detection result of the flow data. Through this device, not only the loss function of the width learning model is improved, but also the enhanced nodes of the width learning model are optimized, resulting in a more efficient and accurate target width learning model, which solves the problems of poor detection real-time performance and accuracy in existing related technologies.

[0109] Figure 6 An example of a physical structure diagram of an electronic device is shown below. Figure 6 As shown, the electronic device may include: a processor 601, a communications interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communications interface 602, and the memory 603 communicate with each other via the communication bus 604. The processor 601 may call the logic instructions in the memory 603 to execute the network traffic anomaly detection method, which includes:

[0110] Build a width learning model, obtain sample data, train the width learning model with the sample data, determine the weight penalty factor of the sample data, and adjust the loss function of the width learning model according to the weight penalty factor;

[0111] Based on the contribution of each enhanced node in the width learning model, the enhanced nodes are pruned and the loss function of the width learning model is updated;

[0112] With the goal of minimizing the model loss of the width learning model, the width learning model is dynamically retrained offline using sample data to obtain the target width learning model.

[0113] Obtain traffic data, perform anomaly detection on the traffic data through the target width learning model, and obtain anomaly detection results of the traffic data.

[0114] In addition, the logic instructions in the above-mentioned memory 603 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0115] On the other hand, the present invention further provides a computer program product, comprising a computer program, which may be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the network traffic anomaly detection method provided by the above methods, which includes:

[0116] Build a width learning model, obtain sample data, train the width learning model with the sample data, determine the weight penalty factor of the sample data, and adjust the loss function of the width learning model according to the weight penalty factor;

[0117] Based on the contribution of each enhanced node in the width learning model, the enhanced nodes are pruned and the loss function of the width learning model is updated;

[0118] With the goal of minimizing the model loss of the width learning model, the width learning model is dynamically retrained offline using sample data to obtain the target width learning model.

[0119] Obtain traffic data, perform anomaly detection on the traffic data through the target width learning model, and obtain anomaly detection results of the traffic data.

[0120] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for detecting anomalies in network traffic provided by the above methods is implemented. The method includes:

[0121] Build a width learning model, obtain sample data, train the width learning model with the sample data, determine the weight penalty factor of the sample data, and adjust the loss function of the width learning model according to the weight penalty factor;

[0122] Based on the contribution of each enhanced node in the width learning model, the enhanced nodes are pruned and the loss function of the width learning model is updated;

[0123] With the goal of minimizing the model loss of the width learning model, the width learning model is dynamically retrained offline using sample data to obtain the target width learning model.

[0124] Obtain traffic data, perform anomaly detection on the traffic data through the target width learning model, and obtain anomaly detection results of the traffic data.

[0125] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0126] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for detecting anomaly in network traffic, characterized in that: include: Constructing a width learning model, obtaining sample data, training the width learning model using the sample data, determining a weight penalty factor for the sample data, and adjusting a loss function of the width learning model according to the weight penalty factor; Based on the contribution of each enhanced node in the width learning model, pruning the enhanced nodes and updating the loss function of the width learning model; With the goal of minimizing the model loss of the width learning model, dynamically retraining the width learning model offline using the sample data to obtain a target width learning model; Flow data is acquired, and anomaly detection is performed on the flow data using the target width learning model to obtain an anomaly detection result of the flow data.

2. The network traffic anomaly detection method according to claim 1, characterized in that: Training the width learning model using the sample data, determining a weight penalty factor for the sample data, and adjusting a loss function of the width learning model according to the weight penalty factor, including: Inputting the sample data into the width learning model to determine the intra-class distance and inter-class distance of the sample features of the sample data; In batch learning and incremental learning scenarios, assigning sample weights to the sample data based on the intra-class distance and inter-class distance of the sample features; The weight penalty factor is determined according to the sample weight of the sample data, and the weight penalty factor is used to improve the loss function of the width learning model.

3. The network traffic anomaly detection method according to claim 2, characterized in that: Determining the intra-class distance and inter-class distance of the sample features of the sample data includes: Determine the characteristic mean of the category to which each sample data belongs; Determining the intra-class distance of the sample feature based on the sample feature and the feature mean of the corresponding class; The inter-class distance of the sample feature is determined according to the mean of the sample feature and the features of other classes.

4. The network traffic anomaly detection method according to claim 3, characterized in that: Allocating sample weights to the sample data according to the intra-class distance and the inter-class distance of the sample features includes: Determining the intra-class distance and inter-class distance of the sample data according to the intra-class distance and inter-class distance of the sample features; Determining a distance factor of the sample data according to the intra-class distance and the inter-class distance of the sample data; The sample weight of the sample data is determined according to the distance factor of the sample data.

5. The network traffic anomaly detection method according to claim 1, wherein: Based on the contribution of each of the enhanced nodes in the width learning model, pruning the enhanced nodes includes: For each of the enhancement nodes, determining a contribution of the enhancement node according to a difference in output of the width learning model before and after removing the enhancement node; In batch learning and incremental learning scenarios, the sample data are divided into categories, and the contribution of the enhancement nodes of the width learning model is ranked within each category; Select a number of enhancement nodes with the lowest contribution corresponding to each category, take the intersection of the enhancement nodes selected for each category, and perform pruning on the enhancement nodes in the intersection.

6. The network traffic anomaly detection method according to claim 5, characterized in that: In batch learning and incremental learning scenarios, the sample data is classified into categories, including: In a batch learning scenario, the sample data is divided according to the label category; In the incremental learning scenario, the sample data is divided according to how new it is.

7. The network traffic anomaly detection method according to claim 5, characterized in that: Pruning the enhanced nodes in the intersection includes: The output weight of the enhancement node is modified to 0.

8. The network traffic anomaly detection method according to claim 1, characterized in that: Updating the loss function of the width learning model includes: For the width learning model after the pruning process, incrementally learning new representations of new class sample data and old class sample data from the sample data; Based on the adjusted loss function of the width learning model, the model loss of the width learning model is updated.

9. The network traffic anomaly detection method according to claim 1, wherein: Dynamically retraining the width learning model offline using the sample data includes: Determining an example set from the sample data through data windowing and flock selection, and training the width learning model through the example set; The model loss of the trained width learning model is evaluated through a dynamic threshold-based model performance evaluation mechanism.

10. A network traffic anomaly detection device, characterized in that: include: A construction module is used to construct a width learning model, obtain sample data, train the width learning model with the sample data, determine a weight penalty factor for the sample data, and adjust the loss function of the width learning model according to the weight penalty factor; An updating module, configured to prune the enhanced nodes based on the contribution of each enhanced node in the width learning model and update the loss function of the width learning model; an evaluation module, configured to dynamically retrain the width learning model offline using the sample data with the goal of minimizing the model loss of the width learning model to obtain a target width learning model; The detection module is used to obtain flow data, perform anomaly detection on the flow data through the target width learning model, and obtain an anomaly detection result of the flow data.

Images