Cybersecurity-oriented explainable cyber data labeling method, system, computing device

Abstract

The application discloses a kind of network security explainable network data marking method, system, computing device, the method includes: simulator carries out simulation to each kind of network attack, corresponding network packet is obtained by packet capturing operation, and on this basis, data is clustered to obtain final dataset;Abnormality detector carries out unified modeling to the network traffic feature information of final dataset and part of the explanation result provided by interpreter, determines a suspicious flow in each interaction with network analyst;Interpreter explains the suspicious flow detected currently based on maximum linear separation, and queries network analyst to judge whether it is abnormal flow or not.The application has the advantages that: the computing resources of interpreter are fully utilized, and the abnormality detector can interact with the network analyst, wherein the interaction quality is ensured by the interpreter, and finally the abnormality detector model has the ability to adapt to dynamic network environment.

Description

Technical Field

[0001] This invention relates to the field of computer network technology, and specifically to a method, system, and computing device for interpretable network data tagging oriented towards network security. Background Technology

[0002] With the rapid development of artificial intelligence and machine learning, the demand for high-quality labeled data is constantly growing. High-quality labeled data can help models more accurately capture patterns and relationships in the data, thereby improving the predictive performance of models on unlabeled data. However, data labeling usually requires a significant investment of manpower, time, and resources, especially when dealing with complex tasks and large-scale datasets. To improve the efficiency and quality of data labeling, researchers and engineers are exploring various techniques, such as automated labeling, semi-supervised learning, transfer learning, and leveraging pre-trained models. In addition, there are specialized data labeling platforms and tools, such as Amazon Mechanical Turk, Figure Eight, and Prodigy, which can help accelerate the data labeling process and improve labeling quality.

[0003] In the field of communication networks, data labeling is commonly used to build supervised machine learning models for network security, network management, and network performance analysis. These models require large amounts of labeled data for training to accurately detect anomalies, identify attacks, or predict network performance in real-world scenarios. Particularly in network security, researchers and engineers utilize machine learning models for real-time monitoring of network data to prevent malicious attacks and anomalous behavior. In this context, data labeling involves assigning a label, such as "normal" or "abnormal," to each data packet or flow using an anomaly detector to train the model for intrusion and anomaly detection.

[0004] Given the sparsity of anomalous traffic and the abundance of normal traffic, it is difficult to learn anomalous patterns from limited network anomalous traffic. Furthermore, due to the imbalance between anomalous and normal traffic, it is challenging to obtain an unbiased classifier to distinguish between a small amount of anomalous traffic accompanying a large amount of normal traffic. To address these issues, most existing research is conducted in an unsupervised or supervised manner, primarily including statistical probability-based methods, proximity-based methods, and shallow machine learning-based methods. Statistical probability-based methods model traffic distribution and detect anomalous traffic based on the degree of deviation of network traffic from normal traffic in the model. However, these traditional methods lack effective statistical features of network traffic and are not adaptable to the current dynamic network environment. With the development of deep learning, a class of deep learning-based methods has been proposed. These methods mainly extract deep features of network traffic through deep learning models and then use these extracted features for anomaly detection. However, these deep learning-based methods lack reasonable interpretations of anomalous traffic and are also unable to adapt to the current dynamic network environment. In fact, network analysts can often improve their ability to adapt to dynamic network environments by describing whether the discovered anomalous traffic provides valuable information for model training. The Human-in-the-Loop approach leverages this characteristic to allow network analysts to interact with the model during training, enabling the model to adapt to dynamically changing network environments. However, because the frequency of interaction between network analysts and the model is far less than the frequency of network traffic generation, this type of method suffers from limited labor resources. Furthermore, the quality of interaction between network analysts and the model depends to some extent on the model's ability to interpret predicted anomalous traffic. The Human-in-the-Loop approach cannot provide a reasonable explanation for detected anomalous traffic, resulting in a decrease in the quality of interaction between network analysts and the model, thereby affecting model performance.

[0005] To integrate anomaly detection and anomaly interpretation, existing technologies employ a unidirectional data flow model. This model interprets the detection results provided by the anomaly detection model to a certain extent, then delivers the interpretation to network analysts for further judgment. However, in this approach, the data flow between the anomaly detector and the interpreter is unidirectional. The anomaly detector cannot fully utilize the interpretation results provided by the interpreter, or may even be unaware of the interpreter's existence. This leads to a waste of computational resources and a loss of model performance. Summary of the Invention

[0006] This invention proposes a method, system, computing device, and storage medium for interpretable network data tagging oriented towards network security, which overcomes or at least partially solves the above-mentioned problems.

[0007] According to one aspect of the present invention, a method for interpretable network data tagging oriented towards network security is provided, characterized in that it includes:

[0008] The simulator simulates each type of network attack, obtains the corresponding network data packets through packet capture, and then performs clustering operations on the data to obtain the final dataset.

[0009] The anomaly detector performs a unified modeling of the network traffic characteristics of the final dataset and the partial interpretation results provided by the interpreter. In each interaction with the network analyst, a suspicious traffic is identified. The feedback from the network analyst is integrated back into the anomaly detector, and the parameters of the anomaly detector are updated through an update strategy. This interaction process will iterate until the number of interactions is exhausted.

[0010] The interpreter interprets the detected suspicious traffic based on the maximum linear separation and queries network analysts to determine whether it is abnormal traffic.

[0011] Furthermore, obtaining the corresponding network data packets through packet capture includes: clustering N traffic streams into a group of K different clusters, where traffic streams within the same cluster share similar network traffic characteristics or conform to the same traffic pattern.

[0012] Furthermore, the anomaly detector performs a unified modeling of the network traffic feature information of the final dataset and the partial interpretation results provided by the interpreter, including:

[0013] The system employs a bidirectional data flow mode, where the anomaly detector transmits the currently detected traffic data to the interpreter, and the interpreter, after interpreting the data, transmits some of the interpretation results back to the anomaly detector.

[0014] Furthermore, identifying a suspicious flow includes selecting the most anomalous suspicious flow based on a tight confidence upper bound of the expected return of selecting a particular arm.

[0015] Furthermore, the parameters of the anomaly detector are updated through an update strategy, including updating the model parameters based on ridge regression.

[0016] Furthermore, identifying a suspicious flow also includes obtaining the contribution of any attribute to the anomalous node in the linear hyperplane through the parameters of each linear hyperplane.

[0017] Furthermore, the interpreter interprets the currently detected suspicious traffic based on maximum linear separation, including:

[0018] For the traffic to be explained, provide its anomaly score;

[0019] For this traffic, give the top two attributes with the highest abnormal scores and their scores;

[0020] Using the two given attributes as the X and Y axes, determine the distribution of the traffic and its context.

[0021] According to another aspect of the present invention, a network data tagging system for network security is provided, characterized in that it comprises:

[0022] The simulator is used to simulate each type of network attack. It obtains the corresponding network data packets by capturing packets and then performs clustering operations on the data to obtain the final dataset.

[0023] An anomaly detector is used to uniformly model the network traffic feature information of the final dataset and the partial interpretation results provided by the interpreter. In each interaction with the network analyst, a suspicious traffic is identified. The feedback from the network analyst is integrated back into the anomaly detector, and the parameters of the anomaly detector are updated through the update strategy. This interaction process will iterate until the number of interactions is exhausted.

[0024] The interpreter is used to interpret the currently detected suspicious traffic based on the maximum linear separation and to query network analysts to determine whether the traffic is abnormal.

[0025] According to another aspect of the present invention, a computing device is provided, characterized in that the device includes: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the network security-oriented interpretable network data tagging method.

[0026] According to another aspect of the present invention, a computer storage medium is provided, characterized in that the storage medium stores at least one executable instruction, the executable instruction causing a processor to perform the operation corresponding to the network security-oriented interpretable network data tagging method.

[0027] As can be seen from the technical solutions provided by the present invention, the framework of the network security interpretable network data tagging method, system, computing device and storage medium provided by the present invention, consisting of an anomaly detector model and an interpreter model, makes full use of the interpreter's computing resources and enables the anomaly detector to interact with network analysts. The interpreter ensures the quality of interaction, and ultimately enables the anomaly detector model to adapt to dynamic network environments.

[0028] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0029] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0030] Figure 1 This is a schematic diagram of an embodiment of the network security-oriented interpretable network data tagging method of the present invention;

[0031] Figure 2 This is a schematic diagram of the bidirectional data flow mode of the present invention;

[0032] Figure 3 This is a schematic diagram illustrating the process of summarizing the anomaly explanation problem in this invention;

[0033] Figure 4 This is a schematic diagram illustrating how the interpreter of the present invention interprets abnormal traffic.

[0034] Figure 5 This is a schematic diagram illustrating how the interpreter of the present invention interprets normal traffic flow;

[0035] Figure 6 This is a structural diagram of an exemplary hardware architecture for a computing device for network security interpretable network data tagging in an embodiment of the present invention. Detailed Implementation

[0036] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0037] Please refer to Figures 1 to 5 The present invention provides a method for interpretable network data tagging for network security, comprising:

[0038] Step S101: The simulator simulates each type of network attack, obtains the corresponding network data packets through packet capture, and performs clustering operations on the data to obtain the final dataset.

[0039] In step S102, the anomaly detector performs a unified modeling of the network traffic feature information of the final dataset and the partial interpretation results provided by the interpreter. In each interaction with the network analyst, a suspicious traffic is identified. The feedback from the network analyst is integrated back into the anomaly detector, and the parameters of the anomaly detector are updated through an update strategy. This interaction process will iterate until the number of interactions is exhausted.

[0040] In step S103, the interpreter interprets the currently detected suspicious traffic based on the maximum linear separation and queries the network analyst to determine whether it is abnormal traffic.

[0041] Specifically, first, the simulator simulates each type of network attack, then obtains the corresponding network data packets through packet capture, and performs clustering operations on the data to obtain the final dataset. In each interaction t with the network analyst, the anomaly detector performs a unified modeling of network traffic characteristic information and partial interpretation results provided by the interpreter, identifies a suspicious traffic, and after final interpretation by the interpreter, queries the network analyst to determine whether it is anomalous traffic. The feedback from the network analyst is integrated back into the anomaly detector, updating the anomaly detector's parameters through an update strategy. This interaction process iterates until the T query budget is exhausted.

[0042] This invention performs anomaly detection based on multi-armed slot machine algorithms. However, unlike multi-armed slot machine algorithms, this invention not only considers the contextual information of traffic, but also incorporates the interpretation results provided by the interpreter during the decision-making process.

[0043] The specific implementation steps of this invention are as follows:

[0044] During the data simulation phase, users can capture data packets based on the current network environment and obtain the corresponding network traffic characteristics.

[0045] This invention clusters N traffic flows into a group of K distinct clusters. Traffic flows within the same cluster share similar network traffic characteristics or conform to the same traffic patterns because they can be considered samples drawn from the same distribution. Therefore, this invention assumes that traffic flows within the same cluster share the same model, which determines how these flows will interact and receive rewards. Specifically, for each flow in the network, its cluster is considered as an arm to be pulled, and its features are considered as the contextual feature vector when pulling a particular arm. This invention uses K-Medoids as the clustering method.

[0046] Based on the network traffic statistics provided by the simulator, this invention aims to enable the anomaly detector to interact with network analysts and make reasonable use of the interpretation results provided by the interpreter. Overall, the research goal of the anomaly detector is to more accurately detect all abnormal network traffic with fewer interactions with network analysts. Since the anomaly detector proposed in this invention is based on a multi-armed slot machine method, the problem can be summarized as: maximizing the number of true anomalies presented to network analysts after a given T interactions.

[0047] In the anomaly detection phase, to fully utilize the interpreter's computational resources, this invention proposes a bidirectional data flow mode based on the original unidirectional data flow mode, such as... Figure 2 As shown, the anomaly detector transmits the currently detected traffic data to the interpreter. After interpretation, the interpreter transmits part of the interpretation results back to the anomaly detector. The anomaly detector obtains the final detection result by comprehensively considering the network traffic characteristics of each traffic and the partial interpretation results provided by the interpreter. Then, after interpretation by the interpreter, the final interpretation result is delivered to network analysts for judgment.

[0048] This invention, based on modeling network traffic characteristics, models the interpretation results provided by the interpreter to achieve efficient utilization of computing resources and further improve model performance. In this invention, the anomaly detector models the attribute scores provided by the interpreter that lead to anomalies. Assume the interpreter's interpretation result for the i-th traffic (judged by the anomaly detector) is y. i Then the expected revenue from choosing the i-th traffic can be expressed as:

[0049]

[0050] Since this invention treats each flow cluster as an arm, the arm a(i) corresponding to the i-th flow is the cluster to which the i-th flow belongs. Where x i θ is the context feature vector of the i-th flow. a(i) ρ is the coefficient vector of the a(i)th arm to which the i-th flow belongs, and ρ is an adjustable parameter of the expected revenue function provided by the control interpreter.

[0051] In each interaction t, this invention selects the most suspicious traffic i based on a tight upper confidence bound of the expected revenue of a specific arm. t :

[0052]

[0053] in, and ||y i ||P -1 These are for θ a(i) and The uncertainty of the estimate It is the tight confidence upper bound for the expected revenue of the i-th flow.

[0054] In determining i t Subsequently, the anomaly detector transmits the detection results to the interpreter, which interprets the results and delivers the final interpretation to network analysts for judgment. After receiving feedback from the network analysts (the benefits of arm selection), this invention updates the model parameters based on ridge regression.

[0055] Since the expected return can be divided into two independent parts, we can refer to these two parts as the first term and the second term. If we consider the second term as a constant, then the expected return has a linear relationship with the coefficient vector θ. Similarly, when the first term is set as a constant, the expected return has a linear relationship with the coefficient vector θ. The relationship is linear. Therefore, according to ridge regression, for any arm a, we can obtain the relationship between θ and... Closed form estimation:

[0056]

[0057]

[0058] Among them, due to the second item This represents the interpretation result provided by the interpreter. This invention does not perform clustering operations on it, so there is no subscript 'a'.

[0059] Specifically, in the t-th interaction, when the i-th... t When a traffic item is selected, the model weight A a(it) , P and q are updated by the following equations respectively:

[0060]

[0061]

[0062]

[0063]

[0064] The anomaly detector within the framework proposed in this invention is described in detail in the following algorithm, in which network traffic characteristic information and the interpretation results provided by the interpreter are clearly modeled into the expected returns. It is noteworthy that as feedback from network analysts is continuously incorporated into the anomaly detector model, the traffic selection strategy can be updated over time according to the network analysts' interests.

[0065] As a specific embodiment, the interpretable network data tagging system for network security of the present invention includes:

[0066] The simulator is used to simulate each type of network attack. It obtains the corresponding network data packets by capturing packets and then performs clustering operations on the data to obtain the final dataset.

[0067] An anomaly detector is used to uniformly model the network traffic feature information of the final dataset and the partial interpretation results provided by the interpreter. In each interaction with the network analyst, a suspicious traffic is identified. The feedback from the network analyst is integrated back into the anomaly detector, and the parameters of the anomaly detector are updated through the update strategy. This interaction process will iterate until the number of interactions is exhausted.

[0068] The interpreter is used to interpret the currently detected suspicious traffic based on the maximum linear separation and to query network analysts to determine whether the traffic is abnormal.

[0069] Specifically, the simulator obtains effective network traffic statistical characteristics through simulation.

[0070] Specifically, the anomaly detector has the ability to adapt to dynamic network environments, which to some extent solves the problem of limited interaction with network analysts.

[0071] Specifically, the interpreter can provide a reasonable explanation for the detected abnormal network traffic.

[0072] Specifically, the anomaly detector algorithm of the present invention includes:

[0073] Input: α, β, λ∈[0,1],

[0074] Let be the set of positive real numbers. It is a set of positive integers. An N*d dimensional real matrix

[0075] Initialization: P←λI, q←0,

[0076] I is the identity matrix. Let P be the inverse matrix for each cluster a∈{a1, ... a2}. K}:

[0077]

[0078] A a , b a These are model parameters, representing the weights.

[0079] For each t, ​​where t is the number of interactions, fort = 1 to Tdo

[0080] For each node,

[0081] In each interaction t, this invention selects the most suspicious traffic based on a tight upper confidence bound of the expected return for a specific arm: y is obtained through an interpreter algorithm. i

[0082] Select abnormal nodes based on:

[0083]

[0084] For nodoi t Get feedback

[0085]

[0086]

[0087]

[0088]

[0089]

[0090]

[0091] endfor

[0092]

[0093] endfor

[0094] Where α, β, ρ, T and λ are adjustable parameters, X is the model input, and the other parameters are considered as model weights.

[0095] Specifically, the quality of interaction between network analysts and models depends to some extent on the model's ability to interpret the predicted abnormal traffic. Therefore, the research goal of the interpreter is to explain, from multiple perspectives, why the suspected abnormal node was classified as an abnormal node after the anomaly detector provides a suspicious abnormal node at the t-th interaction, and to provide network analysts with an intuitive and visual explanation.

[0096] In the anomaly interpretation phase, the problem induction process of the network anomaly traffic interpreter proposed in this invention is as follows: Figure 4 As shown. In the t-th trial, given the dataset Let 'o' represent the i-th traffic that the anomaly detector is currently judging in the t-th trial. This invention first transforms the interpretation task into a global classification problem. Then, the global classification problem across the entire data scale can be divided into a series of local classification problems, each focusing on the context of anomaly traffic. Due to the imbalance between anomaly and normal traffic, this invention downsamples the anomaly traffic to be interpreted, making the number of context traffic for the current traffic to be interpreted similar to the number of traffic obtained after downsampling. Next, a set of local interpreters 'g' can be constructed around the anomaly traffic. Finally, the parameters of the local interpreters 'g' can be used to provide a reasonable interpretation of the anomaly traffic detected by the anomaly detector.

[0097] Specifically, this invention uses an L1-norm SVM (Support Vector Machine) to determine each linear hyperplane. The loss function for determining the linear hyperplane is:

[0098]

[0099] stξ n ≥0, ||w||1≤b

[0100] in It is the sum of the downsampled traffic obtained from the current traffic that needs to be interpreted and the context traffic. (.) + This is the hinge loss function, where b and c are variable parameters. When When setting y n =1, when When setting y n =0.

[0101] Specifically, this invention will explain network anomalies from three aspects: context description, attribute scores leading to anomalies, and anomaly degree scores, and finally propose a visual anomaly explanation method.

[0102] Specifically, regarding "context description," this invention uses Euclidean distance as the distance measurement method and employs the nearest neighbor method to obtain abnormal nodes. Neighbors In obtaining Subsequently, the present invention further... The clustering is divided into multiple disjoint clusters. Specifically, the number of clusters L in each cluster is first determined using the prediction strength method, and then the K-means method is used to cluster the neighboring nodes to obtain the final context description. Where i represents the i-th traffic flow. For smaller clusters, this invention sets if... Then it will be discarded directly in subsequent processing.

[0103] Specifically, regarding the "attributes that cause anomalies," this invention obtains the contribution of the m-th attribute to the anomalous node in each linear hyperplane through the parameter w:

[0104]

[0105] in, It is attribute a m The j-th cluster in the i-th traffic context The present invention sets the standard parameters. equal The average distance along the m-th axis between the flow and its nearest neighbor.

[0106] Then, the present invention, by applying the method to each cluster The m-th attribute a is obtained by performing a weighted average. m The contribution of the i-th traffic anomaly to be explained is:

[0107]

[0108] Regarding the "anomaly score," this invention first obtains the distance from the anomaly node to each linear hyperplane:

[0109] d i (o i )=|g i，l (o i )| / ||w i，l ||2

[0110] Among them, o i This refers to the network traffic characteristic information of the i-th traffic to be explained. It's worth noting that this invention uses an L1 paradigm SVM, which directly results in sparse model parameters w, so ||w|| = 1 / 2.i，l The calculation of ||2 is efficient.

[0111] Then by processing each cluster We obtain o by weighted average i The anomaly score is:

[0112]

[0113] Where γ i，l These are standard parameters; this invention sets γ. i，l equals o i The distance to its nearest neighbor.

[0114] Because of the contextual description, the explanations for the three aspects of anomalies—attribute scores, anomaly scores, and so on—are all numerical. However, network analysts prefer a more visual explanation. To provide network analysts with an intuitive and visual explanation, this invention proposes a visual method for interpreting anomaly traffic. The specific steps are as follows:

[0115] For the traffic to be explained, provide its anomaly score.

[0116] For this traffic, give the top two attributes with the highest abnormal scores and their scores.

[0117] Using the two given attributes as the X and Y axes, plot the distribution of the flow and its context, where each cluster in the context is represented by a color.

[0118] test

[0119] In the t-th interaction with the network analyst, assume there are 100 traffic streams, each with traffic characteristics {x1, x2, ..., xn}. 50 …x 100 For example, where x2 = {332.1, 11.2…0.022…29.028}. Then, the interpreter algorithm interprets each traffic flow, obtaining the interpretation result y for each flow. i Finally, we get {y1, y2, ... y} 50 …y 100 The value of}, for example, where y2 = {0.111, 0.911…0.001…0.028}. This can be expressed by the following formula:

[0120]

[0121] The tight confidence upper bound {91.0, 0.111…11.2…101.22} of the expected revenue for each selected traffic item can be calculated. The traffic item with the largest value is identified as the 34th item, with a tight confidence upper bound of 201.22. This 34th traffic item is interpreted by the interpreter and then given to network anomaly personnel for evaluation. If the personnel consider the traffic abnormal, they provide feedback r=1; if they consider it normal, they provide feedback r=0. After receiving feedback from the network analyst, the model parameters are updated using lines 6-11 of Algorithm 1. The end of this update signifies the end of the interaction.

[0122] For traffic entry 674 (id_outlier = 674), its corresponding attribute value is {0.021, 0.123…0.293…0.008}, which is abnormal traffic. For traffic entry 465 (id_outlier = 465), its corresponding attribute value is {0.111, 0.003…0.093…0.078}, which is normal traffic. As described in the "Detailed Implementation," this invention provides anomaly explanations through three aspects. Figure 4 and Figure 5 The interpreter interprets an abnormal traffic item (id_outlier=674). Figure 4 And interpreting a normal traffic entry (id_outlier=465) Figure 5 The figure also shows the anomaly score and the number of clusters for this traffic. The X and Y axes represent the two attributes that result in the highest anomaly scores. Along each axis, this invention provides the attribute number and the score for that attribute causing the anomaly, which is helpful for network analysts to identify abnormal traffic.

[0123] Figure 6 This is a structural diagram of an exemplary hardware architecture of a computing device 900 for network security interpretable network data tagging, as described in an embodiment of the present invention. The computing device 900 includes an input device 901, an input interface 902, a central processing unit 903, a memory 904, an output interface 905, and an output device 906. The input interface 902, central processing unit 903, memory 904, and output interface 905 are interconnected via a bus 910. The input device 901 and output device 906 are connected to the bus 910 via the input interface 902 and output interface 905, respectively, and are subsequently connected to other components of the computing device 900 for network security interpretable network data tagging.

[0124] Specifically, input device 901 receives input information from the outside and transmits the input information to central processing unit 903 through input interface 902; central processing unit 903 processes the input information based on computer-executable instructions stored in memory 904 to generate output information, temporarily or permanently stores the output information in memory 904, and then transmits the output information to output device 906 through output interface 905; output device 906 outputs the output information to the outside of computing device 900 for network security interpretable network data tagging for user use.

[0125] In other words, Figure 6 The computing device shown for network security interpretable network data tagging can also be implemented as including: a memory storing computer-executable instructions; and a processor that can achieve [combination] when executing the computer-executable instructions. Figure 1 This describes a method for interpretable network data tagging oriented towards network security.

[0126] This invention also provides a computer storage medium storing computer program instructions, which, when executed by a processor, implement the resource scheduling method of this invention.

[0127] In an exemplary embodiment, the computing device 900 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0128] It is understood that the memory 904 in this embodiment can be volatile memory or non-volatile memory, or both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); the magnetic surface memory can be disk storage or magnetic tape storage. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0129] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 904 storing a computer program, which can be executed by the central processing unit 903 of the computing device 900 to complete the steps described in the aforementioned method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0130] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0131] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0132] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the content of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0133] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0134] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0135] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0136] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0137] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The present invention can also be implemented as a device or system program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0138] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several systems, several of these systems may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

[0139] It should also be noted that the exemplary embodiments mentioned in this invention describe methods or systems based on a series of steps or apparatus. However, this invention is not limited to the order of the steps described above; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0140] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0141] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0142] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0143] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0144] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0145] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A method for interpretable network data tagging for network security, characterized in that, include: The simulator simulates each type of network attack, obtains the corresponding network data packets through packet capture, and then performs clustering operations on the data to obtain the final dataset. The anomaly detector performs a unified modeling of the network traffic characteristics of the final dataset and the partial interpretation results provided by the interpreter. In each interaction with the network analyst, a suspicious traffic is identified. The feedback from the network analyst is integrated back into the anomaly detector, and the parameters of the anomaly detector are updated through an update strategy. This interaction process will iterate until the number of interactions is exhausted. The interpreter interprets the detected suspicious traffic based on the maximum linear separation and queries network analysts to determine whether it is abnormal traffic; The process of obtaining the corresponding network data packets through packet capture includes: clustering N traffic streams into a group of K different clusters, where traffic streams within the same cluster share similar network traffic characteristics or conform to the same traffic pattern; The anomaly detector performs a unified modeling of the network traffic feature information of the final dataset and some interpretation results provided by the interpreter, including: The anomaly detector transmits the currently detected traffic data to the interpreter, and the interpreter then transmits part of the interpretation results back to the anomaly detector. Identifying a suspicious flow includes: selecting the most suspicious flow with the highest degree of anomalousness based on a tight confidence upper bound of the expected revenue of selecting a particular arm; The parameters of the anomaly detector are updated through an update strategy, including updating the model parameters based on ridge regression. Identifying a suspicious flow also includes: obtaining the contribution of any attribute to the anomalous node in the linear hyperplane through the parameters of each linear hyperplane; The interpreter interprets the currently detected suspicious traffic based on maximum linear separation, including: For the traffic to be explained, provide its anomaly score; For this traffic, give the top two attributes with the highest abnormal scores and their scores; Using the two given attributes as the X and Y axes, determine the distribution of the traffic and its context; The anomaly detector is based on the multi-armed slot machine method, which treats each traffic cluster as an arm. Therefore, the arm a(i) corresponding to the i-th traffic is the cluster to which the i-th traffic belongs. The interpretation results provided by the interpreter are also incorporated into the decision-making process. The expected revenue from selecting the i-th traffic is: In the formula, x i It is the context feature vector of the i-th flow, y i θa(i) is the interpretation result of the interpreter for the i-th flow, θa(i) is the coefficient vector of the a(i)-th arm to which the i-th flow belongs, and ρ is an adjustable parameter that controls the interpretation result provided by the interpreter on the expected revenue function. At each interaction t, a suspicious traffic i with the highest degree of anomalousness is selected based on a tight confidence upper bound of the expected return of selecting a specific arm. t The algorithm is as follows: It is to select the tight confidence upper bound of the expected revenue of the i-th flow; The anomaly detector employs the following algorithm: Input: α, β, λ∈[0,1], ; Let be the set of positive real numbers. It is a set of positive integers. It is an N*d dimensional real matrix; Initialization: P←λI, q←0, In the formula, and These are respectively for θa(i) and The uncertainty of the estimate The interpreter provides the interpretation results; P and q are the model weights, and t is the number of interactions; I is the identity matrix. Let be the inverse matrix of P, for each cluster a∈{a1, ...,aK}; ； A a , b a These are model parameters, representing weights.

2. A network security-oriented interpretable network data tagging system employing the method described in claim 1, characterized in that, include: The simulator is used to simulate each type of network attack. It obtains the corresponding network data packets by capturing packets and then performs clustering operations on the data to obtain the final dataset. An anomaly detector is used to uniformly model the network traffic feature information of the final dataset and the partial interpretation results provided by the interpreter. In each interaction with the network analyst, a suspicious traffic is identified. The feedback from the network analyst is integrated back into the anomaly detector, and the parameters of the anomaly detector are updated through the update strategy. This interaction process will iterate until the number of interactions is exhausted. The interpreter is used to interpret the currently detected suspicious traffic based on the maximum linear separation and to query network analysts to determine whether the traffic is abnormal.

3. A computing device, characterized in that, The device includes: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the network security-oriented interpretable network data tagging method as described in claim 1.

4. A computer storage medium, characterized in that, The storage medium stores at least one executable instruction that causes the processor to perform the operation corresponding to the network security interpretable network data tagging method as described in claim 1.

Images