Network intrusion detection method, device, equipment, medium and product

Abstract

The invention relates to the technical field of network engineering, and discloses a network intrusion detection method, device and equipment, a medium and a product, and the method comprises the steps: obtaining network flow data, and extracting the network flow features of the network flow data; inputting the network traffic features into a pre-trained semi-supervised classifier to identify intrusion categories of the network traffic data through the semi-supervised classifier, the traffic categories including known categories and unknown categories; when the intrusion category is an unknown category, clustering the known features of the known category and the network flow features together in the same feature space to obtain a clustering result; and performing intrusion category labeling on the data clusters which do not belong to the known category in the clustering result. According to the method, unknown category data are identified based on the semi-supervised classifier, then the data clusters are separately clustered, and then unknown features are separately labeled, so that the problem that unknown traffic intrusion is misjudged as known intrusion is avoided, and the sensitivity of novel traffic intrusion discovery and detection is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of network engineering technology, and in particular to a network intrusion detection method, device, equipment, medium and product. Background Art

[0002] The network intrusion detection system is a technology that actively defends the network. The network environment is dynamic and complex, which also determines the harm and hidden characteristics of network attacks. The current network intrusion detection technology faces very severe network attacks. The current network intrusion detection technology is mainly implemented through pattern recognition, but the training of the model depends on a large amount of labeled data. When the data label is not perfect, the model effect is poor, and the recognition rate is low when encountering new network intrusion data. Summary of the invention

[0003] In view of this, the present invention provides a network intrusion detection method to solve the problem of low recognition rate of new network intrusion data.

[0004] In a first aspect, the present invention provides a network intrusion detection method, the method comprising: obtaining network traffic data and extracting network traffic features of the network traffic data; inputting the network traffic features into a pre-trained semi-supervised classifier to identify the intrusion category of the network traffic data through the semi-supervised classifier, the intrusion category comprising a known category and an unknown category; when the intrusion category is the unknown category, clustering the known features of the known category and the network traffic features together in the same feature space to obtain a clustering result; and labeling the data clusters that do not belong to the known category in the clustering result as intrusion categories.

[0005] In some optional embodiments, the step of training the semi-supervised classifier includes: defining an agent model of the semi-supervised classifier based on reinforcement learning technology; defining a category space, wherein the category space includes the known categories and the unknown categories; defining a reward function of the semi-supervised classifier, wherein the reward function is:

[0006]

[0007] Where r represents the reward value used to adjust the parameters of the agent model, r c is the reward score for classifying the predicted category into the unknown category, r true is the reward score when the predicted category is classified into the known category and the classification is correct, r false is the reward score when the predicted category is classified into the known category and the classification is wrong, a t is the current predicted category, a N+1 Represents the unknown category, a t ′Representing a label of the known category; obtaining a training feature sample set; inputting the training feature sample set into the intelligent agent model, adjusting the parameters of the intelligent agent model according to the predicted category of the intelligent agent model and the reward function, and obtaining the trained semi-supervised classifier.

[0008] In some optional implementations, obtaining the training feature sample set includes: obtaining a network traffic data set; hiding category labels of part of the data in the network traffic data set; and using the processed network traffic data set as the training feature sample set.

[0009] In some optional implementations, the inputting the training feature sample set into the agent model includes: receiving the training feature samples in the training feature sample set in sequence according to time steps; placing the training feature samples received in each time step in a temporary buffer in sequence; and before each round of training, calculating the probability of each training sample in the temporary buffer being extracted according to the following formula:

[0010]

[0011] Among them, P(i) represents the probability of the i-th training sample being extracted, k is the number of training samples, and q i is the priority of training sample i, N i represents the number of times the i-th training sample has been extracted, α is a hyperparameter representing the priority importance weight, and t i Represents the total number of steps after starting sampling in the temporary buffer area; based on the probability of each training sample being extracted, a training sample is randomly extracted from the temporary buffer area and input into the intelligent agent model.

[0012] In some optional embodiments, the agent model for defining a semi-supervised classifier based on reinforcement learning technology includes: defining the agent model using an Actor-Critic network model, the Actor-Critic network model includes a training network and a target network, the training network is used to update model parameters according to the results output by the target network, and after a preset period, the updated model parameters are synchronized to the target network.

[0013] In some optional implementations, the obtaining of network traffic data includes: obtaining original network traffic data; cleaning the original network traffic data for errors; converting character type data in the original network traffic data into numerical values, and encoding and normalizing discrete data therein to obtain preprocessed network traffic data.

[0014] In a second aspect, the present invention provides a network intrusion detection device, which includes: a data feature extraction module, which is used to obtain network traffic data and extract network traffic features of the network traffic data; a classification module, which is used to input the network traffic features into a pre-trained semi-supervised classifier to identify the intrusion category of the network traffic data through the semi-supervised classifier, and the intrusion category includes a known category and an unknown category; a clustering module, which is used to cluster the known features of the known category and the network traffic features in the same feature space when the intrusion category is the unknown category to obtain a clustering result; a new category labeling module, which is used to label the data clusters that do not belong to the known category in the clustering result with intrusion categories.

[0015] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0016] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to cause a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0017] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0018] The technical solution provided by the present invention has the following advantages:

[0019] The present invention trains a semi-supervised classifier through semi-supervised learning, so that it can train labeled traffic data and unlabeled traffic data at the same time, so as to achieve the effect of classifying unlabeled traffic data into unknown categories. In the application stage, the semi-supervised classifier is used to identify the network traffic data acquired in real time. If the intrusion category of the network traffic data is determined to be an unknown category, it is switched to the clustering and labeling module, and the features of the known categories and the network traffic features of the current unknown category are clustered together using a clustering algorithm, so that the data part of the network traffic data that is clustered with the known features can be labeled as a known intrusion category, and the unknown features are clustered into data clusters separately, and then the unknown features are separately labeled based on expert experience, thereby avoiding the problem of misjudging unknown traffic intrusions as known intrusions, and significantly improving the sensitivity of discovering and detecting new types of traffic intrusions. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 is a flow chart of a network intrusion detection method according to an embodiment of the present invention;

[0022] Figure 2 is another flow chart of a network intrusion detection method according to an embodiment of the present invention;

[0023] Figure 3 is another flow chart of a network intrusion detection method according to an embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of the structure of a network intrusion detection device according to an embodiment of the present invention;

[0025] Figure 5 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0027] According to an embodiment of the present invention, an embodiment of a network intrusion detection method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] In this embodiment, a network intrusion detection method is provided. Figure 1 1 is a flow chart of a network intrusion detection method according to an embodiment of the present invention, the flow chart includes the following steps:

[0029] Step S101, acquiring network traffic data and extracting network traffic features of the network traffic data;

[0030] Step S102, inputting the network traffic features into a pre-trained semi-supervised classifier to identify intrusion categories of the network traffic data through the semi-supervised classifier, where the intrusion categories include known categories and unknown categories;

[0031] Step S103, when the intrusion category is an unknown category, clustering the known features of the known category and the network traffic features together in the same feature space to obtain a clustering result;

[0032] Step S104: labeling the data clusters that do not belong to the known categories in the clustering results with intrusion categories.

[0033] Specifically, related technologies often use fully supervised learning models to detect network intrusion traffic. Due to the limitation of training sample labels, new network intrusions are often mistakenly judged as known types, and new network intrusions cannot be detected separately, resulting in inaccurate detection. Based on this, the solution provided by the present invention trains a semi-supervised classifier through semi-supervised learning. The semi-supervised classifier, as a classification module, can simultaneously train labeled traffic data and unlabeled traffic data to achieve the effect of classifying unlabeled traffic data into unknown categories. In the application stage, the semi-supervised classifier is used to identify the network traffic data acquired in real time. If the intrusion category of the network traffic data is determined to be an unknown category, it is switched to the labeling module, and the features of the known categories and the network traffic features of the current unknown category are clustered together using a clustering algorithm, so that the data part of the network traffic data that is clustered with the known features can be marked as a known intrusion category, and the unknown features are clustered into data clusters separately, and then the unknown features are separately labeled based on expert experience.

[0034] In the embodiment of the present invention, the labeling module can use the k-means algorithm based on Euclidean distance to cluster the data features and use professional knowledge to label the unknown samples. ij =||s j -u i || 2 Calculate the distance from the current data to each centroid. The number of centroids is k, and the current training sample is s. j , the center of mass is u i, i∈[i,k]. For example, the original data set has only 5 categories. The present invention sets the number of categories of the clustering algorithm to 7, and sets two additional categories. The algorithm first randomly selects 7 data points as centroids, then calculates the distance between each data point and each centroid, assigns it to the nearest centroid, and then recalculates each centroid. This process is repeated until the stop condition is met (for example, the centroid no longer changes or the maximum number of iterations is reached). After clustering the data features using the clustering algorithm, new data categories will be generated, and the new categories will be labeled by expert knowledge. Through this solution, the problem of misjudging unknown traffic intrusions as known intrusions is avoided, and the sensitivity of discovering and detecting new types of traffic intrusions is significantly improved.

[0035] In an embodiment of the present invention, network traffic features of network traffic data can be extracted by a CNN network and a BiLSTM network, wherein the CNN network is used to complete the extraction of data features, and the BiSTM network completes the extraction of data time series features, so that the data features are more in line with the training rules of the reinforcement learning intelligent agent model.

[0036] In some optional embodiments, the step of training a semi-supervised classifier comprises:

[0037] Step a1, defining an agent model of a semi-supervised classifier based on reinforcement learning technology;

[0038] Step a2, defining a category space, the category space includes known categories and unknown categories;

[0039] Specifically, the present invention creates a semi-supervised classifier based on the theory of reinforcement learning, wherein the neural network model used in the semi-supervised classifier is called an agent model in reinforcement learning, and the agent model includes but is not limited to Q learning, Actor-Critic network model, etc. After creating the model architecture, it is also necessary to define the category space, which is the category that the agent model can output, including known categories and unknown categories, wherein the known categories include a variety of different network intrusion categories, such as denial of service attacks, remote to local attacks, user to root attacks, detection attacks, etc. For example, the defined category space is A={a 1 ,…,a N+1}, where a 1 ~a N Represents a known category, a N+1 Indicates an unknown category.

[0040] Step a3, define the reward function of the semi-supervised classifier, the reward function is:

[0041]

[0042] In the formula, r represents the reward value used to adjust the parameters of the agent model, r cis the reward score for classifying the predicted category into the unknown category, r true is the reward score when the predicted category is classified into the known category and the classification is correct, r false is the reward score when the predicted category is classified into the known category and the classification is wrong, a t is the current predicted category, a N+1 Indicates unknown category, a t ′ Represents the labels of known categories.

[0043] Specifically, the embodiment of the present invention also needs to define a reward function, and the function value of the reward function is used to adjust the reward function of the intelligent model parameters during the training process. The reward function refers to the intelligent model in reinforcement learning. It calculates a probability distribution matrix based on the state (network traffic data) input by the user. The probability distribution matrix includes the probability of classifying the network traffic data into various intrusion categories this time, and then determines an intrusion category of the network traffic data this time based on the probability. After that, the intrusion category is substituted into the reward function to obtain a reward value. The higher the reward value, the more accurate the current output category is, and the lower the reward value, the more inaccurate the current output category is, thereby guiding the intelligent model to adjust the parameter size according to the corresponding reward value.

[0044] In the embodiment of the present invention, r true is the reward score when the predicted category is classified into the known category and the classification is correct, r false It is the reward score when the predicted category is classified into the known category and the classification is wrong, usually r true positive r false is negative, and r c is the bonus score when the predicted category is classified into an unknown category. Since the unknown category has no data label and cannot be verified, the embodiment of the present invention targets r c The value is often assigned to 0, so the agent model provided by the present invention is a semi-supervised classifier. During training, when the network traffic data is classified into an unknown class, the reward is 0, and the parameters of the model are not adjusted. If the network traffic data of unknown class is always classified into a known class with a label, and the condition a t ≠a N+1 ,a t ′ ≠a t It is judged that it does not actually belong to a known category, so the model parameters are updated according to the failure reward. As the number of failures increases, the probability of the unknown class being classified as a known category naturally decreases, thereby increasing the recognition probability of the unknown category, and then achieving the effect of classifying the network traffic data into the unknown category.

[0045] Step a4, obtaining a training feature sample set;

[0046] Step a5: input the training feature sample set into the intelligent agent model, adjust the parameters of the intelligent agent model according to the prediction category and reward function of the intelligent agent model, and obtain a trained semi-supervised classifier.

[0047] Specifically, based on the training principles of steps a1 to a3 above, the acquired training feature sample set is input into the intelligent agent model, the reward function is input according to the predicted category of the intelligent agent model, and the parameters of the intelligent agent model are repeatedly adjusted according to the reward value of the reward function to obtain a trained semi-supervised classifier, which ultimately achieves the effect of distinguishing and detecting new network intrusions from known categories.

[0048] In some optional implementations, the above step a4 includes:

[0049] Step b1, obtaining a network traffic data set;

[0050] Step b2, hiding the category labels of some data in the network traffic data set;

[0051] Step b3, using the processed network traffic data set as a training feature sample set.

[0052] Specifically, the present invention obtains two data sets, NSL_KDD and UNSW_NB15, as network traffic data sets for training. Among them, the NSL_KDD data set contains 41 features and a label, and the features include basic features, content features, and traffic-based features; the label is used to identify whether the connection is normal or attack, and the attack types are divided into four categories: DOS (denial of service attack), R2L (remote to local attack), U2R (user to root attack) and Probe (probe attack). The UNSW_NB15 data set contains 2,540,044 records, which are divided into normal and abnormal (attack) network activities. These records are collected by the IXIA traffic generator using three virtual servers, two of which are configured to distribute normal network traffic and the third server is configured to generate abnormal network traffic. In order to achieve semi-supervised learning, the present invention hides the category labels of part of the data in the dataset as data of unknown categories. For example, the two categories R2L and U2R in NSL_KDD are set as hidden classes, and Fuzzers, Analysis, worms and Dos in UNSW_NB15 are set as hidden classes. Then the processed network traffic dataset is used as a training feature sample set, which facilitates subsequent semi-supervised learning.

[0053] In some optional implementations, the above step a5 includes:

[0054] Step c1, receiving training feature samples in the training feature sample set in sequence according to time steps;

[0055] Step c2, placing the training feature samples received at each time step into a temporary buffer area in sequence;

[0056] Step c3, before each round of training, the probability of each training sample being extracted from the temporary buffer is calculated according to the following formula:

[0057]

[0058] Among them, P(i) represents the probability of the i-th training sample being extracted, k is the number of training samples, and q i is the priority of training sample i, N i represents the number of times the i-th training sample has been extracted, α is a hyperparameter representing the priority importance weight, and t i Indicates the total number of steps after starting sampling in the temporary buffer;

[0059] Step c4, randomly extract a training sample from the temporary buffer area based on the probability of each training sample being extracted and input it into the intelligent agent model.

[0060] Specifically, Figure 2 and Figure 3 As shown, the present invention also processes the training samples based on the experience replay mechanism, initializes the maximum training round and the current training round; initializes the maximum training step size and the current training step size of each round; constructs a temporary buffer area as an agent trajectory experience replay module, for example, the training samples that need to be placed in the temporary buffer area are set to (s i ,a i ,r i ), where s i represents the i-th training sample, a i represents the i-th intrusion category, r i Represents the current reward value. After a round of training is completed, the trajectory experience training samples are summarized in chronological order as {(s 1 ,a 1 ,r 1 ),…,(s A ,a T ,r T )}. Each time you extract experience training samples from the temporary buffer to train the model, you need to first calculate the sampling probability of the data in the experience playback cache module based on the familiarity. The purpose is to gradually reduce the probability of higher priority experience samples being collected, increase the probability of samples not being collected, and allow low priority samples to be collected. The probability calculation formula is:

[0061]

[0062] The agent model is trained using the sampled training samples. If the training data is highly correlated, the direction and amplitude of the update may become unstable. This is because in the time series, the correlation between the data will affect the variance of the gradient estimate, thereby affecting the stability of the learning process. Through the processing of the temporary buffer, samples are randomly drawn from the temporary buffer. The temporary buffer can effectively disrupt the time series correlation of the samples, making the training data more independent and identically distributed. The effect of disrupting the sample correlation is similar to reducing the autocorrelation between samples, thereby improving the stability of the training.

[0063] Through the processing of the embodiment of the present invention, the model training process of the network intrusion detection method provided by the present invention is:

[0064] Input the current traffic features to the semi-supervised classifier, which outputs a probability distribution matrix. The probability distribution matrix is ​​then used to determine the intrusion category. The determined category is then compared with the actual label to determine whether the classification is correct. The reward value is calculated according to the reward function, and the classification module parameters are adjusted according to the reward value. The above process is repeated repeatedly. In particular, the training data is placed in a temporary buffer according to the time series, and a sample is randomly selected from the temporary buffer each time according to the calculated probability to participate in the training to enhance the stability of the model.

[0065] In some optional implementations, the above step a1 includes:

[0066] The Actor-Critic network model is used to define the intelligent agent model. The Actor-Critic network model includes a training network and a target network. The training network is used to update the model parameters according to the output of the target network, and synchronize the updated model parameters to the target network after a preset period.

[0067] Specifically, the intelligent agent model of the embodiment of the present invention builds a deep reinforcement learning Actor-Critic model, including a training network and a target network, the target network is set as a target Actor-Critic network and the training network is set as a training Actor-Critic network.

[0068] Furthermore, the model parameters for training the Actor network are The parameters of the target Actor network are The training Critic network parameter is θ μ , the target Critic network parameter is θ ′ μ, the discount factor is γ, the learning rate of the network is α, and the target network update rate is τ. The Actor-Critic algorithm is a reinforcement learning method that combines policy gradient and temporal difference learning. It consists of two parts: the actor and the critic. The actor is responsible for generating actions and interacting with the environment, while the critic evaluates the value of the current strategy and guides the actor's actions in the next stage. The Actor-Critic algorithm is suitable for problems in continuous action spaces because it does not require discretization of the action space, which makes it more effective in dealing with high-dimensional action spaces.

[0069] In the present invention, the model parameters of the training network are updated in real time, and the parameters of the target network directly copy the model parameters of the training network after a preset period, and the model parameter update process of the training network needs to refer to the category of the target network output. Through this scheme, the training network can learn to predict the output results more accurately and adjust the model parameters accurately. At the same time, the target network remains fixed for a period of time, avoiding the training network chasing the constantly changing target and causing the output results to change too frequently, thereby affecting the training effect. The integrated training network and the target network can improve the training stability of the intelligent model.

[0070] Specifically, the update formula for training the Actor network is:

[0071]

[0072] The update formula for training the Critic network is:

[0073]

[0074] Among them, Q′(s t ,a t ) is the output value of the target Actor network, To train the action strategy output by the Actor network, y t =r t +γmax A Q′(s t ,a t ).

[0075] Furthermore, using the parameters of the current training network μ updates the target network parameters at an update rate of τ and μ′, the update formula is:

[0076]

[0077] μ′←τμ+(1-τ)μ′

[0078] If the current training round is greater than or equal to the maximum training round, the network training ends.

[0079] In some optional implementations, the above step S101 includes:

[0080] Step d1, obtaining original network traffic data;

[0081] Step d2, cleaning the original network traffic data for erroneous data;

[0082] Step d3, converting the character type data in the original network traffic data into numerical values, and encoding and normalizing the discrete data therein to obtain preprocessed network traffic data.

[0083] Specifically, after acquiring the network traffic data, the embodiment of the present invention first cleans and encodes the traffic data, removes obvious erroneous data in the original network traffic data, then converts the character type data in the original network traffic data into numerical values, and encodes and normalizes the discrete data therein to obtain preprocessed network traffic data. Finally, the preprocessed network traffic data is used for model training, thereby indirectly improving the accuracy of model training by improving data quality.

[0084] In this embodiment, a network intrusion detection device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0085] This embodiment provides a network intrusion detection device, such as Figure 4 As shown, including:

[0086] The data feature extraction module 301 is used to obtain network traffic data and extract network traffic features of the network traffic data;

[0087] A classification module 302, used for inputting network traffic features into a pre-trained semi-supervised classifier to identify intrusion categories of network traffic data through the semi-supervised classifier, where the traffic categories include known categories and unknown categories;

[0088] The clustering module 303 is used to cluster the known features of the known categories and the network traffic features together in the same feature space to obtain a clustering result when the intrusion category is an unknown category;

[0089] The new category labeling module 304 is used to label the data clusters that do not belong to the known categories in the clustering results as intrusion categories.

[0090] In some optional implementations, the classification module 302 includes:

[0091] Model definition unit, used to define the agent model of semi-supervised classifier based on reinforcement learning technology;

[0092] A category definition unit is used to define a category space, where the category space includes known categories and unknown categories;

[0093] The reward definition unit is used to define the reward function of the semi-supervised classifier. The reward function is:

[0094]

[0095] In the formula, r represents the final value of the reward score used to adjust the parameters of the agent model, r c is the reward score for classifying the predicted category into the unknown category, r true is the reward score when the predicted category is classified into the known category and the classification is correct, r false is the reward score when the predicted category is classified into the known category and the classification is wrong, a t is the current predicted category, a N+1 Indicates unknown category, a t ′ Labels representing known categories;

[0096] A training sample acquisition unit, used to acquire a training feature sample set;

[0097] The training unit is used to input the training feature sample set into the intelligent agent model, adjust the parameters of the intelligent agent model according to the prediction category and reward function of the intelligent agent model, and obtain a trained semi-supervised classifier.

[0098] In some optional implementations, the training sample acquisition unit includes:

[0099] A data set acquisition unit, used for acquiring a network traffic data set;

[0100] A hidden processing unit, used for hiding the category labels of part of the data in the network traffic data set;

[0101] The training sample set generating unit is used to use the processed network traffic data set as a training feature sample set.

[0102] In some optional implementations, the training unit includes:

[0103] A sample receiving unit receives training feature samples in a training feature sample set in sequence according to time steps;

[0104] A cache unit is used to sequentially place the training feature samples received at each time step into a temporary cache area;

[0105] The probability calculation unit is used to calculate the probability of each training sample being extracted from the temporary buffer according to the following formula before each round of training:

[0106]

[0107] Among them, P(i) represents the probability of the i-th training sample being extracted, k is the number of training samples, and q i is the priority of training sample i, N i represents the number of times the i-th training sample has been extracted, α is a hyperparameter representing the priority importance weight, and t i Indicates the total number of steps after starting sampling in the temporary buffer;

[0108] The extraction unit is used to randomly extract a training sample from the temporary buffer area based on the probability of each training sample being extracted and input it into the intelligent agent model.

[0109] In some optional embodiments, an agent model of a semi-supervised classifier is defined based on reinforcement learning technology, including: using an Actor-Critic network model to define the agent model, the Actor-Critic network model includes a training network and a target network, the training network is used to update model parameters according to the results output by the target network, and after a preset period, the updated model parameters are synchronized to the target network.

[0110] In some optional embodiments, the device further comprises:

[0111] The data preprocessing unit is used to obtain the original network traffic data; clean the original network traffic data for erroneous data; convert the character type data in the original network traffic data into numerical values, and encode and normalize the discrete data therein to obtain the preprocessed network traffic data.

[0112] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0113] The device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0114] The embodiment of the present invention also provides a computer device having the above Figure 4 The device shown.

[0115] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0116] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0117] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0118] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0119] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0120] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0121] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0122] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0123] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A network intrusion detection method, characterized in that: The method comprises: Acquire network traffic data, and extract network traffic features of the network traffic data; Inputting the network traffic features into a pre-trained semi-supervised classifier to identify intrusion categories of the network traffic data through the semi-supervised classifier, wherein the intrusion categories include known categories and unknown categories; When the intrusion category is the unknown category, clustering the known features of the known category and the network traffic features together in the same feature space to obtain a clustering result; The data clusters that do not belong to known categories in the clustering results are labeled with intrusion categories.

2. The method according to claim 1, characterized in that The steps of training the semi-supervised classifier include: Define an agent model for semi-supervised classifiers based on reinforcement learning techniques; defining a category space, wherein the category space includes the known categories and the unknown categories; Define the reward function of the semi-supervised classifier, which is: Where r represents the reward value used to adjust the parameters of the agent model, r c is the reward score for classifying the predicted category into the unknown category, r true is the reward score when the predicted category is classified into the known category and the classification is correct, r false is the reward score when the predicted category is classified into the known category and the classification is wrong, a t is the current predicted category, a N+1 Represents the unknown category, a t ′ a label representing the known category; Obtain a training feature sample set; The training feature sample set is input into the agent model, and the parameters of the agent model are adjusted according to the predicted category of the agent model and the reward function to obtain the trained semi-supervised classifier.

3. The method according to claim 2, characterized in that The step of obtaining a training feature sample set includes: Get the network traffic dataset; Hiding the category labels of part of the data in the network traffic data set; The processed network traffic data set is used as the training feature sample set.

4. The method according to claim 3, characterized in that The step of inputting the training feature sample set into the agent model comprises: receiving the training feature samples in the training feature sample set in sequence according to time steps; The training feature samples received at each time step are sequentially placed into the temporary buffer; Before each round of training, the probability of each training sample being extracted from the temporary buffer is calculated according to the following formula: Among them, P(i) represents the probability of the i-th training sample being extracted, k is the number of training samples, and q i is the priority of training sample i, N i represents the number of times the i-th training sample has been extracted, α is a hyperparameter representing the priority importance weight, and t i Indicates the total number of steps after starting sampling in the temporary buffer; A training sample is randomly selected from the temporary buffer area based on the probability of each training sample being selected and input into the agent model.

5. The method according to claim 2, characterized in that: The intelligent agent model for defining a semi-supervised classifier based on reinforcement learning technology includes: The agent model is defined using an Actor-Critic network model, which includes a training network and a target network. The training network is used to update model parameters according to the results output by the target network, and synchronize the updated model parameters to the target network after a preset period.

6. The method according to claim 1, characterized in that The obtaining of network traffic data includes: Get raw network traffic data; Cleaning the original network traffic data of errors; The character type data in the original network traffic data is converted into numerical values, and the discrete data therein is encoded and normalized to obtain preprocessed network traffic data.

7. A network intrusion detection device, characterized in that: The device comprises: A data feature extraction module, used to obtain network flow data and extract network flow features of the network flow data; A classification module, used for inputting the network traffic features into a pre-trained semi-supervised classifier to identify the intrusion category of the network traffic data through the semi-supervised classifier, wherein the intrusion category includes a known category and an unknown category; A clustering module, used for clustering known features of known categories and the network traffic features together in the same feature space to obtain a clustering result when the intrusion category is the unknown category; The new category labeling module is used to label the intrusion categories of data clusters that do not belong to known categories in the clustering results.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method according to any one of claims 1 to 6.

Images