A method, apparatus, device, medium, and product for network intrusion detection.

Abstract

This invention relates to the field of network engineering technology and discloses a network intrusion detection method, apparatus, device, medium, and product. The method includes: acquiring network traffic data and extracting network traffic features from the network traffic data; inputting the network traffic features into a pre-trained semi-supervised classifier to identify the intrusion category of the network traffic data through the semi-supervised classifier, where the traffic categories include known categories and unknown categories; when the intrusion category is unknown, clustering the known features of the known categories and the network traffic features together in the same feature space to obtain a clustering result; and labeling the data clusters that do not belong to the known categories in the clustering result with intrusion category. This invention identifies unknown category data based on a semi-supervised classifier, then clusters the data clusters separately, and then labels the unknown features separately, thereby avoiding the problem of misjudging unknown traffic intrusions as known intrusions and significantly improving the sensitivity to the discovery and detection of new types of traffic intrusions.

Description

Technical Field

[0001] This invention relates to the field of network engineering technology, specifically to a network intrusion detection method, apparatus, equipment, medium, and product. Background Technology

[0002] Network intrusion detection systems (NICs) are a proactive network defense technology. The dynamic and complex network environment determines the severity and stealth of network attacks, posing a significant challenge to current NIC technology. Current NICs primarily rely on pattern recognition; however, model training depends on large amounts of labeled data. When data labels are incomplete, the model performs poorly, and its recognition rate is low when encountering novel network intrusion data. Summary of the Invention

[0003] In view of this, the present invention provides a network intrusion detection method to solve the problem of low identification rate of novel network intrusion data.

[0004] In a first aspect, the present invention provides a network intrusion detection method, the method comprising: acquiring network traffic data and extracting network traffic features from the network traffic data; inputting the network traffic features into a pre-trained semi-supervised classifier to identify the intrusion category of the network traffic data through the semi-supervised classifier, the intrusion category including known categories and unknown categories; when the intrusion category is the unknown category, clustering the known features of the known category and the network traffic features together in the same feature space to obtain a clustering result; and labeling the data clusters in the clustering result that do not belong to the known category with intrusion category.

[0005] In some optional implementations, the step of training the semi-supervised classifier includes: defining an agent model for the semi-supervised classifier based on reinforcement learning techniques; defining a class space, which includes the known classes and the unknown classes; and defining a reward function for the semi-supervised classifier, wherein the reward function is:

[0006]

[0007] In the formula, r represents the reward value used to adjust the parameters of the agent model, r c The reward score, r, is the score awarded when the predicted category is classified into the unknown category. true The reward score, r, is the score awarded when the predicted category is correctly classified into the known category. false It is the reward score when the predicted category is classified into the known category but is misclassified, a t This is the current prediction category, a N+1 Indicates the unknown category, a t ′The process involves: assigning labels to the known categories; obtaining a training feature sample set; inputting the training feature sample set into the agent model; adjusting the parameters of the agent model based on its predicted categories and the reward function; and obtaining the trained semi-supervised classifier.

[0008] In some optional implementations, obtaining the training feature sample set includes: obtaining a network traffic dataset; hiding the category labels of a portion of the data in the network traffic dataset; and using the processed network traffic dataset as the training feature sample set.

[0009] In some optional implementations, inputting the training feature sample set into the agent model includes: receiving training feature samples from the training feature sample set sequentially according to time steps; placing the training feature samples received at each time step into a temporary buffer sequentially; and calculating the probability of each training sample being extracted from the temporary buffer before each round of training according to the following formula:

[0010]

[0011] Where P(i) represents the probability of the i-th training sample being selected, k is the number of training samples, and q i It is the priority of training sample i. N i t represents the number of times the i-th training sample has been extracted, α is a hyperparameter representing the priority importance weight, and t i This represents the total number of steps since sampling began in the temporary buffer; a training sample is randomly selected from the temporary buffer based on the probability of each training sample being selected and input into the agent model.

[0012] In some optional implementations, the agent model for defining a semi-supervised classifier based on reinforcement learning technology includes: defining the agent model using an Actor-Critic network model, wherein the Actor-Critic network model includes a training network and a target network, wherein the training network is used to update model parameters based on the output of the target network, and after a preset period, synchronizes the updated model parameters to the target network.

[0013] In some optional implementations, the acquisition of network traffic data includes: acquiring raw network traffic data; cleaning the raw network traffic data to remove errors; converting character data in the raw network traffic data into numerical values, and encoding and normalizing the discrete data therein to obtain preprocessed network traffic data.

[0014] Secondly, the present invention provides a network intrusion detection device, the device comprising: a data feature extraction module for acquiring network traffic data and extracting network traffic features from the network traffic data; a classification module for inputting the network traffic features into a pre-trained semi-supervised classifier to identify the intrusion category of the network traffic data through the semi-supervised classifier, the intrusion category including known categories and unknown categories; a clustering module for clustering known features of known categories and network traffic features together in the same feature space when the intrusion category is the unknown category, to obtain a clustering result; and a new category labeling module for labeling data clusters in the clustering result that do not belong to known categories with intrusion categories.

[0015] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method described in the first aspect or any corresponding embodiment thereof.

[0016] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the method described in the first aspect or any corresponding embodiment thereof.

[0017] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to perform the method described in the first aspect or any corresponding embodiment thereof.

[0018] The technical solution provided by this invention has the following advantages:

[0019] This invention trains a semi-supervised classifier using semi-supervised learning, enabling simultaneous training on both labeled and unlabeled traffic data. This allows for the classification of unlabeled traffic data into an unknown category. In the application phase, the semi-supervised classifier identifies real-time network traffic data. If the intrusion category of the network traffic data is determined to be unknown, the system switches to a clustering and labeling module. A clustering algorithm groups features of known categories with those of the currently unknown category of network traffic data together. This allows the portion of the network traffic data clustered with known features to be labeled as a known intrusion category. Unknown features are then clustered separately, and further labeled based on expert experience. This avoids misclassifying unknown traffic intrusions as known intrusions, significantly improving the sensitivity for detecting and identifying novel traffic intrusions. Attached Figure Description

[0020] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a network intrusion detection method according to an embodiment of the present invention;

[0022] Figure 2 This is another flowchart illustrating a network intrusion detection method according to an embodiment of the present invention;

[0023] Figure 3 This is another flowchart illustrating a network intrusion detection method according to an embodiment of the present invention;

[0024] Figure 4 This is a schematic diagram of the structure of a network intrusion detection device according to an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] According to an embodiment of the present invention, a network intrusion detection method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0028] This embodiment provides a network intrusion detection method. Figure 1 This is a flowchart of a network intrusion detection method according to an embodiment of the present invention, the process including the following steps:

[0029] Step S101: Obtain network traffic data and extract network traffic features from the network traffic data;

[0030] Step S102: Input the network traffic features into a pre-trained semi-supervised classifier to identify the intrusion category of the network traffic data through the semi-supervised classifier. The intrusion category includes known categories and unknown categories.

[0031] Step S103: When the intrusion category is unknown, cluster the known features of the known categories and the network traffic features together in the same feature space to obtain the clustering result;

[0032] Step S104: Intrusion category labeling is performed on data clusters that do not belong to known categories in the clustering results.

[0033] Specifically, related technologies often utilize fully supervised learning models to detect network intrusion traffic. However, due to limitations in training sample labels, they frequently misclassify novel network intrusions as known types and fail to detect them independently, leading to inaccurate detection. To address this, the present invention provides a solution that trains a semi-supervised classifier using semi-supervised learning. This semi-supervised classifier, acting as a classification module, can be trained simultaneously on both labeled and unlabeled traffic data, effectively classifying unlabeled traffic data into an unknown category. In the application phase, the semi-supervised classifier is used to identify real-time acquired network traffic data. If the intrusion category of the network traffic data is determined to be unknown, the system switches to a labeling module. A clustering algorithm is used to cluster features of known categories together with the network traffic features of the currently unknown category. The portion of the network traffic data clustered with known features can then be labeled as a known intrusion category. Unknown features are clustered separately into data clusters, and then individually labeled based on expert experience.

[0034] In this embodiment of the invention, the labeling module can use the k-means algorithm based on Euclidean distance to cluster data features and utilize professional knowledge to label unknown samples. In the k-means algorithm, d... ij =||s j -u i ||2. Calculate the distance from the current data to each centroid. Where the number of centroids is k, and the current training sample is s. j The centroid is u iFor example, if the original dataset has only 5 classes, this invention sets the number of classes in the clustering algorithm to 7, with two additional classes. The algorithm first randomly selects 7 data points as centroids, then calculates the distance between each data point and each centroid, assigns it to the nearest centroid, and then recalculates each centroid. This process is repeated until a stopping condition is met (e.g., the centroids no longer change or the maximum number of iterations is reached). After clustering the data features using the clustering algorithm, new data categories are generated, and these new categories are labeled using expert knowledge. This approach avoids the problem of misjudging unknown traffic intrusions as known intrusions, significantly improving the sensitivity to the discovery and detection of new types of traffic intrusions.

[0035] In this embodiment of the invention, network traffic features extracted from network traffic data can be composed of a CNN network and a BiLSTM network. The CNN network is used to extract data features, and the BiLSTM network is used to extract data temporal features, so that the data features are more in line with the training rules of the reinforcement learning agent model.

[0036] In some alternative implementations, the step of training a semi-supervised classifier includes:

[0037] Step a1: Define the agent model of the semi-supervised classifier based on reinforcement learning techniques;

[0038] Step a2: Define the category space, which includes known categories and unknown categories;

[0039] Specifically, this invention creates a semi-supervised classifier based on reinforcement learning theory. The neural network model used in the semi-supervised classifier is called an agent model in reinforcement learning, which includes, but is not limited to, Q-learning and Actor-Critic network models. After creating the model architecture, a class space needs to be defined. The class space represents the classes that the agent model can output, including known and unknown classes. Known classes include various types of network intrusion, such as denial-of-service attacks, remote-to-local attacks, user-to-root attacks, and probing attacks. For example, the defined class space is A = {a1, ..., a...}. N+1}, where a1~a N Indicates a known category. N+1 Indicates an unknown category.

[0040] Step a3, define the reward function for the semi-supervised classifier. The reward function is:

[0041]

[0042] In the formula, r represents the reward value used to adjust the parameters of the agent model. c It is the reward score when the predicted category is classified into an unknown category, rtrue The reward score, r, is the score awarded when the predicted category is correctly classified into the known category. false It is the reward score when the predicted category is classified into a known category but is misclassified, a t This is the current prediction category, a N+1 Indicates an unknown category, a t ′ Labels representing known categories.

[0043] Specifically, this embodiment of the invention also needs to define a reward function. The function value of the reward function is used to adjust the parameters of the agent model during training. The reward function refers to the process in reinforcement learning where the agent model calculates a probability distribution matrix based on the user's input state (network traffic data). The probability distribution matrix includes the probability of classifying the network traffic data into each intrusion category. Then, based on the probability, an intrusion category for the current network traffic data is determined. After that, the intrusion category is substituted into the reward function to obtain the reward value. The higher the reward value, the more accurate the current output category is; the lower the reward value, the less accurate the current output category is. Thus, the agent model is guided to adjust its parameter size based on the corresponding reward value.

[0044] In an embodiment of the present invention, r true The reward score, r, is the score awarded when the predicted category is correctly classified into the known category. false It is the reward score when the predicted category is classified into a known category but is misclassified, usually r true For positive r false It is negative, and r c This is the reward score when a predicted category is classified into an unknown category. Because the unknown category has no data label, it is impossible to verify whether it is correct or incorrect. Therefore, this embodiment of the invention targets r. c The value is often assigned to 0, thus the agent model provided by this invention is a semi-supervised classifier. During training, when network traffic data is classified into an unknown class, the reward is 0, and the model parameters are not adjusted. If network traffic data of unknown categories is always classified into a known labeled class, and this is achieved through condition a... t ≠a N+1 ,a t ′ ≠a t If the system determines that the unknown category does not actually belong to the known category, the model parameters are updated according to the failure reward. As the number of failures increases, the probability of classifying the unknown category as a known category naturally decreases, thereby improving the identification probability of the unknown category and achieving the effect of classifying network traffic data into the unknown category.

[0045] Step a4: Obtain the training feature sample set;

[0046] Step a5: Input the training feature sample set into the agent model, and adjust the parameters of the agent model according to the predicted category and reward function to obtain the trained semi-supervised classifier.

[0047] Specifically, based on the training principles of steps a1 to a3 above, the acquired training feature sample set is input into the agent model, the reward function is input according to the predicted category of the agent model, and the parameters of the agent model are repeatedly adjusted according to the reward value of the reward function to obtain the trained semi-supervised classifier, which ultimately achieves the effect of distinguishing and detecting new network intrusions from known categories.

[0048] In some alternative implementations, step a4 above includes:

[0049] Step b1: Obtain the network traffic dataset;

[0050] Step b2: Hide the category labels of some data in the network traffic dataset;

[0051] Step b3: Use the processed network traffic dataset as the training feature sample set.

[0052] Specifically, this invention acquires two datasets, NSL_KDD and UNSW_NB15, as network traffic datasets for training. The NSL_KDD dataset contains 41 features and one label. Features include basic features, content features, and traffic-based features. The label identifies whether a connection is normal or an attack, with attack types categorized into four types: DoS (Denial of Service), R2L (Remote to Local), U2R (User to Root), and Probe. The UNSW_NB15 dataset contains 2,540,044 records, divided into normal and anomalous (attack) network activity. These records are collected by the IXIA traffic generator using three virtual servers: two servers are configured to distribute normal network traffic, and the third server is configured to generate anomalous network traffic. To achieve semi-supervised learning, this invention hides the category labels of some data in the dataset, treating them as data of unknown categories. For example, the R2L and U2R categories in NSL_KDD are set as hidden classes, and the Fuzzers, Analysis, worms, and Dos categories in UNSW_NB15 are set as hidden classes. Then, the processed network traffic dataset is used as the training feature sample set, which facilitates subsequent semi-supervised learning.

[0053] In some alternative implementations, step a5 above includes:

[0054] Step c1: Receive training feature samples from the training feature sample set sequentially according to the time steps.

[0055] Step c2: Place the training feature samples received at each time step into the temporary buffer area in sequence;

[0056] Step c3: Before each round of training, calculate the probability of each training sample being drawn from the temporary buffer using the following formula:

[0057]

[0058] Where P(i) represents the probability of the i-th training sample being selected, k is the number of training samples, and q i It is the priority of training sample i. N i t represents the number of times the i-th training sample has been extracted, α is a hyperparameter representing the priority importance weight, and t i This represents the total number of steps since sampling began in the temporary buffer.

[0059] Step c4: Randomly select a training sample from the temporary buffer based on the probability of each training sample being selected and input it into the agent model.

[0060] Specifically, such as Figure 2 and Figure 3 As shown, this invention also processes training samples based on an experience replay mechanism, initializing the maximum training rounds and the current training rounds; initializing the maximum training step size and the current training step size for each round; and constructing a temporary buffer as an experience replay module for the agent's trajectory. For example, the training samples that need to be placed in the temporary buffer are set as (s i ,a i ,r i ), where s i Let a represent the i-th training sample. i Let r represent the i-th intrusion category. i This represents the current reward value. After one round of training, the trajectory experience training samples are summarized in chronological order as {(s1,a1,r1),…,(s…}. A ,a T ,r T Each time experience training samples are retrieved from the temporary buffer to train the model, the sampling probability of the data in the experience replay cache module needs to be calculated based on familiarity. The purpose is to gradually decrease the probability of higher-priority experience samples being collected, increase the probability of uncollected samples, and thus ensure that lower-priority samples are also collected. The probability calculation formula is:

[0061]

[0062] The agent model is trained using sampled training data. If the training data are highly correlated, the direction and magnitude of updates may become unstable. This is because, in time series data, the correlation between data affects the variance of gradient estimation, thus impacting the stability of the learning process. By using a temporary buffer, samples are randomly drawn from it. The temporary buffer effectively disrupts the time series correlation of the samples, making the training data more independent and identically distributed. This disruption of sample correlation is similar to reducing autocorrelation between samples, thereby improving the stability of training.

[0063] Through the processing described in this embodiment of the invention, the model training process of the network intrusion detection method provided by this invention is as follows:

[0064] The current traffic features are input into a semi-supervised classifier, which outputs a probability distribution matrix. This matrix is ​​then used to determine the intrusion category. The determined category is compared with the actual label to determine if the classification is correct. A reward value is then calculated according to the reward function, and the classification module parameters are adjusted based on the reward value. This process is repeated. Specifically, training data is placed in a temporary buffer according to time series. Each time, a sample is randomly selected from the temporary buffer based on the calculated probability to participate in training, thereby enhancing the model's stability.

[0065] In some alternative implementations, step a1 above includes:

[0066] The agent model is defined using the Actor-Critic network model, which includes a training network and a target network. The training network is used to update the model parameters based on the output of the target network, and after a preset period, the updated model parameters are synchronized to the target network.

[0067] Specifically, the intelligent agent model in this embodiment of the invention builds a deep reinforcement learning Actor-Critic model, including a training network and a target network. The target network is set as a target Actor-Critic network and the training network is set as a training Actor-Critic network.

[0068] Furthermore, the model parameters for training the Actor network are: The parameters of the target Actor network are The parameters of the Critic network are trained as θ μ The target Critic network parameter is θ ′ μThe discount factor is γ, the network learning rate is α, and the target network update rate is τ. The Actor-Critic algorithm is a reinforcement learning method that combines policy gradient and temporal difference learning. It consists of two parts: an actor and a critic. The actor is responsible for generating actions and interacting with the environment, while the critic evaluates the value of the current policy and guides the actor's next action. The Actor-Critic algorithm is suitable for problems with continuous action spaces because it does not require discretization of the action space, making it more efficient in handling high-dimensional action spaces.

[0069] In this invention, the model parameters of the training network are updated in real time, while the parameters of the target network are directly copied from the training network's model parameters after a preset period. The update process of the training network's model parameters requires reference to the category output by the target network. Through this approach, the training network can more accurately learn and predict output results, and precisely adjust its model parameters. Simultaneously, the target network remains fixed for a period, avoiding the training network chasing a constantly changing target, which would lead to excessively frequent changes in output results and thus affect training effectiveness. By combining the training and target networks, the training stability of the agent model can be improved.

[0070] Specifically, the update formula for training the Actor network is:

[0071]

[0072] The update formula for training the Critic network is:

[0073]

[0074] Where, Q′(s t ,a t ) represents the output value of the target Actor network. To train the action policy output by the Actor network, y t =r t +γmax A Q′(s t ,a t ).

[0075] Furthermore, using the parameters of the currently trained network μ updates the target network parameters at an update rate τ. And μ′, the update formula is:

[0076]

[0077] μ′←τμ+(1-τ)μ′

[0078] If the current training round is greater than or equal to the maximum training round, then the network training ends.

[0079] In some alternative implementations, step S101 includes:

[0080] Step d1: Obtain raw network traffic data;

[0081] Step d2 involves cleaning the raw network traffic data to remove errors.

[0082] Step d3 involves converting the character data in the original network traffic data into numerical data, encoding and normalizing the discrete data to obtain preprocessed network traffic data.

[0083] Specifically, in this embodiment of the invention, after obtaining network traffic data, the traffic data is first cleaned and encoded to remove obvious erroneous data from the original network traffic data. Then, the character data in the original network traffic data is converted into numerical values, and the discrete data is encoded and normalized to obtain preprocessed network traffic data. Finally, the preprocessed network traffic data is used for model training, thereby indirectly improving the accuracy of model training by improving data quality.

[0084] This embodiment also provides a network intrusion detection device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0085] This embodiment provides a network intrusion detection device, such as... Figure 4 As shown, it includes:

[0086] The data feature extraction module 301 is used to acquire network traffic data and extract network traffic features from the network traffic data.

[0087] The classification module 302 is used to input network traffic features into a pre-trained semi-supervised classifier to identify the intrusion category of network traffic data through the semi-supervised classifier. The traffic categories include known categories and unknown categories.

[0088] Clustering module 303 is used to cluster known features of known categories and network traffic features together in the same feature space when the intrusion category is unknown, and obtain clustering results;

[0089] The new category labeling module 304 is used to label intrusion categories for data clusters that do not belong to known categories in the clustering results.

[0090] In some alternative implementations, the classification module 302 includes:

[0091] The model definition unit is used to define an agent model for a semi-supervised classifier based on reinforcement learning techniques.

[0092] A category definition unit is used to define a category space, which includes known categories and unknown categories.

[0093] The reward definition unit is used to define the reward function of the semi-supervised classifier. The reward function is as follows:

[0094]

[0095] In the formula, r represents the final reward score used to adjust the parameters of the agent model. c It is the reward score when the predicted category is classified into an unknown category, r true The reward score, r, is the score awarded when the predicted category is correctly classified into the known category. false It is the reward score when the predicted category is classified into a known category but is misclassified, a t This is the current prediction category, a N+1 Indicates an unknown category, a t ′ Labels indicating known categories;

[0096] The training sample acquisition unit is used to acquire the training feature sample set;

[0097] The training unit is used to input the training feature sample set into the agent model, and adjust the parameters of the agent model according to the predicted category and reward function to obtain the trained semi-supervised classifier.

[0098] In some alternative implementations, the training sample acquisition unit includes:

[0099] Data set acquisition unit, used to acquire network traffic datasets;

[0100] The hiding processing unit is used to hide the category labels of some data in the network traffic dataset;

[0101] The training sample set generation unit is used to take the processed network traffic dataset as the training feature sample set.

[0102] In some alternative implementations, the training unit includes:

[0103] The sample receiving unit receives training feature samples from the training feature sample set sequentially according to the time step.

[0104] The cache unit is used to sequentially place the training feature samples received at each time step into the temporary cache area;

[0105] The probability calculation unit is used to calculate the probability of each training sample being drawn from the temporary buffer before each round of training, according to the following formula:

[0106]

[0107] Where P(i) represents the probability of the i-th training sample being selected, k is the number of training samples, and q i It is the priority of training sample i. N i t represents the number of times the i-th training sample has been extracted, α is a hyperparameter representing the priority importance weight, and t i This represents the total number of steps since sampling began in the temporary buffer.

[0108] The extraction unit is used to randomly extract a training sample from the temporary buffer based on the probability of each training sample being extracted and input it into the agent model.

[0109] In some optional implementations, the agent model of the semi-supervised classifier is defined based on reinforcement learning techniques, including: defining the agent model using an Actor-Critic network model, which includes a training network and a target network. The training network is used to update the model parameters based on the output of the target network, and after a preset period, the updated model parameters are synchronized to the target network.

[0110] In some alternative embodiments, the apparatus further includes:

[0111] The data preprocessing unit is used to acquire raw network traffic data; clean the raw network traffic data of errors; convert the character data in the raw network traffic data into numerical data, and encode and normalize the discrete data therein to obtain preprocessed network traffic data.

[0112] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0113] The device in this embodiment is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0114] This invention also provides a computer device having the above-described features. Figure 4 The apparatus shown.

[0115] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 5 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 5 Take a processor 10 as an example.

[0116] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0117] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0118] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0119] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0120] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0121] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0122] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0123] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A network intrusion detection method, characterized in that, The method includes: Acquire network traffic data and extract network traffic characteristics from the network traffic data; The network traffic features are input into a pre-trained semi-supervised classifier to identify the intrusion category of the network traffic data, which includes known and unknown categories. When the intrusion category is the unknown category, the known features of the known category and the network traffic features are clustered together in the same feature space to obtain the clustering result; Intrusion category labeling is applied to data clusters in the clustering results that do not belong to known categories; The steps for training the semi-supervised classifier include: A smart agent model based on a semi-supervised classifier defined using reinforcement learning techniques; Define a category space, which includes the known categories and the unknown categories; Define the reward function for the semi-supervised classifier as follows: In the formula, r represents the reward value used to adjust the parameters of the agent model, r c The reward score, r, is the score awarded when the predicted category is classified into the unknown category. true The reward score, r, is the score awarded when the predicted category is correctly classified into the known category. false It is the reward score when the predicted category is classified into the known category but is misclassified, a t This is the current prediction category, a N+1 Indicates the unknown category, a t ′ Labels representing the known categories; Obtain the training feature sample set; The training feature sample set is input into the agent model, and the parameters of the agent model are adjusted according to the predicted category of the agent model and the reward function to obtain the trained semi-supervised classifier.

2. The method according to claim 1, characterized in that, The acquisition of the training feature sample set includes: Obtain network traffic dataset; The category labels of some data in the network traffic dataset are hidden. The processed network traffic dataset is used as the training feature sample set.

3. The method according to claim 2, characterized in that, The step of inputting the training feature sample set into the agent model includes: The training feature samples in the training feature sample set are received sequentially according to the time step; The training feature samples received at each time step are sequentially placed into a temporary buffer. Before each round of training, the probability of each training sample being drawn from the temporary buffer is calculated according to the following formula: Where P(i) represents the probability of the i-th training sample being selected, k is the number of training samples, and q i It is the priority of training sample i. N i t represents the number of times the i-th training sample has been extracted, α is a hyperparameter representing the priority importance weight, and t i This represents the total number of steps since sampling began in the temporary buffer. A training sample is randomly selected from the temporary buffer based on the probability of each training sample being selected and input into the agent model.

4. The method according to claim 1, characterized in that, The agent model based on reinforcement learning technology to define a semi-supervised classifier includes: The agent model is defined using an Actor-Critic network model, which includes a training network and a target network. The training network is used to update the model parameters based on the output of the target network, and after a preset period, the updated model parameters are synchronized to the target network.

5. The method according to claim 1, characterized in that, The acquisition of network traffic data includes: Obtain raw network traffic data; The original network traffic data is cleaned to remove errors. The character data in the original network traffic data is converted into numerical values, and the discrete data therein is encoded and normalized to obtain preprocessed network traffic data.

6. A network intrusion detection device, characterized in that, The device includes: The data feature extraction module is used to acquire network traffic data and extract the network traffic features of the network traffic data. A classification module is used to input the network traffic features into a pre-trained semi-supervised classifier to identify the intrusion category of the network traffic data through the semi-supervised classifier. The intrusion category includes known categories and unknown categories. The steps for training the semi-supervised classifier include: A smart agent model based on a semi-supervised classifier defined using reinforcement learning techniques; Define a category space, which includes the known categories and the unknown categories; Define the reward function for the semi-supervised classifier as follows: In the formula, r represents the reward value used to adjust the parameters of the agent model, r c The reward score, r, is the score awarded when the predicted category is classified into the unknown category. true The reward score, r, is the score awarded when the predicted category is correctly classified into the known category. false It is the reward score when the predicted category is classified into the known category but is misclassified, a t This is the current prediction category, a N+1 Indicates the unknown category, a t ′ Labels representing the known categories; Obtain the training feature sample set; The training feature sample set is input into the agent model, and the parameters of the agent model are adjusted according to the predicted category of the agent model and the reward function to obtain the trained semi-supervised classifier. The clustering module is used to cluster the known features of the known categories and the network traffic features together in the same feature space when the intrusion category is the unknown category, so as to obtain the clustering result; The new category labeling module is used to label intrusion categories for data clusters that do not belong to known categories in the clustering results.

7. A computer device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1 to 5.

9. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the method of any one of claims 1 to 5.

Images