Network traffic detection model training method and device, network traffic detection method and device, medium and product

Abstract

The invention discloses a training and detection method and device of a network traffic detection model, a medium and a product, and the method comprises the steps: obtaining a first sample data set which comprises a plurality of network traffic data with N network traffic characteristics and first sample data of a correspondingly labeled network traffic category label; a genetic algorithm is adopted to process the first sample data set, an optimal network traffic feature subset is obtained through calculation, and the subset comprises n key network traffic features; generating a second sample data set containing a plurality of pieces of second sample data; the second sample data are network flow data with the n key network flow characteristics and correspondingly marked network flow category labels; and training a machine learning model by adopting the second sample data set to obtain a network flow detection model. According to the invention, the optimal network traffic feature subset is obtained through the genetic algorithm and is used for training the network traffic detection model, so that the accuracy of identifying the abnormal network traffic is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a training and detection method, apparatus, medium and product for a network traffic detection model. Background Technology

[0002] With the rapid development of network technology, the internet has become an indispensable part of people's lives and work. At the same time, network security issues are becoming increasingly serious, with network attack methods constantly evolving, posing a huge challenge to network security. Anomaly detection of network traffic, as one of the important means of network security protection, can promptly detect abnormal behavior in the network, providing early warning and decision support for network security protection.

[0003] With the development of machine learning technology, machine learning-based methods have shown great potential in the detection of abnormal network traffic. Currently, the most commonly used methods for anomaly detection in network traffic are model-based methods, utilizing neural networks, support vector machines, and other techniques to extract feature data from network traffic to train detection models, thereby achieving the goal of identifying abnormal network traffic. However, network traffic features can number in the hundreds. Inputting too many features during model training leads to excessive computational load. Furthermore, selecting only a few key features for training has limitations, as the selection of network traffic features may not be optimal, potentially affecting the accuracy of model identification. Summary of the Invention

[0004] The purpose of this invention is to provide a training and detection method, device, medium, and product for a network traffic detection model. It obtains an optimal subset of network traffic features through a genetic algorithm to train the network traffic detection model, thereby effectively improving the accuracy of identifying abnormal network traffic.

[0005] To achieve the above objectives, embodiments of the present invention provide a training method for a network traffic detection model, comprising: Obtain a first sample dataset containing several first sample data; wherein, the first sample data is network traffic data with N network traffic features and corresponding labeled network traffic category labels, the network traffic categories are divided into normal categories and abnormal categories, the abnormal categories include several specific abnormal types; N>1; A genetic algorithm is used to process the first sample dataset to calculate the optimal subset of network traffic features; wherein the optimal subset of network traffic features includes n key network traffic features, n < N; Generate a second sample dataset containing several second sample data; wherein, the second sample data is network traffic data with the n key network traffic features and corresponding labeled network traffic category labels; The second sample dataset is used to train a preset machine learning model to obtain a trained network traffic detection model; wherein the input of the network traffic detection model is the network traffic data to be tested, and the output is the network traffic category.

[0006] As an improvement to the above scheme, the step of using a genetic algorithm to process the first sample dataset and calculate the optimal subset of network traffic features includes: Based on the first sample data, several chromosomes are generated using binary encoding as the initial population; wherein, in each chromosome, 1 indicates that the corresponding network traffic feature is selected, and 0 indicates that the corresponding network traffic feature is not selected, and the length of the chromosome is N; Genetic operations are performed on the current population in an iterative manner to generate a new generation of population; wherein the genetic operations include selection, crossover and mutation. When the preset iteration termination condition is met, the chromosome with the highest fitness value in the final population is obtained as the optimal chromosome. Based on the network traffic features selected by the optimal chromosome, an optimal subset of network traffic features is obtained.

[0007] As an improvement to the above scheme, the step of performing genetic operations on the current population iteratively to generate a new generation of population includes: Construct a machine learning model; wherein the input of the machine learning model is the network traffic data to be tested, and the output is the network traffic category; The machine learning model is trained using chromosomes in the current population as training data, and the fitness value of each chromosome is calculated based on the training results of the machine learning model. Based on the fitness value of the chromosome and a preset selection algorithm, the parent chromosome is selected; A preset crossover algorithm is used to exchange some gene segments in the parent chromosome to generate the offspring chromosome; Using a preset mutation algorithm, some gene loci in the offspring chromosome are flipped to obtain new offspring chromosomes; The chromosomes in the current population and the new offspring chromosomes are combined to form a new generation population.

[0008] As an improvement to the above scheme, the step of training the machine learning model using chromosomes in the current population as training data, and calculating the fitness value of each chromosome based on the training results of the machine learning model, includes: The chromosomes in the current population are divided into three datasets: dataset 1, dataset 2, and dataset 3. Using any two datasets as the training set and another dataset as the test set, three training combinations are obtained. The machine learning model is trained using the training set and test set in each training combination to obtain the average precision and average error rate of each chromosome after classification by the machine learning model; The fitness value of the chromosome is calculated based on the preset precision weight, the preset error rate correction weight, the average precision, and the average error rate.

[0009] As an improvement to the above scheme, the machine learning model is a multi-layer SVM classifier, and the number of layers in the multi-layer SVM classifier is equal to the number of specific anomaly types; The specific anomaly types include denial-of-service attacks, port scanning attacks, unauthorized remote access attacks, and other attacks.

[0010] This invention also provides a network traffic detection method, comprising: Obtain network traffic data to be tested; The network traffic data to be tested is input into a preset network traffic detection model for classification, and the network traffic category output by the network traffic model is obtained; wherein, the network traffic detection model is trained using the training method of the network traffic detection model described above.

[0011] This invention also provides a training apparatus for a network traffic detection model, comprising: The first dataset acquisition module is used to acquire a first sample dataset containing several first sample data; wherein, the first sample data is network traffic data with N network traffic features and corresponding labeled network traffic category labels, the network traffic categories are divided into normal categories and abnormal categories, the abnormal categories include several specific abnormal types; N>1; The optimal feature subset calculation module is used to process the first sample dataset using a genetic algorithm to calculate the optimal network traffic feature subset; wherein, the optimal network traffic feature subset includes n key network traffic features, n < N; The second dataset acquisition module is used to generate a second sample dataset containing several second sample data; wherein, the second sample data is network traffic data with the n key network traffic features and corresponding labeled network traffic category labels; The detection model training module is used to train a preset machine learning model using the second sample dataset to obtain a trained network traffic detection model; wherein, the input of the network traffic detection model is the network traffic data to be tested, and the output is the network traffic category.

[0012] This invention also provides a network traffic detection device, comprising: The network traffic acquisition module is used to acquire network traffic data under test. The network traffic detection module is used to input the network traffic data to be tested into a preset network traffic detection model for classification, and obtain the network traffic category output by the network traffic model; wherein, the network traffic detection model is trained using the training method of the network traffic detection model described above.

[0013] This invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a training method for a network traffic detection model as described in any of the above claims, or a network traffic detection method as described above.

[0014] This invention also provides a computer-readable storage medium, which includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute a training method for a network traffic detection model as described in any one of the above claims, or a network traffic detection method as described above.

[0015] This invention also provides a computer program product, which includes a computer program or computer instructions. When the computer program or computer instructions are executed by a processor, they implement a training method for a network traffic detection model as described above, or a network traffic detection method as described above.

[0016] Compared with existing technologies, the network traffic detection model training, detection method, device, medium, and product disclosed in this invention use a genetic algorithm to process network traffic data samples with a large number of network traffic features, iteratively selecting a smaller subset of optimal network traffic features with better classification performance for the construction and training of the network traffic detection model. This solves the problem that traditional feature selection methods are difficult to find globally optimal features, balances feature simplification with recognition rate requirements, and can eliminate redundant and irrelevant features, reducing the computational load and time cost of model training. It also avoids overfitting, allowing the model to maintain stable detection performance in the dynamic traffic environment of the live network, effectively improving the accuracy of the network traffic detection model in identifying abnormal network traffic. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a training method for a network traffic detection model provided in an embodiment of the present invention. Figure 2 This is a classification diagram of the SVM multilayer classifier in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the genetic algorithm in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the principle of crossover operation in the genetic algorithm in this embodiment of the invention; Figure 5 This is a schematic diagram illustrating the principle of the mutation operation in the genetic algorithm in this embodiment of the invention; Figure 6 This is a flowchart illustrating a network traffic detection method provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of a training device for a network traffic detection model provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of a network traffic detection device provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0020] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0021] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0022] See Figure 1 This is a flowchart illustrating a training method for a network traffic detection model provided in an embodiment of the present invention. The embodiment of the present invention provides a training method for a network traffic detection model, including steps S11 to S14: S11. Obtain a first sample dataset containing several first sample data; wherein, the first sample data is network traffic data with N network traffic features and corresponding labeled network traffic category labels, the network traffic categories are divided into normal categories and abnormal categories, the abnormal categories include several specific abnormal types; N>1; S12. The first sample dataset is processed using a genetic algorithm to calculate the optimal network traffic feature subset; wherein the optimal network traffic feature subset includes n key network traffic features, n < N. S13. Generate a second sample dataset containing several second sample data; wherein, the second sample data is network traffic data with the n key network traffic features and corresponding labeled network traffic category labels; S14. The second sample dataset is used to train the preset machine learning model to obtain the trained network traffic detection model; wherein, the input of the network traffic detection model is the network traffic data to be tested, and the output is the network traffic category.

[0023] This invention identifies abnormal network traffic data by training a network traffic detection model. During the training process, several network traffic data points are acquired as sample data, and their corresponding network traffic categories are labeled. A machine learning model learns the network traffic characteristics and labels of the sample data to acquire the ability to identify whether network traffic data is abnormal. These network traffic characteristics include, but are not limited to, protocol type, packet length, and connection duration.

[0024] Due to the large number of network traffic features, in order to reduce the computational resources required for model training while ensuring the accuracy of model training, this embodiment of the invention uses a genetic algorithm to obtain the optimal subset of network traffic features from the network traffic data. A genetic algorithm (GA) is a computational model that simulates the biological evolutionary process of natural selection and genetic mechanisms; it is a method for searching for optimal solutions by simulating natural evolution. Furthermore, based on n key network traffic features from the optimal subset of network traffic features, the network traffic detection model is trained, thereby using the fewest and most optimal network traffic features to obtain a network traffic detection model with a high recognition rate.

[0025] By employing the technical means of this invention, a genetic algorithm is used to process network traffic data samples with a large number of network traffic characteristics. The algorithm iteratively selects a smaller subset of optimal network traffic features with better classification performance for the construction and training of a network traffic detection model. This solves the problem that traditional feature selection methods struggle to find globally optimal features, balances feature simplification with recognition rate requirements, and eliminates redundant and irrelevant features, reducing the computational and time costs of model training. It also avoids overfitting, allowing the model to maintain stable detection performance in the dynamic traffic environment of the live network, effectively improving the accuracy of the network traffic detection model in identifying abnormal network traffic.

[0026] Specifically, in this embodiment of the invention, a first sample dataset is first obtained. The first sample dataset includes several first sample data points, which are network traffic data with N network traffic characteristics. The first sample data points are then labeled with corresponding network traffic categories. The network traffic categories are divided into normal categories and abnormal categories, and the abnormal categories include several specific abnormal types.

[0027] Preferably, the specific anomaly types include denial-of-service (DoS) attacks, port scanning attacks (Probe), unauthorized remote access attacks (U2L), and other attacks. Unauthorized remote access attacks (U2L) refer to attacks that attempt to log in and access a host that is not authorized by the system.

[0028] Optionally, the first sample dataset can be a publicly available dataset, such as UNSW-NB15 or NSL-KDD. These datasets are based on real network data and can be used to train models. Alternatively, the first sample dataset can also be generated from network traffic data collected from the current system network. No specific limitations are imposed here.

[0029] As an example, the UNSW-NB15 public dataset is selected. This dataset contains network traffic data of the normal category (Normal) and network traffic data of four abnormal categories: Probe, DoS, U2L, and Other, with a total of 25,400 samples. Each sample contains 49 network traffic features (such as protocol type, packet length, connection duration, etc.). The dataset is divided into a training set (17,780 samples) and a test set (7,620 samples) in a 7:3 ratio, and each sample is labeled with the corresponding category label.

[0030] After obtaining the first sample dataset, data preprocessing is performed on the sample data in the first sample dataset. This preprocessing includes numerical conversion and normalization. Since network traffic data contains various feature formats, including character and discrete types, non-numerical features cannot be used for training. Therefore, numerical processing of the network traffic data is necessary. For example, the traffic protocol feature `protocol_type` includes three types: `icmp`, `tcp`, and `udp`. One-hot encoding can be used to convert this type of non-numerical data into a numerical type, thus encoding it as 100, 010, 001. Similarly, other character-type features such as "service" and "flag" are encoded.

[0031] In addition, the label column is also a non-numeric type, and one-hot encoding is used to replace the five different categories of labels: Normal, Probe, Dos, R2L, and Other. The label data after one-hot encoding is shown in Table 1.

[0032] Table 1. One-hot encoding for label categories

[0033] Furthermore, based on practical findings, the classifier exhibits bias in classifying larger numerical features. Therefore, it is necessary to normalize all feature data. The standard deviation normalization method is used for normalization, and the formula for normalization is as follows:

[0034] in It is the j-th network traffic feature of the i-th network traffic data in the network sample. It is the calculated new network traffic characteristic value. It is the sample mean of the j-th network traffic feature. Let be the sample variance of the j-th network traffic feature. The above formula ensures that each feature follows a standard normal distribution with a mean of 0 and a variance of 1.

[0035] As a preferred embodiment, the machine learning model is a multi-layer SVM classifier.

[0036] It's important to note that Support Vector Machines (SVMs) are a type of generalized linear classifier that performs binary classification of data using supervised learning. As a binary classification algorithm, SVMs are widely used for anomaly traffic detection. Specifically, the method involves training a classification model using features extracted from training data as input. This trained model is then used to detect abnormal network traffic. Features from current network traffic are input into the trained model, and the output is used to determine whether abnormal traffic exists.

[0037] SVM is a binary classification algorithm. To identify multi-class abnormal traffic, this embodiment of the invention constructs a multi-layer SVM classifier based on the SVM algorithm. The number of layers in the multi-layer SVM classifier is equal to the number of specific anomaly types. The first layer of the multi-layer SVM classifier is used to classify normal and abnormal categories, and the k-th layer is used to classify the (k-1)-th specific anomaly type and other anomaly categories besides the first k-1 specific anomaly types, where k > 1.

[0038] Taking network traffic categories including the Normal category and four specific anomaly types—Probe, DoS, U2L, and Other—as an example, see [link to relevant documentation]. Figure 2 This is a classification diagram of the SVM multi-layer classifier in this embodiment of the invention. The first layer distinguishes between two traffic categories: normal and abnormal. The second layer distinguishes between Probe attack type and other non-Probe abnormal types. The third layer distinguishes between DoS attack type and other non-DoS and non-Probe abnormal types. The fourth layer distinguishes between U2L attack type and other non-DoS, non-Probe, and non-U2L attack abnormal types, denoted as Other.

[0039] By employing the technical means of this invention, sample datasets are obtained through publicly available datasets or by collecting data from the current system network. Preprocessing the sample datasets, such as numericalization and normalization, is beneficial for the learning of machine learning models and improves the training accuracy of network traffic detection models.

[0040] As a preferred embodiment, the training process of the network traffic detection model is further optimized and explained in this invention. It should be noted that the genetic algorithm is a search algorithm that simulates the principles of natural selection and genetics, used to solve optimization problems; see [link to relevant documentation]. Figure 3 This is a flowchart illustrating the genetic algorithm in an embodiment of the present invention. Typically, the steps of a genetic algorithm include: Step 1: Initialize the population: Randomly generate a set of candidate solutions as the initial population. Each candidate solution is called a chromosome, which encodes a potential solution to the problem.

[0041] Step 2: Calculate the fitness function: The fitness function is used to evaluate the performance of each chromosome, that is, its ability to solve problems. The higher the fitness, the greater the chance that the chromosome will be selected for reproduction.

[0042] Step 3, Selection: Individuals are selected from the current population for reproduction based on chromosome fitness. Common selection methods include roulette wheel selection, tournament selection, and elite selection (selection operators).

[0043] Step 4, Crossover: The selected individuals produce new offspring through a crossover operation. Crossover is a step in genetic algorithms that simulates the biological reproduction process, generating new combinations by exchanging partial genes from two chromosomes (crossover operator). The crossover operator is used to generate new individuals from the parents, inspired by the idea that offspring inherit the best traits from their parents.

[0044] Step 5, Mutation: This involves making small, random changes to the chromosomes to increase the genetic diversity of the population and prevent the algorithm from prematurely converging to a local optimum. Mutation operations typically involve randomly altering one or more genes in the chromosome (mutation operators).

[0045] Step 6: Formation of the new generation population: Offspring generated through selection, crossover, and mutation form a new generation population with some individuals from the current population. The formation of the new generation population can occur in various ways, such as complete replacement, partial replacement, or fitness-based mixing.

[0046] Step 7: Evaluate the new generation population: Use the fitness function to evaluate the performance of each individual in the new generation population.

[0047] Step 8: Termination Condition Check: Check if the termination condition is met, such as reaching the maximum number of iterations, the fitness no longer significantly improving, or finding a satisfactory solution. If the termination condition is met, the algorithm ends; otherwise, return to step 3 to continue iterating.

[0048] Step 9: Output the optimal solution: Select the individual with the highest fitness from the final population as the optimal solution to the problem.

[0049] Based on this, step S12 above, namely, processing the first sample dataset using a genetic algorithm to calculate the optimal network traffic feature subset, includes steps S121 to S124: S121. Based on the first sample data, several chromosomes are generated using binary encoding as the initial population; wherein, in the gene position of each chromosome, 1 indicates that the corresponding network traffic feature is selected, and 0 indicates that the corresponding network traffic feature is not selected, and the length of the chromosome is N. S122. Genetic operations are performed on the current population in an iterative manner to generate a new generation of population; wherein, the genetic operations include selection, crossover and mutation operations; S123. When the preset iteration termination condition is met, the chromosome with the highest fitness value in the final population is obtained as the optimal chromosome. S124. Based on the network traffic features selected by the optimal chromosome, obtain the optimal network traffic feature subset.

[0050] Preferably, step S122 above, that is, the iterative genetic operation on the current population to generate a new generation of population, includes steps S1221 to S1226: S1221. Construct a machine learning model; wherein the input of the machine learning model is the network traffic data to be tested, and the output is the network traffic category; S1222. Using chromosomes in the current population as training data, train the machine learning model, and calculate the fitness value of each chromosome based on the training results of the machine learning model. S1223. Select the parent chromosome based on the fitness value of the chromosome and the preset selection algorithm; S1224. Using a preset crossover algorithm, some gene segments in the parent chromosome are exchanged to generate offspring chromosomes; S1225. Using a preset mutation algorithm, some gene loci in the offspring chromosome are flipped to obtain a new offspring chromosome; S1226. Combine the chromosomes in the current population with the new offspring chromosomes to form a new generation population.

[0051] Preferably, the preset iteration termination condition is: the current iteration count reaches a preset threshold, or the ratio of the fitness value of the previous generation to the fitness value of the current generation is greater than a preset fitness threshold; wherein, the fitness value of the previous generation refers to the maximum fitness value of all chromosomes in the population in the previous iteration, and the fitness value of the current generation refers to the maximum fitness value of all chromosomes in the population in the current iteration.

[0052] In this embodiment of the invention, binary encoding is used during chromosome encoding, where 1 indicates that the feature value is selected and 0 indicates that the feature value is not selected. For example, the feature chromosome is: 100100110110...01. The chromosome length is consistent with the number of feature values ​​collected in the data.

[0053] Crossover operator: The crossover operator is used to generate new individuals from the parent generation. Crossover combines the codes of two parents to complete the crossover operation. The purpose of crossover is to combine the genes of superior parents to obtain superior offspring, such as... Figure 4As shown, Figure 4 This is a schematic diagram illustrating the principle of crossover operation in the genetic algorithm in this embodiment of the invention.

[0054] The crossover probability is usually a fixed value, set to 0.8. When the average fitness of the population is close to the optimal solution, the crossover probability should be reduced. Therefore, the crossover probability is optimized to an adaptive value. The optimized adaptive crossover probability is:

[0055] in, It is the crossover probability of the current generation. This is the crossover probability from the previous iteration, with an initial value of 0.8. It is the minimum fitness value up to the current generation. It is the maximum fitness value up to the current generation.

[0056] Mutation operators: Mutation operators are also used to simulate mutation phenomena in biology. Mutations typically produce new individuals that differ from their parents, such as... Figure 5 As shown, Figure 5 This is a schematic diagram illustrating the principle of the mutation operation in the genetic algorithm in this embodiment of the invention.

[0057] The mutation operator is generally a fixed value; in this embodiment, it is set to 0.2. Similar to the crossover probability, as the number of generations increases, a higher mutation probability generates new individuals, affecting the algorithm's convergence. Therefore, this embodiment also optimizes the mutation probability; the optimized mutation probability is:

[0058] in, It is the mutation probability of the current generation. It is the mutation probability of the previous iteration, with an initial value of 0.2.

[0059] In a preferred embodiment, the fitness function of chromosomes is improved. Step S1222, namely, training the machine learning model using chromosomes in the current population as training data, and calculating the fitness value of each chromosome based on the training results of the machine learning model, includes: The chromosomes in the current population are divided into three datasets: dataset 1, dataset 2, and dataset 3. Using any two datasets as the training set and another dataset as the test set, three training combinations are obtained. The machine learning model is trained using the training set and test set in each training combination to obtain the average precision and average error rate of each chromosome after classification by the machine learning model; The fitness value of the chromosome is calculated based on the preset precision weight, the preset error rate correction weight, the average precision, and the average error rate.

[0060] Preferably, the machine learning model is a multi-layer SVM classifier. The step of training the machine learning model using the training and test sets in each training combination to obtain the average precision and average error rate of each chromosome after classification by the machine learning model specifically involves: Each layer of the multi-layer SVM classifier is trained using the training and test sets in each training combination to obtain the average accuracy of each chromosome after classification by each layer of the multi-layer SVM classifier. and average error rate ; i Indicates the layer number of the classifier; The step of calculating the fitness value of the chromosome based on a preset precision weight, a preset error rate correction weight, the average precision, and the average error rate is as follows: The average accuracy of the chromosome after classification in each layer of a multi-level SVM classifier. Sum the results to obtain the final accuracy. The average error rate of the chromosome after classification in each layer of a multi-level SVM classifier. Sum the results to obtain the final error rate; The final precision and the final error rate are weighted and summed according to the preset precision weight and the preset error rate correction weight to obtain the fitness value of the chromosome.

[0061] In this embodiment of the invention, the fitness function is used to evaluate the survival quality of an individual. This embodiment combines a multi-layer SVM classifier and proposes the following fitness function formula:

[0062] in, As a preset accuracy weight, Adjust the weights according to the preset error rate. Let be the precision of the i-th layer SVM classifier, i.e., the classification prediction accuracy. Let m be the error rate of the i-th layer SVM classifier, i.e., the classification prediction error rate. m is the number of SVM classifier layers. Taking the network traffic categories including the normal category and the four specific anomaly types Probe, DoS, U2L, and Other as an example, m=4.

[0063] Precision refers to the proportion of correctly classified positive classes out of all positive classes. A value closer to 1 indicates a better classifier.

[0064] Error rate refers to the proportion of samples that the classifier mispredicts out of all samples.

[0065] Optionally, Take 60% / number of SVM classifier layers. The value is 40% per SVM classifier layer. The confusion matrix is ​​shown in Table 2.

[0066] Table 2. Binary Confusion Matrix

[0067] This invention employs 3-fold cross-validation, dividing the dataset into three equal parts (a, b, and c), as shown in Table 3. Two parts are selected as the training set each time, and the remaining part is used as the test set. Within the training set, for any training set a, b, or c, the data is partitioned and labeled. Taking network traffic categories including a normal category and four specific anomaly types (Probe, DoS, U2L, and Other) as an example, the first training set uses all data, labeled Normal and Non-Normal; the second training set uses the training set excluding Normal, labeled Probe and Non-Probe; the third training set uses the training set excluding Normal and Probe, labeled DoS and Non-DoS; and the fourth training set removes Normal, Probe, and DoS, labeled U2L and Other.

[0068] Three validation results were obtained through 3-fold cross-validation. The precision and error rate of each combination were calculated. The fitness value of each evolution was calculated according to the chromosome fitness function formula. The average fitness value of the three combinations was taken as the final fitness value of this evolution.

[0069] Table 3 Dataset Partitioning

[0070] This invention employs a genetic algorithm that optimizes the cross-compilation operator to iteratively evolve the optimal feature subset, thereby obtaining a network traffic detection model using a multi-layer SVM classifier. The specific steps are as follows: Initialize the population to generate chromosome data; Decoding the chromosome yields training data based on it (this training data is a feature subset of the original training data). Data and labels are then partitioned according to the SVM model. The first SVM layer trains Normal and non-Normal attack types; the second SVM layer trains Probe and other non-Probe types; the third layer trains DoS and other non-DoS types; and the fourth layer trains U2L and Other attack type traffic data. The classifier is evaluated using 3-fold cross-validation on the test set, and the fitness value is calculated using the fitness function formula. To determine whether the evolution has terminated, the termination condition is that the number of iterations is greater than 100, or the ratio of the fitness value of the previous generation to the fitness value of the current generation is greater than 99.9%. Otherwise, continue to the next step. Selecting, crossing over, and mutating the parents yields the next generation population. Step 2 continues with decoding the chromosome to obtain training data based on that chromosome. The algorithm terminates to obtain the optimal chromosome, and decoding yields the optimal subset of network traffic features.

[0071] The training process of the network traffic detection model in this embodiment of the invention will be explained and illustrated with specific implementation methods.

[0072] Initialize the population: Binary encoding is used, with a chromosome length of 49 (consistent with the number of features). 1 indicates that the corresponding feature is selected, and 0 indicates that it is not selected. 50 chromosomes are randomly generated as the initial population. For example, a chromosome may be 100110...01 (49 bits in total, where bits 1, 4, 5...49 are 1, corresponding to the selection of 12 features).

[0073] Training a multi-layer SVM classifier (based on a single chromosome feature subset): Decode the current chromosome to obtain training data for 12 features, and train using a four-layer SVM structure: First layer SVM: Input the full training data, and divide the labels into "NORMAL (10000)" and "non-NORMAL (0xxxx)". Train a binary classification model to distinguish between normal and abnormal traffic.

[0074] The second layer of SVM is input with "non-NORMAL" training data, and the labels are divided into "Probe (01000)" and "non-Probe (0x0xx)". The training distinguishes between Probe attacks and other anomalies.

[0075] The third layer of SVM: Input training data that is "non-NORMAL, non-Probe", and the labels are divided into "DoS (00100)" and "non-DoS (0xx0x)". The training distinguishes between DoS attacks and residual anomalies.

[0076] Fourth layer SVM: Input training data that is “non-NORMAL, non-Probe, non-DoS”, and label it as “U2L (00010)” and “Other (00001)”. The training distinguishes between U2L and Other attacks.

[0077] 3-fold cross-validation is used to calculate fitness values: The training set was divided into three groups, a, b, and c, according to Table 3 (approximately 5927 records in each group), and the results were validated using the three combinations.

[0078] Combination 1: Training set a+b, test set c, calculate the precision and error of the four-layer SVM. Combination 2: Training set a+c, test set b, repeat the above metrics; Combination 3: Training set b+c, test set a, repeat the above metrics.

[0079] The fitness value is calculated using the fitness function formula, where =60% / 4=15%, =40% / 4=10% (SVM layers are 4). Assuming a four-layer SVM model corresponding to a certain chromosome, in a 3-fold validation, the average Pre1=92%, Pre2=88%, Pre3=90%, Pre4=85%, and the average Error1=3%, Error2=5%, Error3=4%, Error4=6%, then: Fscore=15%×(92%+88%+90%+85%)+10%×(1-3%+1-5%+1-4%+1-6%)=15%×355%+10%×382%=53.25%+38.2%=91.45%.

[0080] Iterative optimization using genetic algorithms: (1) Selection operation: The tournament selection method is adopted. Five chromosomes are randomly selected from the current population, and the two with the highest fitness are selected as the parents. This is repeated 25 times to obtain 50 parent individuals.

[0081] (2) Crossover operation: Initial crossover probability =0.8, and adaptively adjust according to the formula. Assume that in the current iteration, the minimum fitness of the population is... =82%, maximum fitness =91.45%, then the current crossover probability is... =(0.8+0.8×(82% / 91.45%)) / 2≈0.76. For the selected parent chromosome, a crossover point is randomly chosen (e.g., the 20th chromosome), and the gene segments after the crossover point are exchanged, for example: Parent generation 1: 100110...01 (first 20 bits: 100110...10, last 29 bits: 01...01); Parent generation 2: 011001...10 (first 20 bits: 011001...01, last 29 bits: 10...10); Child generation 1: 100110...10 (the first 20 inherit from parent generation 1, and the last 29 inherit from parent generation 2); Child generation 2: 011001...01 (the first 20 characters inherit from parent generation 2, and the last 29 characters inherit from parent generation 1); (3) Mutation operation: Initial mutation probability =0.2, adaptively adjusted according to the formula, in the current iteration ≈0.18. For the offspring chromosomes after crossing over, randomly select 1-2 gene positions to flip (flip 0 to 1 or 1 to 0), for example, flip 0 to 1 in the 15th position of offspring 1 to obtain a new chromosome.

[0082] (4) Termination condition judgment: If the number of iterations has not reached 100, and the fitness value of the previous generation / the fitness value of the current generation is ≤99.9%, then the previous genetic operation steps are repeated with the new generation population; if the fitness value of a certain chromosome reaches 95.2% when the iteration reaches 80, and the fitness value of the previous generation is 95.1% / the fitness value of the current generation is 95.2%≈99.89% (≤99.9%), the termination condition is met.

[0083] Output the optimal feature subset and the final model: The chromosome with the highest fitness at the end of decoding is obtained as the optimal feature subset. Based on this feature subset, the four-layer SVM model is retrained, which is the final network abnormal traffic classification model.

[0084] This invention employs the technical means of an embodiment of the present invention. Based on the optimized crossover algorithm and mutation operator of a genetic algorithm, it proposes a training method for a network traffic detection model using a multi-layer SVM classifier, incorporating the fitness function of the multi-layer SVM classifier. The optimal feature subset is obtained through iterative evolution using a genetic algorithm and used to train the network traffic detection model. This addresses the problem of traditional feature selection methods struggling to find globally optimal features, balancing feature simplification with recognition rate requirements. It also eliminates redundant and irrelevant features, reducing the computational and time costs of model training, while avoiding overfitting and ensuring stable detection performance in dynamic network traffic environments. By adaptively adjusting the crossover and mutation probabilities, premature or slow convergence of the algorithm is avoided. Combined with the performance feedback of the multi-layer SVM, the optimal solution is quickly located, reducing the number of generations.

[0085] Furthermore, this embodiment of the invention employs a multi-layer SVM classifier as the network traffic detection model, overcoming the limitations of binary SVM classification and efficiently achieving accurate identification of multiple types of abnormal traffic. It solves the problems of complex construction and low accuracy of traditional multi-layer SVM models. Each layer of SVM focuses on only two types of targets, reducing the classification complexity of a single model and minimizing category interference. Combined with feature subsets optimized by genetic algorithms, it further improves the accuracy of each layer's classification, reduces the error rate, and effectively enhances the recognition performance and accuracy of the network traffic detection model.

[0086] See Figure 6 This is a flowchart illustrating a network traffic detection method provided in an embodiment of the present invention. The embodiment of the present invention also provides a network traffic detection method, including steps S21 to S22: S21. Obtain network traffic data to be tested; S22. Input the network traffic data to be tested into a preset network traffic detection model for classification to obtain the network traffic category output by the network traffic model; wherein, the network traffic detection model is trained using the training method of the network traffic detection model as described in any of the above embodiments.

[0087] It should be noted that the training method of the network traffic detection model provided in the embodiments of the present invention corresponds one-to-one with all the process steps of the training method of the network traffic detection model in the above embodiments. The working principle and beneficial effects of the two are the same, so they will not be described again.

[0088] As an example, 1,000 data points of live network traffic from a cloud phone are collected as network traffic data to be tested. After data preprocessing, the data is input into the final network traffic detection model.

[0089] Taking the network traffic detection model as an SMV multilayer classifier, with network traffic categories including normal and four specific abnormal types (Probe, DoS, U2L, and Other) as an example, after the first layer SVM judgment, 850 traffic items are classified as normal and 150 as abnormal. Of the 150 abnormal traffic items, 40 are identified as Probe attacks after the second layer SVM judgment. Of the remaining 110 traffic items, 55 are identified as DoS attacks after the third layer SVM judgment. Finally, of the last 55 traffic items, 20 are identified as U2L attacks and 35 as Other attacks after the fourth layer SVM judgment. The classification results are output for network maintainers to process.

[0090] By employing the technical means of this invention, a multi-layer SVM classifier network traffic detection model trained with the optimal feature subset selected by a genetic algorithm is used to classify and detect the network traffic data to be tested. This effectively improves the accuracy of identifying the network traffic category of the network traffic data to be tested, and can provide network maintainers with accurate and effective data information, thereby improving network security.

[0091] See Figure 7 This is a schematic diagram of the structure of a training device for a network traffic detection model provided in an embodiment of the present invention. The embodiment of the present invention provides a training device 10 for a network traffic detection model, comprising: The first dataset acquisition module 11 is used to acquire a first sample dataset containing a number of first sample data; wherein, the first sample data is network traffic data with N network traffic features and corresponding labeled network traffic category labels, the network traffic categories are divided into normal categories and abnormal categories, the abnormal categories include a number of specific abnormal types; N>1; The optimal feature subset calculation module 12 is used to process the first sample dataset using a genetic algorithm to calculate the optimal network traffic feature subset; wherein, the optimal network traffic feature subset includes n key network traffic features, n < N. The second dataset acquisition module 13 is used to generate a second sample dataset containing several second sample data; wherein, the second sample data is network traffic data with the n key network traffic features and corresponding labeled network traffic category labels; The detection model training module 14 is used to train a preset machine learning model using the second sample dataset to obtain a trained network traffic detection model; wherein, the input of the network traffic detection model is the network traffic data to be tested, and the output is the network traffic category.

[0092] It should be noted that the training device for a network traffic detection model provided in this embodiment of the invention is used to execute all the process steps of the training method for a network traffic detection model in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0093] See Figure 8 This is a schematic diagram of the structure of a network traffic detection device provided in an embodiment of the present invention. The embodiment of the present invention provides a network traffic detection device 20, comprising: The network traffic acquisition module 21 is used to acquire network traffic data under test. The network traffic detection module 22 is used to input the network traffic data to be tested into a preset network traffic detection model for classification, and obtain the network traffic category output by the network traffic model; wherein, the network traffic detection model is trained using the training method of the network traffic detection model as described in any of the above embodiments.

[0094] It should be noted that the network traffic detection device provided in this embodiment of the invention is used to execute all the process steps of the network traffic detection method in the above embodiment. The working principle and beneficial effects of the two are one-to-one, so they will not be described again.

[0095] This invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a training method for a network traffic detection model as described in any of the above embodiments, or implements a network traffic detection method as described in any of the above embodiments.

[0096] This invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform a training method for a network traffic detection model as described in any of the above embodiments, or a network traffic detection method as described in any of the above embodiments.

[0097] This invention also provides a computer program product, which includes a computer program or computer instructions. When the computer program or computer instructions are executed by a processor, they implement the training method for the network traffic detection model as described in any of the above embodiments, or the network traffic detection method as described in any of the above embodiments.

[0098] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0099] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A training method for a network traffic detection model, characterized in that, include: Obtain a first sample dataset containing several first sample data; wherein, the first sample data is network traffic data with N network traffic features and corresponding labeled network traffic category labels, the network traffic categories are divided into normal categories and abnormal categories, the abnormal categories include several specific abnormal types; N>1; A genetic algorithm is used to process the first sample dataset to calculate the optimal subset of network traffic features; wherein the optimal subset of network traffic features includes n key network traffic features, n < N; Generate a second sample dataset containing several second sample data; wherein, the second sample data is network traffic data with the n key network traffic features and corresponding labeled network traffic category labels; The second sample dataset is used to train a preset machine learning model to obtain a trained network traffic detection model; wherein the input of the network traffic detection model is the network traffic data to be tested, and the output is the network traffic category.

2. The training method for the network traffic detection model as described in claim 1, characterized in that, The step of processing the first sample dataset using a genetic algorithm to calculate the optimal subset of network traffic features includes: Based on the first sample data, several chromosomes are generated using binary encoding as the initial population; wherein, in each chromosome, 1 indicates that the corresponding network traffic feature is selected, and 0 indicates that the corresponding network traffic feature is not selected, and the length of the chromosome is N; Genetic operations are performed on the current population in an iterative manner to generate a new generation of population; wherein the genetic operations include selection, crossover and mutation. When the preset iteration termination condition is met, the chromosome with the highest fitness value in the final population is obtained as the optimal chromosome. Based on the network traffic features selected by the optimal chromosome, an optimal subset of network traffic features is obtained.

3. The training method for the network traffic detection model as described in claim 2, characterized in that, The method of performing genetic operations on the current population in an iterative manner to generate a new generation of population includes: Construct a machine learning model; wherein the input of the machine learning model is the network traffic data to be tested, and the output is the network traffic category; The machine learning model is trained using chromosomes in the current population as training data, and the fitness value of each chromosome is calculated based on the training results of the machine learning model. Based on the fitness value of the chromosome and a preset selection algorithm, the parent chromosome is selected; A preset crossover algorithm is used to exchange some gene segments in the parent chromosome to generate the offspring chromosome; Using a preset mutation algorithm, some gene loci in the offspring chromosome are flipped to obtain new offspring chromosomes; The chromosomes in the current population and the new offspring chromosomes are combined to form a new generation population.

4. The training method for the network traffic detection model as described in claim 3, characterized in that, The step of training the machine learning model using chromosomes from the current population as training data, and calculating the fitness value of each chromosome based on the training results of the machine learning model, includes: The chromosomes in the current population are divided into three datasets: dataset 1, dataset 2, and dataset 3. Using any two datasets as the training set and another dataset as the test set, three training combinations are obtained. The machine learning model is trained using the training set and test set in each training combination to obtain the average precision and average error rate of each chromosome after classification by the machine learning model; The fitness value of the chromosome is calculated based on the preset precision weight, the preset error rate correction weight, the average precision, and the average error rate.

5. The training method for the network traffic detection model as described in claim 1 or 4, characterized in that, The machine learning model is a multi-layer SVM classifier, and the number of layers in the multi-layer SVM classifier is equal to the number of specific anomaly types. The specific anomaly types include denial-of-service attacks, port scanning attacks, unauthorized remote access attacks, and other attacks.

6. A method for detecting network traffic, characterized in that, include: Obtain network traffic data to be tested; The network traffic data to be tested is input into a preset network traffic detection model for classification to obtain the network traffic category output by the network traffic model; wherein, the network traffic detection model is trained using the training method of the network traffic detection model as described in any one of claims 1 to 5.

7. A training device for a network traffic detection model, characterized in that, include: The first dataset acquisition module is used to acquire a first sample dataset containing several first sample data; wherein, the first sample data is network traffic data with N network traffic features and corresponding labeled network traffic category labels, the network traffic categories are divided into normal categories and abnormal categories, the abnormal categories include several specific abnormal types; N>1; The optimal feature subset calculation module is used to process the first sample dataset using a genetic algorithm to calculate the optimal network traffic feature subset; wherein, the optimal network traffic feature subset includes n key network traffic features, n < N; The second dataset acquisition module is used to generate a second sample dataset containing several second sample data; wherein, the second sample data is network traffic data with the n key network traffic features and corresponding labeled network traffic category labels; The detection model training module is used to train a preset machine learning model using the second sample dataset to obtain a trained network traffic detection model; wherein, the input of the network traffic detection model is the network traffic data to be tested, and the output is the network traffic category.

8. A network traffic detection device, characterized in that, include: The network traffic acquisition module is used to acquire network traffic data under test. The network traffic detection module is used to input the network traffic data to be tested into a preset network traffic detection model for classification, and obtain the network traffic category output by the network traffic model; wherein, the network traffic detection model is trained using the training method of the network traffic detection model as described in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform a training method for a network traffic detection model as described in any one of claims 1 to 5, or a network traffic detection method as described in claim 6.

10. A computer program product, characterized in that, The computer program product includes a computer program or computer instructions, which, when executed by a processor, implement the training method for the network traffic detection model as described in any one of claims 1 to 5, or the network traffic detection method as described in claim 6.

Images