A network intrusion detection method, system and electronic device

Abstract

The application discloses a network intrusion detection method and system and electronic equipment, and relates to the technical field of network intrusion detection. The network intrusion detection method provided by the application calculates the relative distance of abnormal data in the data set after spatial modeling after obtaining the data set, calculates the cleaning radius based on the relative distance between the abnormal data, and screens the normal data around the abnormal data according to the cleaning radius, so that the abnormal data has higher attention, and the measurement accuracy of the isolated core can be improved. Moreover, the isolated core can accurately obtain the network intrusion detection result based on the network traffic data to be detected.

Description

Technical Field

[0001] This invention relates to the field of network intrusion detection technology, and in particular to a network intrusion detection method, system and electronic device. Background Technology

[0002] The emergence of massive databases, such as commercial data warehouses and automated data collection methods, along with rapid access to vast amounts of data, has spurred the rapid development of data mining technology. Due to the abundance of data available for widespread use, this data can be transformed into useful information and knowledge. Data mining first appeared at the 11th International Conference on Artificial Intelligence in 1989. The concepts behind data mining originate from machine learning, database systems, pattern recognition, and statistics. Machine learning, with its excellent accuracy and time performance, is widely used, particularly in data similarity measurement. Currently, data mining is widely applied in various fields, such as finance, healthcare, telecommunications, retail, and e-commerce. In the telecommunications sector, network intrusion detection refers to the use of computer network technology to monitor networks in real time, promptly detecting network security vulnerabilities and abnormal behaviors to ensure network security and integrity.

[0003] Network intrusion detection is a crucial part of cybersecurity and a hot research topic. Traditional rule-based and signature-based intrusion detection techniques are no longer sufficient for practical needs. In reality, the data to be processed has diverse forms and characteristics, and measuring data similarity is a vital aspect of network intrusion detection. For these different data types, it's impossible to find a universally applicable similarity measurement algorithm; in fact, each type of data has its own corresponding similarity measurement standard. Similarity measurement is mainly based on four data attributes: nominal attributes, binary attributes, ordinal attributes, and numerical attributes. The main measurement methods used are: nominal attribute similarity measurement, binary attribute similarity measurement, numerical attribute similarity measurement, ordinal attribute similarity measurement, and mixed-type attribute similarity measurement. Selecting the appropriate measurement method based on different data attributes is a crucial step in data similarity measurement. In addition, there are methods that measure similarity based on the spatial distribution of data features, such as the isolation kernel function proposed by Dr. Zhu Ye. The isolation kernel function is a similarity measurement method that adapts to the local density distribution of data. It can amplify or shrink the similarity between data points in different regions. That is, the similarity between two data points placed in a high-density area will decrease, and vice versa. In other words, it increases the similarity between objects in low-density areas and decreases the similarity between objects in high-density areas.

[0004] When using isolated kernel functions for network intrusion detection, similarity measurement is a crucial step. Isolated kernel functions utilize the kernel concept to measure similarity, mapping samples to a high-dimensional space for processing, resulting in more accurate and robust detection. Specifically, in network intrusion detection using isolated kernel functions, network traffic data can be represented as a feature vector. Then, a kernel function is used to map this feature vector to a high-dimensional space, yielding a more accurate similarity measure. Subsequently, the isolated kernel function is used to calculate the outlier degree of the samples, thereby identifying network intrusion behavior. This method not only detects intrusion behavior but also helps analyze the characteristics of network traffic data, providing more reference information for network security defense.

[0005] In summary, the similarity measurement capability of isolated kernel functions is one of the important technical foundations for the application of this algorithm in the field of network intrusion detection. By using isolated kernel functions to map data into a high-dimensional space, the inherent structure of the data can be better described, thereby improving the accuracy and robustness of network intrusion detection. Experimental comparisons show that isolated kernel functions have a significant advantage in handling problems with obvious local density distributions in the data. However, in some data similarity measurement problems, due to the excessively high overlap density of positive and negative samples in the dataset and the uneven partitioning area between outlier and normal data points, the training effect of isolated kernel functions is not ideal, which leads to certain errors in the detection results. Summary of the Invention

[0006] To address the aforementioned problems in the existing technology, this invention provides a network intrusion detection method, system, and electronic device.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] A network intrusion detection method, comprising:

[0009] Obtain the dataset and perform feature space modeling on the dataset;

[0010] Calculate the relative distance between outliers in the spatially modeled dataset;

[0011] Calculate the cleanup radius based on the relative distance between abnormal data;

[0012] The normal data surrounding the abnormal data is filtered based on the cleanup radius; the normal data surrounding the abnormal data refers to the normal data within a defined range centered on the abnormal data.

[0013] Obtain network traffic data to be detected;

[0014] The network traffic data to be detected is input into the isolated core to obtain the network intrusion detection result.

[0015] Optionally, before obtaining the dataset, the method further includes:

[0016] Obtain a network traffic dataset and extract features from each traffic data in the network traffic dataset; the network traffic dataset includes normal traffic data and abnormal traffic data;

[0017] An original dataset is generated based on the aforementioned features and the network traffic dataset. The original dataset includes negative samples and positive samples. The negative samples consist of abnormal network data and the features corresponding to the abnormal network data. The positive samples consist of normal network data and the features corresponding to the normal network data.

[0018] All negative samples are selected from the original dataset to form a discriminant dataset;

[0019] Determine the Euclidean distance between any two adjacent negative samples in the discriminant dataset;

[0020] Using any negative sample as the center and the smallest Euclidean distance corresponding to this negative sample as the data cleaning radius, positive sample cleaning is performed on the regions in the original dataset where the positive sample density meets the preset density requirements, thus obtaining the training dataset.

[0021] Optionally, the classification decision boundary between any two adjacent data points is the perpendicular bisector of the line connecting these two adjacent data points in the feature space.

[0022] Optionally, an isolated partition can be used as a cell to form a Venn diagram.

[0023] Optionally, the cells in the Venn diagram are represented as:

[0024]

[0025] In the formula, f p (xz) is a distance function, z is the center point of the cell, x is any point within the cell, D is the training dataset, and R is the distance between the points. d Let be the feature space of the dataset.

[0026] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0027] The network intrusion detection method provided by this invention, after acquiring the dataset, calculates the relative distance of abnormal data in the spatially modeled dataset, and filters normal data surrounding the abnormal data based on the cleanup radius, thus increasing the focus on abnormal data and improving the measurement accuracy of the isolated kernel. Furthermore, using the isolated kernel based on the network traffic data to be detected can accurately obtain network intrusion detection results.

[0028] Furthermore, the present invention also provides the following implementation structure:

[0029] A network intrusion detection system, applied to the network intrusion detection method provided above; the system includes:

[0030] The data acquisition module is used to acquire datasets and perform feature space modeling on the datasets.

[0031] The relative distance calculation module is used to calculate the relative distance of outliers in the spatially modeled dataset.

[0032] The cleanup radius calculation module is used to calculate the cleanup radius based on the relative distance between abnormal data.

[0033] The data filtering module is used to filter normal data around abnormal data based on the cleaning radius; normal data around abnormal data refers to normal data within a defined range centered on the abnormal data.

[0034] The isolated arc establishment module is used to partition the dataset into isolated regions based on the filtered dataset and train the isolated kernel.

[0035] The traffic data acquisition module is used to acquire network traffic data to be detected.

[0036] A network detection model is used to input the network traffic data to be detected into the isolated kernel to obtain network intrusion detection results.

[0037] An electronic device, comprising:

[0038] Memory, used to store computer programs;

[0039] A processor, connected to the memory, is used to retrieve and execute the computer program to implement the network intrusion detection method described above.

[0040] Optionally, the memory is a computer-readable storage medium.

[0041] Since the technical effects achieved by the above-described implementation structure provided by the present invention are the same as those achieved by the network intrusion detection method provided by the present invention, they will not be described again here. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1A flowchart of the network intrusion detection method provided by the present invention;

[0044] Figure 2 The flowchart for constructing an isolated kernel using the partitioning method provided by this invention;

[0045] Figure 3 This is a schematic diagram of the isolation partition provided by the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] The purpose of this invention is to provide a network intrusion detection method, system, and electronic device that can improve the accuracy of network intrusion detection.

[0048] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] like Figure 1 As shown, the network intrusion detection method provided by the present invention includes:

[0050] Step 100: Obtain the dataset and perform feature space modeling on the dataset.

[0051] Step 101: Calculate the relative distances between outlier data points in the spatially modeled dataset. The distance between outlier data points is calculated using Euclidean distance, and only the distance between adjacent outlier data points is calculated.

[0052] Step 102: Calculate the cleanup radius based on the relative distances between outlier data points. The cleanup radius is selected by choosing the minimum Euclidean distance between adjacent outlier data points.

[0053] Step 103: Filter the normal data surrounding the abnormal data according to the cleanup radius. When filtering the normal data, it is necessary to divide the cleanup range in the feature space with each abnormal data point as the center and the cleanup radius, and delete all normal data within the range to reduce the sample overlap in areas with high positive and negative sample density.

[0054] Step 104: Divide the processed dataset into isolated partitions and train isolated kernels. When dividing the dataset into isolated partitions, the nearest neighbor partitioning method is used, resulting in a Venn diagram, which avoids uneven distribution of isolated partition areas.

[0055] Step 104 largely inherits the nearest neighbor partitioning method, but incorporates an improvement before the isolation partitioning: a method for filtering and cleaning some normal data. By deleting some normal data, the sample density in highly overlapping positive and negative sample regions is reduced, thereby increasing the dissimilarity of the isolation partitions and ultimately improving the measurement accuracy of the isolated kernel function.

[0056] Step 105: Obtain the network traffic data to be detected.

[0057] Step 106: Input the network traffic data to be detected into the trained isolated kernel to obtain the network intrusion detection results. In this step, the network intrusion detection results are used as keys to mark abnormal traffic data.

[0058] Based on this, the present invention has the following characteristics compared to the prior art:

[0059] i) This invention introduces a data filtering method to increase the isolation area of ​​negative samples during partitioning, thereby increasing the attention given to abnormal data in this area and improving the measurement accuracy of the algorithm model.

[0060] ii) Reducing the density of overlapping region samples in the training data improves the training speed and measurement accuracy of isolated kernels, effectively improving algorithm efficiency.

[0061] The following is a specific case to illustrate the implementation process of the network intrusion detection method described above in this invention.

[0062] like Figure 2 As shown, the main process is as follows:

[0063] 1. Collect network traffic datasets from the server, including both normal and abnormal traffic. For each network traffic data point, meaningful features need to be extracted. Feature extraction can use various techniques, such as n-gram methods and spectrum analysis, to extract useful information from the data. It can also extract features from requests, such as the source IP address, destination IP address, and request type.

[0064] 2. The collected dataset and extracted features are used to form the original dataset. All negative samples from the original dataset are selected to form the discriminant dataset. In the discriminant dataset, the Euclidean distance between any two adjacent negative samples is calculated. This process is repeated for all negative samples in the discriminant dataset, calculating the Euclidean distance between any two adjacent negative sample points. The Euclidean distance with the smallest value is selected as the cleaning radius μ for data cleaning. Negative samples consist of anomalous network data and their corresponding features. Positive samples consist of normal network data and their corresponding features.

[0065] 3. In the original dataset, perform data cleaning on regions with high density of positive and negative samples by selecting all negative sample points in the dataset. For example, construct a circular cleaning region with each sample point yi as the center and μ as the radius, and remove all positive samples within the cleaning region to obtain the training dataset. Let D = x1, x2, ..., x n x i ∈R d It is the dataset obtained after processing, i.e., the training dataset.

[0066] 4. Using Euclidean distance as the distance metric, the nearest neighbor classification method is employed. The classification decision boundary between any two adjacent sample points is the perpendicular bisector of the line connecting the two sample points in the feature space. A data sample point can be partitioned into an isolation partition by defining the decision boundary with its neighboring sample points.

[0067] 5. Continue to select other sample points in the dataset and construct isolation partitions according to step 3. Repeat this process multiple times to obtain multiple isolation partitions, each containing one data sample point.

[0068] 6. The sample point within an isolation partition can also be called the center of that partition. The graph formed by all isolation partitions can be considered a Voronoi diagram. An isolation partition is equivalent to a cell in a Voronoi diagram, and a Voronoi diagram cell centered at a specific point is:

[0069]

[0070] In the formula, f p (xz) is a distance function, z is the center point of the cell, x is any point within the cell, D is the training dataset, and R is the distance between the points. d Let be the feature space of the dataset.

[0071] The resulting Voronoi diagram possesses the properties required for a spatial partitioning mechanism, generating large partitions in sparse regions and small partitions in dense regions. This leads to the characteristic of isolating similarity: two points in a sparse region are more similar than two points equidistant from each other in a dense region.

[0072] 7. Train the isolated kernel function based on the generated isolation partitions, and test the trained isolated kernel on the data according to the different isolation partition densities for each training point. During testing, if two points fall within the same isolation partition, they are considered similar. Due to the different areas of the isolation partitions, points in sparse regions are more likely to be similar than those in dense regions at the same distance.

[0073] 8. After training, the isolated kernel is obtained. To further improve intrusion detection accuracy, it needs to be evaluated using a test dataset. The test dataset should contain both normal and abnormal traffic that did not appear in the training dataset. For each data point in the test dataset, the isolated kernel calculates its distance to other data points and uses a threshold to determine if it is an outlier. The evaluation results can be assessed using metrics such as precision, recall, and F1 score.

[0074] 9. Once the orphaned kernel performs well on the test dataset, it can be deployed to real-world applications, such as on a company server. During runtime, the orphaned kernel will monitor network traffic on the server in real time and flag abnormal traffic, thereby improving server security.

[0075] The above describes the process of training isolated kernels using the new partitioning method and applying it to network intrusion detection. The main optimization is the area of ​​each partition when generating isolated kernel partitions, thereby improving the measurement accuracy of the isolated kernels. Figure 3 This is a schematic diagram of the resulting isolated partitions. The resulting isolated kernels improve both classification accuracy and training time, thereby enhancing the overall performance of the algorithm.

[0076] Furthermore, the present invention also provides the following implementation structure:

[0077] A network intrusion detection system is provided, which applies the network intrusion detection method described above. The system includes:

[0078] The data acquisition module is used to acquire a dataset and perform feature space modeling on the dataset.

[0079] The relative distance calculation module is used to calculate the relative distance of abnormal data in the dataset after spatial modeling.

[0080] The cleanup radius calculation module is used to calculate the cleanup radius based on the relative distance between abnormal data.

[0081] The data filtering module is used to filter normal data surrounding abnormal data based on a cleanup radius. Normal data surrounding abnormal data refers to normal data within a defined range centered on the abnormal data.

[0082] The isolated arc establishment module is used to partition the dataset into isolated regions based on the filtered dataset and train the isolated kernel.

[0083] The traffic data acquisition module is used to acquire network traffic data to be detected.

[0084] The network detection model is used to input the network traffic data to be detected into an isolated kernel to obtain network intrusion detection results.

[0085] An electronic device, comprising:

[0086] Memory is used to store computer programs.

[0087] The processor, connected to the memory, is used to retrieve and execute computer programs to implement the network intrusion detection methods described above.

[0088] Furthermore, when the computer program in the aforementioned memory is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0089] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0090] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A network intrusion detection method, characterized by, The method comprises: acquiring a data set and modeling a feature space of the data set; acquiring a network traffic data set and extracting features of each traffic data in the network traffic data set; the network traffic data set comprises normal traffic data and abnormal traffic data; generating an original data set based on the features and the network traffic data set; the original data set comprises negative samples and positive samples; the positive samples are composed of normal network data and features corresponding to the normal network data; the negative samples are composed of abnormal network data and features corresponding to the abnormal network data; selecting all negative samples from the original data set to form a discrimination data set; calculating the Euclidean distance between any two adjacent negative samples in the discrimination data set, and calculating the Euclidean distance between adjacent two negative sample points in the discrimination data set, and selecting the minimum Euclidean distance as the cleaning radius of data cleaning; screening normal data around abnormal data in the data set after space modeling according to the cleaning radius; the normal data around the abnormal data refers to the normal data within a set range with the abnormal data as the center; isolating and partitioning the data set after screening to train an isolated kernel; acquiring network traffic data to be detected; inputting the network traffic data to be detected into the isolated kernel to obtain a network intrusion detection result.

2. The network intrusion detection method of claim 1, wherein, The classification decision boundary between any two adjacent data is the perpendicular bisector of the connecting line between the two adjacent data in the feature space.

3. The network intrusion detection method of claim 1, wherein, A Venn diagram is formed by taking one isolated partition as a unit cell.

4. The network intrusion detection method of claim 3, wherein, The cells in the Venn diagram are represented as : ； wherein, is a distance function, z is a center point of a cell, x is an arbitrary point within a cell, D is a training data set, R d is a feature space of a data set.

5. A network intrusion detection system characterized by, The system is applied to the network intrusion detection method according to any one of claims 1-4; the system comprises: a data acquisition module for acquiring a data set and modeling a feature space of the data set; a relative distance calculation module for calculating the relative distance of abnormal data in the data set after space modeling; a cleaning radius calculation module for calculating the cleaning radius according to the relative distance between the abnormal data; a data screening module for screening normal data around the abnormal data according to the cleaning radius; the normal data around the abnormal data refers to the normal data within a set range with the abnormal data as the center; an isolated arc establishment module for isolating and partitioning the data set after screening to train an isolated kernel; a traffic data acquisition module for acquiring network traffic data to be detected; a network detection model for inputting the network traffic data to be detected into the isolated kernel to obtain a network intrusion detection result.

6. An electronic device, comprising: The method comprises: a memory for storing a computer program; a processor connected with the memory, for calling and executing the computer program to implement the network intrusion detection method according to any one of claims 1-4.

7. The electronic device of claim 6, wherein, The memory is a computer readable storage medium.

Images