Intrusion traffic detection method and device, computer device and readable storage medium
By constructing a training dataset for differential screening and a self-learning detection model, the problem of decreased detection speed of traditional IDS in complex network environments is solved, and fast and accurate intrusion traffic identification is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA TELECOM CORP LTD TECHNOLOGY INNOVATION CENTER
- Filing Date
- 2024-09-23
- Publication Date
- 2026-05-19
AI Technical Summary
Traditional intrusion detection systems suffer from reduced detection speed and increased computational complexity when faced with complex network environments and diverse network traffic, making it difficult to effectively identify intrusion traffic.
By constructing a training dataset, training traffic samples under each traffic label are selected to ensure that the difference between each sample is greater than the difference threshold, thereby reducing the number of highly similar samples. A detection model with self-learning capabilities is then trained to optimize the detection process.
It improves the speed and accuracy of intrusion traffic detection, simplifies the detection process, reduces the need for manual analysis and intervention, and improves detection efficiency.
Smart Images
Figure CN119276553B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of network security technology, and in particular to an intrusion traffic detection method, apparatus, computer device, and readable storage medium. Background Technology
[0002] An Intrusion Detection System (IDS) is a network security device or software specifically designed to detect, identify, and respond to unauthorized network access, malicious activities, and other behaviors that may pose a threat to system and network security. With the rapid development of 5G core networks, the complexity of the network environment and the dramatic increase in traffic have brought unprecedented challenges to IDS.
[0003] Traditional IDS typically employs pattern matching algorithms to detect network attacks. However, with increasing network traffic and the diversification of attack types, the computational complexity of pattern matching algorithms also increases, leading to a decrease in detection speed. Therefore, improvements are urgently needed. Summary of the Invention
[0004] Therefore, it is necessary to provide an intrusion traffic detection method, apparatus, computer equipment, and readable storage medium that can improve the speed of intrusion traffic detection methods in response to the above-mentioned technical problems.
[0005] Firstly, this application provides an intrusion traffic detection method, the method comprising:
[0006] Acquire the traffic to be detected and extract its traffic characteristics;
[0007] Input the traffic characteristics into the detection model to obtain the intrusion detection results output by the detection model;
[0008] The detection model is trained on the training dataset. The training dataset contains training traffic samples under each traffic label selected from the original dataset. The difference between different training traffic samples under each traffic label is greater than the difference threshold.
[0009] In one embodiment, the training dataset is obtained in the following way:
[0010] Based on the flow characteristics of each original flow sample in the original dataset, determine the flow sample difference value between each original flow sample;
[0011] The original dataset is filtered based on the differences between the original traffic samples and the traffic labels of each original traffic sample to obtain the training dataset.
[0012] In one embodiment, the original dataset is filtered based on the traffic sample difference values between each original traffic sample and the traffic labels of each original traffic sample to obtain a training dataset, including:
[0013] Select a target traffic sample from the unlabeled raw traffic samples in the original dataset, and treat all raw traffic samples other than the target traffic sample in the unlabeled raw traffic as other traffic samples;
[0014] Based on the difference between the target flow sample and other flow samples, the neighboring flow samples of the target flow sample are determined from the other flow samples; wherein the difference between the target flow sample and its corresponding neighboring flow samples is less than or equal to the difference threshold.
[0015] Based on the traffic labels of the target traffic sample and the traffic labels of neighboring traffic samples, select samples to be labeled and samples to be removed from the target traffic sample and neighboring traffic samples;
[0016] The original dataset is updated based on the samples to be labeled and the samples to be removed;
[0017] If the updated original dataset contains only labeled traffic samples or contains one unlabeled original traffic sample, then the updated original dataset will be used as the training dataset.
[0018] In one embodiment, selecting samples to be labeled and samples to be removed from the target traffic sample and neighboring traffic samples based on the traffic label of the target traffic sample and the traffic label of neighboring traffic samples includes:
[0019] If the traffic label of the target traffic sample in the original dataset is the same as the traffic label of the neighboring traffic samples, then the target traffic sample is taken as the sample to be labeled, and the neighboring traffic samples are taken as the sample to be removed.
[0020] If the traffic label of the target traffic sample in the original dataset is inconsistent with the traffic label of the neighboring traffic samples, then the target traffic sample will be used as the sample to be labeled.
[0021] In one embodiment, the original dataset is updated based on the samples to be labeled and the samples to be removed, including:
[0022] In the original dataset, the samples to be labeled are labeled, and the samples to be removed are removed, resulting in the updated original dataset.
[0023] In one embodiment, the method further includes:
[0024] If there are at least two unlabeled original traffic samples in the updated original dataset, then based on the updated original dataset, return to the step of selecting a target traffic sample from the unlabeled original traffic samples in the original dataset.
[0025] In one embodiment, the traffic characteristics of the original traffic sample include at least the average packet length, the variance of the packet length, the average packet arrival time, the standard deviation of the packet arrival time, the number of packets arriving per second, and the duration of the flow.
[0026] Secondly, this application also provides an intrusion traffic detection method apparatus, the apparatus comprising:
[0027] The acquisition module is used to acquire the traffic to be detected and extract the traffic characteristics of the traffic to be detected;
[0028] The detection module is used to input traffic characteristics into the detection model and obtain the intrusion detection results output by the detection model;
[0029] The detection model is trained on the training dataset. The training dataset contains training traffic samples under each traffic label selected from the original dataset. The difference between different training traffic samples under each traffic label is greater than the difference threshold.
[0030] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0031] Acquire the traffic to be detected and extract its traffic characteristics;
[0032] Input the traffic characteristics into the detection model to obtain the intrusion detection results output by the detection model;
[0033] The detection model is trained on the training dataset. The training dataset contains training traffic samples under each traffic label selected from the original dataset. The difference between different training traffic samples under each traffic label is greater than the difference threshold.
[0034] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:
[0035] Acquire the traffic to be detected and extract its traffic characteristics;
[0036] Input the traffic characteristics into the detection model to obtain the intrusion detection results output by the detection model;
[0037] The detection model is trained on the training dataset. The training dataset contains training traffic samples under each traffic label selected from the original dataset. The difference between different training traffic samples under each traffic label is greater than the difference threshold.
[0038] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:
[0039] Acquire the traffic to be detected and extract its traffic characteristics;
[0040] Input the traffic characteristics into the detection model to obtain the intrusion detection results output by the detection model;
[0041] The detection model is trained on the training dataset. The training dataset contains training traffic samples under each traffic label selected from the original dataset. The difference between different training traffic samples under each traffic label is greater than the difference threshold.
[0042] The aforementioned intrusion traffic detection method, apparatus, computer equipment, and readable storage medium, when constructing the training dataset, select training traffic samples under each traffic label from the original dataset and ensure that the difference between each sample is greater than a preset difference threshold. This method reduces the number of highly similar samples, preventing the model from being interfered with by redundant information during training. It ensures that the training dataset contains diverse samples, covering various situations under different traffic labels. Increasing the difference between samples of different categories helps the model better learn and distinguish different traffic types, especially the difference between intrusion traffic and normal traffic. Based on this, the detection model is trained on the optimized training dataset, and through its self-learning ability, the model can learn the characteristics of different traffic types.
[0043] During training, the model continuously adjusts its parameters to minimize prediction errors and improve the accuracy of identifying intrusion traffic. Since the detection model has been fully trained on an optimized training dataset, in practical applications, only the features of the traffic to be detected need to be input into the model to quickly obtain intrusion detection results. This method simplifies the intrusion traffic detection process, reduces the need for manual analysis and intervention, and improves detection efficiency. The detection model fully learns the characteristics of different traffic types during training, thus enabling it to more accurately identify intrusion traffic in real-world applications. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0045] Figure 1 This is a flowchart illustrating an intrusion traffic detection method in one embodiment;
[0046] Figure 2 This is a flowchart illustrating the steps of filtering the original dataset to obtain the training dataset in one embodiment.
[0047] Figure 3 This is a flowchart illustrating the steps for updating the original dataset in one embodiment;
[0048] Figure 4 This is a flowchart illustrating the steps of selecting samples to be labeled and samples to be removed from the target traffic sample and neighboring traffic samples in one embodiment.
[0049] Figure 5 This is a flowchart illustrating an intrusion traffic detection method in another embodiment;
[0050] Figure 6 This is a structural block diagram of an intrusion traffic detection method apparatus in one embodiment;
[0051] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0053] In one exemplary embodiment, such as Figure 1 As shown, an intrusion traffic detection method is provided. Taking the application of this method to a computer device as an example, the method includes the following steps S101 to S102. Wherein:
[0054] S101, acquire the traffic to be detected and extract the traffic characteristics of the traffic to be detected.
[0055] The traffic to be detected refers to real-time or recent network traffic data collected from the network environment. This traffic data may include normal network communication as well as potential intrusion attempts.
[0056] Optionally, the collected traffic data can be processed to extract key information that represents its characteristics. These features may include the source address, destination address, protocol used, packet size, transmission speed, connection duration, etc. The purpose of feature extraction is to allow this information to be subsequently input into the detection model for analysis.
[0057] S102, input the traffic characteristics into the detection model to obtain the intrusion detection results output by the detection model.
[0058] Specifically, the traffic features extracted from S101 are used as input data and fed into a pre-trained detection model. The detection model will output a judgment result regarding whether an intrusion has occurred based on the input traffic features. This result may be binary (e.g., "intrusion" or "non-intrusion") or multi-class (e.g., specifying the type of intrusion).
[0059] The detection model is trained based on the training dataset.
[0060] Furthermore, the training dataset contains training traffic samples under each traffic label selected from the original dataset, and the difference between different training traffic samples under each traffic label is greater than the difference threshold.
[0061] Optionally, training traffic samples under different labels can be selected from the original dataset based on traffic labels (i.e., traffic type or category). These samples are used to train the detection model.
[0062] For each traffic label, ensure that the difference between the selected training traffic samples is greater than a preset difference threshold. This is done to guarantee the diversity and representativeness of the training samples, thereby improving the generalization ability of the detection model.
[0063] Specifically, when selecting training traffic samples under each traffic label from the original dataset, the following process can be adopted: (1) Data cleaning and preprocessing: Remove noise, outliers, and incomplete records from the original dataset. This helps improve the quality of the training dataset and reduce interference during model training. Standardize the cleaned data, such as scaling and normalization, to ensure comparability between different features. In addition, feature selection can be performed to retain features that have an important impact on the classification task and remove redundant features. (2) Data sampling: Stratify according to traffic labels to ensure that the number of samples under each label is relatively balanced. This helps to deal with the imbalance problem in the original dataset and improve the generalization ability of the model.
[0064] In the aforementioned intrusion traffic detection method, when constructing the training dataset, training traffic samples under each traffic label are selected from the original dataset, ensuring that the difference between each sample is greater than a preset difference threshold. This method reduces the number of highly similar samples, preventing the model from being interfered with by redundant information during training. It also ensures that the training dataset contains diverse samples, covering various situations under different traffic labels. Increasing the difference between samples of different categories helps the model better learn and distinguish different traffic types, especially the difference between intrusion traffic and normal traffic.
[0065] Based on this, the detection model is trained on an optimized training dataset. Through its self-learning capability, the model can learn the characteristics of different traffic types. During training, the model continuously adjusts its parameters to minimize prediction errors and improve the accuracy of identifying intrusion traffic. Since the detection model has been fully trained on the optimized training dataset, in practical applications, only the characteristics of the traffic to be detected need to be input into the model to quickly obtain intrusion detection results. This method simplifies the intrusion traffic detection process, reduces the need for manual analysis and intervention, and improves detection efficiency. Because the detection model fully learns the characteristics of different traffic types during training, it can more accurately identify intrusion traffic in practical applications.
[0066] In one exemplary embodiment, such as Figure 2 As shown, the training dataset was obtained in the following way:
[0067] S201, Based on the flow characteristics of each original flow sample in the original dataset, determine the flow sample difference value between each original flow sample.
[0068] The flow characteristics of the original flow sample include at least the average packet length, the variance of the packet length, the average packet arrival time, the standard deviation of the packet arrival time, the number of packets arriving per second, and the duration of the flow.
[0069] Understandably, the average packet length refers to the average length of all packets within a specific traffic sample or flow. Intrusive traffic may exhibit a different packet length distribution than normal traffic, thus the average packet length can serve as a useful feature.
[0070] Packet length variance: Variance measures the degree of dispersion between packet length and the average. For some types of intrusions, such as DDoS attacks, packet length may exhibit high variance because attackers may use packets of different sizes to simulate normal traffic.
[0071] Average packet arrival time: This refers to the average time interval between the arrival of data packets. Intrusion traffic may exhibit different arrival time patterns than normal traffic, such as faster arrival rates or more irregular intervals.
[0072] Standard deviation of packet arrival time: The standard deviation measures the dispersion of packet arrival time from the average value. A large standard deviation may indicate instability in traffic patterns, which could be a sign of intrusion.
[0073] Packets arriving per second: This is a fundamental metric for measuring network traffic, specifically the number of data packets transmitted per second. Intrusive traffic can cause a significant increase in this number, especially during flood attacks or DDoS attacks.
[0074] Flow duration: Flow duration refers to the time from the start to the end of a specific network connection or transmission session. Certain types of intrusions may exhibit duration patterns different from normal traffic; for example, short, frequent connections may be an indication of a scanning attack or an attempt to intrude.
[0075] S202, based on the traffic sample difference values between each original traffic sample and the traffic label of each original traffic sample, the original dataset is filtered to obtain the training dataset.
[0076] In one feasible approach, for each original traffic sample under a traffic label, the difference between them is calculated. This difference can be based on various metrics, such as Euclidean distance, Manhattan distance, and cosine similarity; the specific method chosen depends on the traffic characteristics and the requirements of the classification task. By setting a difference threshold, sample pairs with differences greater than this threshold can be filtered out to ensure the diversity and representativeness of the training dataset. The original dataset is then filtered based on the traffic sample difference values and the traffic labels.
[0077] Optionally, firstly, remove sample pairs with excessively small differences to reduce the number of highly similar samples. Secondly, ensure that there are sufficient samples under each traffic label to support model training. To address potential data imbalance, stratified sampling can be used. This involves stratifying the original dataset according to traffic labels and then sampling independently within each stratum. This ensures that each traffic label has an appropriate representation in the training dataset.
[0078] In another possible implementation, such as Figure 3 As shown, the original dataset is filtered based on the differences between the original traffic samples and the traffic labels of each original traffic sample to obtain the training dataset, which includes:
[0079] S301, select a target traffic sample from the unlabeled raw traffic samples in the original dataset, and treat all raw traffic samples other than the target traffic sample in the unlabeled raw traffic as other traffic samples.
[0080] In this context, labeled raw traffic samples refer to raw traffic samples that do not need to participate in the subsequent screening process and can be directly used as training traffic samples; correspondingly, unlabeled raw traffic samples refer to raw traffic samples that need to participate in the subsequent screening process to verify whether they can be used as training traffic samples.
[0081] S302, based on the difference value between the target flow sample and each other flow sample, determine the neighboring flow sample of the target flow sample from each other flow sample.
[0082] Among them, the difference between the target flow sample and the neighboring flow samples corresponding to the target flow sample is less than or equal to the difference threshold.
[0083] KNN (K-Nearest Neighbors) is a commonly used supervised learning algorithm, primarily for classification and regression problems. In classification problems, KNN predicts the class of a target traffic sample by finding its K nearest neighbors and based on the classes of these neighbors. The following are the steps of the KNN algorithm to determine the neighbors of a target traffic sample:
[0084] In the KNN algorithm, distance metrics are crucial for determining the similarity or proximity between samples. Common distance metrics include: 1) Euclidean distance: measures the straight-line distance between two samples in multidimensional space. 2) Manhattan distance: measures the distance between two samples in multidimensional space, representing city blocks. 3) Cosine similarity: measures the cosine of the angle between two sample vectors, typically used in scenarios such as text classification. In this embodiment, the KNN algorithm is used to determine the neighboring traffic samples of the target traffic sample. Specifically, for a given target traffic sample, the distances between all other traffic samples need to be calculated. Based on the sorted distances, the K closest other traffic samples are selected as the neighboring samples of the target traffic sample. These neighboring samples will be used for subsequent category determination or regression prediction.
[0085] S303, based on the traffic label of the target traffic sample and the traffic label of the neighboring traffic samples, select samples to be labeled and samples to be removed from the target traffic sample and the neighboring traffic samples.
[0086] Among them, the sample to be labeled refers to the sample that needs to be labeled, and the labeled sample will be used directly as the training traffic sample; the sample to be removed refers to the sample that needs to be removed from the original dataset.
[0087] S304 Updates the original dataset based on the samples to be labeled and the samples to be removed.
[0088] Specifically, in the original dataset, the samples to be labeled are labeled, and the samples to be removed are removed, resulting in the updated original dataset.
[0089] S305, if there are at least two unlabeled original traffic samples in the updated original dataset, then based on the updated original dataset, return to the step of selecting a target traffic sample from the unlabeled original traffic samples in the original dataset.
[0090] Understandably, if there are at least two unlabeled original traffic samples in the updated original dataset, it means that there may be sample pairs that are adjacent to each other among the at least two unlabeled original traffic samples. In this case, it is necessary to further filter the at least two unlabeled original traffic samples, that is, return to the step of selecting a target traffic sample from the unlabeled original traffic samples in the original dataset, until the updated original dataset contains only labeled traffic samples or there is one unlabeled original traffic sample.
[0091] For example, in this embodiment, three nearest neighbor traffic samples can be selected for each target traffic sample. Accordingly, when the three nearest neighbor traffic samples of a target traffic sample A are all the same traffic label as A, these three nearest neighbor traffic samples can be deleted from the original dataset. This process is repeated until no sample in the original dataset has three nearest neighbor traffic samples that are the same traffic label as itself.
[0092] S306. If the updated original dataset contains only labeled traffic samples or contains one unlabeled original traffic sample, then the updated original dataset shall be used as the training dataset.
[0093] Correspondingly, if the updated original dataset only contains labeled traffic samples, it means that there are no samples in the original dataset that need to be filtered. In this case, the original dataset is directly used as the training dataset, and the training dataset is classified according to different traffic labels.
[0094] Furthermore, if the updated original dataset contains only one unlabeled original traffic sample, it means that all other original traffic samples in the updated original dataset are labeled original traffic samples. It is understood that the labeled original traffic samples do not need to be filtered again, while the unlabeled original traffic sample is a neighboring traffic sample of the target traffic sample in this round of filtering, and the traffic label of the neighboring traffic sample is inconsistent with that of the target traffic sample. In this case, it means that although the neighboring traffic sample and the target traffic sample have a large difference, they can be used as training traffic samples for their respective traffic labels. In this case, the original dataset containing only one unlabeled original traffic sample is directly used as the training dataset, and the training dataset is classified according to different traffic labels.
[0095] In this embodiment, neighboring samples corresponding to each target traffic sample are filtered from the sample feature dimension and the sample label dimension to update the original dataset, resulting in greater differences between samples in the updated original dataset.
[0096] In one exemplary embodiment, such as Figure 4 As shown, based on the traffic labels of the target traffic sample and the traffic labels of neighboring traffic samples, samples to be labeled and samples to be removed are selected from the target traffic sample and neighboring traffic samples, including:
[0097] S401, if the traffic label of the target traffic sample in the original dataset is the same as the traffic label of the neighboring traffic samples, then the target traffic sample is taken as the sample to be labeled, and the neighboring traffic samples are taken as the sample to be removed.
[0098] It is understandable that if the traffic label of the target traffic sample and the traffic label of the neighboring traffic sample in the original dataset are the same, it means that the similarity between the target traffic sample and the neighboring traffic sample is high (high similarity in both the traffic feature dimension and the traffic label dimension). In this case, it is only necessary to choose between the target traffic sample and the neighboring traffic sample. Optionally, the target traffic sample can be retained, and the neighboring traffic sample can be removed as a sample to be removed.
[0099] S402, if the traffic label of the target traffic sample in the original dataset is inconsistent with the traffic label of the neighboring traffic samples, then the target traffic sample will be used as the sample to be labeled.
[0100] It is understandable that if the traffic labels of the target traffic sample and the neighboring traffic samples in the original dataset are inconsistent, it means that the target traffic sample and the neighboring traffic samples are only similar in the dimension of traffic features, but differ greatly in the dimension of traffic labels. They can be used as training traffic samples for their respective traffic labels.
[0101] In this embodiment, the similarity between samples is measured from the perspective of traffic labels, based on the traffic labels of the target traffic sample and the traffic labels of the neighboring traffic samples in the original dataset, so as to achieve accurate screening of the target traffic sample and the neighboring traffic sample.
[0102] In one exemplary embodiment, such as Figure 5 As shown, this embodiment provides an intrusion traffic detection method, including:
[0103] S501, label each of the collected raw traffic samples to obtain the raw dataset.
[0104] The flow characteristics of each original flow sample include at least the average packet length, the variance of the packet length, the average packet arrival time, the standard deviation of the packet arrival time, the number of packets arriving per second, and the duration of the flow.
[0105] S502, Based on the flow characteristics of each original flow sample in the original dataset, determine the flow sample difference value between each original flow sample.
[0106] S503: Select a target traffic sample from the unlabeled raw traffic samples in the original dataset, and treat all raw traffic samples other than the target traffic sample in the unlabeled raw traffic as other traffic samples.
[0107] S504, based on the difference value between the target flow sample and each other flow sample, determine the neighboring flow sample of the target flow sample from each other flow sample.
[0108] S505, based on the traffic labels of the target traffic sample and the traffic labels of neighboring traffic samples, select samples to be labeled and samples to be removed from the target traffic sample and neighboring traffic samples.
[0109] S506 updates the original dataset based on the samples to be labeled and the samples to be removed.
[0110] S507, if there are at least two unlabeled original traffic samples in the updated original dataset, then based on the updated original dataset, return to the step of selecting a target traffic sample from the unlabeled original traffic samples in the original dataset.
[0111] S508. If the updated original dataset contains only labeled traffic samples or contains one unlabeled original traffic sample, then the updated original dataset will be used as the training dataset.
[0112] S509, based on the training dataset, trains the initial model to obtain the detection model.
[0113] S510 acquires the traffic to be detected and extracts the traffic characteristics of the traffic to be detected.
[0114] S511, input the traffic characteristics into the detection model to obtain the intrusion detection results output by the detection model.
[0115] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0116] Based on the same inventive concept, this application also provides an intrusion traffic detection device for implementing the aforementioned method. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more intrusion traffic detection method device embodiments provided below can be found in the limitations of the intrusion traffic detection method described above, and will not be repeated here.
[0117] In one exemplary embodiment, such as Figure 6 As shown, an intrusion traffic detection method apparatus is provided, comprising: an acquisition module 11 and a detection module 12, wherein:
[0118] The acquisition module 11 is used to acquire the traffic to be detected and extract the traffic characteristics of the traffic to be detected;
[0119] Detection module 12 is used to input traffic characteristics into the detection model and obtain the intrusion detection results output by the detection model;
[0120] The detection model is trained on the training dataset. The training dataset contains training traffic samples under each traffic label selected from the original dataset. The difference between different training traffic samples under each traffic label is greater than the difference threshold.
[0121] In one embodiment, the apparatus further includes a construction module for: determining the traffic sample difference value between the original traffic samples based on the traffic characteristics of each original traffic sample in the original dataset;
[0122] The original dataset is filtered based on the differences between the original traffic samples and the traffic labels of each original traffic sample to obtain the training dataset.
[0123] In one embodiment, the construction module is configured to: select a target traffic sample from the unlabeled raw traffic samples in the raw dataset, and treat all raw traffic samples other than the target traffic sample in the unlabeled raw traffic as other traffic samples;
[0124] Based on the difference between the target flow sample and other flow samples, the neighboring flow samples of the target flow sample are determined from the other flow samples; wherein the difference between the target flow sample and its corresponding neighboring flow samples is less than or equal to the difference threshold.
[0125] Based on the traffic labels of the target traffic sample and the traffic labels of neighboring traffic samples, select samples to be labeled and samples to be removed from the target traffic sample and neighboring traffic samples;
[0126] The original dataset is updated based on the samples to be labeled and the samples to be removed;
[0127] If the updated original dataset contains only labeled traffic samples or contains one unlabeled original traffic sample, then the updated original dataset will be used as the training dataset.
[0128] In one embodiment, the construction module is configured to: if the traffic label of the target traffic sample in the original dataset is consistent with the traffic label of the neighboring traffic samples, then the target traffic sample is used as the sample to be labeled and the neighboring traffic samples are used as the samples to be removed.
[0129] If the traffic label of the target traffic sample in the original dataset is inconsistent with the traffic label of the neighboring traffic samples, then the target traffic sample will be used as the sample to be labeled.
[0130] In one embodiment, the construction module is configured to: label samples to be labeled in the original dataset and remove samples to be removed, thereby obtaining an updated original dataset.
[0131] In one embodiment, the construction module is configured to: if there are at least two unlabeled original traffic samples in the updated original dataset, then based on the updated original dataset, return to the step of selecting a target traffic sample from the unlabeled original traffic samples in the original dataset.
[0132] In one embodiment, the traffic characteristics of the original traffic sample include at least the average packet length, the variance of the packet length, the average packet arrival time, the standard deviation of the packet arrival time, the number of packets arriving per second, and the duration of the flow.
[0133] Each module in the aforementioned intrusion traffic detection method and device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.
[0134] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores XX data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements an intrusion traffic detection method.
[0135] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0136] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0137] Acquire the traffic to be detected and extract its traffic characteristics;
[0138] Input the traffic characteristics into the detection model to obtain the intrusion detection results output by the detection model;
[0139] The detection model is trained on the training dataset. The training dataset contains training traffic samples under each traffic label selected from the original dataset. The difference between different training traffic samples under each traffic label is greater than the difference threshold.
[0140] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0141] Acquire the traffic to be detected and extract its traffic characteristics;
[0142] Input the traffic characteristics into the detection model to obtain the intrusion detection results output by the detection model;
[0143] The detection model is trained on the training dataset. The training dataset contains training traffic samples under each traffic label selected from the original dataset. The difference between different training traffic samples under each traffic label is greater than the difference threshold.
[0144] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:
[0145] Acquire the traffic to be detected and extract its traffic characteristics;
[0146] Input the traffic characteristics into the detection model to obtain the intrusion detection results output by the detection model;
[0147] The detection model is trained on the training dataset. The training dataset contains training traffic samples under each traffic label selected from the original dataset. The difference between different training traffic samples under each traffic label is greater than the difference threshold.
[0148] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0149] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0150] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for detecting intrusion traffic, characterized in that, The method includes: Acquire the traffic to be detected and extract the traffic characteristics of the traffic to be detected; The traffic characteristics are input into the detection model to obtain the intrusion detection results output by the detection model; The detection model is trained on a training dataset; the training dataset contains training traffic samples under each traffic label selected from the original dataset, and the difference between different training traffic samples under each traffic label is greater than a difference threshold; the training dataset is obtained in the following way: Based on the traffic characteristics of each original traffic sample in the original dataset, the traffic sample difference value between each original traffic sample is determined; a target traffic sample is selected from the unlabeled original traffic samples in the original dataset, and all other original traffic samples in the unlabeled original traffic samples are taken as other traffic samples; based on the traffic sample difference value between the target traffic sample and each other traffic sample, the neighboring traffic samples of the target traffic sample are determined from each other traffic sample; wherein, the traffic sample difference value between the target traffic sample and the corresponding neighboring traffic sample is less than or equal to the difference threshold; based on the traffic label of the target traffic sample and the traffic label of the neighboring traffic samples, samples to be labeled and samples to be removed are selected from the target traffic sample and the neighboring traffic samples; based on the samples to be labeled and the samples to be removed, the original dataset is updated; if the updated original dataset contains only labeled traffic samples or contains one unlabeled original traffic sample, the updated original dataset is used as the training dataset.
2. The method according to claim 1, characterized in that, The step of selecting samples to be labeled and samples to be removed from the target traffic sample and the neighboring traffic samples based on the traffic label of the target traffic sample and the traffic label of the neighboring traffic samples includes: If the traffic label of the target traffic sample in the original dataset is the same as the traffic label of the neighboring traffic sample, then the target traffic sample is used as the sample to be labeled, and the neighboring traffic sample is used as the sample to be removed. If the traffic label of the target traffic sample in the original dataset is inconsistent with the traffic label of the neighboring traffic samples, then the target traffic sample is used as the sample to be labeled.
3. The method according to claim 1, characterized in that, The step of updating the original dataset based on the samples to be labeled and the samples to be removed includes: In the original dataset, the samples to be labeled are labeled, and the samples to be removed are removed to obtain the updated original dataset.
4. The method according to claim 1, characterized in that, The method further includes: If there are at least two unlabeled original traffic samples in the updated original dataset, then based on the updated original dataset, return to the step of selecting a target traffic sample from the unlabeled original traffic samples in the original dataset.
5. The method according to claim 1, characterized in that, The flow characteristics of the original flow sample include at least the average packet length, the variance of the packet length, the average packet arrival time, the standard deviation of the packet arrival time, the number of packets arriving per second, and the duration of the flow.
6. The method according to claim 1, characterized in that, The traffic to be detected includes real-time network traffic data collected from the network environment, which includes normal network communication behavior and / or potential intrusion behavior.
7. An intrusion flow detection method and apparatus, characterized in that, The device includes: The acquisition module is used to acquire the traffic to be detected and extract the traffic characteristics of the traffic to be detected; The detection module is used to input the traffic characteristics into the detection model and obtain the intrusion detection results output by the detection model; The detection model is trained on a training dataset; the training dataset contains training traffic samples under each traffic label selected from the original dataset, and the difference between different training traffic samples under each traffic label is greater than a difference threshold; the training dataset is obtained in the following way: Based on the traffic characteristics of each original traffic sample in the original dataset, the traffic sample difference value between each original traffic sample is determined; a target traffic sample is selected from the unlabeled original traffic samples in the original dataset, and all other original traffic samples in the unlabeled original traffic samples are taken as other traffic samples; based on the traffic sample difference value between the target traffic sample and each other traffic sample, the neighboring traffic samples of the target traffic sample are determined from each other traffic sample; wherein, the traffic sample difference value between the target traffic sample and the corresponding neighboring traffic sample is less than or equal to the difference threshold; based on the traffic label of the target traffic sample and the traffic label of the neighboring traffic samples, samples to be labeled and samples to be removed are selected from the target traffic sample and the neighboring traffic samples; based on the samples to be labeled and the samples to be removed, the original dataset is updated; if the updated original dataset contains only labeled traffic samples or contains one unlabeled original traffic sample, the updated original dataset is used as the training dataset.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.