Network intrusion anomaly detection method, device and equipment based on network performance

Abstract

This invention provides a method, apparatus, and device for detecting network intrusion anomalies based on network performance. The method includes: selecting a minimum set of observation events for the target network performance; obtaining observation results of the minimum set of observation events for the target network performance based on kernel observability technology; inputting the observation results into a pre-trained anomaly detection model to obtain an anomaly score deviation value corresponding to the target network performance; if the anomaly score deviation value exceeds an anomaly deviation threshold, determining that the target network performance is abnormal and extracting network performance anomaly features; determining the type of network performance anomaly (network failure, network intrusion, overload, or hardware failure) based on the network performance anomaly features; if it is a network intrusion type, determining the cause of the anomaly detection result based on the prediction error of the observed events, thereby achieving the detection of network anomaly behavior based on network performance while reducing system resource overhead.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a method, apparatus, and device for detecting network intrusion anomalies based on network performance. Background Technology

[0002] Significant progress has been made in network security protection, but lightweight, real-time network intrusion anomaly detection remains a problem that has not yet been effectively solved.

[0003] Traditional network anomaly detection methods typically rely on predefined thresholds and rules. Manually configuring and managing these rules is not only labor-intensive but also struggles to maintain efficiency in the face of constantly changing network environments. As networks grow in scale and complexity, traditional methods can no longer meet the demands for real-time detection and handling of network intrusion anomalies. The main issues are response time, detection accuracy, adaptability, and data processing capabilities. With machine learning gaining traction in network traffic-based anomaly detection, this method collects and analyzes massive amounts of network traffic feature data, utilizing unsupervised deep learning algorithms to automatically identify normal and abnormal behavior patterns, thus achieving more intelligent and efficient network anomaly detection. It can detect both known and unknown security attacks. However, this method is limited by the large volume of traffic data and high processing and storage requirements. Detection in encrypted traffic and high-traffic environments may be limited. It also has certain shortcomings in terms of lightweightness and real-time performance. Summary of the Invention

[0004] This invention provides a network intrusion anomaly detection method, apparatus, and device based on network performance, which addresses the shortcomings of existing network intrusion anomaly detection technologies in terms of lightweightness and real-time performance, and achieves lightweight and real-time network intrusion anomaly detection.

[0005] This invention provides a network anomaly detection method based on network performance, comprising the following steps: Select the minimum set of observation events for the target network performance; Based on kernel observability technology, the observation results of the minimum set of observation events for the target network performance are obtained; The observation results are input into the pre-trained anomaly detection model to obtain the anomaly score deviation value corresponding to the target network performance; wherein, the anomaly detection model is obtained by unsupervised training based on normal index data of the target network performance over a set period of time; In the anomaly detection model, if the anomaly score deviation value corresponding to the target network performance exceeds the threshold of anomaly deviation, the target network performance is determined to be abnormal, and network performance anomaly features are extracted. Based on the network performance anomaly characteristics, determine the type of the target network performance anomaly; wherein, the type of the target network performance anomaly includes network fault anomaly type, network intrusion anomaly type, load overload anomaly type, or hardware fault anomaly type; If the type causing the abnormal performance of the target network is a network intrusion anomaly, the cause of the abnormal detection result of the target network performance is determined based on the prediction error of the observed event.

[0006] According to the network intrusion anomaly detection method based on network performance provided by the present invention, the step of selecting the minimum set of observation events for the target network performance includes: Obtain multi-dimensional network performance metrics of the host through kernel observability techniques; Select a first multidimensional network performance index for the entire network protocol stack and a second multidimensional network performance index for each protocol stack layer from the multidimensional network performance indices. Principal component analysis or correlation analysis techniques are used to select important observation events characterizing network performance from the multidimensional first network performance index and the multidimensional second network performance index; The multidimensional first network performance index and the multidimensional second network performance index selected from the important observation events are used as the minimum set of observation events.

[0007] According to the present invention, a network intrusion anomaly detection method based on network performance is provided. The minimum observation event set of the target network performance includes network performance indicators for network performance and resource indicators for resources. The network performance indicators include packet loss, latency, bandwidth and jitter of the network protocol stack globally and at each protocol stack layer of the target network performance. The resource indicators include memory utilization and CPU utilization.

[0008] According to the network intrusion anomaly detection method based on network performance provided by the present invention, the step of determining the cause of the anomaly detection result of the target network performance based on the prediction result error of the observed event includes: Based on the prediction error of the observed events, the reconstruction error between the prediction result of each observed event and the corresponding benchmark observation result is calculated; Based on the reconstruction error, the observation events in the minimum observation event set are sorted in descending order, and the top N observation events are selected as the reasons for the abnormal detection results of the target network performance, where N is a positive integer.

[0009] According to the network performance-based network intrusion anomaly detection method provided by the present invention, the training process of the anomaly detection model includes: Obtain multi-dimensional network performance sample metrics over a set time period using kernel observability techniques; Extract the performance sample features of each network performance sample index from the multidimensional network performance sample index; The performance sample features are input into the LSTM-VAE model for unsupervised iterative training until the LSTM-VAE model converges, thus obtaining the anomaly detection model.

[0010] According to the network intrusion anomaly detection method based on network performance provided by the present invention, the observation results of obtaining the minimum set of observation events of the target network performance based on kernel observability technology include: Based on kernel observability technology, the rate of change of each observation event in the minimum observation event set is obtained, and the rate of change is used as the observation result.

[0011] According to the present invention, a network intrusion anomaly detection method based on network performance is provided, wherein determining the type causing the target network performance anomaly based on the network performance anomaly characteristics includes: Based on the aforementioned network performance anomaly characteristics, the SHAP values ​​corresponding to network failure, network intrusion, overload, and hardware failure are calculated respectively. The type of abnormality causing the target network performance is determined based on the SHAP value.

[0012] According to a network intrusion anomaly detection method based on network performance provided by the present invention, the method is applied to a host. After determining the cause of the anomaly detection result of the target network performance based on the prediction result error of the observed event, the method further includes: If the anomaly detection model meets the update conditions, train the target anomaly detection model and replace the current anomaly detection model with the target anomaly detection model; The update conditions include at least one of the following: a change in the network topology corresponding to the host, an adjustment in the services provided by the host, a change in the application scenario or services of the host that meets a preset change condition, or the running time of the anomaly detection model reaching a preset running time requirement.

[0013] The present invention also provides a network intrusion anomaly detection device based on network performance, comprising: The selection module is used to select the minimum set of observation events for the target network performance. The first acquisition module is used to acquire the observation results of the minimum set of observation events for the performance of the target network based on kernel observability technology. The input module is used to input the observation results into the pre-trained anomaly detection model to obtain the anomaly score deviation value corresponding to the target network performance; wherein, the anomaly detection model is obtained by unsupervised training based on normal index data of the target network performance over a set period of time; An extraction module is used to determine that the target network performance is abnormal if the abnormal score deviation value corresponding to the target network performance exceeds the threshold of abnormal deviation, and to extract the abnormal network performance features. The first determining module is used to determine, in the anomaly detection model, the type causing the target network performance anomaly based on the network performance anomaly characteristics; wherein, the type causing the target network performance anomaly includes network fault anomaly type, network intrusion anomaly type, load overload anomaly type, or hardware fault anomaly type; The second determining module is used to determine the cause of the abnormal detection result of the target network performance based on the prediction result error of the observed event if the type causing the abnormality of the target network performance is a network intrusion anomaly type.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the network intrusion anomaly detection method based on network performance as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the network intrusion anomaly detection method based on network performance as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the network intrusion anomaly detection method based on network performance as described above.

[0017] The present invention provides a network intrusion anomaly detection method, apparatus, and device based on network performance. It selects a minimum set of observation events representing the target network performance, and then, based on kernel observability technology, obtains the observation results of this minimum set of observation events for subsequent anomaly detection processes. This reduces the number of observation results input into the anomaly detection model, achieving lightweight network intrusion anomaly detection. After determining the minimum set of observation events, the observation results are acquired in real time and input into the anomaly detection model for detection, obtaining the anomaly score deviation value corresponding to the target network performance. The anomaly detection model is obtained through unsupervised training on indicator data over a set period, making it applicable to network intrusion anomaly detection for different services. It features loose coupling and is suitable for various host network services. Furthermore, by using the anomaly detection model to learn and extract latent feature rules, rather than relying on specific rule construction, it is more suitable for detecting unknown anomalies, improving detection accuracy and robustness, efficiently performing network intrusion anomaly detection, and avoiding excessive system resource overhead. When the abnormal score deviation value exceeds the abnormal deviation threshold, the network performance of the target network is determined to be abnormal. The abnormal network performance features are further extracted, and the type of abnormal target network performance caused by the abnormal network performance features is determined. When the type of abnormal target network performance is detected to be a network intrusion anomaly, the cause of the abnormal detection result of the target network performance is determined based on the prediction result error of the observed event, so as to realize the detection of possible network intrusion anomalies. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts of the network intrusion anomaly detection method based on network performance provided by the present invention.

[0020] Figure 2 This is the second flowchart of the network intrusion anomaly detection method based on network performance provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the network intrusion anomaly detection method based on network performance provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the network intrusion anomaly detection device based on network performance provided by the present invention.

[0023] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] The following is combined with Figures 1 to 3 This invention describes a network intrusion anomaly detection method based on network performance.

[0026] Figure 1 This is a flowchart illustrating a network intrusion anomaly detection method based on network performance, according to an exemplary embodiment. Figure 1 As shown, this network intrusion anomaly detection method based on network performance includes steps 110 to 160, which are described in detail below.

[0027] Step 110: Select the minimum set of observation events for the target network performance.

[0028] In this embodiment of the invention, the target network performance has observation events with multiple dimensions. The observation events that best characterize the network performance are extracted from the observation events with multiple dimensions to form a minimum set of observation events.

[0029] Step 120: Based on kernel observability technology, obtain the observation results of the minimum set of observation events for the target network performance.

[0030] In this embodiment of the invention, after selecting the minimum set of observed events, subsequent detection only needs to acquire the observation results of each observation event in the minimum set in real time, without needing to acquire the observation results of all observation events. The time interval between acquiring the observation results and inputting the observation results into the anomaly detection model for detection can be set to 1 second.

[0031] The embodiments of the present invention extract only the observation events that best characterize network performance for subsequent detection processes, resulting in fewer observation results being input into the anomaly detection model, thus achieving lightweight network intrusion anomaly detection.

[0032] Step 130: Input the observation results into the trained anomaly detection model to obtain the anomaly score deviation value corresponding to the target network performance; wherein, the anomaly detection model is obtained by unsupervised training based on the normal index data of the target network performance over a set period of time.

[0033] In this embodiment of the application, after obtaining the observation results, the observation results are then input into the anomaly detection model for detection to obtain the corresponding anomaly score deviation value. The anomaly score deviation value can characterize whether the network performance of the corresponding observation event is abnormal.

[0034] The anomaly detection model can predict the next moment's result based on the currently acquired observation results, calculate the reconstruction error of each observation event based on the prediction results, and calculate the anomaly score deviation value based on the reconstruction error of all observation events.

[0035] The anomaly detection model is trained based on network performance metrics data from unlabeled normal states over a preset time period. This allows the model to be applied to network anomaly detection for various services, exhibiting loose coupling and applicability to various host network services, thus improving the flexibility and versatility of the method provided in this invention. Furthermore, by learning and extracting latent feature rules using the anomaly detection model, rather than relying on specific rule construction, it is more suitable for detecting unknown anomalies, improving detection accuracy and robustness.

[0036] Step 140: In the anomaly detection model, if the anomaly score deviation value corresponding to the target network performance exceeds the threshold of the anomaly deviation, the target network performance is determined to be abnormal, and the network performance anomaly features are extracted.

[0037] In this embodiment of the invention, in the anomaly detection model, after determining the anomaly score deviation value, the anomaly score deviation value is compared with a preset anomaly deviation threshold. If the anomaly score deviation value is greater than the anomaly deviation threshold, it can be determined that the target network performance is abnormal; otherwise, it can be determined that the target network performance is normal.

[0038] After identifying the target network performance anomaly, further extract the network performance anomaly features. Specifically, the type of anomaly causing the target network performance anomaly can be detected based on the network performance anomaly features, such as network failure, network intrusion, load overload, and hardware failure. Each type of network performance anomaly has network performance anomaly features that can characterize the corresponding characteristics.

[0039] For network fault anomaly types, they are characterized by high packet loss rate, increased latency, and unavailability of specific paths or devices. Therefore, network performance anomaly characteristics such as Ping response time can be used to check whether it belongs to a network fault anomaly type.

[0040] Network intrusion anomalies are characterized by sudden surges in traffic, abnormal connections, and port scanning. Therefore, network performance anomalies such as traffic performance peaks and surges in traffic performance on specific ports can be used to check whether a network intrusion anomaly is involved.

[0041] For overload anomalies, the characteristic is that resource utilization is consistently high (such as CPU, memory, and bandwidth). Therefore, network performance anomalies such as CPU utilization, memory utilization, and network bandwidth can be used to check whether it belongs to the overload anomaly type.

[0042] Hardware fault anomaly types are characterized by a high interface error rate. Therefore, abnormal features such as interface error statistics can be used to check whether it belongs to the hardware fault anomaly type.

[0043] Step 150: Determine the type of network performance anomaly caused by the network performance anomaly based on the network performance anomaly characteristics; wherein the type of network performance anomaly caused by the network performance anomaly includes network fault anomaly type, network intrusion anomaly type, load overload anomaly type, or hardware fault anomaly type.

[0044] In this embodiment, feature importance analysis (such as SHAP value) is used to distinguish between network performance anomalies such as network failure, network intrusion, overload, and hardware failure. This can help understand the basis for model decision-making and classify anomaly types according to the importance of different features.

[0045] Step 160: If the type causing the abnormal performance of the target network is a network intrusion anomaly, determine the cause of the abnormal detection result of the target network performance based on the prediction result error of the observed event.

[0046] In this embodiment of the application, if the detected abnormality type of the target network performance is a network intrusion abnormality type, the cause of the abnormality detection result of the target network performance is determined based on the prediction result error of the observed event.

[0047] In this embodiment of the invention, a minimum set of observation events for the target network performance is selected. Then, based on kernel observability technology, the observation results of this minimum set of observation events are obtained for subsequent detection processes. This reduces the number of observation results input into the anomaly detection model, achieving lightweight network anomaly detection. After determining the minimum set of observation events, the observation results are acquired in real time and input into the anomaly detection model for detection, obtaining the anomaly score deviation value corresponding to the target network performance. The anomaly detection model is obtained through unsupervised training on indicator data over a set period, making it applicable to network intrusion anomaly detection for different services. It features loose coupling and is suitable for various host network services. Furthermore, the anomaly detection model learns and extracts latent feature rules instead of relying on specific rule construction, making it more suitable for detecting unknown anomalies, improving detection accuracy and robustness, and efficiently performing network intrusion anomaly detection while avoiding excessive system resource overhead. When the abnormal score deviation value exceeds the abnormal deviation threshold, the target network performance is determined to be abnormal. The abnormal network performance features are further extracted, and the type causing the abnormal target network performance is determined based on the abnormal network performance features. When the type causing the abnormal target network performance is detected to be a network intrusion anomaly, the abnormal detection result of the target network performance is determined based on the observation results, thereby realizing the detection of possible network intrusion anomalies.

[0048] In an exemplary embodiment of this application, step 110, which involves selecting the minimum set of observation events for the target network performance, includes the following steps, which are detailed below.

[0049] Obtain multidimensional network performance metrics through kernel observability techniques.

[0050] From the multidimensional network performance indicators, select a first multidimensional network performance indicator for the entire network protocol stack and a second multidimensional network performance indicator for each protocol stack layer of the network protocol stack.

[0051] Principal component analysis or correlation analysis techniques are used to select important observation events characterizing network performance from the multidimensional first network performance index and the multidimensional second network performance index.

[0052] The multidimensional first network performance index and the multidimensional second network performance index selected from the important observation events are used as the minimum set of observation events.

[0053] In this embodiment of the invention, kernel observability technology is used to capture network performance indicators. Kernel observability technology can employ methods such as kprobe / tracepoint. Kernel observability technology does not require modification of the kernel source code, which facilitates the maintenance and updating of the system ecosystem.

[0054] Multidimensional network performance metrics are obtained through kernel observability techniques, and then multidimensional first network performance metrics for the entire network protocol stack and multidimensional second network performance metrics for each protocol stack layer are extracted from the multidimensional network performance metrics.

[0055] Specifically, correlation analysis or feature importance scoring is used to select the most representative multidimensional first network performance index and multidimensional second network performance index as important observation events.

[0056] Correlation analysis refers to the analysis of two or more correlated variables to measure the degree of correlation between the two variables. When selecting multidimensional first-level and second-level network performance indicators through correlation analysis, correlation analysis is performed on multiple first-level network performance indicators of the network protocol stack with network performance status. Based on the results of the correlation analysis, several first-level network performance indicators with high correlation are selected to form the multidimensional first-level network performance indicator. Simultaneously, correlation analysis is performed on multiple second-level network performance indicators of each layer of the network protocol stack with network performance status. Based on the results of the correlation analysis, several second-level network performance indicators with high correlation are selected to form the multidimensional second-level network performance indicator.

[0057] When selecting multidimensional first and second network performance indicators using feature importance scoring, a pre-set feature importance detection model is used via machine learning. Multiple first network performance indicators for the network protocol stack are input into the feature importance detection model to calculate their corresponding feature importance values. These values ​​are then sorted in descending order, and the top M (M being a positive integer) first network performance indicators are selected to form the multidimensional first network performance indicator. Simultaneously, multiple second network performance indicators for each layer of the network protocol stack are input into the feature importance detection model to calculate their corresponding feature importance values. These values ​​are then sorted in descending order, and the top M second network performance indicators are selected to form the multidimensional second network performance indicator. The selected multidimensional first and second network performance indicators are then combined to form a minimum set of observed events.

[0058] In this embodiment of the invention, by fully utilizing the high-performance characteristics of kernel observability technology, combined with the ingenious selection of the minimum observation event set and the dimensionality reduction strategy of the unsupervised model, a lightweight performance advantage is successfully achieved, significantly reducing system resource overhead. This invention focuses more on network performance metrics, providing the system with more accurate and effective monitoring and analysis.

[0059] In this embodiment of the invention, a multi-dimensional first network performance index is considered, taking into account both the overall network protocol stack's network performance and the multi-dimensional second network performance index for each layer (application layer, transport layer, network layer, and data link layer) within the network protocol stack. This embodiment forms a minimum set of observation events from two perspectives: performance categories and the performance of different layers within the network.

[0060] In an exemplary embodiment of the present invention, the minimum set of observation events for the target network performance includes network performance metrics for network performance and resource metrics for resources. The network performance metrics include packet loss, latency, bandwidth, and jitter of the network protocol stack globally and at each protocol stack layer of the target network performance. The resource metrics include memory utilization and CPU utilization.

[0061] In this embodiment of the invention, the multidimensional first network performance indicator and the multidimensional second network performance indicator respectively include packet loss, latency, jitter, and bandwidth. These network performance indicators can more easily locate anomalies in network intrusions, helping administrators quickly identify security issues and take corresponding measures. The minimum set of observed events, consisting of the multidimensional first network performance indicator, the multidimensional second network performance indicator, memory usage, and CPU usage, is input into the anomaly detection model for detection, thereby obtaining an anomaly score deviation value.

[0062] In an exemplary embodiment of the present invention, determining the cause of the abnormal detection result of the target network performance based on the prediction result error of the observed event includes the following steps, which are described in detail below.

[0063] Based on the prediction error of the observed events, the reconstruction error between the prediction result of each observed event and the corresponding baseline observation result is calculated.

[0064] Based on the reconstruction error, the observation events in the minimum observation event set are sorted in descending order, and the top N observation events are selected as the reasons for the abnormal detection results of the target network performance, where N is a positive integer.

[0065] In this embodiment, the reconstruction error between the predicted result of each observed event and the corresponding baseline observation result is calculated. The baseline observation result is the observation result of the network performance sample index corresponding to the normal network performance of the target network when training the anomaly detection model. The reconstruction error can be calculated by mean squared error or cross-entropy loss, thereby quantifying the difference between the current network performance index and the normal network performance index. After calculating the reconstruction error corresponding to each observed event, the observed events are sorted in descending order based on the reconstruction error. According to the sorting order, the top N observed events are selected as the causes of the anomaly detection results of the target network performance and output. These are then stored in the system log to facilitate security personnel in quickly locating the problem.

[0066] In an exemplary embodiment of the present invention, the training process of the anomaly detection model includes the following steps, which are described in detail below.

[0067] Obtain multidimensional network performance sample metrics over a set time period using kernel observability techniques.

[0068] Extract the performance sample features of each network performance sample index from the multidimensional network performance sample index.

[0069] The performance sample features are input into the LSTM-VAE model for unsupervised iterative training until the LSTM-VAE model converges, thus obtaining the anomaly detection model.

[0070] In this embodiment of the invention, unlabeled multidimensional network performance sample indicators of the host's normal behavior over a set period of time (24 hours or 72 hours) are obtained using kernel observability technology. These multidimensional network performance sample indicators are then normalized or standardized to ensure the stability of the model training. Performance sample features of each network performance sample indicator are then extracted from the multidimensional network performance sample indicators; these performance sample features are also the rates of change described later. The extracted performance sample features are then input into the LSTM-VAE model for unsupervised iterative training. Utilizing unsupervised learning, no labeling of the network performance sample indicators is required during training, resulting in a more widely applicable anomaly detection model while reducing the workload of manual labeling.

[0071] In the LSTM-VAE model provided in this embodiment, VAE encoding is first performed within an original time window and embedded into the LSTM to predict the embedding of the next window. Decoding is then performed to obtain the reconstructed window. During this process, the anomaly detection value represents the prediction error. The advantages of this implementation are dimensionality reduction, lower computational complexity, and enhanced robustness. Furthermore, the model combines local and global information, improving detection accuracy. During training, the default ELBO loss function is used.

[0072] After confirming that the LSTM-VAE model has converged, the trained model is validated using an independent validation set. During the validation process, overfitting and other performance metrics are monitored. Once the validation is successful, the anomaly detection model is obtained. If the validation fails, the model is trained again until it is successfully validated.

[0073] In the LSTM-VAE model, LSTM (Long Short Term Memory) considers the correlation of time, while VAE (Variational Auto Encoder) reduces the dimensionality of network performance observation events, thus achieving lightweight operation.

[0074] In an exemplary embodiment of the present invention, the observation results of obtaining the minimum set of observation events for the target network performance based on kernel observability technology include the following steps, which are described in detail below.

[0075] Based on kernel observability technology, the rate of change of each observation event in the minimum observation event set is obtained, and the rate of change is used as the observation result.

[0076] In this embodiment of the invention, kernel observability technology is used to calculate the rate of change of each observation event in the minimum set of observation events. This rate of change is calculated by subtracting the previously acquired network performance metric from the currently acquired data and dividing the result by the previously acquired data. The rate of change of each observation event is then used as the corresponding performance feature, i.e., the corresponding observation result.

[0077] In an exemplary embodiment of the present invention, determining the type of network performance anomaly based on the network performance anomaly characteristics includes the following steps, which are described in detail below.

[0078] Based on the aforementioned network performance anomaly characteristics, the SHAP values ​​corresponding to network failure, network intrusion, overload, and hardware failure are calculated respectively.

[0079] The type of abnormality causing the target network performance is determined based on the SHAP value.

[0080] In this embodiment, the SHAP (SHapley Additive exPlanations) value is a unified feature importance metric that can explain the contribution of each feature to determining the anomaly type. For each extracted network performance anomaly feature, its SHAP value is calculated to determine which features have the greatest impact on the anomaly detection results. Features with high SHAP values ​​indicate that the feature has a significant impact on the detected anomaly. Therefore, when a target network performance anomaly is detected, the SHAP value of the network performance feature corresponding to the network performance anomaly type is calculated in real time. Based on the SHAP value, important network performance features for each network performance anomaly type are determined, and the target network performance anomaly type is determined based on the determined important network performance features.

[0081] In an exemplary embodiment of the present invention, the network anomaly detection method based on network performance is applied to a host. After determining the cause of the anomaly detection result of the target network performance based on the prediction result error of the observed event, the method further includes the following steps, which are described in detail below.

[0082] If the anomaly detection model meets the update conditions, a target anomaly detection model is trained, and the current anomaly detection model is replaced with the target anomaly detection model.

[0083] The update conditions include at least one of the following: a change in the network topology corresponding to the host, an adjustment in the services provided by the host, a change in the application scenario or services of the host that meets a preset change condition, or the running time of the anomaly detection model reaching a preset running time requirement.

[0084] In this embodiment of the invention, if the network topology of the host changes, the host's services are adjusted, the host's application scenarios and services change significantly (such as an increase in the number of existing LAN servers or a company re-planning the network), or the anomaly detection model's running time reaches the preset running time requirement (three months), the anomaly detection model is retrained to obtain the target anomaly detection model. The currently used anomaly detection model is then replaced with the target anomaly detection model to maintain the model's adaptability. By retraining the anomaly detection module, the model can adapt to the new network environment and maintain its flexibility and accuracy in the face of changes.

[0085] In one exemplary embodiment of this application, please refer to Figure 2 and Figure 3The process involves extracting the minimum observation set of the host and using kernel observability technology to acquire multidimensional network performance sample indicators corresponding to this minimum observation set over a set time. The rate of change of each network performance sample indicator is then extracted as its performance sample feature. Data preprocessing is performed to form a training dataset, which is then input into an LSTM-VAE model for unsupervised iterative training. The training loss value is calculated, and the training completion status of the LSTM-VAE model is determined based on the loss value. If training is incomplete, iterative training continues. If training is complete, an anomaly detection model is obtained. This model is then used for online network anomaly detection to obtain the corresponding target network performance. The system detects whether the abnormal score deviation value is greater than the abnormal deviation threshold. If it is, the target network performance is determined to be abnormal. The abnormal network performance features are extracted, and the type causing the abnormal network performance is determined based on the extracted features. If the type causing the abnormal network performance is a network intrusion anomaly, the reconstruction error corresponding to the observed events in the minimum observation event set is calculated based on the observation results. The observation events in the minimum observation event set are sorted in descending order based on the reconstruction error, and the top 5 observation events are selected as the abnormal detection results of the target network performance. The system checks whether the online anomaly detection should end. If it does not end, the system returns to the online network anomaly detection steps; otherwise, the network anomaly detection ends.

[0086] The following describes the network intrusion anomaly detection device based on network performance provided by the present invention. The network intrusion anomaly detection device based on network performance described below can be referred to in correspondence with the network anomaly detection method based on network performance described above. It should be noted that the device provided in the following embodiments belongs to the same concept as the method provided in the above embodiments, and the specific way in which each module and unit performs its operation has been described in detail in the method embodiments, and will not be repeated here.

[0087] In one exemplary embodiment of the present invention, please refer to Figure 4 , Figure 4 This is an exemplary embodiment of a network intrusion anomaly detection device based on network performance, comprising the following modules.

[0088] Module 410 is used to select the minimum set of observation events for the target network performance.

[0089] The first acquisition module 420 is used to acquire the observation results of the minimum set of observation events for the target network performance based on kernel observability technology.

[0090] The input module 430 is used to input the observation results into the pre-trained anomaly detection model to obtain the anomaly score deviation value corresponding to the target network performance; wherein, the anomaly detection model is obtained by unsupervised training based on the normal index data of the target network performance over a set period of time.

[0091] The first determining module 440 is used to determine that the target network performance is abnormal and extract network performance abnormal features if the abnormal score deviation value corresponding to the target network performance exceeds the threshold of abnormal deviation in the anomaly detection model.

[0092] The second determining module 450 is used to determine the type of target network performance abnormality based on the network performance abnormality characteristics; wherein the type of target network performance abnormality includes network fault abnormality type, network intrusion abnormality type, load overload abnormality type or hardware fault abnormality type.

[0093] The third determining module 460 is used to determine the cause of the abnormal detection result of the target network performance based on the prediction error of the observed event if the type causing the abnormality of the target network performance is a network intrusion anomaly type.

[0094] In an exemplary embodiment of the present invention, module 410 is selected, including the following sub-modules.

[0095] The Acquisition submodule is used to acquire multidimensional network performance metrics through kernel observability techniques.

[0096] The extraction submodule is used to select a first multidimensional network performance indicator for the entire network protocol stack and a second multidimensional network performance indicator for each protocol stack layer of the network protocol stack from the multidimensional network performance indicators.

[0097] The first selection submodule is used to select important observation events characterizing network performance from the multidimensional first network performance index and the multidimensional second network performance index using principal component analysis or correlation analysis techniques.

[0098] As a submodule, it is used to take the multidimensional first network performance index and the multidimensional second network performance index selected for the important observation events as the minimum set of observation events.

[0099] In an exemplary embodiment of the present invention, the minimum set of observation events for the target network performance includes network performance metrics for network performance and resource metrics for resources. The network performance metrics include packet loss, latency, bandwidth, and jitter of the network protocol stack globally and at each protocol stack layer of the target network performance. The resource metrics include memory utilization and CPU utilization.

[0100] In one exemplary embodiment of the present invention, the third determining module 460 includes the following sub-modules.

[0101] The first calculation submodule is used to calculate the reconstruction error between the prediction result of each observation event and the corresponding benchmark observation result based on the prediction error of the observation event.

[0102] The second selection submodule is used to sort the observation events in the minimum observation event set in descending order according to the reconstruction error, and select the top N observation events as the reasons for the abnormal detection results of the target network performance, where N is a positive integer.

[0103] In an exemplary embodiment of the present invention, the training process of the anomaly detection model includes the following modules.

[0104] The second acquisition module is used to acquire multidimensional network performance sample metrics over a set time period through kernel observability technology.

[0105] The extraction module is used to extract the performance sample features of each network performance sample index from the multidimensional network performance sample index.

[0106] The training module is used to input the performance sample features into the LSTM-VAE model for unsupervised iterative training until the LSTM-VAE model converges, thereby obtaining the anomaly detection model.

[0107] In an exemplary embodiment of the present invention, the first acquisition module 420 includes the following sub-modules.

[0108] The second calculation submodule is used to obtain the rate of change of each observation event in the minimum observation event set based on kernel observability technology, and use the rate of change as the observation result.

[0109] In an exemplary embodiment of the present invention, the second determining module 450 includes the following sub-modules.

[0110] The third calculation submodule is used to calculate the SHAP values ​​corresponding to network failure, network intrusion, overload and hardware failure based on the network performance anomaly characteristics. The determination submodule is used to determine the type of abnormality causing the target network performance based on the SHAP value.

[0111] In an exemplary embodiment of the present invention, the network anomaly intrusion detection device based on network performance further includes the following modules.

[0112] The replacement module is used to train a target anomaly detection model and replace the current anomaly detection model with the target anomaly detection model if the anomaly detection model meets the update conditions. The update conditions include at least one of the following: a change in the network topology corresponding to the host, an adjustment in the services provided by the host, a change in the application scenario or services of the host that meets a preset change condition, or the running time of the anomaly detection model reaching a preset running time requirement.

[0113] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logic instructions in the memory 530 to execute a network intrusion anomaly detection method based on network performance. This method includes: selecting a minimum set of observation events for the target network performance; obtaining observation results of the minimum set of observation events for the target network performance based on kernel observability technology; inputting the observation results into a pre-trained anomaly detection model to obtain an anomaly score deviation value corresponding to the target network performance; wherein the anomaly detection model is obtained through unsupervised training based on normal index data of the target network performance over a set period; in the anomaly detection model, if the anomaly score deviation value corresponding to the target network performance exceeds an anomaly deviation threshold, the target network performance is determined to be abnormal, and network performance anomaly features are extracted; based on the network performance anomaly features, the type causing the target network performance anomaly is determined; wherein the type causing the target network performance anomaly includes network fault anomaly type, network intrusion anomaly type, load overload anomaly type, or hardware fault anomaly type; if the type causing the target network performance anomaly is a network intrusion anomaly type, the cause of the anomaly detection result of the target network performance is determined based on the prediction result error of the observed events.

[0114] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0115] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the network intrusion anomaly detection method based on network performance provided by the above methods. The method includes: selecting a minimum set of observation events for the target network performance; obtaining the observation results of the minimum set of observation events for the target network performance based on kernel observability technology; inputting the observation results into a pre-trained anomaly detection model to obtain an anomaly score deviation value corresponding to the target network performance; wherein, the anomaly detection model is based on the target network performance at a set time. The abnormality detection model is obtained through unsupervised training on long-term normal indicator data. If the abnormal score deviation value corresponding to the target network performance exceeds the abnormal deviation threshold, the target network performance is determined to be abnormal, and abnormal network performance features are extracted. Based on the abnormal network performance features, the type causing the abnormality is determined. The types causing the abnormality include network fault anomalies, network intrusion anomalies, overload anomalies, or hardware fault anomalies. If the type causing the abnormality is a network intrusion anomaly, the cause of the abnormality detection result is determined based on the prediction error of the observed event.

[0116] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the network intrusion anomaly detection method based on network performance provided by the above methods. This method includes: selecting a minimum set of observation events for target network performance; obtaining observation results of the minimum set of observation events for target network performance based on kernel observability technology; inputting the observation results into a pre-trained anomaly detection model to obtain an anomaly score deviation value corresponding to the target network performance; wherein the anomaly detection model is obtained through unsupervised training based on normal index data of the target network performance over a set period; in the anomaly detection model, if the anomaly score deviation value corresponding to the target network performance exceeds an anomaly deviation threshold, determining that the target network performance is abnormal, and extracting network performance anomaly features; determining the type causing the target network performance anomaly based on the network performance anomaly features; wherein the type causing the target network performance anomaly includes network fault anomaly type, network intrusion anomaly type, load overload anomaly type, or hardware fault anomaly type; if the type causing the target network performance anomaly is a network intrusion anomaly type, determining the cause of the anomaly detection result of the target network performance based on the prediction result error of the observed events.

[0117] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A network intrusion anomaly detection method based on network performance, characterized in that, include: Select the minimum set of observation events for the target network performance; Based on kernel observability technology, the observation results of the minimum set of observation events for the target network performance are obtained; The observation results are input into the pre-trained anomaly detection model to obtain the anomaly score deviation value corresponding to the target network performance; wherein, the anomaly detection model is obtained by unsupervised training based on normal index data of the target network performance over a set period of time; the anomaly detection model is an LSTM-VAE model, which performs VAE encoding within an original time window and embeds it into LSTM, predicts the embedding of the next window, and then decodes to obtain the reconstructed window; In the anomaly detection model, if the anomaly score deviation value corresponding to the target network performance exceeds the threshold of anomaly deviation, the target network performance is determined to be abnormal, and network performance anomaly features are extracted. Based on the network performance anomaly characteristics, determine the type of the target network performance anomaly; wherein, the type of the target network performance anomaly includes network fault anomaly type, network intrusion anomaly type, load overload anomaly type, or hardware fault anomaly type; If the type causing the abnormal performance of the target network is a network intrusion anomaly, the cause of the abnormal detection result of the target network performance is determined based on the prediction result error of the observed event; The minimum set of observation events for selecting the target network performance includes: Obtain multi-dimensional network performance metrics through kernel observability techniques; Select a first multidimensional network performance index for the entire network protocol stack and a second multidimensional network performance index for each protocol stack layer from the multidimensional network performance indices. Principal component analysis or correlation analysis techniques are used to select important observation events characterizing network performance from the multidimensional first network performance index and the multidimensional second network performance index. The multidimensional first network performance index and the multidimensional second network performance index selected for the important observation events are used as the minimum set of observation events; The step of determining the type causing the target network performance anomaly based on the network performance anomaly characteristics includes: Based on the aforementioned network performance anomaly characteristics, the SHAP values ​​corresponding to network failure, network intrusion, overload, and hardware failure are calculated respectively. The type of abnormality causing the target network performance is determined based on the SHAP value.

2. The network intrusion anomaly detection method based on network performance according to claim 1, characterized in that, The minimum set of observation events for the target network performance includes network performance metrics for network performance and resource metrics for resources. The network performance metrics include packet loss, latency, bandwidth, and jitter of the network protocol stack globally and at each protocol stack layer of the target network performance. The resource metrics include memory utilization and CPU utilization.

3. The network intrusion anomaly detection method based on network performance according to claim 1, characterized in that, The determination of the cause of the abnormal detection result of the target network performance based on the prediction result error of the observed event includes: Based on the prediction error of the observed events, the reconstruction error between the prediction result of each observed event and the corresponding benchmark observation result is calculated. Based on the reconstruction error, the observation events in the minimum observation event set are sorted in descending order, and the top N observation events are selected as the reasons for the abnormal detection results of the target network performance, where N is a positive integer.

4. The network intrusion anomaly detection method based on network performance according to claim 1, characterized in that, The training process of the anomaly detection model includes: Obtain multi-dimensional network performance sample metrics over a set time period using kernel observability techniques; Extract the performance sample features of each network performance sample index from the multidimensional network performance sample index; The performance sample features are input into the LSTM-VAE model for unsupervised iterative training until the LSTM-VAE model converges, thus obtaining the anomaly detection model.

5. The network intrusion anomaly detection method based on network performance according to claim 1, characterized in that, The observation results for obtaining the minimum set of observation events for the target network performance based on kernel observability technology include: Based on kernel observability technology, the rate of change of each observation event in the minimum observation event set is obtained, and the rate of change is used as the observation result.

6. The network intrusion anomaly detection method based on network performance according to any one of claims 1 to 5, characterized in that, The network intrusion anomaly detection method based on network performance is applied to a host. After determining the cause of the anomaly detection result of the target network performance based on the prediction result error of the observed event, the method further includes: If the anomaly detection model meets the update conditions, train the target anomaly detection model and replace the current anomaly detection model with the target anomaly detection model; The update conditions include at least one of the following: a change in the network topology corresponding to the host, an adjustment in the services provided by the host, a change in the application scenario or services of the host that meets a preset change condition, or the running time of the anomaly detection model reaching a preset running time requirement.

7. A network intrusion anomaly detection device based on network performance, characterized in that, include: The selection module is used to select the minimum set of observation events for the target network performance. The first acquisition module is used to acquire the observation results of the minimum set of observation events for the performance of the target network based on kernel observability technology. The input module is used to input the observation results into the pre-trained anomaly detection model to obtain the anomaly score deviation value corresponding to the target network performance; wherein, the anomaly detection model is obtained by unsupervised training based on normal index data of the target network performance over a set period of time; the anomaly detection model is an LSTM-VAE model, which performs VAE encoding within an original time window and embeds it into an LSTM, predicts the embedding of the next window, and then decodes it to obtain the reconstructed window; The first determining module is used to determine that the target network performance is abnormal and extract network performance abnormal features if the abnormal score deviation value corresponding to the target network performance exceeds the threshold of abnormal deviation in the anomaly detection model. The second determining module is used to determine the type of network performance anomaly caused by the network performance anomaly based on the network performance anomaly characteristics; wherein the type of network performance anomaly caused by the network performance anomaly includes network fault anomaly type, network intrusion anomaly type, load overload anomaly type, or hardware fault anomaly type; The third determining module is used to determine the cause of the abnormal detection result of the target network performance based on the prediction result error of the observed event if the type causing the abnormal performance of the target network is a network intrusion anomaly. Select modules, including: The acquisition submodule is used to acquire multi-dimensional network performance metrics through kernel observability techniques; The extraction submodule is used to select a multi-dimensional first network performance indicator for the entire network protocol stack and a multi-dimensional second network performance indicator for each protocol stack layer of the network protocol stack from the multi-dimensional network performance indicators. The first selection submodule is used to select important observation events characterizing network performance from the multidimensional first network performance index and the multidimensional second network performance index using principal component analysis or correlation analysis techniques. The second determining module includes: The third calculation submodule is used to calculate the SHAP values ​​corresponding to network failure, network intrusion, overload and hardware failure based on the network performance anomaly characteristics. The determination submodule is used to determine the type of abnormality causing the target network performance based on the SHAP value.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the network intrusion anomaly detection method based on network performance as described in any one of claims 1 to 6.

Images