Network operation and maintenance monitoring system based on data analysis

Abstract

The invention relates to the technical field of network monitoring, in particular to a network operation and maintenance monitoring system based on data analysis, and the system comprises a data collection module which is used for collecting various types of data in a network in real time, and the various types of data comprise but are not limited to flow data, performance indexes and log files; the data processing module is used for processing various kinds of collected data, the data processing module is used for carrying out standardized processing on features, and the data analysis module is used for identifying abnormal modes in a network based on various kinds of data processed by the data processing module and sending the abnormal modes to the data processing module. The data collection unit is used for collecting data, judging whether the data is abnormal or not and analyzing the collected data by adopting a machine learning algorithm, the anomaly detection unit is used for identifying abnormal behaviors in the network, and the trend prediction unit is used for predicting the network performance trend; and the early warning module is used for issuing early warning information based on a result of the data judgment module.

Description

Technical Field

[0001] The present invention relates to the technical field of network monitoring, and in particular to a network operation and maintenance monitoring system based on data analysis. Background Art

[0002] With the popularization of the Internet and the expansion of enterprise networks, the scale and complexity of networks are increasing. Modern networks involve multiple geographical locations and the connection of hundreds or even thousands of devices, including servers, routers, switches, firewalls, etc. Monitoring these devices and the interactions between them, and maintaining the operation and maintenance monitoring network have become the key to ensuring the stable operation of the network;

[0003] A computer security operation and maintenance system based on big data is disclosed in announcement number CN118573433A, including a computer security operation and maintenance system, a big data monitoring system, a big data analysis system, a risk processing system and a security protection update system. The big data monitoring system is arranged inside the computer security operation and maintenance system, and the big data analysis system is arranged inside the computer security operation and maintenance system.

[0004] Network operation and maintenance monitoring includes monitoring network devices, servers, applications, and services. With the development of network technology, the amount of data that network operation and maintenance monitoring systems need to process is increasing. Traditional monitoring systems can no longer meet current needs and lack effective data analysis capabilities, making it impossible to identify and respond to abnormal situations in the network in a timely and accurate manner.

[0005] In view of the above technical defects, a network operation and maintenance monitoring system based on data analysis is proposed. Summary of the invention

[0006] The purpose of the present invention is to provide a network operation and maintenance monitoring system based on data analysis to solve the above-mentioned technical defects.

[0007] The purpose of the present invention can be achieved through the following technical solutions: a network operation and maintenance monitoring system based on data analysis, comprising

[0008] Data collection module: The data collection module is used to collect various types of data in the network in real time, including but not limited to flow data, performance indicators, and log files;

[0009] Data processing module: The data processing module is used to process various types of collected data. The data processing module processes the data by standardizing the features.

[0010] Data analysis module: The data analysis module identifies abnormal patterns in the network based on various types of data processed by the data processing module, and determines whether the data is abnormal. It uses machine learning algorithms to analyze the collected data. The anomaly detection unit is used to identify abnormal behaviors in the network, and the trend prediction unit is used to predict network performance trends;

[0011] Early warning module: The early warning module issues early warning information based on the results of the data judgment module.

[0012] Preferably, the data analysis module identifies abnormal patterns in the network, the abnormal patterns including: abnormal network traffic, abnormal network packet loss rate and abnormal device performance;

[0013] The abnormal risks of the network are judged based on abnormal network traffic, abnormal network packet loss rate and abnormal device performance. The abnormal risks include Class I risk, Class II risk and Class III risk.

[0014] Among them, the harmfulness is that Class I risk is less than Class II risk and less than Class III risk. Class I risk includes network traffic anomaly; Class II risk includes network traffic anomaly and device performance anomaly; Class III risk includes network traffic anomaly, network packet loss rate anomaly and device performance anomaly;

[0015] The early warning module outputs different early warning information based on Class I risk, Class II risk and Class III risk, and the different early warning information includes Class A information, Class B information and Class C information;

[0016] Preferably, the data analysis module in the risk category I analyzes whether the network data is abnormal in the following manner:

[0017] S1. Collect network traffic data through the data collection module. The network traffic data includes inbound and outbound traffic, packet size, and transport layer protocol type; (use network monitoring tools to capture network traffic data);

[0018] S2. Extract features from the raw traffic data through the data processing module, such as traffic size (such as bits per second), number of packets, session duration, specific port traffic, and specific IP address traffic; process the traffic data through data analysis tools, such as Python's Pandas library;

[0019] S3, Standardization: Standardize the features to eliminate the dimensional effects between different features;

[0020] S4. The data analysis module calculates the distance between the test data point and the nearest neighbor data point in the training set. If the distance from the test data point to the nearest neighbor is greater than the set threshold, the data point is abnormal, and the early warning module outputs Class A early warning information. The KNN algorithm is used to calculate the distance between the test data point and the known data point.

[0021] Preferably, based on step S4, the specific calculation method of the distance between the test data point and the nearest neighbor data point in the training set is:

[0022] Obtain the characteristic value x of the test data point through the data acquisition module i , the eigenvalue y of the nearest neighbor sample i , calculated by the formula where x i is the ith eigenvalue of the test data point, y i is the ith eigenvalue of the nearest neighbor sample, σ i is the standard deviation of the ith feature, D1 is the distance between the test data point and the nearest neighbor data point in the training set. If D1 is greater than the set threshold, it is considered abnormal. It should be noted that this threshold can be set based on historical data and business needs. It should also be noted that the eigenvalue is the data that describes the various dimensions of the data point. The eigenvalue can be various indicators of network traffic data, such as traffic size, number of data packets, session duration, connection rate, port usage, etc.

[0023] Preferably, if the network traffic is abnormal and the warning module outputs Class A warning information, the data analysis module is further used to continue analyzing whether the device performance is abnormal, specifically:

[0024] According to the data collection module, the CPU usage rate, memory usage rate, etc. of the device performance are collected, and a data set [P1, P2, ..., P n ];

[0025] For each data point P i Calculate the LOF (P) of the corresponding data point i ) value, if LOF(P i ) is greater than the preset LOF threshold, then the corresponding data point P i If an abnormality occurs, that is, a sudden increase in CPU or memory usage, the warning module will output a Class B warning message; this may indicate that the device is overloaded or there is a security threat;

[0026] Otherwise, the warning module continues to output Class A warning information.

[0027] Preferably, the LOF(P i ) value is calculated as follows: First, for each data point P i , calculate its local reachable density loc(P i ), that is, in the neighborhood of P, other points to P i The reciprocal of the average distance;

[0028] Then for each data point P i, calculate the average local reachability density avg(P i );

[0029] Where avg(P i ) is the data point P i The average local reachability density of k points in the neighborhood, loc(P i ) is the data point P i The local reachable density; if LOF(P i ) is greater than the preset LOF threshold, the warning module outputs a warning signal, and the data analysis module detects abnormal points P i Conduct further analysis to determine whether the performance anomaly is caused by device overload, configuration errors, or security threats, and take appropriate measures based on the analysis results, such as optimizing configuration, increasing resources, or isolating suspicious activities.

[0030] Preferably, based on the early warning module output LOF (P i ) is greater than the preset LOF threshold, and the detected abnormal point P i Further analysis includes the following steps:

[0031] S1. Analyze device overload: Check the monitoring data of CPU and memory usage to determine whether there is a sudden increase in usage. If there is a sudden increase, increase hardware resources such as CPU, memory or upgrade network equipment to solve it; if there is no sudden increase, continue to analyze whether the configuration is wrong; Performance test: Use key performance indicators (such as network, disk, processor, memory) to perform stress testing to evaluate the performance of the system under high load;

[0032] S2. Whether the configuration is wrong includes: checking the system and network configuration to confirm whether there is an improper configuration that may cause performance problems. If there is an improper configuration, adjust the database configuration parameters, such as increasing the memory limit, adjusting the number of concurrent connections, etc.; if there is no improper configuration, continue to analyze security threats; performance problem analysis: analyze the database buffer pool, index, query statement, etc. to determine whether there is a performance bottleneck caused by improper configuration;

[0033] S3. The analysis of security threats includes: analyzing security logs, looking for signs of unauthorized access attempts, abnormal traffic, and the appearance of unknown files, identifying and isolating infected systems, and using firewall rules or physical disconnection methods if security threats are identified.

[0034] Preferably, the data analysis module in the risk category III analyzes whether the packet loss rate is abnormal in the following specific steps:

[0035] Use the Isolation Forest Algorithm to detect network packet loss rate: First, set the parameters of the Isolation Forest Algorithm, such as the number of trees and the maximum depth of each tree; then build multiple random trees, each of which randomly selects features and split points to isolate data points; finally, for each data point, calculate its isolation depth in each tree, that is, the path length from the root node to the node where the data point is isolated;

[0036] If the path length from the root node to the node where the data point is isolated is greater than a preset threshold, the warning module outputs a Class C warning message; otherwise, the warning module outputs a Class B warning message.

[0037] Preferably, the specific calculation method of the path length from the root node to the node where the data point is isolated is:

[0038] Get the data point L; calculate according to the formula: L is the data point, B is the depth of the decision tree, b is the decision tree, i max (L, b) is the decision that makes the data point L reach the maximum depth in the bth decision tree; by monitoring network traffic, device performance and network packet loss rate, if all three are abnormal, it is a Class III risk and a Class C warning information is output. The solution is to increase the bandwidth of the network link to reduce congestion and packet loss in data transmission.

[0039] Preferably, the network operation and maintenance monitoring system also includes a data center platform, and the data center platform stores various types of network data.

[0040] The beneficial effects of the present invention are as follows:

[0041] By monitoring network traffic in real time, when Class A warning information is output, timely Class A protection measures are taken, including increasing bandwidth or replacing faulty equipment, to avoid subsequent impacts, that is, to avoid excessive consumption of network resources, which in turn leads to interruption of normal business, and to avoid abnormal traffic occupying a large amount of network resources, which may lead to degradation of network performance, such as increased network delay and reduced throughput; abnormal traffic may be a precursor to an attack by an attacker, such as port scanning, brute force cracking, etc., which may lead to system breach, resulting in data leakage or system paralysis. Such threats are often hidden and highly destructive, and need to be detected and defended in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present invention will be further described below in conjunction with the accompanying drawings;

[0043] Figure 1 It is a system flow chart of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Network operation and maintenance monitoring includes monitoring network devices, servers, applications, and services. With the development of network technology, the amount of data that network operation and maintenance monitoring systems need to process is increasing. Traditional monitoring systems can no longer meet current needs. They lack effective data analysis capabilities and cannot identify and respond to abnormal situations in the network in a timely and accurate manner.

[0046] See also Figure 1 As shown, the present invention is a network operation and maintenance monitoring system based on data analysis, including a data collection module, a data processing module, a data analysis module and an early warning module;

[0047] Data collection module: The data collection module is used to collect various types of data in the network in real time, including but not limited to flow data, performance indicators, and log files;

[0048] Data processing module: The data processing module is used to process various types of collected data. The data processing module processes the data by standardizing the features.

[0049] Data analysis module: The data analysis module identifies abnormal patterns in the network based on various types of data processed by the data processing module, and determines whether the data is abnormal. It uses machine learning algorithms to analyze the collected data. The anomaly detection unit is used to identify abnormal behaviors in the network, and the trend prediction unit is used to predict network performance trends;

[0050] Early warning module: The early warning module issues early warning information based on the results of the data judgment module.

[0051] The data analysis module identifies abnormal patterns in the network, the abnormal patterns including: abnormal network traffic, abnormal network packet loss rate, and abnormal device performance;

[0052] The abnormal risks of the network are judged based on abnormal network traffic, abnormal network packet loss rate and abnormal device performance. The abnormal risks include Class I risk, Class II risk and Class III risk.

[0053] Among them, the harmfulness is that Class I risk is less than Class II risk and less than Class III risk. Class I risk includes network traffic anomaly; Class II risk includes network traffic anomaly and device performance anomaly; Class III risk includes network traffic anomaly, network packet loss rate anomaly and device performance anomaly;

[0054] The early warning module outputs different early warning information based on Class I risk, Class II risk and Class III risk, and the different early warning information includes Class A information, Class B information and Class C information;

[0055] Specifically, when an abnormality occurs in the network, different abnormalities are divided into three types of risks, among which network traffic risk has the lightest risk, network traffic and device performance are second, and network traffic, device performance and network packet loss rate are the most serious. The early warning module can output different early warning information according to these three types of risks, so as to make different repair measures. This design has a strong pertinence for network detection and can directly detect in real time which specific aspects have abnormalities.

[0056] Network traffic anomalies are usually the first to be detected, because traffic data can be collected in real time and is one of the basic data for network monitoring. Network traffic anomalies may cause device performance anomalies, which in turn may cause network packet loss rate anomalies. Therefore, a comprehensive network anomaly detection system can monitor these three aspects at the same time and take corresponding response measures based on the detection results.

[0057] The data analysis module described in Category I risk is used to analyze whether network data is abnormal:

[0058] S1. Collect network traffic data through the data collection module. The network traffic data includes inbound and outbound traffic, packet size, and transport layer protocol type; (Use network monitoring tools to capture network traffic data)

[0059] S2. Extract features from the raw traffic data through a data processing module, such as traffic size (e.g., bits per second), number of packets, session duration, specific port traffic, and specific IP address traffic; process the traffic data through a data analysis tool, such as Python's Pandas library;

[0060] S3, Standardization: Standardize the features to eliminate the dimensional effects between different features;

[0061] The normalization formula is x is the original data, u is the mean, σ is the standard deviation, x 1 It is the standardized data;

[0062] S4. The data analysis module calculates the distance between the test data point and the nearest neighbor data point in the training set. If the distance from the test data point to the nearest neighbor is greater than the set threshold, the data point is abnormal, and the warning module outputs Class A warning information. The KNN algorithm is used to calculate the distance between the test data point and the known data point.

[0063] It should be noted that in the network operation and maintenance monitoring system, when network traffic is abnormal, that is, the network traffic suddenly increases or decreases, it may indicate a DDoS attack or network configuration error, which needs to be dealt with in a timely manner.

[0064] Based on step S4, the specific calculation method of the distance between the test data point and the nearest neighbor data point in the training set is:

[0065] Obtain the characteristic value x of the test data point through the data acquisition module i , the eigenvalue y of the nearest neighbor sample i , calculated by the formula where x i is the ith eigenvalue of the test data point, y i is the ith eigenvalue of the nearest neighbor sample, σ i is the standard deviation of the ith feature, D1 is the distance between the test data point and the nearest neighbor data point in the training set. If D1 is greater than the set threshold, it is considered abnormal.

[0066] It should be noted that this threshold can be set based on historical data and business needs; it should also be noted that the feature value is the data that describes the various dimensions of the data point. The feature value can be various indicators of network traffic data, such as traffic size, number of data packets, session duration, connection rate, port usage, etc.

[0067] Specifically, by real-time monitoring of network traffic, when Class A warning information is output, timely Class A protection measures are taken, including increasing bandwidth or replacing faulty equipment, to avoid subsequent impacts, that is, to avoid excessive consumption of network resources, which in turn leads to interruption of normal business, and to avoid abnormal traffic occupying a large amount of network resources, which may lead to degradation of network performance, such as increased network latency, reduced throughput, etc.; abnormal traffic may be a precursor to an attack by an attacker, such as port scanning, brute force cracking, etc., which may lead to system breach, resulting in data leakage or system paralysis. Such threats are often hidden and highly destructive, and need to be detected and defended in a timely manner.

[0068] If the network traffic is abnormal and the warning module outputs a Class A warning message, the data analysis module is further used to continue analyzing whether the device performance is abnormal, specifically:

[0069] According to the data collection module, the CPU usage rate, memory usage rate, etc. of the device performance are collected, and a data set [P1, P2, ..., P n ];

[0070] For each data point P i Calculate the LOF (P) of the corresponding data point i ) value, if LOF(P i) is greater than the preset LOF threshold, then the corresponding data point P i If an abnormality occurs, that is, a sudden increase in CPU or memory usage, the warning module will output a Class B warning message; this may indicate that the device is overloaded or there is a security threat;

[0071] Otherwise, the warning module continues to output Class A warning information;

[0072] Specifically, the Local Outlier Factor (LOF) algorithm is a machine learning algorithm for anomaly detection. It identifies anomalies based on the density of data points. The core idea of ​​the LOF algorithm is that the density of anomalies in their neighborhood is significantly lower than the density of neighboring points in their neighborhood. The preset LOF threshold is set based on historical data and business needs.

[0073] If the corresponding data point P i When an abnormality occurs, that is, when the network traffic is abnormal, if the device performance is also abnormal, then the output is a Class B warning message. If only the network traffic is abnormal, but the device performance is not abnormal, then it is a Class A warning message. This makes it convenient to make different solutions for different warning messages later.

[0074] The LOF (P i ) value is calculated as follows: First, for each data point P i , calculate its local reachable density loc(P i ), that is, in the neighborhood of P, other points to P i The reciprocal of the average distance;

[0075] Then for each data point P i , calculate the average local reachability density avg(P i );

[0076] Where avg(P i ) is the data point P i The average local reachability density of k points in the neighborhood, loc(P i ) is the data point P i The local reachable density; if LOF(P i ) is greater than the preset LOF threshold, the warning module outputs a warning signal, and the data analysis module detects abnormal points P i Conduct further analysis to determine whether the performance anomaly is caused by device overload, configuration errors, or security threats, and take appropriate measures based on the analysis results, such as optimizing configuration, adding resources, or isolating suspicious activities;

[0077] It should be noted that after the CPU usage and memory usage of device performance are collected, the data needs to be processed, cleaned, missing values ​​and outliers processed, and features standardized so that each feature has the same dimension.

[0078] The proportional factor coefficient is used to correct the deviation of various parameters in the process of formula calculation, so as to make the calculation result more accurate;

[0079] The threshold is set to facilitate comparison. The threshold depends on the amount of sample data and the number of bases set by technicians in this field for each set of sample data. It does not affect the proportional relationship between the parameter and the quantized value.

[0080] The size of the coefficient is to quantify each parameter to obtain a specific value for subsequent comparison. The size of the coefficient depends on the amount of sample data and the preliminary setting of the corresponding operating coefficient for each set of sample data by technical personnel in this field; as long as it does not affect the proportional relationship between the parameter and the quantized value.

[0081] Embodiment 2:

[0082] Based on the early warning module output LOF (P i ) is greater than the preset LOF threshold, and the detected abnormal point P i Further analysis includes the following steps:

[0083] S1. Analyze device overload: Check the monitoring data of CPU and memory usage to determine whether there is a sudden increase in usage. If there is a sudden increase, increase hardware resources such as CPU, memory or upgrade network equipment to solve it; if there is no sudden increase, continue to analyze whether the configuration is wrong; Performance test: Use key performance indicators (such as network, disk, processor, memory) to perform stress testing to evaluate the performance of the system under high load;

[0084] S2. Whether the configuration is wrong includes: checking the system and network configuration to confirm whether there is an improper configuration that may cause performance problems. If there is an improper configuration, adjust the database configuration parameters, such as increasing the memory limit, adjusting the number of concurrent connections, etc.; if there is no improper configuration, continue to analyze security threats; performance problem analysis: analyze the database buffer pool, index, query statement, etc. to determine whether there is a performance bottleneck caused by improper configuration;

[0085] S3. The analysis of security threats includes: analyzing security logs to find signs of unauthorized access attempts, abnormal traffic, and the appearance of unknown files, identifying and isolating infected systems, and using firewall rules or physical disconnection methods if security threats are identified;

[0086] It should be noted that, through this monitoring system, not only can the device performance be analyzed in real time to see if there is a threat, but also different solutions can be formulated through a variety of analysis methods, which improves the security of network operation, saves a lot of workload and avoids the problem of difficulty in analysis caused by the impact between different data; Class B warning information measures include: increasing hardware resources, adjusting database configuration parameters, using firewall rules or physical disconnection methods.

[0087] The data analysis module in the risk category III is used to analyze whether the packet loss rate is abnormal. Specifically:

[0088] Use the Isolation Forest Algorithm to detect network packet loss rate: First, set the parameters of the Isolation Forest Algorithm, such as the number of trees and the maximum depth of each tree; then build multiple random trees, each of which randomly selects features and split points to isolate data points; finally, for each data point, calculate its isolation depth in each tree, that is, the path length from the root node to the node where the data point is isolated;

[0089] If the path length from the root node to the node where the data point is isolated is greater than the preset threshold, the warning module outputs a Class C warning message; otherwise, the warning module outputs a Class B warning message.

[0090] Abnormal packet loss rate: Packet loss rate is a key indicator to measure the quality of network transmission. A high packet loss rate affects network congestion or equipment failure. Abnormal packet loss rate: Abnormal behavior: A sudden increase in network packet loss rate may mean network congestion or equipment failure, and data transmission abnormalities caused by improper configuration or unexpected operations, such as repeated data packet sending, data transmission failure, etc. These abnormalities often occur when network protocols are inconsistent or the network environment is unstable, which will affect data integrity and transmission efficiency. Real-time monitoring of the network packet loss rate facilitates network monitoring.

[0091] The specific calculation method of the path length from the root node to the node where the data point is isolated is:

[0092] Get the data point L; calculate according to the formula: L is the data point, B is the depth of the decision tree, b is the decision tree, i max (L, b) is the decision that makes data point L reach the maximum depth in the b-th decision tree;

[0093] By monitoring network traffic, device performance, and network packet loss rate, if all three are abnormal, it is a Class III risk and a Class C warning message is output. The solution is to increase the bandwidth of the network link to reduce congestion and packet loss in data transmission.

[0094] The network operation and maintenance monitoring system also includes a data center platform, which stores various network data.

[0095] The above formulas are obtained by collecting a large amount of data for software simulation and selecting a formula that is close to the actual value. The coefficients in the formula are set by technical personnel in this field according to actual conditions. The above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited to this. Any technical personnel familiar with the technical field within the technical scope disclosed by the present invention, according to the technical solution and the inventive concept of the present invention, make equivalent replacement or change, which should be covered within the protection scope of the present invention.

Claims

1. A network operation and maintenance monitoring system based on data analysis, characterized in that: include Data collection module: The data collection module is used to collect various types of data in the network in real time, including but not limited to flow data, performance indicators, and log files; Data processing module: The data processing module is used to process various types of collected data; Data analysis module: The data analysis module identifies abnormal patterns in the network based on various types of data processed by the data processing module, and determines whether the data is abnormal; Early warning module: The early warning module issues early warning information based on the results of the data judgment module.

2. The network operation and maintenance monitoring system based on data analysis according to claim 1 is characterized in that: The data analysis module identifies abnormal patterns in the network, the abnormal patterns including: abnormal network traffic, abnormal network packet loss rate, and abnormal device performance; The abnormal risks of the network are judged based on abnormal network traffic, abnormal network packet loss rate and abnormal device performance. The abnormal risks include Class I risk, Class II risk and Class III risk. Among them, the harmfulness is that Class I risk is less than Class II risk and less than Class III risk. Class I risk includes network traffic anomaly; Class II risk includes network traffic anomaly and device performance anomaly; Class III risk includes network traffic anomaly, network packet loss rate anomaly and device performance anomaly; The early warning module outputs different early warning information based on Class I risk, Class II risk and Class III risk, and the different early warning information includes Class A information, Class B information and Class C information.

3. The network operation and maintenance monitoring system based on data analysis according to claim 2 is characterized in that: The data analysis module described in Category I risk is used to analyze whether network data is abnormal: S1. Collect network traffic data through a data collection module, where the network traffic data includes inbound and outbound traffic, packet size, and transport layer protocol type; S2, extracting features from the raw traffic data through a data processing module, such as traffic size, number of packets, session duration, specific port traffic, and specific IP address traffic; S3, Standardization: Standardize the features; S4. The data analysis module calculates the distance between the test data point and the nearest neighbor data point in the training set. If the distance from the test data point to the nearest neighbor is greater than the set threshold, the data point is abnormal, and the warning module outputs a Class A warning message.

4. The network operation and maintenance monitoring system based on data analysis according to claim 3 is characterized in that: Based on step S4, the specific calculation method of the distance between the test data point and the nearest neighbor data point in the training set is: Obtain the characteristic value x of the test data point through the data acquisition module i , the eigenvalue y of the nearest neighbor sample i , calculated by the formula where x i is the ith eigenvalue of the test data point, y i is the ith eigenvalue of the nearest neighbor sample, σ i is the standard deviation of the i-th feature, D1 is the distance between the test data point and the nearest neighbor data point in the training set. If D1 is greater than the set threshold, it is considered abnormal.

5. The network operation and maintenance monitoring system based on data analysis according to claim 3 is characterized in that: If the network traffic is abnormal and the warning module outputs Class A warning information, the data analysis module is further used to continue analyzing whether the device performance is abnormal, specifically: According to the data collection module, the CPU usage rate, memory usage rate, etc. of the device performance are collected, and a data set [P1, P2, ..., P n ]; For each data point P i Calculate the LOF (P) of the corresponding data point i ) value, if LOF(P i ) is greater than the preset LOF threshold, then the corresponding data point P i If an abnormality occurs, that is, a sudden increase in CPU or memory usage, the warning module will output a Class B warning message; Otherwise, the warning module continues to output Class A warning information.

6. The network operation and maintenance monitoring system based on data analysis according to claim 5 is characterized in that: The LOF (P i ) value is calculated as follows: First, for each data point P i , calculate its local reachable density loc(P i ), that is, in the neighborhood of P, other points to P i The reciprocal of the average distance; Then for each data point P i , calculate the average local reachability density avg(P i ); Where avg(P i ) is the data point P i The average local reachability density of k points in the neighborhood, loc(P i ) is the data point P i The local reachable density of If LOF(P i ) is greater than the preset LOF threshold, the warning module outputs B warning information, and the data analysis module detects abnormal points P i Conduct further analysis to determine whether the performance anomaly is caused by device overload, configuration errors, or security threats, and take appropriate measures based on the analysis results, such as optimizing configuration, increasing resources, or isolating suspicious activities.

7. The network operation and maintenance monitoring system based on data analysis according to claim 6 is characterized in that: Based on the early warning module output LOF (P i ) is greater than the preset LOF threshold, and the detected abnormal point P i Further analysis includes the following steps: S1. Analyze device overload: Check the monitoring data of CPU and memory usage to determine whether there is a sudden increase in usage. If there is a sudden increase, increase hardware resources such as CPU, memory or upgrade network equipment to solve the problem; If there is no sudden increase, continue to analyze whether the configuration is wrong; S2. Whether the configuration is wrong includes: checking the system and network configuration to confirm whether there is an improper configuration that may cause performance problems. If there is an improper configuration, adjust the database configuration parameters, such as increasing the memory limit, adjusting the number of concurrent connections, etc.; if there is no improper configuration, continue to analyze security threats; S3. The analysis of security threats includes: analyzing security logs, looking for signs of unauthorized access attempts, abnormal traffic, and the appearance of unknown files, identifying and isolating infected systems, and using firewall rules or physical disconnection methods if security threats are identified.

8. The network operation and maintenance monitoring system based on data analysis according to claim 5 is characterized in that: The data analysis module in the risk category III is used to analyze whether the packet loss rate is abnormal. Specifically: Use the Isolation Forest Algorithm to detect network packet loss rate: First, set the parameters of the Isolation Forest Algorithm, such as the number of trees and the maximum depth of each tree; then build multiple random trees, each of which randomly selects features and split points to isolate data points; Finally, for each data point, calculate its isolation depth in each tree, that is, the path length from the root node to the node where the data point is isolated; If the path length from the root node to the node where the data point is isolated is greater than a preset threshold, the warning module outputs a Class C warning message; Otherwise, the warning module outputs Class B warning information.

9. The network operation and maintenance monitoring system based on data analysis according to claim 8, characterized in that: The specific calculation method of the path length from the root node to the node where the data point is isolated is: Get the data point L; calculate according to the formula: L is the data point, B is the depth of the decision tree, b is the decision tree, i max (L, b) is the decision that makes data point L reach the maximum depth in the b-th decision tree.

10. The network operation and maintenance monitoring system based on data analysis according to claim 1, characterized in that: The network operation and maintenance monitoring system also includes a data center platform, which stores various network data.

Images