Network anomaly detection and automatic handling methods, systems, storage media and devices

Abstract

This invention relates to the field of network detection and optimization technology, and discloses a method, system, storage medium, and device for network anomaly detection and automatic processing. The method includes: collecting multi-dimensional data related to network performance; preprocessing the collected data and constructing feature engineering on the preprocessed data; constructing a multi-dimensional anomaly detection model; performing time-series anomaly detection, multi-dimensional feature clustering analysis, and rule engine detection through the multi-dimensional anomaly detection model; classifying and automatically processing the anomalies detected by the multi-dimensional anomaly detection model; embedding data points on network performance indicators and anomaly events; monitoring the status of each layer of the network in real time and tracking and recording anomaly events; simultaneously performing predictive analysis on network performance data; mining data value through trend and correlation analysis; and generating optimization suggestions based on the predictive analysis results. Through the above method, a network anomaly detection and automatic processing system is realized, improving network quality and user experience.

Description

Technical Field

[0001] This invention relates to the field of network detection and optimization technology, and in particular to a method, system, storage medium, and device for network anomaly detection and automatic processing. Background Technology

[0002] With the development of 6G network technology, network architecture is becoming increasingly complex, especially network management under the same SSID scenario faces many challenges.

[0003] Existing technologies mainly suffer from the following problems: First, traditional network monitoring solutions are primarily designed for 4G / 5G networks and do not fully consider the characteristics and requirements of 6G networks, resulting in issues such as untimely network switching, connection drops due to signal interference, and imperfect automatic reconnection mechanisms. Second, anomaly detection methods are too simplistic, relying solely on single indicator thresholds for judgment, and cannot effectively identify complex anomaly patterns. Third, network anomaly handling relies mainly on manual intervention, resulting in slow response times and a high risk of misoperation. Finally, the lack of a systematic data collection and analysis mechanism makes it difficult to support continuous optimization of network performance.

[0004] This demonstrates that existing network monitoring solutions lack anomaly detection capabilities specific to 6G scenarios, and most adopt a passive response mode, failing to provide early warnings and automatic handling of network anomalies, thus impacting user experience and network performance. These issues severely affect the service quality and user experience of 6G networks. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of the existing technology by providing a network anomaly detection and automatic processing method, system, storage medium, and device. This method establishes a multi-dimensional anomaly detection model through real-time monitoring of network performance indicators and combines it with machine learning algorithms to achieve anomaly early warning. Furthermore, an intelligent automatic processing flow is designed, which can automatically perform operations such as network switching and signal optimization based on the anomaly type, and collect anomaly data through data collection points for continuous optimization.

[0006] On the one hand, a method for network anomaly detection and automatic handling is provided, including the following steps:

[0007] S1: Collect multi-dimensional data related to network performance, perform preprocessing on the collected data including noise removal, outlier handling, and data standardization, and construct feature engineering on the preprocessed data;

[0008] S2: Construct a multidimensional anomaly detection model based on deep learning and machine learning algorithms, and use the multidimensional anomaly detection model to perform time-series anomaly detection based on Long Short-Term Memory Network (LSTM), multidimensional feature clustering analysis based on Density Clustering Algorithm (DBSCAN), and rule engine detection based on expert rules.

[0009] S3: Based on the hierarchical classification strategy, classify the anomalies detected by the multidimensional anomaly detection model, and execute the corresponding automatic processing strategy according to the anomaly type;

[0010] S4: Multi-level data tracking is implemented for network performance indicators and abnormal events to monitor the status of each layer of the network in real time and to track and record abnormal events. At the same time, trend prediction analysis of network performance data is performed using the exponential smoothing method, and correlation analysis is performed based on the Pearson correlation coefficient to mine data value. Based on the prediction analysis results, optimization suggestions are generated by combining the decision tree algorithm.

[0011] Furthermore, in step S1, the multi-dimensional data includes basic network metrics and advanced network metrics, wherein...

[0012] The basic metrics collected from the network include:

[0013] The signal strength RSSI of the network is continuously collected at configurable time intervals to assess the degree of attenuation of the wireless signal during transmission.

[0014] Network latency (RTT) is obtained by actively probing and recording the time difference between sending a specific probe request packet to the target server and receiving the response packet returned by the target server.

[0015] At the sending end, the total number of data packets sent within a specific time period is recorded based on a sliding window mechanism to obtain the data packet loss rate;

[0016] Obtain network interface traffic statistics through the management interface provided by the network device to obtain bandwidth utilization.

[0017] The Channel Quality Index (CQI) value is calculated in real time based on the characteristics of the received signal.

[0018] Advanced metrics for data collection networks include:

[0019] The Network Stability Index (NSI) is obtained by normalizing the basic indicators and weighting them together based on an adaptive weighting mechanism. The NSI is used to comprehensively evaluate the stability of the network, and the weighting coefficients are adjusted according to the actual situation.

[0020] The anomaly rate is calculated by using a sliding window method with configurable size to determine the ratio of the number of outlier samples to the total number of samples, and is used to evaluate the anomaly status of the network.

[0021] Furthermore, in step S1, the preprocessing of the collected data, including noise removal, outlier handling, and data standardization, further includes:

[0022] Statistical methods were used to perform preliminary anomaly detection on the data, including using Z-score and box plot methods to remove obvious noise and outliers from the data, and using interpolation and nearest neighbor interpolation to fill in missing values.

[0023] The Z-score standardization method is used to transform data features of different ranges and magnitudes to the same scale, normalize and standardize the data features, and avoid the imbalance of the contribution of different features to the anomaly detection model due to differences in magnitude.

[0024] Furthermore, in step S1, the construction of feature engineering on the preprocessed data further includes:

[0025] Based on random forest and mutual information evaluation methods, relevant features are selected from the collected raw data, and the most valuable features for anomaly detection are screened through feature importance evaluation methods.

[0026] Time features, including hour, minute, second and periodic features of timestamps, are extracted from time series data. The interactive feature generation method is used to combine features of multidimensional data to capture the nonlinear relationship between different dimensions.

[0027] Further, in step S2, the multidimensional anomaly detection model includes a temporal anomaly detection model, a multidimensional feature clustering analysis model, and a rule engine detection model, wherein constructing the temporal anomaly detection model includes:

[0028] Based on the characteristics of time series data, the time series data is organized into a three-dimensional tensor format including the number of samples, time step, and number of features. The time series data is then split into training set, validation set, and test set according to a set ratio.

[0029] A deep neural network model including a multi-layer long short-term memory (LSTM) network structure was constructed, and the model was trained using a training set and the backpropagation algorithm.

[0030] The hyperparameters, including the learning rate, number of LSTM units, and Dropout ratio, are adjusted by inputting the validation set using grid search and cross-validation methods.

[0031] The performance of the time-series anomaly detection model is evaluated using metrics including recall, precision, and F1 score by performing prediction tests on the input test set.

[0032] Preferably, constructing the multidimensional feature clustering analysis model includes:

[0033] Based on feature correlation analysis, data from multiple dimensions are merged into a feature matrix to create a sample dataset that includes multidimensional features;

[0034] The multidimensional feature clustering analysis model is constructed based on the density clustering algorithm DBSCAN. The optimal neighborhood radius and minimum number of samples are determined by grid search, and the clustering quality is evaluated using the silhouette coefficient and DTW distance. At the same time, the rationality of outliers is evaluated by combining expert experience.

[0035] More preferably, constructing the rule engine detection model includes:

[0036] Based on expert experience and historical anomalies, various rules are encoded into executable logic, and a rule base including rule priority, triggering conditions and execution actions is established.

[0037] The rule engine detection model is constructed through a rule conflict detection and priority ranking mechanism. The rule engine detection model is then evaluated and optimized online based on the number of anomalies triggered in actual applications, the processing effect, and the resource consumption.

[0038] Further, in step S2, the step of performing time-series anomaly detection based on Long Short-Term Memory (LSTM) network, multi-dimensional feature clustering analysis based on the density clustering algorithm DBSCAN, and rule engine detection based on expert rules through the multi-dimensional anomaly detection model further includes:

[0039] Based on the distribution characteristics of historical data, the standard deviation threshold is determined using the 3σ criterion and a quantile-based method. The standard deviation is used as an important indicator to measure the deviation between the predicted and actual values. The normal network performance index range is predicted through the time-series anomaly detection model.

[0040] Multidimensional data is organized into a feature matrix according to time series. The multidimensional feature clustering model is used to divide the data into normal clusters and outliers based on density thresholds. The density thresholds are dynamically adjusted according to historical data.

[0041] The rule engine performs detection based on priority order, matching the input network performance data with the rules in the rule base. When the rule conditions are met, the corresponding abnormal status and severity are returned.

[0042] Furthermore, in step S2, the classification of anomalies detected by the multidimensional anomaly detection model based on a hierarchical classification strategy further includes:

[0043] The classification of anomalies is determined based on a hierarchical classification strategy. These anomaly classifications include:

[0044] Network congestion anomalies are characterized by high network latency, high packet loss rate, and low bandwidth utilization.

[0045] Signal interference anomalies are characterized by low signal strength, low channel quality indicators, and high bit error rate.

[0046] Hardware failure-related anomalies are characterized by sudden disconnections, abnormal restarts, and configuration errors.

[0047] At the same time, severity levels are set for each type of anomaly, and priority is assigned based on the duration, scope of impact, and degree of business impact of the anomaly.

[0048] Furthermore, in step S3, the execution of the corresponding automatic processing strategy based on the exception type further includes:

[0049] A strategy matching algorithm based on multi-dimensional feature similarity calculates the similarity score between the current anomaly and historical cases, and generates a set of candidate processing strategies.

[0050] Based on the applicable conditions of the strategy, historical execution results and resource consumption, the optimal strategy is selected from the processing strategy library, including network switching, parameter optimization and fault self-healing, and the rollback point of strategy execution is set.

[0051] The effectiveness of the processing strategy is quantitatively evaluated by calculating the relative improvement of key performance indicators. If the improvement does not meet expectations, a rollback mechanism is triggered.

[0052] A policy optimization algorithm based on deep reinforcement learning is adopted to update the weights of the processing policy by calculating the reward value of policy execution, thereby realizing online learning and optimization of the processing policy.

[0053] Furthermore, in step S4, the multi-level tracking of network performance indicators and abnormal events further includes:

[0054] The sampling frequency of the data points is set based on a hierarchical sampling strategy, with high-frequency sampling for core indicators and low-frequency sampling for secondary indicators.

[0055] It collects and stores data from all layers of the network, including the physical layer, protocol layer, and application layer, in real time.

[0056] Physical layer data includes: network interface status, physical connection status, operating speed, and duplex mode;

[0057] Protocol layer data includes: TCP / IP performance metrics, protocol stack status, and network protocol interaction data;

[0058] Application layer data includes: service response time, business success rate, and user experience data;

[0059] Real-time recording of abnormal events and their complete lifecycle, including network switching events, parameter adjustment events, and fault handling events. The recorded content includes the event occurrence time, type determination, processing procedure, execution result, and effect evaluation.

[0060] Furthermore, in step S4, the generation of optimization suggestions further includes:

[0061] The exponential smoothing method based on adaptive parameters is used to analyze and predict network performance trends. Correlation analysis is performed by combining Pearson correlation coefficient and mutual information index to evaluate the correlation between network performance indicators.

[0062] A decision tree algorithm based on information gain is adopted to construct a network optimization decision model by using significantly relevant indicators as decision nodes.

[0063] The optimization suggestions are comprehensively scored and prioritized based on the following factors:

[0064] The level of urgency is based on the severity and scope of the anomaly.

[0065] Resource consumption is based on the required resources and technological conditions.

[0066] Scope of impact, based on historical optimization results and business impact assessment;

[0067] At the same time, an implementation feedback mechanism for optimization suggestions should be established to continuously evaluate and improve the optimization effect.

[0068] On the other hand, a network anomaly detection and automatic processing system is provided, including:

[0069] The indicator acquisition module is used to collect multi-dimensional data related to network performance. It performs preprocessing on the collected data, including noise removal, outlier handling, and data standardization, and constructs feature engineering on the preprocessed data.

[0070] An anomaly detection module is used to construct a multidimensional anomaly detection model based on deep learning and machine learning algorithms. The multidimensional anomaly detection model is used to perform time-series anomaly detection based on Long Short-Term Memory (LSTM) network, multidimensional feature clustering analysis based on the density clustering algorithm DBSCAN, and rule engine detection based on expert rules.

[0071] The strategy execution model is used to classify the anomalies detected by the multidimensional anomaly detection model based on the hierarchical classification strategy, and to execute the corresponding automatic processing strategy according to the anomaly type.

[0072] The data analysis module is used to perform multi-level tracking of network performance indicators and abnormal events, monitor the status of each layer of the network in real time, and track and record abnormal events. At the same time, it performs trend prediction analysis on network performance data through exponential smoothing, performs correlation analysis based on Pearson correlation coefficient to mine data value, and generates optimization suggestions based on the prediction analysis results by combining decision tree algorithm.

[0073] Meanwhile, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the network anomaly detection and automatic processing method described above.

[0074] Compared with the prior art, the beneficial effects of the present invention are:

[0075] This invention employs a multi-dimensional indicator collection and analysis system, which achieves comprehensive perception of network status by real-time monitoring of core indicators such as signal strength, network latency, and packet loss rate, combined with an innovative calculation method for the Network Stability Index (NSI).

[0076] This invention integrates a triple detection mechanism of LSTM time series analysis, DBSCAN clustering, and expert rule engine at the anomaly detection level, which greatly improves the accuracy and reliability of anomaly identification.

[0077] In terms of automatic processing, this invention designs an intelligent policy generation system based on reinforcement learning, which can automatically perform processing measures such as network switching, parameter optimization or fault self-healing according to the anomaly type, significantly reducing the need for manual intervention.

[0078] This invention establishes a complete data tracking and analysis system, continuously optimizing system performance through exponential smoothing prediction and decision tree analysis. This closed-loop optimization mechanism enables the system to continuously evolve and adapt to dynamic changes in the network environment. Attached Figure Description

[0079] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0080] Figure 1 This is a flowchart of a network anomaly detection and automatic processing method according to the present invention;

[0081] Figure 2 This is a schematic diagram of a network performance indicator collection process according to the present invention;

[0082] Figure 3 This is a block diagram of a multidimensional anomaly detection model according to the present invention;

[0083] Figure 4 This is a schematic diagram of an anomaly detection process according to the present invention;

[0084] Figure 5 This is a schematic diagram of an automatic exception handling process according to the present invention;

[0085] Figure 6 This is a schematic diagram of a data embedding design according to the present invention;

[0086] Figure 7 This is a schematic diagram illustrating the generation of indicator data analysis and optimization suggestions according to the present invention;

[0087] Figure 8 This is a block diagram of the integrated deployment structure of a network anomaly detection and automatic processing system according to the present invention;

[0088] Figure 9 This is a schematic diagram of an embodiment of an electronic device according to the present invention. Detailed Implementation

[0089] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0090] This invention presents an innovative method for calculating the Network Stability Index (NSI). By real-time monitoring of multiple dimensions such as signal strength, network latency, and packet loss rate, and combining LSTM time-series analysis, DBSCAN clustering, and an expert rule engine, a triple anomaly detection mechanism is constructed. This multi-layered detection architecture is unprecedented in existing technologies. Particularly in anomaly handling, the solution introduces a reinforcement learning-based intelligent policy generation system, capable of adaptively executing network switching, parameter optimization, or fault self-healing measures based on different types of anomalies.

[0091] The specific embodiments of the present invention will be described below with reference to the accompanying drawings and examples.

[0092] First Embodiment

[0093] Please see Figure 1 This embodiment provides a method for network anomaly detection and automatic handling, the technical solution of which includes the following steps:

[0094] S1: Collect multi-dimensional data related to network performance, perform preprocessing on the collected data including noise removal, outlier handling, and data standardization, and construct feature engineering on the preprocessed data;

[0095] S2: Construct a multidimensional anomaly detection model based on deep learning and machine learning algorithms, and use the multidimensional anomaly detection model to perform time-series anomaly detection based on Long Short-Term Memory Network (LSTM), multidimensional feature clustering analysis based on Density Clustering Algorithm (DBSCAN), and rule engine detection based on expert rules.

[0096] S3: Based on the hierarchical classification strategy, classify the anomalies detected by the multidimensional anomaly detection model, and execute the corresponding automatic processing strategy according to the anomaly type;

[0097] S4: Multi-level data tracking is implemented for network performance indicators and abnormal events to monitor the status of each layer of the network in real time and to track and record abnormal events. At the same time, trend prediction analysis of network performance data is performed using the exponential smoothing method, and correlation analysis is performed based on the Pearson correlation coefficient to mine data value. Based on the prediction analysis results, optimization suggestions are generated by combining the decision tree algorithm.

[0098] In step S1, the multi-dimensional data includes basic network metrics and advanced network metrics, such as... Figure 2 As shown, where,

[0099] In this embodiment, the basic network metrics collected include:

[0100] According to the set time interval, the network signal strength RSSI is collected every 100ms to assess the degree of attenuation of the wireless signal during transmission.

[0101] Network latency RTT is collected every 500ms using an active probing method. The network latency RTT is obtained by recording the time difference between sending a specific probe request packet to the target server and receiving the response packet returned by the target server.

[0102] At the sending end, the total number of data packets sent is recorded every 1 second using a sliding window mechanism to obtain the packet loss rate (PacketLoss).

[0103] The bandwidth utilization rate is collected every 1 second. The network interface traffic statistics information is obtained through the management interface provided by the network device to obtain the bandwidth utilization rate (Bandwidth).

[0104] The Channel Quality Index (CQI) is collected every 200ms, and the CQI value is calculated based on the characteristics of the received signal.

[0105] In addition, advanced metrics for the collected network include:

[0106] The Network Stability Index (NSI) is obtained by normalizing the basic indicators and weighting them using an adaptive weighting mechanism. The NSI is used to comprehensively evaluate the stability of the network, and its calculation formula is as follows:

[0107] NSI = w1×RSSI_norm + w2×RTT_norm + w3×PacketLoss_norm + w4×Bandwidth_norm + w5×CQI_norm,

[0108] Among them, the weight coefficients w1~w5 are adjusted according to the actual situation, and xx_norm represents the normalized index value.

[0109] The anomaly rate is calculated by using a configurable sliding window method to determine the ratio of outlier samples to the total number of samples. This rate is used to assess the anomaly status of the network. The calculation formula is as follows:

[0110] AnomalyRate = (Number of outliers / Total number of samples) × 100%,

[0111] In this embodiment, the sliding window size is 5 minutes.

[0112] First, ensure that the collected data has a sufficient time span and a large enough sample size to cover different network states, including normal states and possible abnormal states. Then, preprocess the collected data, including:

[0113] Use statistical methods to perform preliminary anomaly detection on the data, such as Z-score, box plot, and standard deviation or IQR methods to remove obvious noise and outliers in the data for more accurate anomaly detection, and fill missing values ​​by interpolation and nearest neighbor interpolation.

[0114] By transforming data features of different ranges and magnitudes to the same scale and normalizing and standardizing the data features, the contribution of different features to the anomaly detection model is prevented from becoming unbalanced due to differences in magnitude. In this embodiment, the data is scaled to the range [0, 1] using min-max normalization or Z-score standardization.

[0115] Then, feature engineering is performed on the preprocessed data, including:

[0116] This embodiment selects relevant features from the collected raw data based on random forest and mutual information evaluation methods. Correlation analysis, such as Pearson correlation coefficient, is used to evaluate the correlation between different features, avoiding the use of highly correlated features to prevent redundant information. Then, feature importance evaluation methods, such as feature importance in random forest, are used to screen the features most valuable for anomaly detection.

[0117] Extract time features from time series data, including hour, minute, second, and periodic features of timestamps, to reflect the time patterns of the data; and combine features of multidimensional data through interactive feature generation methods, capturing nonlinear relationships between different dimensions by calculating the ratios or differences between features.

[0118] The multidimensional anomaly detection model includes a time-series anomaly detection model, a multidimensional feature clustering analysis model, and a rule engine detection model, such as... Figure 3 As shown.

[0119] The construction of the time-series anomaly detection model includes:

[0120] Based on the characteristics of time series data, the time series data is organized into a format suitable for the input of the time series anomaly detection model, including a three-dimensional tensor of the number of samples, time step, and number of features, and the time series data is split into training set, validation set, and test set.

[0121] Construct a deep neural network model that includes a multi-layer Long Short-Term Memory (LSTM) network structure. The LSTM network consists of an input layer, one or more LSTM layers, a Dropout layer (to prevent overfitting), and an output layer. Use Keras to build a simple LSTM model and train the model using a training set and the backpropagation algorithm.

[0122] Hyperparameters, including learning rate, number of LSTM units, and Dropout ratio, are adjusted based on the input validation set using grid search and cross-validation methods.

[0123] The time-series anomaly detection model is tested by inputting a test set for prediction, and its performance is evaluated using metrics including recall, precision, and F1 score.

[0124] Secondly, constructing the multidimensional feature clustering analysis model includes:

[0125] Combine data from multiple dimensions into a feature matrix to create a sample dataset, where each sample includes multiple features;

[0126] The multidimensional feature clustering analysis model is constructed based on the density clustering algorithm DBSCAN. The optimal neighborhood radius and minimum number of samples are determined through grid search. The sample data is input into the multidimensional feature clustering analysis model, and the silhouette coefficient is used to evaluate the clustering quality. At the same time, the proportion and rationality of outliers are statistically analyzed in combination with expert experience.

[0127] Furthermore, constructing the rule engine detection model includes:

[0128] Based on expert experience and historical anomalies, various rules are encoded into executable logic, and a rule base including rule priority, triggering conditions and execution actions is established.

[0129] The rule engine detection model is constructed through a rule conflict detection and priority ranking mechanism. The rule engine detection model is then evaluated and optimized online based on the number of anomalies triggered in actual applications, the processing effect, and the resource consumption.

[0130] Then, the multidimensional anomaly detection model is used for temporal anomaly detection based on the Long Short-Term Memory (LSTM) network, multidimensional feature clustering analysis based on the density clustering algorithm DBSCAN, and rule engine detection based on expert rules, such as... Figure 4 As shown, it specifically includes:

[0131] S10: Based on the distribution characteristics of historical data, the standard deviation threshold is determined using the 3σ criterion and a quantile-based method. When performing time-series anomaly detection using the trained time-series anomaly detection model, the standard deviation is used as an important indicator to measure the deviation between the predicted and actual values, predicting the range of normal network performance indicators. In this embodiment, the anomaly determination conditions are as follows:

[0132] If |actual value - predicted value| > 3 × σ (standard deviation), it is considered an anomaly.

[0133] For the RTT metric, if the predicted value is 50ms and the standard deviation is 10ms, an actual value exceeding 80ms is considered abnormal.

[0134] S20: Using multi-dimensional data as features, data from different time points are organized into a multi-dimensional data matrix. Each sample point includes feature values ​​of multiple dimensions. The multi-dimensional feature clustering model divides regions with sufficiently high density into clusters, and points in low-density regions are considered noise or outliers. In this embodiment, the core parameters are: ε=0.5, MinPts=4. A DBSCAN object is created, where ε is the neighborhood radius and MinPts is the minimum number of samples. The feature matrix X is input into the model for clustering, and the cluster to which each sample belongs is predicted. -1 represents an outlier, i.e., an anomaly.

[0135] S30: The rule engine performs detection based on priority order, matching the input network performance data with rules in the rule base. That is, the input network performance data is judged using different rules. If a rule condition is met, the corresponding abnormal status is returned; otherwise, normal status is returned. In this embodiment, the rule base is as follows:

[0136] Rule 1: If RTT > 200ms and duration > 30s, it is considered abnormal;

[0137] Rule 2: If PacketLoss > 5% and duration > 15s, it is considered an anomaly;

[0138] Rule 3: If the bandwidth utilization is less than 20% and the signal strength is normal, it is considered abnormal.

[0139] In step S2, classifying the detected anomalies further includes:

[0140] The classification of anomalies is determined based on a hierarchical classification strategy. These anomaly classifications include:

[0141] Network congestion anomalies are characterized by high network latency, high packet loss rate, and low bandwidth utilization.

[0142] Signal interference anomalies are characterized by low signal strength, low channel quality indicators, and high bit error rate.

[0143] Hardware failure-related anomalies are characterized by sudden disconnections, abnormal restarts, and configuration errors.

[0144] Simultaneously, a severity level is assigned to each type of anomaly, and priority is ranked based on the anomaly's duration, scope of impact, and degree of business impact. This is represented as follows:

[0145] A. Network congestion-related anomalies:

[0146] Feature vector = [high RTT, high packet loss rate, low bandwidth utilization];

[0147] B. Signal interference type anomalies:

[0148] Feature vector = [low RSSI, low CQI, high bit error rate];

[0149] C. Hardware failure type exceptions:

[0150] Feature vector = [Sudden disconnection, abnormal restart, configuration error].

[0151] Then, the anomalies detected and output by the multidimensional anomaly detection model are automatically processed, such as... Figure 5 As shown, it includes:

[0152] S100: A strategy matching algorithm based on multi-dimensional feature similarity. It calculates the similarity score between the current anomaly detection result and historical cases across multiple dimensions, and generates a processing strategy. In this embodiment, the algorithm formula is as follows:

[0153] Score(strategy i) = Σ(wj × similarity j),

[0154] Where wj represents the weight of each dimension, and similarity j represents the similarity between the current anomaly and historical cases;

[0155] S200: Based on the calculated processing strategy, its applicable conditions, historical execution results, and resource consumption, select the optimal strategy from the processing strategy library, including network switching, parameter optimization, and fault self-healing, and set the rollback point for strategy execution. In this embodiment, the processing strategy library is as follows:

[0156] Strategy 1: Network Switching

[0157] Applicable conditions: Current network quality score < 60 and backup network available.

[0158] Execution steps:

[0159] a) Scan for available networks.

[0160] b) Assess the quality of the backup network.

[0161] c) Perform a smooth switch;

[0162] Strategy 2: Parameter Optimization

[0163] Applicable conditions: Network quality score ≤ 60 < 80

[0164] Execution steps:

[0165] a) Adjust the transmission power.

[0166] b) Optimize channel configuration.

[0167] c) Update QoS policy;

[0168] Strategy 3: Fault Self-Healing

[0169] Applicable conditions: Hardware-level anomalies detected.

[0170] Execution steps:

[0171] a) Try resetting the network interface.

[0172] b) Restore the default configuration.

[0173] c) Activate the backup module;

[0174] S300: The effectiveness of the processing strategy is quantitatively evaluated by calculating the relative improvement of key performance indicators, that is, by calculating the relative change of the network performance indicator value after the implementation of the processing strategy compared with the initial value before processing. If the improvement does not meet expectations, a rollback mechanism is triggered. The calculation formula is as follows:

[0175] EffectScore = (Post-processing index value - Pre-processing index value) / Pre-processing index value × 100%;

[0176] S400: Employs a policy optimization algorithm based on deep reinforcement learning. By calculating the deviation between the actual execution effect and the expected effect of the processing policy, and combining the feature values, the weights of the processing policy are updated to optimize subsequent policy selection. The policy optimization formula is as follows:

[0177] w'i = wi + α × (actual effect - expected effect) × i,

[0178] Where α is the learning rate, with a value of 0.1, and i is the feature value.

[0179] Furthermore, in step S4, the multi-level data collection of network performance indicators and abnormal events is as follows: Figure 6 As shown, it specifically includes:

[0180] Based on the importance of the hierarchical sampling strategy, the sampling frequency of the data points is set. High-frequency sampling is used for core indicators and low-frequency sampling is used for secondary indicators, with a range of 1 second to 5 minutes. The network interface status (point 1), protocol stack performance (point 2), and application layer experience (point 3) are obtained in real time. The physical connection status, working speed, and duplex mode of the network interface are monitored, the performance of different protocol layers is evaluated, and the performance of the application is evaluated from the user's perspective.

[0181] It collects and stores data from all layers of the network, including the physical layer, protocol layer, and application layer, in real time.

[0182] Physical layer data includes: network interface status, physical connection status, operating speed, and duplex mode;

[0183] Protocol layer data includes: TCP / IP performance metrics, protocol stack status, and network protocol interaction data;

[0184] Application layer data includes: service response time, business success rate, and user experience data;

[0185] Real-time recording of abnormal events and their complete lifecycle, including network switching events, parameter adjustment events, and fault handling events. The recorded content includes the event occurrence time, type determination, processing procedure, execution result, and effect evaluation.

[0186] Next, as Figure 7 As shown, the generated optimization suggestions further include:

[0187] S101: The exponential smoothing method based on adaptive parameters is used to analyze and predict network performance trends. The formula is as follows:

[0188] St = α × Yt + (1-α) × St-1, where α is the smoothing coefficient with a value of 0.3, Yt is the actual observed value at time t, and St is the smoothed value at time t;

[0189] Then, correlation analysis is performed by combining the Pearson correlation coefficient and mutual information index to evaluate the correlation between network performance indicators, as shown in the following formula:

[0190] r = Σ((Xi- (Yi- )) / √(Σ(Xi- )²×Σ(Yi- )²),

[0191] in, and The mean of two variables X and Y is given by the Pearson correlation coefficient r, which measures the linear correlation between the two variables X and Y, ranging from -1 to 1, to assess the relationship between network performance indicators. Here, r = 1 indicates a perfect positive correlation, r = -1 indicates a perfect negative correlation, and r = 0 indicates no linear correlation.

[0192] S201: Construct a decision tree using the CART algorithm based on information gain, and use significantly relevant indicators as decision nodes to build a network optimization decision model;

[0193] S301: Optimization suggestions will be comprehensively scored and prioritized based on the following factors:

[0194] The level of urgency is based on the severity and scope of the anomaly.

[0195] Resource consumption is based on the required resources and technological conditions.

[0196] Scope of impact, based on historical optimization results and business impact assessment;

[0197] Simultaneously, an implementation feedback mechanism for the optimization suggestions will be established to continuously evaluate and improve the optimization effect. The formula is as follows:

[0198] Priority = Urgency × Impact × Resource,

[0199] Among them, Urgency: level of urgency (1-5 points), Impact: scope of impact (1-5 points), Resource: resource consumption (1-5 points).

[0200] In summary, the method presented in this embodiment provides a complete solution for network anomaly detection and automatic processing through systematic design and implementation. Combining technologies such as machine learning, expert systems, and automated control, it effectively improves network service quality and user experience. It overcomes the technical bottleneck of passive response in traditional network monitoring, constructing a closed-loop intelligent network management system. Firstly, at the indicator collection level, the solution designs a multi-dimensional data collection system based on the NetGuard framework and introduces a sliding window mechanism for dynamic evaluation. Secondly, at the anomaly identification level, the integration of multiple machine learning algorithms significantly improves the accuracy and reliability of anomaly detection. Thirdly, at the automatic processing level, by establishing a complete data tracking and analysis system, continuous system optimization and self-evolution are achieved.

[0201] This embodiment also provides a network anomaly detection and automatic processing system, including:

[0202] The indicator acquisition module is used to collect multi-dimensional data related to network performance. It performs preprocessing on the collected data, including noise removal, outlier handling, and data standardization, and constructs feature engineering on the preprocessed data.

[0203] An anomaly detection module is used to construct a multidimensional anomaly detection model based on deep learning and machine learning algorithms. The multidimensional anomaly detection model is used to perform time-series anomaly detection based on Long Short-Term Memory (LSTM) network, multidimensional feature clustering analysis based on the density clustering algorithm DBSCAN, and rule engine detection based on expert rules.

[0204] The strategy execution model is used to classify the anomalies detected by the multidimensional anomaly detection model based on the hierarchical classification strategy, and to execute the corresponding automatic processing strategy according to the anomaly type.

[0205] The data analysis module is used to perform multi-level tracking of network performance indicators and abnormal events, monitor the status of each layer of the network in real time, and track and record abnormal events. At the same time, it performs trend prediction analysis on network performance data through exponential smoothing, performs correlation analysis based on Pearson correlation coefficient to mine data value, and generates optimization suggestions based on the prediction analysis results by combining decision tree algorithm.

[0206] Among them, the legal entity integration and deployment structure of the network anomaly detection and automatic processing system is as follows: Figure 8 As shown, to achieve performance optimization, resource consumption was optimized, including memory pool management, thread pool optimization, and queue management. At the same time, processing latency was optimized, including asynchronous processing, batch processing, and priority scheduling.

[0207] The functions of each module and unit correspond to the steps in the above-mentioned network anomaly detection and automatic processing method embodiment, and their functions and implementation processes will not be described in detail here.

[0208] This embodiment also provides an electronic device, such as... Figure 9 As shown, the electronic device includes a processor 14 and a memory 13. The memory 13 stores machine-executable instructions that can be executed by the processor 14, which executes the machine-executable instructions to implement the above-described audio control method.

[0209] Furthermore, Figure 9 The electronic device shown also includes a bus 12 and a communication interface 11, with the processor 14, the communication interface 11 and the memory 13 connected via the bus 12.

[0210] The memory 13 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 11 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The bus 12 may be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0211] Processor 14 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 14 or by instructions in software form. Processor 14 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in this embodiment. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this embodiment can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 1001. Processor 1000 reads the information in memory 1001 and combines it with its hardware to complete the steps of the audio control method.

[0212] This disclosure also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, storing a computer program that, when run on a computer, causes the computer to perform the steps of an audio control method.

[0213] Finally, it should be noted that the above description is only a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be pointed out that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

[0214] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0215] Finally, it should be noted that the above description is only a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be pointed out that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

[0216] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A network anomaly detection and automatic handling method, characterized in that, The method comprises the following steps: S1: Collecting multi-dimensional data related to network performance, preprocessing the collected data including noise removal, outlier processing, data standardization, and constructing feature engineering of the preprocessed data; S2: Building a multi-dimensional anomaly detection model based on deep learning and machine learning algorithms, performing time series anomaly detection based on long short-term memory network LSTM, multi-dimensional feature clustering analysis based on density-based spatial clustering algorithm DBSCAN, and rule engine detection based on expert rules through the multi-dimensional anomaly detection model; S3: Classifying the anomalies output by the multi-dimensional anomaly detection model based on a hierarchical classification strategy, and executing corresponding automatic processing strategies according to the type of anomaly; S4: Multi-level burying points for network performance indicators and abnormal events, setting the sampling frequency of the burying points based on a hierarchical sampling strategy, using high-frequency sampling for core indicators and low-frequency sampling for secondary indicators, real-time monitoring of network state at each layer and tracking of abnormal events, trend prediction analysis of network performance data through exponential smoothing method, correlation analysis based on Pearson correlation coefficient to mine data value, and generating optimization suggestions based on the prediction analysis results combined with decision tree algorithm; In step S3, the corresponding automatic processing strategy according to the type of anomaly comprises: A strategy matching algorithm based on multi-dimensional feature similarity is used to calculate the similarity score of the current anomaly and historical cases, and a candidate processing strategy set is generated; The optimal strategy is selected from the processing strategy library according to the strategy application conditions, historical execution effect and resource consumption, including network switching, parameter optimization and fault self-healing, and the rollback point of strategy execution is set; The execution effect of the processing strategy is quantitatively evaluated by calculating the relative improvement degree of the key performance indicators, and the rollback mechanism is triggered if the improvement degree does not meet the expectation; A strategy optimization algorithm based on deep reinforcement learning is used to update the weight of the processing strategy by calculating the reward value of the strategy execution, realizing online learning and optimization of the processing strategy.

2. The network anomaly detection and automatic handling method of claim 1, wherein, In step S1, the multi-dimensional data includes network basic indicators and network advanced indicators, wherein, The network basic indicators include: The received signal strength indicator RSSI of the network is continuously collected according to the configurable time interval to evaluate the weakening degree of the wireless signal in transmission; The network delay RTT is obtained by recording the time difference between sending a specific probe request packet to the target server and receiving the response packet returned by the target server through active probe; The total number of data packets sent in a specific time period is recorded based on the sliding window mechanism at the sending end to obtain the data packet loss rate; The traffic statistics information of the network interface is obtained through the management interface provided by the network device to obtain the broadband utilization rate; The channel quality indicator CQI value is calculated in real time according to the received signal characteristics; The network advanced indicators include: The network stability index NSI is obtained by normalizing and weighted summing the basic indicators based on an adaptive weight mechanism, which is used to comprehensively evaluate the stability of the network, wherein the weight coefficients are adjusted according to the actual situation; The abnormal rate is calculated by the ratio of the number of abnormal samples to the total number of samples through a sliding window method with a configurable size, and is used to evaluate the abnormal situation of the network.

3. The network anomaly detection and automatic handling method of claim 1, wherein, In step S1, the preprocessing of the collected data includes noise removal, outlier processing, and data standardization further includes: Using statistical methods for preliminary anomaly detection, including using Z-score method and box plot method to remove obvious noise and outliers in the data, and filling missing values by interpolation method and nearest neighbor interpolation method; Using Z-score standardization method to convert data features of different ranges and magnitudes to the same scale, normalizing and standardizing the data features, and avoiding the imbalance of different features to the anomaly detection model due to the difference in magnitude.

4. The network anomaly detection and automatic handling method of claim 1, wherein, In step S1, the construction of feature engineering of the preprocessed data further includes: Based on random forest and mutual information evaluation method, relevant features are selected from the collected original data, and the most valuable features for anomaly detection are selected through feature importance evaluation method; Extracting time features including hour, minute, second and period features of time stamp in time series data, and combining multi-dimensional data through interactive feature generation method to capture the non-linear relationship between different dimensions.

5. The network anomaly detection and automatic handling method of claim 4, wherein, In step S2, the multi-dimensional anomaly detection model includes a time series anomaly detection model, a multi-dimensional feature clustering analysis model and a rule engine detection model, wherein the time series anomaly detection model is constructed as follows: Based on the characteristics of time series data, the time series data is arranged into a three-dimensional tensor format including sample number, time step and feature number, and the time series data is split into training set, validation set and test set according to the set proportion; A deep neural network model including multi-layer long short-term memory network (LSTM) structure is constructed, and the model is trained using training set and back propagation algorithm; Based on grid search and cross-validation method, the validation set is input to adjust the hyperparameters, including learning rate, LSTM unit number and Dropout proportion; Through input test set for prediction test, the performance of the time series anomaly detection model is evaluated by using indicators including recall rate, precision rate and F1 score.

6. The network anomaly detection and automatic handling method of claim 5, wherein, The construction of the multi-dimensional feature clustering analysis model includes: Based on feature correlation analysis, multiple dimensions of data are merged into a feature matrix, and a sample data set including multi-dimensional features is created; Based on the density clustering algorithm DBSCAN, the multi-dimensional feature clustering analysis model is constructed, the optimal neighborhood radius and minimum sample number parameters are determined by grid search, and the clustering quality is evaluated by using contour coefficient and DTW distance, and the rationality of outliers is evaluated combined with expert experience.

7. The network anomaly detection and automatic handling method of claim 5, wherein, The construction of the rule engine detection model includes: Based on expert experience and historical abnormal cases, various rules are encoded into executable logic to establish a rule base including rule priority, trigger condition and execution action; Through rule conflict detection and priority sorting mechanism, the rule engine detection model is constructed, and online evaluation and optimization of the rule engine detection model are carried out according to the number of triggered anomalies, processing effect and resource consumption in actual application.

8. The network anomaly detection and automatic handling method according to any of claims 5 to 7, characterized in that, In step S2, the long short-term memory network LSTM-based time series anomaly detection, the density-based spatial clustering of applications with noise DBSCAN-based multi-dimensional feature clustering analysis and the rule engine detection based on expert rules further include: Based on the distribution characteristics of historical data, the standard deviation threshold is determined using the 3σ criterion and the quantile-based method, and the standard deviation is used as an important indicator to measure the deviation between predicted values and actual values. The normal network performance indicator range is predicted by the time series anomaly detection model; The multi-dimensional data is arranged in time series into a feature matrix, and the data is divided into normal clusters and abnormal points based on the density threshold value determined by the multi-dimensional feature clustering model, wherein the density threshold value is dynamically adjusted according to historical data; Based on the priority order, the rule engine detection is performed, and the input network performance data is matched with the rules in the rule library, and when the rule conditions are met, the corresponding abnormal state and severity are returned.

9. The network anomaly detection and automatic handling method of claim 1, wherein, In step S2, the classification of the anomaly output by the multi-dimensional anomaly detection model based on the hierarchical classification strategy further includes: Determine the classification to which the abnormal item belongs based on the hierarchical classification strategy, and the classification of the anomaly includes: Network congestion class anomaly, the feature vector of which is high network delay, high packet loss rate and low bandwidth utilization; Signal interference class anomaly, the feature vector of which is low signal strength, low channel quality indicator and high bit error rate; And hardware failure class anomaly, the feature vector of which is sudden disconnection, abnormal restart and configuration error; At the same time, the severity level of each type of anomaly is set, and the priority is sorted based on the duration, impact range and business impact of the anomaly.

10. The network anomaly detection and automatic handling method of claim 1, wherein, In step S4, the multi-level burying of network performance indicators and abnormal events further includes: Real-time collection and storage of data of each layer of the network including the physical layer, the protocol layer and the application layer, wherein, The physical layer data includes: network interface state, physical connection state, working rate and duplex mode; The protocol layer data includes: TCP / IP performance indicators, protocol stack state and network protocol interaction data; The application layer data includes: service response time, business success rate and user experience data; Real-time recording of abnormal events and their complete life cycle, the abnormal events including network switching events, parameter adjustment events and fault handling events, and the recorded content including event occurrence time, type determination, processing process, execution result and effect evaluation.

11. The network anomaly detection and automatic handling method of claim 1, wherein, In step S4, the generation of optimization suggestions further includes: Based on the adaptive parameter exponential smoothing method, the network performance trend is analyzed, the correlation analysis is performed combining the Pearson correlation coefficient and the mutual information indicator, and the correlation between the network performance indicators is evaluated; Using the decision tree algorithm based on information gain, the significantly correlated indicators are used as decision nodes to construct a network optimization decision model; According to the following factors, the optimization suggestions are comprehensively scored and prioritized: Urgency, based on the severity and impact range of the anomaly; Resource consumption, based on the required resources and technical conditions; Impact range, based on historical optimization effects and business impact evaluation; At the same time, an execution feedback mechanism for optimization suggestions is established to continuously evaluate and improve the optimization effect.

12. A network anomaly detection and automatic handling system, characterized in that, Comprise: An index collection module for collecting multi-dimensional data related to network performance, pre-processing the collected data including noise removal, outlier processing, and data standardization, and constructing feature engineering of the pre-processed data; An anomaly detection module for constructing a multi-dimensional anomaly detection model based on deep learning and machine learning algorithms, performing time series anomaly detection based on long short-term memory network (LSTM), multi-dimensional feature clustering analysis based on density-based spatial clustering of applications with noise (DBSCAN), and rule engine detection based on expert rules through the multi-dimensional anomaly detection model; A strategy execution model for classifying anomalies output by the multi-dimensional anomaly detection model based on a hierarchical classification strategy, and executing corresponding automatic processing strategies according to the anomaly type; A data analysis module for multi-level burying of network performance indicators and abnormal events, setting the sampling frequency of the buried points based on a hierarchical sampling strategy, using high-frequency sampling for core indicators and low-frequency sampling for secondary indicators, real-time monitoring of network state and tracking of abnormal events, trend prediction analysis of network performance data through exponential smoothing method, correlation analysis based on Pearson correlation coefficient to mine data value, and generation of optimization suggestions based on the prediction analysis results combined with decision tree algorithm; Wherein, the corresponding automatic processing strategy according to the anomaly type comprises: A strategy matching algorithm based on multi-dimensional feature similarity, calculating the similarity score of the current anomaly and historical cases to generate a candidate processing strategy set; Selecting the optimal strategy from the processing strategy library according to the strategy applicability conditions, historical execution effect and resource consumption, including network switching, parameter optimization and fault self-healing, and setting the rollback point of strategy execution; Quantitative evaluation of the execution effect of the processing strategy by calculating the relative improvement degree of the key performance indicators, and triggering the rollback mechanism if the improvement degree does not meet the expectation; Using a strategy optimization algorithm based on deep reinforcement learning to update the weight of the processing strategy by calculating the reward value of the strategy execution, realizing online learning and optimization of the processing strategy.

13. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the network anomaly detection and automatic processing method of any one of claims 1-11.

14. An electronic device, comprising: Comprise: One or more processors; Storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, so that the one or more processors realize the network anomaly detection and automatic processing method of any one of claims 1-11.

Images