Network anomaly detection and automatic processing method and system, storage medium and equipment

Abstract

The invention relates to the technical field of network detection optimization, and discloses a network anomaly detection and automatic processing method and system, a storage medium and equipment, and the method comprises the steps: collecting multi-dimensional data related to network performance, carrying out the preprocessing of the collected data, and carrying out the feature engineering construction of the preprocessed data; constructing a multi-dimensional anomaly detection model, and performing time sequence anomaly detection, multi-dimensional feature clustering analysis and rule engine detection through the multi-dimensional anomaly detection model; performing classification and automatic processing on anomalies detected and output through the multi-dimensional anomaly detection model; point burying is carried out on network performance indexes and abnormal events, states of all layers of the network are monitored in real time, the abnormal events are tracked and recorded, meanwhile, prediction analysis is carried out on network performance data, data values are mined through trend and correlation analysis, and optimization suggestions are generated based on prediction analysis results. Through the method, a network anomaly detection and automatic processing system is realized, and the network quality and experience are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of network detection and optimization, and in particular to a network anomaly detection and automatic processing method, system, storage medium and equipment. Background Art

[0002] With the development of 6G network technology, network architecture is becoming increasingly complex, especially in the same SSID scenario, network management faces many challenges.

[0003] The existing technology mainly has the following problems: First, traditional network monitoring solutions are mainly designed for 4G / 5G networks, and have not fully considered the characteristics and needs of 6G networks. There are problems such as untimely network switching, disconnection caused by signal interference, and imperfect automatic reconnection mechanism; Second, the anomaly detection method is too simple, based only on a single indicator threshold judgment, and cannot effectively identify complex abnormal patterns; Third, network anomaly handling is mainly based on manual intervention, with slow response speed and prone to misoperation; Finally, there is a lack of systematic data collection and analysis mechanism, which makes it difficult to support the continuous optimization of network performance.

[0004] It can be seen that the existing network monitoring solutions lack the ability to detect anomalies for 6G specific scenarios, and most of them adopt a passive response mode, which cannot achieve early warning and automatic processing of network anomalies, affecting user experience and network performance. These problems seriously affect the service quality and user experience of 6G networks. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the above-mentioned prior art, and to provide a network anomaly detection and automatic processing method, system, storage medium and device, which can realize anomaly warning by real-time monitoring of network performance indicators, establishing a multi-dimensional anomaly detection model, and combining machine learning algorithms. At the same time, an intelligent automatic processing process is designed, which can automatically perform network switching, signal optimization and other operations according to the anomaly type, and collect abnormal data through buried points for continuous optimization.

[0006] On the one hand, a method for network anomaly detection and automatic processing is provided, comprising the following steps: S1: Collect multi-dimensional data related to network performance, pre-process the collected data including noise removal, outlier processing, and data standardization, and construct feature engineering for the pre-processed data; S2: Building a multidimensional anomaly detection model based on deep learning and machine learning algorithms, and performing time series anomaly detection based on long short-term memory network LSTM, multidimensional feature clustering analysis based on density clustering algorithm DBSCAN, and rule engine detection based on expert rules through the multidimensional anomaly detection model; S3: classifying the anomalies detected and output by the multidimensional anomaly detection model based on a hierarchical classification strategy, and executing a corresponding automatic processing strategy according to the anomaly type; S4: Carry out multi-level tracking of network performance indicators and abnormal events, monitor the status of each layer of the network in real time, and track and record abnormal events. At the same time, perform trend forecasting and analysis on network performance data through exponential smoothing method, perform correlation analysis based on Pearson correlation coefficient to explore data value, and generate optimization suggestions based on the prediction and analysis results by combining decision tree algorithm.

[0007] Furthermore, in step S1, the multi-dimensional data includes basic network indicators and advanced network indicators, wherein: The basic indicators of the collection network include: Continuously collect the network signal strength RSSI according to the configurable time interval to evaluate the attenuation of the wireless signal during transmission; The network delay RTT is obtained by actively detecting the time difference between sending a specific detection request packet to the target server and receiving the response packet returned by the target server. At the sending end, the total number of data packets sent in a specific time period is recorded based on the sliding window mechanism to obtain the data packet loss rate; Obtain traffic statistics of network interfaces through the management interface provided by network devices to obtain bandwidth utilization; Calculate the channel quality index CQI value in real time according to the received signal characteristics; The collected network advanced indicators include: Based on the adaptive weight mechanism, the basic indicators are normalized and weighted summed to obtain the network stability index NSI, which is used to comprehensively evaluate the stability of the network, wherein the weight coefficient is adjusted according to the actual situation; The anomaly rate is obtained by calculating the ratio of the number of abnormal samples to the total number of samples using a sliding window method with a configurable size, which is used to evaluate the abnormality of the network.

[0008] Furthermore, in step S1, the preprocessing of the collected data including noise removal, outlier processing, and data standardization further includes: Use statistical methods to perform preliminary anomaly detection on the data, including using the Z-score method and box plot method to remove obvious noise and outliers in the data, and fill in missing values ​​through interpolation and nearest neighbor interpolation; The Z-score standardization method is used to convert data features of different ranges and magnitudes to the same scale, normalize and standardize the data features, and avoid the imbalance of the contribution of different features to the anomaly detection model due to differences in magnitude.

[0009] Furthermore, in step S1, the construction of feature engineering for the preprocessed data further includes: Select relevant features from the collected raw data based on random forest and mutual information evaluation methods, and screen the most valuable features for anomaly detection through feature importance evaluation methods; Time features including the hour, minute, second and period features of the timestamp are extracted from the time series data, and the features of the multidimensional data are combined through the interactive feature generation method to capture the nonlinear relationship between different dimensions.

[0010] Further, in step S2, the multidimensional anomaly detection model includes a time series anomaly detection model, a multidimensional feature clustering analysis model and a rule engine detection model, wherein constructing the time series anomaly detection model includes: Based on the characteristics of time series data, the time series data is organized into a three-dimensional tensor format including the number of samples, time steps and number of features, and the time series data is split into training set, validation set and test set according to the set ratio; Construct a deep neural network model including a multi-layer long short-term memory network LSTM structure, and use the training set and back-propagation algorithm to train the model; Input validation set based on grid search and cross-validation methods to adjust hyperparameters, including learning rate, number of LSTM units, and dropout ratio; The prediction test is performed by inputting the test set, and the performance of the time series anomaly detection model is evaluated using indicators including recall, precision, and F1 score.

[0011] Preferably, constructing the multidimensional feature clustering analysis model includes: Based on feature correlation analysis, data from multiple dimensions are combined into a feature matrix to create a sample data set including multi-dimensional features; The multi-dimensional feature clustering analysis model is constructed based on the density clustering algorithm DBSCAN. The optimal neighborhood radius and minimum sample number parameters are determined through grid search. The silhouette coefficient and DTW distance are used to evaluate the clustering quality. At the same time, the rationality of outliers is evaluated in combination with expert experience.

[0012] Preferably, constructing the rule engine detection model includes: Encode various rules into executable logic based on expert experience and historical exception cases, and establish a rule library including rule priority, trigger conditions and execution actions; The rule engine detection model is constructed through rule conflict detection and priority sorting mechanism, and the rule engine detection model is evaluated and optimized online according to the number of exceptions triggered in actual applications, processing effects and resource consumption.

[0013] Furthermore, in step S2, the multidimensional anomaly detection model is used to perform time series anomaly detection based on long short-term memory network LSTM, multidimensional feature clustering analysis based on density clustering algorithm DBSCAN, and rule engine detection based on expert rules, further including: Based on the distribution characteristics of historical data, the standard deviation threshold is determined using the 3σ criterion and the quantile-based method, the standard deviation is used as an important indicator to measure the deviation between the predicted value and the actual value, and the normal network performance indicator range is predicted through the time series anomaly detection model; Arrange the multidimensional data into a feature matrix according to the time series, and divide the data into normal clusters and abnormal points based on the density threshold through the multidimensional feature clustering model, wherein the density threshold is dynamically adjusted according to the historical data; The rule engine detection is executed based on the priority order, the input network performance data is matched with the rules in the rule base, and the corresponding abnormal status and severity are returned when the rule conditions are met.

[0014] Furthermore, in step S2, the classifying the anomalies detected and output by the multidimensional anomaly detection model based on the hierarchical classification strategy further includes: The classification to which the abnormal item belongs is determined based on a hierarchical classification strategy, and the abnormal classification includes: Network congestion anomalies, whose characteristic vectors are high network delay, high packet loss rate, and low bandwidth utilization; Signal interference anomalies, whose characteristic vectors are low signal strength, low channel quality index, and high bit error rate; and hardware failure anomalies, whose feature vectors are sudden disconnection, abnormal restart, and configuration errors; At the same time, set a severity level for each type of anomaly and prioritize them based on the duration, scope of impact, and business impact of the anomaly.

[0015] Furthermore, in step S3, executing a corresponding automatic processing strategy according to the exception type further includes: A strategy matching algorithm based on multi-dimensional feature similarity calculates the similarity score between the current anomaly and historical cases and generates a set of candidate processing strategies; According to the policy application conditions, historical execution results and resource consumption, the optimal policy is selected from the processing policy library, including network switching, parameter optimization and fault self-healing, and the rollback point of policy execution is set; The execution effect of the processing strategy is quantitatively evaluated by calculating the relative improvement of key performance indicators. If the improvement does not meet expectations, a rollback mechanism is triggered; A policy optimization algorithm based on deep reinforcement learning is adopted to update the weight of the processing strategy by calculating the reward value of strategy execution, so as to realize online learning and optimization of the processing strategy.

[0016] Furthermore, in step S4, the multi-level tracking of network performance indicators and abnormal events further includes: The sampling frequency of the embedding points is set based on the stratified sampling strategy, with high-frequency sampling for core indicators and low-frequency sampling for secondary indicators; Collect and store data from all layers of the network including the physical layer, protocol layer and application layer in real time. Physical layer data includes: network interface status, physical connection status, working speed and duplex mode; Protocol layer data includes: TCP / IP performance indicators, protocol stack status, and network protocol interaction data; Application layer data includes: service response time, business success rate and user experience data; Real-time recording of abnormal events and their complete life cycle, including network switching events, parameter adjustment events and fault handling events, and the recorded content includes event occurrence time, type determination, handling process, execution results and effect evaluation.

[0017] Furthermore, in step S4, generating optimization suggestions further includes: The exponential smoothing method based on adaptive parameters is used to analyze and predict the trend of network performance. The correlation analysis is performed by combining the Pearson correlation coefficient and mutual information index to evaluate the correlation between network performance indicators. Adopting the decision tree algorithm based on information gain, taking significantly related indicators as decision nodes, and constructing a network optimization decision model; The optimization suggestions are scored and prioritized based on the following factors: Urgency, based on the severity and scope of the anomaly; Resource consumption, based on required resources and technical conditions; Impact scope, based on historical optimization results and business impact assessment; At the same time, an execution feedback mechanism for optimization suggestions will be established to continuously evaluate and improve optimization effects.

[0018] On the other hand, a network anomaly detection and automatic processing system is provided, comprising: The indicator collection module is used to collect multi-dimensional data related to network performance, pre-process the collected data including noise removal, outlier processing, and data standardization, and construct feature engineering for the pre-processed data; Anomaly detection module, used to build a multidimensional anomaly detection model based on deep learning and machine learning algorithms, and to perform time series anomaly detection based on long short-term memory network LSTM, multidimensional feature clustering analysis based on density clustering algorithm DBSCAN, and rule engine detection based on expert rules through the multidimensional anomaly detection model; A strategy execution model, used to classify the anomalies detected and output by the multidimensional anomaly detection model based on a hierarchical classification strategy, and execute a corresponding automatic processing strategy according to the anomaly type; The data analysis module is used to perform multi-level tracking of network performance indicators and abnormal events, monitor the status of each layer of the network in real time, and track and record abnormal events. At the same time, it uses the exponential smoothing method to perform trend forecasting and analysis on network performance data, performs correlation analysis based on the Pearson correlation coefficient to explore the value of data, and combines the decision tree algorithm to generate optimization suggestions based on the forecasting and analysis results.

[0019] At the same time, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement any of the above-mentioned network anomaly detection and automatic processing methods.

[0020] Compared with the prior art, the present invention has the following beneficial effects: The present invention adopts a multi-dimensional indicator collection and analysis system, which realizes comprehensive perception of network status by real-time monitoring of core indicators such as signal strength, network delay, and packet loss rate, combined with an innovative calculation method of the Network Stability Index (NSI); At the anomaly detection level, the present invention integrates the triple detection mechanism of LSTM time series analysis, DBSCAN clustering and expert rule engine, which greatly improves the accuracy and reliability of anomaly identification; In terms of automatic processing, the present invention designs an intelligent strategy generation system based on reinforcement learning, which can automatically execute network switching, parameter optimization or fault self-healing according to the type of anomaly, significantly reducing the need for manual intervention; The present invention establishes a complete data point and analysis system, and continuously optimizes system performance through exponential smoothing prediction and decision tree analysis. This closed-loop optimization mechanism enables the system to continuously evolve and adapt to dynamic changes in the network environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 A flow chart of a network anomaly detection and automatic processing method of the present invention; Figure 2 A schematic diagram of a network performance indicator collection process of the present invention; Figure 3 This is a structural block diagram of a multidimensional anomaly detection model of the present invention; Figure 4 A schematic diagram of an abnormality detection process of the present invention; Figure 5 This is a schematic diagram of an automatic abnormality processing flow of the present invention; Figure 6 A schematic diagram of a data embedding design of the present invention; Figure 7 A schematic diagram of an indicator data analysis and optimization suggestion generation of the present invention; Figure 8 This is a structural block diagram of the network anomaly detection and automatic processing system integration and deployment of the present invention; Fig. 9 The figure is a schematic diagram of an embodiment of an electronic device of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] The embodiment of the present invention designs an innovative calculation method for the network stability index (NSI), which builds a triple anomaly detection mechanism by real-time monitoring of multiple dimensions such as signal strength, network delay, and packet loss rate, and combines LSTM timing analysis, DBSCAN clustering, and expert rule engine. This multi-level detection architecture has not been seen in existing technical solutions. In particular, in terms of anomaly handling, the solution introduces an intelligent strategy generation system based on reinforcement learning, which can adaptively perform network switching, parameter optimization, or fault self-healing according to different types of anomalies.

[0024] The specific implementation of the present invention is described below with reference to the accompanying drawings and embodiments.

[0025] First embodiment See also Figure 1 This embodiment provides a method for network anomaly detection and automatic processing, and the technical solution includes the following steps: S1: Collect multi-dimensional data related to network performance, pre-process the collected data including noise removal, outlier processing, and data standardization, and construct feature engineering for the pre-processed data; S2: Building a multidimensional anomaly detection model based on deep learning and machine learning algorithms, and performing time series anomaly detection based on long short-term memory network LSTM, multidimensional feature clustering analysis based on density clustering algorithm DBSCAN, and rule engine detection based on expert rules through the multidimensional anomaly detection model; S3: classifying the anomalies detected and output by the multidimensional anomaly detection model based on a hierarchical classification strategy, and executing a corresponding automatic processing strategy according to the anomaly type; S4: Carry out multi-level tracking of network performance indicators and abnormal events, monitor the status of each layer of the network in real time, and track and record abnormal events. At the same time, perform trend forecasting and analysis on network performance data through exponential smoothing method, perform correlation analysis based on Pearson correlation coefficient to explore data value, and generate optimization suggestions based on the prediction and analysis results by combining decision tree algorithm.

[0026] Among them, in step S1, the multi-dimensional data includes network basic indicators and network advanced indicators, such as Figure 2 As shown, In this embodiment, collecting basic network indicators includes: According to the set time interval, the network signal strength RSSI is collected every 100ms to evaluate the attenuation of the wireless signal during transmission; The network delay RTT is collected every 500ms through active detection. The network delay RTT is obtained by recording the time difference between sending a specific detection request packet to the target server and receiving the response packet returned by the target server. At the sending end, based on the sliding window mechanism, the total number of packets sent is calculated every 1 second to obtain the packet loss rate PacketLoss; Collect the bandwidth utilization rate every 1 second, and obtain the traffic statistics of the network interface through the management interface provided by the network device to obtain the bandwidth utilization rate; The channel quality index CQI is collected every 200ms, and the channel quality index CQI value is calculated according to the received signal characteristics; In addition, advanced network metrics are collected including: Based on the adaptive weight mechanism, the basic indicators are normalized and weighted summed to obtain the network stability index NSI, which is used to comprehensively evaluate the stability of the network. The calculation formula is as follows: NSI = w1×RSSI_norm + w2×RTT_norm + w3×PacketLoss_norm + w4×Bandwidth_norm + w5×CQI_norm, Among them, the weight coefficients w1~w5 are adjusted according to the actual situation, and xx_norm represents the normalized indicator value of the indicator; The ratio of the number of abnormal samples to the total number of samples is calculated using a sliding window method with a configurable size to obtain the abnormal rate, which is used to evaluate the abnormality of the network. The calculation formula is as follows: AnomalyRate = (number of abnormal samples / total number of samples) × 100%, In this embodiment, the window size of the sliding window is 5 minutes.

[0027] First, ensure that the collected data has a sufficient time span and a sufficient number of samples to cover different network states, including normal states and possible abnormal states. Then preprocess the collected data, including: Use statistical methods to perform preliminary anomaly detection on the data, such as using the Z-score method, box plot method, and standard deviation or IQR method to remove obvious noise and outliers in the data for more accurate anomaly detection, and fill in missing values ​​through interpolation and nearest neighbor interpolation; The data features of different ranges and magnitudes are converted to the same scale, and the data features are normalized and standardized to avoid the imbalance of the contribution of different features to the anomaly detection model due to magnitude differences. In this embodiment, the minimum-maximum normalization or Z-score normalization method is used to scale the data to the range of [0, 1].

[0028] Then, the preprocessed data is subjected to feature engineering, including: Based on the random forest and mutual information evaluation methods, relevant features are selected from the collected raw data. This embodiment uses correlation analysis, such as the Pearson correlation coefficient, to evaluate the correlation between different features, avoiding the use of highly correlated features to avoid generating redundant information. Then, the most valuable features for anomaly detection are screened through feature importance evaluation methods, such as feature importance in random forests.

[0029] Extract the time features of time series data, including the hour, minute, second and period features of the timestamp, to reflect the time regularity of the data; and combine the features of multidimensional data through interactive feature generation methods, and capture the nonlinear relationship between different dimensions by calculating the ratio or difference between the features.

[0030] The multidimensional anomaly detection model includes a time series anomaly detection model, a multidimensional feature clustering analysis model and a rule engine detection model. Figure 3 shown.

[0031] Wherein, constructing the time series anomaly detection model includes: Based on the characteristics of the time series data, the time series data is organized into a format suitable for the input of the time series anomaly detection model, including a three-dimensional tensor of the number of samples, time steps, and the number of features, and the time series data is split into a training set, a validation set, and a test set; Build a deep neural network model with a multi-layer LSTM structure. The LSTM includes an input layer, one or more LSTM layers, a Dropout layer (to prevent overfitting), and an output layer. Use Keras to build a simple LSTM model and use the training set and back-propagation algorithm to train the model. Based on the grid search and cross-validation methods, the validation set is input to adjust the hyperparameters, including the learning rate, the number of LSTM units, and the Dropout ratio; The test set is input for prediction testing, and the performance of the time series anomaly detection model is evaluated using indicators including recall, precision, and F1 score.

[0032] Secondly, constructing the multidimensional feature clustering analysis model includes: Merge data of multiple dimensions into a feature matrix to create a sample data set where each sample includes multiple features; The multidimensional feature clustering analysis model is constructed based on the density clustering algorithm DBSCAN, the optimal neighborhood radius and minimum sample number parameters are determined through grid search, the sample data are input into the multidimensional feature clustering analysis model, the silhouette coefficient is used to evaluate the clustering quality, and the proportion and rationality of outliers are statistically analyzed in combination with expert experience.

[0033] Again, building the rule engine detection model includes: Encode various rules into executable logic based on expert experience and historical exception cases, and establish a rule library including rule priority, trigger conditions and execution actions; The rule engine detection model is constructed through rule conflict detection and priority sorting mechanism, and the rule engine detection model is evaluated and optimized online according to the number of exceptions triggered in actual applications, processing effects and resource consumption.

[0034] Then, the multidimensional anomaly detection model is used to perform time series anomaly detection based on long short-term memory network LSTM, multidimensional feature clustering analysis based on density clustering algorithm DBSCAN, and rule engine detection based on expert rules, such as Figure 4 As shown, specifically including: S10: Based on the historical data distribution characteristics, the standard deviation threshold is determined using the 3σ criterion and the quantile-based method. When performing time series anomaly detection through the trained time series anomaly detection model, the standard deviation is used as an important indicator to measure the deviation between the predicted value and the actual value, and the normal network performance indicator range is predicted. In this embodiment, the abnormality determination conditions are as follows: If |actual value - predicted value| > 3×σ (standard deviation), it is considered abnormal. For the RTT indicator, if the predicted value is 50ms and the standard deviation is 10ms, it is considered abnormal when the actual value exceeds 80ms; S20: Using data of multiple dimensions as features, organizing data at different time points into a multidimensional data matrix, each sample point includes feature values ​​of multiple dimensions, and dividing areas with sufficiently high density into clusters through the multidimensional feature clustering model, and treating points in low-density areas as noise or outliers. In this embodiment, the core parameters are: ε=0.5, MinPts= 4, creating a DBSCAN object, where ε is the neighborhood radius and MinPts is the minimum number of samples, inputting the feature matrix X into the model for clustering, and predicting the cluster to which each sample belongs, and -1 represents an outlier, i.e., an abnormal point; S30: Execute the rule engine detection based on the priority order, and match the input network performance data with the rules in the rule base, that is, judge the input network performance data through different rules, and return the corresponding abnormal state if a certain rule condition is met, otherwise return to normal. In this embodiment, the rule base is as follows: Rule 1: If RTT > 200ms and duration > 30s, it is considered abnormal; Rule 2: If PacketLoss > 5% and duration > 15s, it is considered abnormal; Rule 3: If the bandwidth utilization is < 20% and the signal strength is normal, it is considered abnormal.

[0035] In step S2, the classifying of the detected anomalies further comprises: The classification to which the abnormal item belongs is determined based on a hierarchical classification strategy, and the abnormal classification includes: Network congestion anomalies, whose characteristic vectors are high network delay, high packet loss rate, and low bandwidth utilization; Signal interference anomalies, whose characteristic vectors are low signal strength, low channel quality index, and high bit error rate; and hardware failure anomalies, whose feature vectors are sudden disconnection, abnormal restart, and configuration errors; At the same time, set a severity level for each type of exception and prioritize them based on the duration, scope of impact, and business impact of the exception. This is shown as follows: A. Network congestion exception: Feature vector = [high RTT, high packet loss rate, low bandwidth utilization]; B. Signal interference anomalies: Feature vector = [low RSSI, low CQI, high BER]; C. Hardware failure exception: Feature vector = [sudden disconnection, abnormal restart, configuration error].

[0036] Then, the anomalies detected and output by the multidimensional anomaly detection model are automatically processed, such as Figure 5 As shown, including: S100: A strategy matching algorithm based on multi-dimensional feature similarity calculates the similarity score between the current anomaly detection result and the historical case according to the similarity between the current anomaly detection result and the historical case in multiple dimensions, and generates a processing strategy. In this embodiment, the algorithm formula is as follows: Score(strategy i) = Σ(wj × similarity j), Among them, wj is the weight of each dimension, and similarity j is the similarity between the current anomaly and the historical case; S200: According to the calculated processing strategy, the policy applicable conditions, the historical execution effect and the resource consumption, the optimal strategy is selected from the processing strategy library, including network switching, parameter optimization and fault self-healing, and a rollback point for the policy execution is set. In this embodiment, the processing strategy library is as follows: Strategy 1: Network switching Applicable conditions: Current network quality score < 60 and backup network is available Execution steps: a) Scan for available networks, b) assess the quality of the backup network, c) perform smooth switching; Strategy 2: Parameter Optimization Applicable conditions: 60 ≤ Network quality score < 80 Execution steps: a) Adjust the transmit power, b) Optimize channel configuration, c) Update QoS policy; Strategy 3: Fault self-healing Applicable conditions: Hardware-level anomalies detected Execution steps: a) Try resetting the network interface, b) Restore the default configuration, c) Start the backup module; S300: By calculating the relative improvement of key performance indicators, that is, calculating the relative change of the network performance indicator value after the processing strategy is executed and the initial value before the processing, the effectiveness of the processing strategy is quantitatively evaluated. If the improvement degree does not meet expectations, the rollback mechanism is triggered. The calculation formula is as follows: EffectScore = (index value after treatment - index value before treatment) / index value before treatment × 100%; S400: A strategy optimization algorithm based on deep reinforcement learning is used to calculate the deviation between the execution effect of the actual processing strategy and the expected effect, and the weight of the processing strategy is updated in combination with the characteristic value to optimize the subsequent strategy selection. The strategy optimization formula is as follows: w'i = wi + α × (actual effect - expected effect) × i, Among them, α is the learning rate, which is 0.1, and i is the eigenvalue.

[0037] Further, in step S4, the network performance indicators and abnormal events are buried at multiple levels. Figure 6 As shown, specifically including: The sampling frequency of the tracking points is set based on the importance of the stratified sampling strategy. High-frequency sampling is used for core indicators, and low-frequency sampling is used for secondary indicators. The sampling range is 1s-5min. The network interface status (point 1), protocol stack performance (point 2) and application layer experience (point 3) are obtained in real time. The physical connection status, working rate and duplex mode of the network interface are monitored, and the performance of different protocol layers is evaluated. The performance of the application is evaluated from the user's perspective. Collect and store data from all layers of the network including the physical layer, protocol layer and application layer in real time. Physical layer data includes: network interface status, physical connection status, working speed and duplex mode; Protocol layer data includes: TCP / IP performance indicators, protocol stack status, and network protocol interaction data; Application layer data includes: service response time, business success rate and user experience data; Real-time recording of abnormal events and their complete life cycle, including network switching events, parameter adjustment events and fault handling events, and the recorded content includes event occurrence time, type determination, handling process, execution results and effect evaluation.

[0038] Then, if Figure 7 As shown, generating optimization suggestions further includes: S101: Analyze and predict network performance trends based on exponential smoothing method with adaptive parameters. The formula is as follows: St = α × Yt + (1-α) × St-1, where α is the smoothing coefficient, which is 0.3, Yt is the actual observation value at time t, and St is the smoothed value at time t; Then, the Pearson correlation coefficient and mutual information index are combined to perform correlation analysis to evaluate the correlation between network performance indicators. The formula is as follows: r = Σ((Xi- )(Yi- )) / √(Σ(Xi- )²×Σ(Yi- )²), in, and is the mean of two variables X and Y. The Pearson correlation coefficient r measures the linear correlation between two variables X and Y, ranging from -1 to 1, to evaluate the relationship between network performance indicators, where r = 1 indicates a perfect positive correlation, r = -1 indicates a perfect negative correlation, and r = 0 indicates no linear correlation; S201: Use the CART algorithm based on information gain to build a decision tree, use significantly related indicators as decision nodes, and build a network optimization decision model; S301: Comprehensively score and prioritize the optimization suggestions based on the following factors: Urgency, based on the severity and scope of the anomaly; Resource consumption, based on required resources and technical conditions; Impact scope, based on historical optimization results and business impact assessment; At the same time, an execution feedback mechanism for optimization suggestions is established to continuously evaluate and improve the optimization effect. The formula is as follows: Priority = Urgency × Impact × Resource, Among them, Urgency: urgency (1-5 points), Impact: impact range (1-5 points), Resource: resource consumption (1-5 points).

[0039] In summary, the method of this embodiment provides a complete network anomaly detection and automatic processing solution through systematic design and implementation. Combining technologies such as machine learning, expert systems, and automated control, it can effectively improve network service quality and user experience. It breaks through the technical bottleneck of passive response of traditional network monitoring and builds a closed-loop intelligent network management system. First, at the indicator collection level, the solution designs a multi-dimensional data collection system based on the netguard framework, and introduces a sliding window mechanism for dynamic evaluation. Secondly, at the anomaly identification level, by integrating multiple machine learning algorithms, the accuracy and reliability of anomaly detection are significantly improved. Thirdly, at the automatic processing level, by establishing a complete data embedding and analysis system, continuous optimization and self-evolution of the system are achieved.

[0040] This embodiment also provides a network anomaly detection and automatic processing system, including: The indicator collection module is used to collect multi-dimensional data related to network performance, pre-process the collected data including noise removal, outlier processing, and data standardization, and construct feature engineering for the pre-processed data; Anomaly detection module, used to build a multidimensional anomaly detection model based on deep learning and machine learning algorithms, and to perform time series anomaly detection based on long short-term memory network LSTM, multidimensional feature clustering analysis based on density clustering algorithm DBSCAN, and rule engine detection based on expert rules through the multidimensional anomaly detection model; A strategy execution model, for classifying anomalies detected and output by the multidimensional anomaly detection model based on a hierarchical classification strategy, and executing a corresponding automatic processing strategy according to the anomaly type; The data analysis module is used to perform multi-level tracking of network performance indicators and abnormal events, monitor the status of each layer of the network in real time, and track and record abnormal events. At the same time, it uses the exponential smoothing method to perform trend forecasting and analysis on network performance data, performs correlation analysis based on the Pearson correlation coefficient to explore the value of data, and combines the decision tree algorithm to generate optimization suggestions based on the forecasting and analysis results.

[0041] Among them, the legal person integration and deployment structure of the network anomaly detection and automatic processing system is as follows Figure 8 As shown, in order to achieve performance optimization, resource usage is optimized, including memory pool management, thread pool optimization and queue management. At the same time, processing delay is optimized, including asynchronous processing, batch processing and priority scheduling.

[0042] The functional implementation of each module and unit corresponds to the steps in the above-mentioned network anomaly detection and automatic processing method embodiment, and its functions and implementation processes are not repeated here one by one.

[0043] This embodiment also provides an electronic device, such as Fig. 9 As shown, the electronic device includes a processor 14 and a memory 13. The memory 13 stores machine executable instructions that can be executed by the processor 14. The processor 14 executes the machine executable instructions to implement the above audio control method.

[0044] Further, Fig. 9 The electronic device shown further includes a bus 12 and a communication interface 11 , and the processor 14 , the communication interface 11 and the memory 13 are connected via the bus 12 .

[0045] The memory 13 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 11 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 12 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0046] The processor 14 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 14. The above processor 14 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in this embodiment can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in this embodiment can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1001, and the processor 1000 reads the information in the memory 1001 and completes the steps of the audio control method in combination with its hardware.

[0047] The present disclosure also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program runs on a computer, the computer executes the steps of the audio control method.

[0048] Finally, it should be noted that the above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

[0049] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0050] Finally, it should be noted that the above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

[0051] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A network anomaly detection and automatic processing method, characterized in that: The steps include: S1: Collect multi-dimensional data related to network performance, pre-process the collected data including noise removal, outlier processing, and data standardization, and construct feature engineering for the pre-processed data; S2: Building a multidimensional anomaly detection model based on deep learning and machine learning algorithms, and performing time series anomaly detection based on long short-term memory network LSTM, multidimensional feature clustering analysis based on density clustering algorithm DBSCAN, and rule engine detection based on expert rules through the multidimensional anomaly detection model; S3: classifying the anomalies detected and output by the multidimensional anomaly detection model based on a hierarchical classification strategy, and executing a corresponding automatic processing strategy according to the anomaly type; S4: Carry out multi-level tracking of network performance indicators and abnormal events, monitor the status of each layer of the network in real time, and track and record abnormal events. At the same time, perform trend forecasting and analysis on network performance data through exponential smoothing method, perform correlation analysis based on Pearson correlation coefficient to explore data value, and generate optimization suggestions based on the prediction and analysis results by combining decision tree algorithm.

2. The network anomaly detection and automatic processing method according to claim 1 is characterized in that: In step S1, the multi-dimensional data includes basic network indicators and advanced network indicators, wherein: The basic indicators of the collection network include: Continuously collect the network signal strength RSSI according to the configurable time interval to evaluate the attenuation of the wireless signal during transmission; The network delay RTT is obtained by actively detecting the time difference between sending a specific detection request packet to the target server and receiving the response packet returned by the target server. At the sending end, the total number of data packets sent in a specific time period is recorded based on the sliding window mechanism to obtain the data packet loss rate; Obtain traffic statistics of network interfaces through the management interface provided by network devices to obtain bandwidth utilization; Calculate the channel quality index CQI value in real time according to the received signal characteristics; The collected network advanced indicators include: Based on the adaptive weight mechanism, the basic indicators are normalized and weighted summed to obtain the network stability index NSI, which is used to comprehensively evaluate the stability of the network, wherein the weight coefficient is adjusted according to the actual situation; The anomaly rate is obtained by calculating the ratio of the number of abnormal samples to the total number of samples using a sliding window method with a configurable size, which is used to evaluate the abnormality of the network.

3. The network anomaly detection and automatic processing method according to claim 1 is characterized in that: In step S1, the preprocessing of the collected data including noise removal, outlier processing, and data standardization further includes: Use statistical methods to perform preliminary anomaly detection on the data, including using the Z-score method and box plot method to remove obvious noise and outliers in the data, and fill in missing values ​​through interpolation and nearest neighbor interpolation; The Z-score standardization method is used to convert data features of different ranges and magnitudes to the same scale, normalize and standardize the data features, and avoid the imbalance of the contribution of different features to the anomaly detection model due to differences in magnitude.

4. The network anomaly detection and automatic processing method according to claim 1 is characterized in that: In step S1, the construction of feature engineering for the preprocessed data further includes: Select relevant features from the collected raw data based on random forest and mutual information evaluation methods, and screen the most valuable features for anomaly detection through feature importance evaluation methods; Time features including the hour, minute, second and period features of the timestamp are extracted from the time series data, and the features of the multidimensional data are combined through the interactive feature generation method to capture the nonlinear relationship between different dimensions.

5. The network anomaly detection and automatic processing method according to claim 4 is characterized in that: In step S2, the multidimensional anomaly detection model includes a time series anomaly detection model, a multidimensional feature clustering analysis model and a rule engine detection model, wherein constructing the time series anomaly detection model includes: Based on the characteristics of time series data, the time series data is organized into a three-dimensional tensor format including the number of samples, time steps and number of features, and the time series data is split into training set, validation set and test set according to the set ratio; Construct a deep neural network model including a multi-layer long short-term memory network LSTM structure, and use the training set and back-propagation algorithm to train the model; Input validation set based on grid search and cross-validation methods to adjust hyperparameters, including learning rate, number of LSTM units, and dropout ratio; The prediction test is performed by inputting the test set, and the performance of the time series anomaly detection model is evaluated using indicators including recall, precision, and F1 score.

6. The network anomaly detection and automatic processing method according to claim 5 is characterized in that: Constructing the multidimensional feature clustering analysis model includes: Based on feature correlation analysis, data from multiple dimensions are combined into a feature matrix to create a sample data set including multi-dimensional features; The multi-dimensional feature clustering analysis model is constructed based on the density clustering algorithm DBSCAN. The optimal neighborhood radius and minimum sample number parameters are determined through grid search. The silhouette coefficient and DTW distance are used to evaluate the clustering quality. At the same time, the rationality of outliers is evaluated in combination with expert experience.

7. The network anomaly detection and automatic processing method according to claim 5 is characterized in that: Building the rule engine detection model includes: Encode various rules into executable logic based on expert experience and historical exception cases, and establish a rule library including rule priority, trigger conditions and execution actions; The rule engine detection model is constructed through rule conflict detection and priority sorting mechanism, and the rule engine detection model is evaluated and optimized online according to the number of exceptions triggered in actual applications, processing effects and resource consumption.

8. The network anomaly detection and automatic processing method according to any one of claims 5 to 7, characterized in that: In step S2, the multidimensional anomaly detection model further includes: performing time series anomaly detection based on long short-term memory network LSTM, multidimensional feature clustering analysis based on density clustering algorithm DBSCAN, and rule engine detection based on expert rules: Based on the distribution characteristics of historical data, the standard deviation threshold is determined using the 3σ criterion and the quantile-based method, the standard deviation is used as an important indicator to measure the deviation between the predicted value and the actual value, and the normal network performance indicator range is predicted through the time series anomaly detection model; Arrange the multidimensional data into a feature matrix according to the time series, and divide the data into normal clusters and abnormal points based on the density threshold through the multidimensional feature clustering model, wherein the density threshold is dynamically adjusted according to the historical data; The rule engine detection is executed based on the priority order, and the input network performance data is matched with the rules in the rule base. When the rule conditions are met, the corresponding abnormal status and severity are returned.

9. The network anomaly detection and automatic processing method according to claim 1, characterized in that: In step S2, the classifying the anomalies detected and output by the multidimensional anomaly detection model based on the hierarchical classification strategy further includes: The classification to which the abnormal item belongs is determined based on a hierarchical classification strategy, and the abnormal classification includes: Network congestion anomalies, whose characteristic vectors are high network delay, high packet loss rate, and low bandwidth utilization; Signal interference anomalies, whose characteristic vectors are low signal strength, low channel quality index, and high bit error rate; and hardware failure anomalies, whose feature vectors are sudden disconnection, abnormal restart, and configuration errors; At the same time, set a severity level for each type of anomaly and prioritize them based on the duration, scope of impact, and business impact of the anomaly.

10. The network anomaly detection and automatic processing method according to claim 1, characterized in that: In step S3, executing the corresponding automatic processing strategy according to the exception type further includes: A strategy matching algorithm based on multi-dimensional feature similarity calculates the similarity score between the current anomaly and historical cases and generates a set of candidate processing strategies; According to the policy application conditions, historical execution results and resource consumption, the optimal policy is selected from the processing policy library, including network switching, parameter optimization and fault self-healing, and the rollback point of policy execution is set; The execution effect of the processing strategy is quantitatively evaluated by calculating the relative improvement of key performance indicators. If the improvement does not meet expectations, a rollback mechanism is triggered; A policy optimization algorithm based on deep reinforcement learning is adopted to update the weight of the processing strategy by calculating the reward value of strategy execution, so as to realize online learning and optimization of the processing strategy.

11. The network anomaly detection and automatic processing method according to claim 1, characterized in that: In step S4, the multi-level tracking of network performance indicators and abnormal events further includes: The sampling frequency of the embedding points is set based on the stratified sampling strategy, with high-frequency sampling for core indicators and low-frequency sampling for secondary indicators; Collect and store data from all layers of the network including the physical layer, protocol layer and application layer in real time. Physical layer data includes: network interface status, physical connection status, working speed and duplex mode; Protocol layer data includes: TCP / IP performance indicators, protocol stack status, and network protocol interaction data; Application layer data includes: service response time, business success rate and user experience data; Real-time recording of abnormal events and their complete life cycle, including network switching events, parameter adjustment events and fault handling events, and the recorded content includes event occurrence time, type determination, handling process, execution results and effect evaluation.

12. The network anomaly detection and automatic processing method according to claim 1, characterized in that: In step S4, generating optimization suggestions further includes: The exponential smoothing method based on adaptive parameters is used to analyze and predict the trend of network performance. The correlation analysis is performed by combining the Pearson correlation coefficient and mutual information index to evaluate the correlation between network performance indicators. Adopting the decision tree algorithm based on information gain, taking significantly related indicators as decision nodes, and constructing a network optimization decision model; The optimization suggestions are scored and prioritized based on the following factors: Urgency, based on the severity and scope of the anomaly; Resource consumption, based on required resources and technical conditions; Impact scope, based on historical optimization results and business impact assessment; At the same time, an execution feedback mechanism for optimization suggestions will be established to continuously evaluate and improve optimization effects.

13. A network anomaly detection and automatic processing system, characterized in that: include: The indicator collection module is used to collect multi-dimensional data related to network performance, pre-process the collected data including noise removal, outlier processing, and data standardization, and construct feature engineering for the pre-processed data; Anomaly detection module, used to build a multidimensional anomaly detection model based on deep learning and machine learning algorithms, and to perform time series anomaly detection based on long short-term memory network LSTM, multidimensional feature clustering analysis based on density clustering algorithm DBSCAN, and rule engine detection based on expert rules through the multidimensional anomaly detection model; A strategy execution model, used to classify the anomalies detected and output by the multidimensional anomaly detection model based on a hierarchical classification strategy, and execute a corresponding automatic processing strategy according to the anomaly type; The data analysis module is used to perform multi-level tracking of network performance indicators and abnormal events, monitor the status of each layer of the network in real time, and track and record abnormal events. At the same time, it uses the exponential smoothing method to perform trend forecasting and analysis on network performance data, performs correlation analysis based on the Pearson correlation coefficient to explore the value of data, and combines the decision tree algorithm to generate optimization suggestions based on the forecasting and analysis results.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the network anomaly detection and automatic processing method according to any one of claims 1 to 12 is implemented.

15. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the network anomaly detection and automatic processing method as described in any one of claims 1-12.

Images