Broadband user-based optical modem power failure and fiber breaking alarm real-time monitoring method

Abstract

The invention relates to the field of broadband alarm, in particular to a broadband user-based optical modem power failure and fiber break alarm real-time monitoring method, which comprises the following steps of: S1, monitoring and recording the online state, the light attenuation level, the flow information, the packet loss rate, the network delay and the log of an optical modem under OLT (Optical Line Terminal) equipment in real time; s2, automatically identifying an abnormal mode and a potential fault according to a model established by a machine learning algorithm and a deep learning algorithm; s3, automatically diagnosing problem reasons according to the collected data, and optimizing the knowledge base in combination with an expert system; s4, performing deep mining and analysis on historical fault data, and predicting a potential fault risk based on the historical data and a real-time monitoring result; and S5, continuously updating and optimizing a fault detection and diagnosis model through a self-learning algorithm model, and uniformly monitoring and managing all data acquisition, analysis and alarm modules. The fault detection accuracy and timeliness can be improved, and potential fault prediction and preventive maintenance are realized.

Description

Technical Field

[0001] The present invention relates to the field of broadband alarms, and particularly to a method for real-time monitoring of power-off and fiber-disconnection alarms of broadband user optical network terminals (ONTs). Background Art

[0002] With the wide application of the fiber to the home (FTTH) technology, the requirements of home and enterprise users for network stability and continuity are increasing day by day. As a key device for home broadband access, the operation status of the ONT directly affects the user's network experience. Traditional monitoring methods mainly rely on periodic manual inspections or simple online monitoring. This method is inefficient, inaccurate, and difficult to detect potential faults in a timely manner. It is impossible to discover and locate power-off or fiber-disconnection problems in the first time, resulting in service response delays and a decline in user satisfaction. In a modern and complex network environment, ONT failures may not only be caused by problems with the device itself, but also be affected by external factors such as fiber breaks and power failures. Therefore, how to achieve real-time monitoring of ONT power-off and fiber-disconnection problems, discover and handle faults in a timely manner, has become the key to improving network service quality. Summary of the Invention

[0003] To solve the above problems, the present invention provides a method for real-time monitoring of power-off and fiber-disconnection alarms of broadband user ONTs, which can improve the accuracy and timeliness of fault detection, realize potential fault prediction and preventive maintenance, enhance network management efficiency and unity, and strengthen network stability and reliability.

[0004] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] A method for real-time monitoring of power-off and fiber-disconnection alarms of broadband user ONTs, the method comprising:

[0006] S1: Real-time monitor and record the online status, optical attenuation level, traffic information, packet loss rate, network latency, and logs of the ONTs under the OLT device, and real-time monitor the power status and fiber connection status of the ONTs through a probe system;

[0007] S2: According to the OLT device data, user offline data, and probe detection data, as well as a model composed of machine learning algorithms and deep learning algorithms, automatically identify abnormal patterns and potential faults, and trigger an alarm mechanism;

[0008] S3: Automatically diagnose the cause of the problem based on the collected data, combine with an expert system to provide fault analysis and handling suggestions, record each processing process and result, and optimize the knowledge base;

[0009] S4: Optimize the system's fault prediction and prevention capabilities by deeply mining and analyzing historical fault data, visually display the trends and outliers of historical data, predict potential fault risks based on historical data and real-time monitoring results, and propose preventive maintenance suggestions;

[0010] S5: Continuously update and optimize the fault detection and diagnosis model through a self-learning algorithm model, automatically identify new fault patterns, and uniformly monitor and manage all data collection, analysis, and alarm modules according to the integrated management platform.

[0011] Furthermore, the online status, optical attenuation level, traffic information, packet loss rate, network latency, and logs of the optical network terminals (ONTs) under the OLT device are monitored and recorded in real time, and the power status and fiber connection status of the ONTs are monitored in real time through a probe system; it includes the following steps:

[0012] S11: Automatically scan for OLT devices and ONTs in the network through the device discovery function of the probe system, add them to the monitoring list, and collect the status information of the ONTs under the OLT device in real time through the SNMP protocol and the probe system;

[0013] S12: The probe system regularly collects various parameter data of the ONTs under the OLT device at set time intervals, judges whether the online status and optical attenuation level of the ONTs are normal according to the collected data, and whether the traffic, packet loss rate, and network latency are within a reasonable range, and detects the power status and fiber connection status of the ONTs;

[0014] S13: Record the collected various parameter data and log information in the database, regularly analyze the recorded data, and optimize and adjust the network according to the data analysis results.

[0015] Furthermore, the model formed according to the OLT device data, user offline data, and probe detection data, as well as machine learning algorithms and deep learning algorithms, automatically identifies abnormal patterns and potential faults and triggers an alarm mechanism; it includes the following steps:

[0016] S21: Collect the offline records of users through the business system, record the time and frequency information of user offline, associate the probe detection data with timestamps, and perform data cleaning, feature extraction, and data standardization on the OLT device data, user offline data, and probe detection data;

[0017] S22: Integrate the support vector machine and deep neural network algorithms according to the framework structure, divide the sorted data into a training set and a test set, and train, evaluate, and optimize the model;

[0018] S23: The OLT device data, user offline data, and probe detection data collected in real time are input into the trained model. The model automatically identifies abnormal patterns based on the input data and further detects potential faults by combining the abnormal patterns with other data features;

[0019] S24: When the model detects that the abnormal pattern or potential fault conforms to the alarm rule, it automatically triggers the alarm mechanism and promptly notifies the network administrator and relevant technical support personnel.

[0020] Furthermore, integrating the support vector machine and the deep neural network algorithm according to the framework structure, dividing the sorted data into a training set and a test set, and training, evaluating, and optimizing the model; including the following steps:

[0021] S221: Divide the sorted data into a training set and a test set, and divide the training set into a training set and a validation set. The division process maintains the independence and randomness of the data;

[0022] S222: Optimize the parameters of the support vector machine through the training set data, train the deep neural network, adjust the weights and biases of the network, optimize the performance of the network, and monitor the performance and convergence of the model during the training process;

[0023] S223: Evaluate the trained model through the test set, calculate the indicators such as the accuracy rate, recall rate, and F1 value of the model, and optimize and adjust the model according to the evaluation results;

[0024] S224: The framework structure adopts a hierarchical structure. Among them, the support vector machine is used as the first layer, and the deep neural network is used as the second layer. The output of the support vector machine is used as the input of the deep neural network;

[0025] S225: Integrate the support vector machine and the deep neural network through the weighted average method, and adjust the weights of the support vector machine and the deep neural network during the integration process.

[0026] Furthermore, the OLT device data, user offline data, and probe detection data collected in real time are input into the trained model. The model automatically identifies abnormal patterns based on the input data and further detects potential faults by combining the abnormal patterns with other data features; including the following steps:

[0027] S231: Through the data transmission protocol, transmit the collected data to the central server, store the data using the distributed storage technology, and perform lossless compression and regular backup;

[0028] S232: The support vector machine makes a preliminary classification of the input data to determine whether the data is abnormal. If the support vector machine classifies the data as abnormal, input the data into the deep neural network for analysis;

[0029] S233: The deep neural network outputs the type and probability of an anomaly based on the input data. When the anomaly probability exceeds a set threshold, it is determined as an abnormal mode.

[0030] S234: Detect potential faults through a clustering analysis method based on the abnormal mode and data characteristics. Select the clusters that may have potential faults from the clustering analysis results, and identify the periodic changes and sudden anomalies in the data through a time series analysis method.

[0031] Furthermore, the detecting of potential faults through a clustering analysis method based on the abnormal mode and data characteristics, selecting the clusters that may have potential faults from the clustering analysis results, and identifying the periodic changes and sudden anomalies in the data through a time series analysis method; includes the following steps:

[0032] S2341: Determine the value of K by using the silhouette coefficient method through the K-Means clustering algorithm. Randomly select K initial clustering centers, calculate the distance between each data point and each clustering center, assign the data point to the closest cluster, recalculate the center of each cluster, and repeat the calculation until the clustering center no longer changes or reaches a certain number of iterations.

[0033] S2342: Calculate the statistical characteristics of each cluster, compare the characteristic differences between different clusters, and judge whether the cluster represents a normal state or a potential fault. If the data points in a cluster have significantly different characteristics from other clusters, or the scale of the cluster is small, it indicates the existence of a potential fault.

[0034] S2343: Select the time series data corresponding to the clusters that may have potential faults from the clustering analysis results, ensure that the time series data is arranged in chronological order and perform preprocessing.

[0035] S2344: Convert the time series data to the frequency domain, analyze the spectrogram to determine the periodic components of specific frequencies existing in the data. Calculate the spectrum of the time series data through the fast Fourier transform algorithm. If peaks of specific frequencies are found in the spectrogram, it indicates the existence of periodic changes of the corresponding periods in the data.

[0036] S2345: Calculate the moving average and moving standard deviation of the time series data through a method based on moving average and standard deviation. If the data value at a certain time point exceeds the range of the moving average plus or minus a certain multiple of the moving standard deviation, it is judged as a sudden anomaly.

[0037] Furthermore, recording each processing procedure and result and optimizing the knowledge base are based on the analysis results of S2, combined with the known network architecture and device characteristics, inferring the rule base according to the problem causes, and performing fault analysis and recommendation generation according to the expert system containing the expert knowledge and experience in the network field. When network administrators or technical support personnel perform fault handling according to the recommendations of the expert system, record the entire handling process and the results of the fault handling. Update and improve the knowledge base according to the records and results of each fault handling, and use machine learning technology to learn and extract knowledge from historical handling records.

[0038] Furthermore, by deeply mining and analyzing historical fault data, optimizing the system's fault prediction and prevention capabilities, and visually displaying the trends and abnormal points of historical data, predicting potential fault risks based on historical data and real-time monitoring results, and proposing preventive maintenance suggestions; including the following steps:

[0039] S41: Extract historical fault data from the database, including OLT device data, user offline data, probe detection data, and the records and results of each fault handling, preprocess the data, and store the preprocessed historical fault data in the historical data warehouse;

[0040] S42: Represent the data in the form of a transaction dataset, where each transaction represents a fault case. Use the FP-Growth algorithm to construct a frequent pattern tree, mine frequent item sets from the frequent pattern tree, and interpret and analyze the possibility of parameter combination faults for the mined frequent item sets;

[0041] S43: Display the historical fault data in an intuitive chart form through PowerBI, design visual charts according to the analysis purposes and requirements, implement interactive visualization functions, and allow users to deeply explore the data through interactive operations;

[0042] S44: Establish a fault prediction model according to machine learning algorithms, evaluate and optimize it, and apply the trained prediction model to real-time monitoring data to predict potential fault risks in real time;

[0043] S45: Analyze potential fault risks according to the output results of the fault prediction model, combine with the analysis results of historical fault data, understand the causes and handling methods of faults, and propose specific preventive maintenance suggestions according to the analysis results and in combination with the knowledge base.

[0044] Furthermore, establishing a fault prediction model according to machine learning algorithms, evaluating and optimizing it, and applying the trained prediction model to real-time monitoring data to predict potential fault risks in real time; including the following steps:

[0045] S441: Collect historical fault data and real-time monitoring results as input data for regression analysis, and preprocess the data;

[0046] S442: Use the historical fault data to train a regression model through polynomial regression analysis. Divide the historical fault data into a training set and a test set, use the training set data to train the regression model, and adjust the model parameters;

[0047] S443: Use the mean squared error, mean absolute error, and coefficient of determination as evaluation metrics, use the test set data to evaluate the trained model, and optimize the regression model according to the evaluation results;

[0048] S444: Apply the trained regression model to the real-time monitoring results to predict potential fault risks. If the predicted parameter value exceeds the threshold or is similar to the fault pattern in the historical data, it is determined that there is a potential fault risk.

[0049] Furthermore, the fault detection and diagnosis model is continuously updated and optimized through a self-learning algorithm model, new fault patterns are automatically identified, and all data acquisition, analysis, and alarm modules are uniformly monitored and managed according to the integrated management platform; including the following steps:

[0050] S51: Define the network system state as the environment of deep reinforcement learning. The environmental state is represented by a vector. According to the deep neural network, determine the agent whose input is the environmental state vector and output is a set of action probability distributions, and determine the reward function according to the accuracy, timeliness of fault detection and diagnosis, and the impact on network performance;

[0051] S52: Train the agent through the deep Q-network algorithm. During the training process, the agent interacts with the environment, continuously tries different actions, and adjusts its strategy according to the feedback of the reward function. As the agent interacts with the environment, it learns new fault patterns and coping strategies;

[0052] S53: The integrated management platform is responsible for collecting various network data and preprocessing them. The deep Q-network agent conducts data analysis and decision-making in the integrated management platform. The alarm module of the integrated management platform conducts alarm management according to the decision-making results of the agent, and the user interface of the integrated management platform;

[0053] S54: Regularly evaluate the performance of the deep Q-network agent and the integrated management platform. According to the evaluation results, optimize and adjust the model parameters of the agent, the reward function, and each module of the integrated management platform.

[0054] The beneficial effects of the present invention are as follows:

[0055] 1. Through a variety of data collection means, including OLT device data, user offline data, and probe detection data, network status information can be comprehensively obtained, and the operating status of the optical modem can be monitored from multiple perspectives, greatly improving the accuracy of fault detection. By using machine learning and deep learning algorithms, a combination of support vector machines and deep neural networks can automatically identify abnormal patterns and potential faults, greatly improving the efficiency and timeliness of fault detection compared with traditional manual inspection methods.

[0056] 2. Deeply mine and analyze historical fault data, use the FP-Growth algorithm for association analysis of the relationships between different parameters, and use time series analysis methods to identify periodic changes and sudden anomalies in the data, which can discover potential fault trends and patterns. Combine with an expert system to provide fault analysis and handling suggestions, record each processing process and result, and continuously optimize the knowledge base to make fault handling more scientific and efficient.

[0057] 3. Through the integrated management platform, all data collection, analysis, and alarm modules are uniformly monitored and managed, achieving real-time mastery of the network status. The self-learning algorithm model continuously updates and optimizes the fault detection and diagnosis model, automatically identifying new fault patterns and adapting to the changing network environment.

[0058] 4. Discover and handle faults in a timely manner, predict potential faults and take preventive maintenance measures, which can effectively reduce the occurrence frequency and duration of network faults, improve the stability of the network. The combination of the integrated management platform and the self-learning algorithm provides a comprehensive and efficient solution for network fault management, which can effectively improve the reliability of the network, ensure the continuous and stable operation of the network, and meet the needs of users for high-quality network services. Brief Description of the Drawings

[0059] Figure 1 It is a flowchart of a method for real-time monitoring of power-off and fiber-disconnection alarms of optical modems for broadband users. Detailed Embodiments

[0060] Please refer to Figure 1 As shown, the present invention relates to a method for real-time monitoring of power-off and fiber-disconnection alarms of optical modems for broadband users, and this method includes:

[0061] S1: Real-time monitor and record the online status, optical attenuation level, traffic information, packet loss rate, network latency, and logs of the optical modems under the OLT device, and real-time monitor the power status and fiber connection status of the optical modems through the probe system;

[0062] S2: According to the OLT device data, user offline data, and probe detection data, as well as the model composed of machine learning algorithms and deep learning algorithms, automatically identify abnormal patterns and potential faults, and trigger the alarm mechanism;

[0063] S3: Automatically diagnose the problem causes based on the collected data, combine with the expert system to provide fault analysis and handling suggestions, record each handling process and result, and optimize the knowledge base;

[0064] S4: Through in-depth mining and analysis of historical fault data, optimize the system's fault prediction and prevention capabilities, visually display the trends and abnormal points of historical data, predict potential fault risks based on historical data and real-time monitoring results, and propose preventive maintenance suggestions;

[0065] S5: Continuously update and optimize the fault detection and diagnosis model through the self-learning algorithm model, automatically identify new fault modes, and according to the integrated management platform, uniformly monitor and manage all data collection, analysis and alarm modules.

[0066] Furthermore, the online status, optical attenuation level, traffic information, packet loss rate, network latency and logs of the optical network terminals (ONTs) under the OLT device are monitored and recorded in real time, and the power status and optical fiber connection status of the ONTs are monitored in real time through the probe system; the method includes the following steps:

[0067] S11: Through the device discovery function of the probe system, automatically scan the OLT devices and ONTs in the network, add them to the monitoring list, and collect the status information of the ONTs under the OLT device in real time through the SNMP protocol and the probe system;

[0068] S12: The probe system regularly collects various parameter data of the ONTs under the OLT device at set time intervals, and based on the collected data, determines whether the online status and optical attenuation level of the ONT are normal, and whether the traffic, packet loss rate and network latency are within a reasonable range, and detects the power status and optical fiber connection status of the ONT;

[0069] S13: Record the collected various parameter data and log information into the database, regularly analyze the recorded data, and optimize and adjust the network according to the data analysis results.

[0070] Specifically, in step S11, for the online status of the optical network terminal (ONT), it is determined by regularly querying the response of the ONT or detecting the network connection status; the optical attenuation level is judged by reading the optical power information reported by the optical line terminal (OLT) device or the ONT; the traffic information is obtained by monitoring the network traffic statistics data, including upload and download traffic. The packet loss rate and network latency are calculated by sending test packets and measuring the response time. For the log data, it is necessary to determine the storage location and format of the log, and the log should include various parameter data, event time, device information, etc. monitored, which is convenient for subsequent analysis and query. In the configuration of the probe system for monitoring the power supply status and optical fiber connection status of the ONT, for the power supply status, it is achieved by monitoring the power indicator of the ONT or using a dedicated power monitoring device; for the optical fiber connection status, it is judged by detecting the optical signal strength of the ONT or using an optical fiber monitoring device.

[0071] In step S12, the setting of the time interval should be adjusted according to actual needs, ensuring both the timeliness of the data and avoiding excessive burden on the network. Determine whether the ONT is online by regularly querying the response of the ONT or detecting the network connection status. If no response from the ONT is received or the network connection is interrupted within a certain period of time, it is judged that the ONT is in an offline state. Read the optical power information reported by the OLT device or the ONT, and compare it with the preset optical attenuation level threshold. If the optical power is lower than the threshold, it is judged that the optical attenuation level is too large, which may affect the network performance. Calculate the packet loss rate by sending test packets and measuring the response time, and compare the packet loss rate with the preset threshold. If the packet loss rate exceeds the threshold, it is judged that there is a packet loss problem in the network. Similarly, calculate the network latency by sending test packets and measuring the response time, and compare the network latency with the preset threshold. If the latency is too high, it may affect the real-time performance and performance of the network. Detect the optical signal strength of the ONT or use an optical fiber monitoring device to judge the optical fiber connection status. If the optical signal strength is too low or the optical fiber monitoring device detects an optical fiber interruption, it is judged that the ONT has a fiber break.

[0072] In step S13, the recorded data should include device information, timestamp, parameter values, etc., so as to accurately understand the operation status of the network and the status changes of the ONT.

[0073] Furthermore, the model established based on the OLT device data, user offline data, probe detection data, as well as machine learning algorithms and deep learning algorithms automatically identifies abnormal patterns and potential faults and triggers an alarm mechanism, including the following steps:

[0074] S21: Collect the offline records of users through the business system, record the time and frequency information of user offline, associate the probe detection data with the timestamp, and perform data cleaning, feature extraction, and data standardization on the OLT device data, user offline data, and probe detection data.

[0075] S22: Integrate the support vector machine and the deep neural network algorithm according to the framework structure, divide the sorted data into a training set and a test set, and train, evaluate, and optimize the model;

[0076] S23: Input the OLT device data, user offline data, and probe detection data collected in real time into the trained model. The model automatically identifies abnormal patterns according to the input data, and further detects potential faults in combination with the abnormal patterns and other data features;

[0077] S24: When the model detects that the abnormal pattern or potential fault conforms to the alarm rule, automatically trigger the alarm mechanism and notify the network administrator and relevant technical support personnel in a timely manner.

[0078] In step S21, use the SNMP protocol to obtain the data of the optical modem from the OLT device, classify and label the user offline data, ensure the normal operation of the probe system, and continuously collect data such as the power status, fiber connection status, packet loss rate, and network latency of the optical modem. Associate the probe detection data with the time stamp to facilitate the analysis of the time series characteristics of the data. For data cleaning, remove outliers and incorrect data, and handle missing values; for feature extraction, extract meaningful features from the original data. For example, for the optical attenuation level data, extract features such as the optical attenuation change rate and the optical attenuation fluctuation amplitude, and perform feature engineering on the time series data; for data standardization, standardize different types of data so that they have the same scale and range.

[0079] In step S24, set corresponding alarm rules according to the types of abnormal patterns and potential faults. For serious faults, immediately trigger an emergency alarm; for minor abnormalities, set a delayed alarm or a periodic summary alarm. Ensure that relevant personnel can receive the alarm information in a timely manner through sounds, pop - up windows, text messages, emails, etc. The alarm information should include detailed content such as the fault type, occurrence time, and device information to facilitate quickly locating and handling problems.

[0080] Furthermore, the integrating the support vector machine and the deep neural network algorithm according to the framework structure, dividing the sorted data into a training set and a test set, and training, evaluating, and optimizing the model includes the following steps:

[0081] S221: Divide the sorted data into a training set and a test set, and further divide the training set into a training set and a validation set. The division process maintains the independence and randomness of the data;

[0082] S222: Optimize the parameters of the support vector machine through the training set data, train the deep neural network, adjust the weights and biases of the network, optimize the performance of the network, and monitor the performance and convergence of the model during the training process;

[0083] S223: Evaluate the trained model using the test set, calculate metrics such as the accuracy, recall rate, and F1 value of the model, and optimize and adjust the model according to the evaluation results;

[0084] S224: The framework structure adopts a hierarchical structure. Among them, the support vector machine serves as the first layer, and the deep neural network serves as the second layer. The output of the support vector machine is used as the input of the deep neural network;

[0085] S225: Integrate the support vector machine and the deep neural network through the weighted average method, and adjust the weights of the support vector machine and the deep neural network during the integration process.

[0086] Specifically, in step S221, the data is divided into a training set and a test set for model training and evaluation. The random division method is adopted to ensure that the training set and the test set are representative. The training set is further divided into a training set and a validation set for model parameter adjustment and performance evaluation. The validation set is used to monitor the performance of the model during model training, and the model parameters are adjusted in a timely manner to avoid overfitting.

[0087] In step S222, the training set data is used to optimize the parameters of the support vector machine, the kernel function parameters and the penalty coefficient. The cross-validation method is adopted to select the optimal parameter combination. The training set data is used to train the deep neural network, adjust the weights and biases of the network, and optimize the performance of the network. The stochastic gradient descent optimization algorithm is used to accelerate the model training process. Monitor the performance and convergence of the model, and use metrics such as the accuracy and loss function of the training set and the validation set to evaluate the performance of the model. Adjust the model parameters and structure in a timely manner to avoid overfitting and underfitting, and adopt regularization techniques to prevent overfitting and improve the generalization ability of the model.

[0088] In step S223, the evaluation metrics reflect the classification performance and generalization ability of the model, and are important criteria for evaluating the quality of the model. When optimizing and adjusting the model, if the performance of the model is not ideal, try methods such as adjusting parameters or increasing the amount of data to improve the performance of the model. For example, increase the kernel function parameters of the SVM, adjust the network structure and number of layers of the DNN, and increase the quantity and diversity of the training data.

[0089] In step S224, the support vector machine serves as the first layer, and the deep neural network serves as the second layer. The support vector machine is used to preliminarily classify and screen the data, dividing the data into normal and abnormal categories. The deep neural network is used to further analyze and diagnose the abnormal data to determine the type and cause of the abnormality.

[0090] In step S225, the advantages of each model are integrated through model fusion to improve the accuracy and generalization ability of fault detection and diagnosis. Adjusting the weights is to balance their performance and contribution.

[0091] Furthermore, the OLT device data, user offline data, and probe detection data collected in real time are input into the trained model. The model automatically identifies abnormal patterns based on the input data and further detects potential faults by combining the abnormal patterns with other data features. The steps are as follows:

[0092] S231: Transmit the collected data to the central server through a data transmission protocol, store the data using a distributed storage technology, and perform lossless compression and regular backup;

[0093] S232: The support vector machine preliminarily classifies the input data to determine whether the data is abnormal. If the support vector machine classifies it as abnormal, the data is input into the deep neural network for analysis;

[0094] S233: The deep neural network outputs the type and probability of the abnormality according to the input data. When the abnormality probability exceeds the set threshold, it is determined as an abnormal pattern;

[0095] S234: Detect potential faults by means of cluster analysis based on the abnormal pattern and data features. Select the clusters that may have potential faults from the cluster analysis results, and identify the periodic changes and sudden abnormalities in the data through time series analysis methods.

[0096] Furthermore, the step of detecting potential faults by means of cluster analysis based on the abnormal pattern and data features, selecting the clusters that may have potential faults from the cluster analysis results, and identifying the periodic changes and sudden abnormalities in the data through time series analysis methods includes the following steps:

[0097] S2341: Determine the value of K by using the silhouette coefficient method through the K-Means clustering algorithm. Randomly select K initial cluster centers, calculate the distances between each data point and each cluster center, assign the data points to the nearest cluster, recalculate the center of each cluster, and repeat the calculation until the cluster centers no longer change or reach a certain number of iterations;

[0098] S2342: Calculate the statistical features of each cluster, compare the feature differences between different clusters, and determine whether the cluster represents a normal state or a potential fault. If the data points in a cluster have significantly different features from other clusters, or the scale of the cluster is small, it indicates the existence of a potential fault;

[0099] S2343: Select the time series data corresponding to the clusters that may have potential faults from the cluster analysis results, ensure that the time series data is arranged in chronological order and perform preprocessing;

[0100] S2344: Convert the time series data to the frequency domain, analyze the spectrogram to determine the periodic components of specific frequencies existing in the data, calculate the spectrum of the time series data through the fast Fourier transform algorithm. If a peak of a specific frequency is found in the spectrogram, it indicates that there are periodic changes of the corresponding period in the data;

[0101] S2345: Calculate the moving average and moving standard deviation of the time series data through a method based on moving average and standard deviation. If the data value at a certain time point exceeds the range of the moving average plus or minus a certain multiple of the moving standard deviation, it is judged as a sudden anomaly.

[0102] Specifically, in step S2342, calculate the statistical features of each cluster, such as mean, standard deviation, median, etc. If it is found that the optical attenuation level in a cluster is significantly higher than that in other clusters, and at the same time the online status of the optical network terminal in this cluster is unstable and the packet loss rate is relatively high, it is preliminarily judged that this cluster may represent potential faults such as optical network terminal hardware failures or optical fiber connection problems.

[0103] In step S2343, if it is found through cluster analysis that the optical network terminal traffic data in a certain cluster is abnormal, then select the traffic time series data of the optical network terminal in this cluster for further analysis. For preprocessing, including removing outliers, filling in missing values, etc., use the same method as in cluster analysis for data preprocessing to ensure the quality and continuity of the data.

[0104] In step S2344, the Fourier transform analysis method is used to identify the periodic changes in the data.

[0105] In step S2345, the method based on moving average and standard deviation needs to adjust the multiple according to the actual situation to improve the sensitivity and accuracy of anomaly detection. Analyze the results of periodic changes and sudden anomalies, and combine other data features and anomaly patterns to further detect potential faults. If it is found that the periodic changes in the data do not match the known network activities or device behaviors, it may indicate the existence of potential faults. For example, if the traffic of the optical network terminal shows periodic peaks at unexpected times, it may be caused by a fault or abnormal behavior of a certain device in the network. Sudden anomalies may be a direct indication of potential faults. For example, if the packet loss rate suddenly increases significantly, it may indicate problems with the optical fiber connection or network device failures. By comprehensively analyzing the results of time series analysis and other data features, potential faults can be detected more accurately, and corresponding measures can be taken in a timely manner for repair and optimization.

[0106] Furthermore, recording each processing process and result and optimizing the knowledge base are based on the analysis results of S2, combined with known network architectures and device characteristics, inferring the rule base according to the problem causes, and performing fault analysis and suggestion generation according to the expert system containing expert knowledge and experience in the network field. When network administrators or technical support personnel perform fault handling according to the suggestions of the expert system, record the entire handling process and the results of the fault handling. Update and improve the knowledge base according to the records and results of each fault handling, and use machine learning technology to learn and extract knowledge from historical handling records.

[0107] Specifically, for machine learning technology, data mining methods are used to discover patterns and rules hidden in the data, providing a basis for optimizing the knowledge base. For example, by analyzing a large number of fault handling records, the association between certain fault types and specific handling methods can be automatically discovered, thereby optimizing the inference rules and suggestion generation mechanism of the expert system. Establish a feedback mechanism to collect feedback from users and technicians on fault analysis and handling suggestions, and improve and optimize the expert system according to the feedback.

[0108] Furthermore, by deeply mining and analyzing historical fault data, optimize the system's fault prediction and prevention capabilities, and visually display the trends and abnormal points of historical data, predict potential fault risks based on historical data and real-time monitoring results, and put forward preventive maintenance suggestions; including the following steps:

[0109] S41: Extract historical fault data from the database, including OLT device data, user offline data, probe detection data, and the records and results of each fault handling, preprocess the data, and store the preprocessed historical fault data in the historical data warehouse;

[0110] S42: Represent the data in the form of a transaction dataset, where each transaction represents a fault case, use the FP-Growth algorithm to construct a frequent pattern tree, mine frequent item sets from the frequent pattern tree, and interpret and analyze the possibility of parameter combination faults occurring for the mined frequent item sets;

[0111] S43: Display the historical fault data in an intuitive chart form through PowerBI, design visual charts according to the analysis purposes and requirements, implement interactive visualization functions, and allow users to deeply explore the data through interactive operations;

[0112] S44: Establish a fault prediction model according to machine learning algorithms, evaluate and optimize it, and apply the trained prediction model to real-time monitoring data to predict potential fault risks in real time;

[0113] S45: Analyze potential failure risks based on the output results of the failure prediction model. Combine with the analysis results of historical failure data to understand the causes and handling methods of failures. Based on the analysis results and combined with the knowledge base, propose specific preventive maintenance suggestions.

[0114] Specifically, in step S41, for extracting historical failure data, ensure the integrity and accuracy of the data, covering different types of failures and time ranges. Set up regular data extraction tasks to ensure timely acquisition of the latest historical failure data. Clean the historical failure data to remove noise, outliers, and duplicate data, and perform feature extraction and standardization processing to make the historical failure data have a consistent feature representation and scale with the real-time data. Store it in a dedicated historical data warehouse for subsequent in-depth mining and analysis, and use data warehouse technology to improve the storage and query efficiency of the data.

[0115] In step S42, the in-depth mining and analysis of historical failure data are achieved through the FP-Growth algorithm in the association rule mining algorithm. Scan the transaction dataset once, count the occurrence frequency of each item, and sort the items in descending order of frequency. Scan the transaction dataset again, sort each transaction in descending order of the item frequency, and construct a frequent pattern tree. Through the recursive mining of the frequent pattern tree, discover all frequent item sets that meet the minimum support threshold. The minimum support threshold is a preset parameter used to control the quantity and importance of the mined frequent item sets. Support refers to the frequency of an item set appearing in the transaction dataset. By analyzing the frequent item sets, find out which parameter combinations are more likely to lead to failures. For example, if the frequent item set (large optical attenuation level, high packet loss rate) is mined, it means that the two parameters of large optical attenuation level and high packet loss rate often appear simultaneously, and there may be a certain correlation. This correlation provides an important basis for failure prediction and prevention.

[0116] In step S43, use line charts to show the time series trend of traffic information; use scatter plots to show the relationship between optical attenuation level and packet loss rate; use box plots to show the network delay distribution in different time periods, etc. Through visual design, clearly show the trends and outliers of historical data, facilitating users to quickly understand and analyze the data. In the interactive visualization function, users can obtain detailed information by clicking on the data points in the chart; view the data distribution under specific conditions through filtering and sorting functions.

[0117] In step S45, collect users' feedback on preventive maintenance suggestions, understand the implementation effects and existing problems of the suggestions, and optimize and improve the maintenance suggestions according to the feedback.

[0118] Further, a fault prediction model is established according to a machine learning algorithm, evaluated and optimized, and the trained prediction model is applied to real-time monitoring data to predict potential fault risks in real time; the method includes the following steps:

[0119] S441: Collect historical fault data and real-time monitoring results as input data for regression analysis, and preprocess the data;

[0120] S442: Use the historical fault data to train a regression model through polynomial regression analysis. Divide the historical fault data into a training set and a test set, use the training set data to train the regression model, and adjust the parameters of the model;

[0121] S443: Use the mean squared error, mean absolute error, and coefficient of determination as evaluation indicators, use the test set data to evaluate the trained model, and optimize the regression model according to the evaluation results;

[0122] S444: Apply the trained regression model to the real-time monitoring results to predict potential fault risks. If the predicted parameter value exceeds the threshold or is similar to the fault mode in the historical data, it is determined that there is a potential fault risk.

[0123] Specifically, in step S441, the historical fault data includes OLT device data, user offline data, probe detection data, and fault handling records and results within a past period of time. The real-time monitoring results are the network parameter values at the current moment, such as optical attenuation level, packet loss rate, network latency, etc. The preprocessing includes removing outliers, standardization processing, etc., to ensure the quality and consistency of the data for better establishing the regression model.

[0124] In step S443, the mean squared error and mean absolute error measure the error between the model prediction value and the actual value, and the coefficient of determination measures the fitting degree of the model to the data. Optimizing the regression model is achieved by adjusting the parameters of the model, selecting different regression methods, adding features, etc., to improve the prediction accuracy and generalization ability of the model.

[0125] In step S444, the real-time monitoring system continuously collects the network parameter values at the current moment, inputs the network parameter values at the current moment into the regression model, and obtains the predicted future parameter values. When there is a potential fault risk, a warning is issued in a timely manner to remind relevant personnel to take preventive measures, such as checking the device status, optimizing the network configuration, etc.

[0126] Further, the fault detection and diagnosis model is continuously updated and optimized through a self-learning algorithm model to automatically identify new fault modes, and according to the integrated management platform, all data collection, analysis, and alarm modules are uniformly monitored and managed; the method includes the following steps:

[0127] S51: Define the network system state as the environment of deep reinforcement learning. The environment state is represented by a vector. According to the deep neural network, determine an agent whose input is the environment state vector and output is a set of action probability distributions. Determine the reward function based on the accuracy, timeliness of fault detection and diagnosis, and the impact on network performance.

[0128] S52: Train the agent through the deep Q-network algorithm. During the training process, the agent interacts with the environment, continuously tries different actions, and adjusts its strategy according to the feedback of the reward function. As the agent and the environment continuously interact, it learns new fault patterns and coping strategies.

[0129] S53: The integrated management platform is responsible for collecting various network data and performing preprocessing. The deep Q-network agent performs data analysis and decision-making in the integrated management platform. The alarm module of the integrated management platform performs alarm management according to the decision results of the agent, and the user interface of the integrated management platform.

[0130] S54: Regularly evaluate the performance of the deep Q-network agent and the integrated management platform. According to the evaluation results, optimize and adjust the model parameters of the agent, the reward function, and each module of the integrated management platform.

[0131] Specifically, in step S51, the environment includes various network parameters such as OLT device data, user offline data, probe detection data, etc., as well as information such as whether there is a current fault and the fault type. The environment state is represented by a vector, which contains the values of various network parameters and the encoding of the fault state. For example, some elements in the vector represent the optical attenuation level, traffic information, packet loss rate, etc., and other elements represent whether there are faults such as power failure and fiber breakage. The agent is a deep neural network whose input is the environment state vector and output is a set of action probability distributions. Actions include performing various operations on the network, such as adjusting the optical modem configuration, sending alarm notifications, and starting a fault diagnosis program. The agent affects the environment state by selecting different actions, so as to achieve the goal of fault detection and diagnosis. The reward function is used to evaluate the behavior of the agent. For example, if the agent can accurately detect a fault and take effective diagnostic measures in a timely manner, a positive reward is given; if the agent has incorrect detection or diagnosis results, or the measures taken have a negative impact on network performance, a negative reward is given.

[0132] In step S52, the deep Q network algorithm effectively handles the problem of high-dimensional state space and continuous action space, improving the learning efficiency and performance of the agent. While continuously learning new fault modes and response strategies, the agent automatically updates its model parameters based on new data and experience, improving the accuracy and adaptability of fault detection and diagnosis, and setting regular model update tasks to allow the agent to retrain on new data to maintain its advanced performance.

[0133] In step S53, preprocessing is to make the data suitable for the input of the deep reinforcement learning algorithm. Data collection is achieved through SNMP. Preprocessing includes steps such as data cleaning, feature extraction and standardization. The agent receives the preprocessed network data as the input of the environment state, and then selects an action according to the current strategy. The action triggers data collection for more detailed data collection, starts data analysis for in-depth fault diagnosis, and sends alarm notifications to relevant personnel. In alarm management, if the agent determines that there is a serious fault risk, the alarm module will issue an alarm notification in time to remind relevant personnel to take measures. The alarm module sets different alarm levels and methods according to different fault types and severity, such as SMS, email, instant messaging, etc. The user interface is used to monitor the network status and the decision-making process of the agent. The user can view the real-time values of various network parameters, the results of fault detection and diagnosis, and the historical records of alarms through the interface. The user interface also provides some management functions, such as setting data collection parameters, adjusting alarm thresholds, managing the training and update of the agent, etc.

[0134] In step S54, the evaluation indicators include the accuracy of fault detection, the timeliness of diagnosis, the accuracy of alarms, etc. As the network environment changes and new faults appear, the deep reinforcement learning agent automatically identifies new fault modes. When the agent encounters an environmental state different from the past, it will try different actions and learn new fault modes and response strategies based on the feedback of the reward function. The integrated management platform records new fault modes and incorporates them into the training data of the agent, allowing the agent to continuously learn and adapt to new situations. The integrated management platform is integrated with other related systems through the API interface, such as the network management system, operation and maintenance management system, etc., to achieve more comprehensive network monitoring and management.

[0135] The above implementation modes are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary engineering and technical personnel in the field shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for real-time monitoring of power-off and fiber-disconnection alarms of broadband user optical network terminals, characterized in that, This method includes: S1: Real-time monitor and record the online status, optical attenuation level, traffic information, packet loss rate, network latency, and logs of the optical network terminals (ONTs) connected to the OLT device. The power status and optical fiber connection status of the ONTs are monitored in real time through a probe system; S2: Based on the data of the OLT device, user offline data, and probe detection data, as well as models established by machine learning algorithms and deep learning algorithms, automatically identify abnormal patterns and potential faults, and trigger an alarm mechanism; S3: Automatically diagnose the cause of the problem according to the collected data, combine with the expert system to provide fault analysis and handling suggestions, record the process and results of each handling, and optimize the knowledge base; S4: Through in-depth mining and analysis of historical fault data, optimize the system's fault prediction and prevention capabilities, visually display the trends and abnormal points of historical data, predict potential fault risks based on historical data and real-time monitoring results, and propose preventive maintenance suggestions; S5: Continuously update and optimize the fault detection and diagnosis model through a self-learning algorithm model, automatically identify new fault patterns, and uniformly monitor and manage all data collection, analysis, and alarm modules according to the integrated management platform.

2. The method for real-time monitoring of power-off and fiber-disconnection alarms of broadband user optical network terminals according to claim 1, wherein, The real-time monitoring and recording of the online status, optical attenuation level, traffic information, packet loss rate, network latency, and logs of the ONTs connected to the OLT device, and the real-time monitoring of the power status and optical fiber connection status of the ONTs through a probe system; includes the following steps: S11: Through the device discovery function of the probe system, automatically scan for OLT devices and ONTs in the network, add them to the monitoring list, and collect the status information of the ONTs connected to the OLT device in real time through the SNMP protocol and the probe system; S12: The probe system regularly collects various parameter data of the ONTs connected to the OLT device at set time intervals. According to the collected data, judge whether the online status and optical attenuation level of the ONTs are normal, whether the traffic, packet loss rate, and network latency are within a reasonable range, and detect the power status and optical fiber connection status of the ONTs; S13: Record the collected parameter data and log information into the database, regularly analyze the recorded data, and optimize and adjust the network according to the data analysis results.

3. The method for real-time monitoring of power-off and fiber-disconnection alarms of broadband user optical modems according to claim 1, characterized in that, Based on the data of the OLT device, user offline data, and probe detection data, as well as models established by machine learning algorithms and deep learning algorithms, automatically identify abnormal patterns and potential faults, and trigger an alarm mechanism; Includes the following steps: S21: Collect the offline records of users through the business system, record the time and frequency information of user offline, associate the probe detection data with timestamps, and perform data cleaning, feature extraction, and data standardization on the OLT device data, user offline data, and probe detection data; S22: Integrate the support vector machine and deep neural network algorithms according to the framework structure, divide the sorted data into training sets and test sets, and train, evaluate, and optimize the model; Input the OLT device data, user offline data, and probe detection data collected in real time into the trained model. The model automatically identifies abnormal patterns according to the input data, and further detects potential faults in combination with the abnormal patterns and other data features; S24: When the model detects that an abnormal pattern or potential fault conforms to the alarm rule, it automatically triggers the alarm mechanism and promptly notifies the network administrator and relevant technical support personnel.

4. The method for real-time monitoring of power-off and fiber-disconnection alarms of broadband user optical network terminals according to claim 3, wherein, Integrating the support vector machine and the deep neural network algorithm according to the framework structure, dividing the sorted data into a training set and a test set, and training, evaluating, and optimizing the model; including the following steps: S221: Divide the sorted data into a training set and a test set, and further divide the training set into a training set and a validation set, while maintaining the independence and randomness of the data during the division process. S222: Optimize the parameters of the support vector machine using the training set data, train the deep neural network, adjust the weights and biases of the network, optimize the performance of the network, and monitor the performance and convergence of the model during the training process. S223: Evaluate the trained model using the test set, calculate the accuracy, recall rate, F1 value and other metrics of the model, and optimize and adjust the model according to the evaluation results. S224: The framework structure adopts a hierarchical structure. Among them, the support vector machine is used as the first layer, and the deep neural network is used as the second layer. The output of the support vector machine is used as the input of the deep neural network. S225: Integrate the support vector machine and the deep neural network through the weighted average method, and adjust the weights of the support vector machine and the deep neural network during the integration process.

5. The method for real-time monitoring of power-off and fiber-disconnection alarms of broadband user optical network terminals according to claim 3, wherein, The real-time collected OLT device data, user offline data, and probe detection data are input into the trained model. The model automatically identifies abnormal patterns according to the input data, and further detects potential faults by combining the abnormal patterns and other data features. Including the following steps: S231: Transmit the collected data to the central server through the data transmission protocol, store the data using the distributed storage technology, and perform lossless compression and regular backup. S232: The support vector machine makes a preliminary classification of the input data to determine whether the data is abnormal. If the support vector machine classifies it as abnormal, the data is input into the deep neural network for analysis. S233: The deep neural network outputs the type and probability of the abnormality according to the input data. When the abnormality probability exceeds the set threshold, it is judged as an abnormal pattern. S234: Detect potential faults through the clustering analysis method according to the abnormal pattern and data features, select the clusters that may have potential faults from the clustering analysis results, and identify the periodic changes and sudden abnormalities in the data through the time series analysis method.

6. The method for real-time monitoring of power-off and fiber-disconnection alarms of broadband user optical network terminals according to claim 5, wherein, Detect potential faults through the clustering analysis method according to the abnormal pattern and data features, select the clusters that may have potential faults from the clustering analysis results, and identify the periodic changes and sudden abnormalities in the data through the time series analysis method. Including the following steps: S2341: Determine the value of K using the silhouette coefficient method through the K-Means clustering algorithm, randomly select K initial clustering centers, calculate the distance between each data point and each clustering center, assign the data points to the closest cluster, recalculate the center of each cluster, and repeat the calculation until the clustering center no longer changes or reaches a certain number of iterations. S2342: Calculate the statistical features of each cluster, compare the feature differences between different clusters, and determine whether the cluster represents a normal state or a potential fault. If the data points in a cluster have significantly different features from those of other clusters, or the scale of the cluster is small, it indicates a potential fault. S2343: From the clustering analysis results, select the time series data corresponding to the clusters that may have potential faults, ensure that the time series data is arranged in chronological order and perform preprocessing. S2344: Transform the time series data into the frequency domain, analyze the spectrogram to determine the periodic components of specific frequencies existing in the data, calculate the spectrum of the time series data through the fast Fourier transform algorithm. If peaks of specific frequencies are found in the spectrogram, it indicates that there are periodic changes of the corresponding periods in the data. S2345: Calculate the moving average and moving standard deviation of the time series data through a method based on moving average and standard deviation. If the data value at a certain time point exceeds the range of the moving average plus or minus a certain multiple of the moving standard deviation, it is judged as a sudden anomaly.

7. The method for real-time monitoring based on power-off and fiber-disconnection alarms of broadband user optical network terminals according to claim 1, wherein The step of recording each processing process and result and optimizing the knowledge base is based on the analysis results of S2, combined with the known network architecture and device characteristics, infer the rule base according to the problem causes, and perform fault analysis and recommendation generation according to the expert system containing the expert knowledge and experience in the network field. When the network administrator or technical support personnel perform fault handling according to the recommendations of the expert system, record the entire processing process and the results of the fault handling. Update and improve the knowledge base according to the records and results of each fault handling, and use machine learning technology to learn and extract knowledge from the historical processing records.

8. The method for real-time monitoring of power-off and fiber-disconnection alarms of broadband user optical network terminals according to claim 1, wherein, The step of optimizing the system's fault prediction and prevention capabilities through in-depth mining and analysis of historical fault data, and visually displaying the trends and anomaly points of historical data, predicting potential fault risks based on historical data and real-time monitoring results, and proposing preventive maintenance suggestions; includes the following steps: S41: Extract historical fault data from the database, including OLT device data, user offline data, probe detection data, and the records and results of each fault handling, preprocess the data, and store the preprocessed historical fault data in the historical data warehouse. S42: Represent the data in the form of a transaction dataset, where each transaction represents a fault case. Use the FP-Growth algorithm to construct a frequent pattern tree, mine frequent item sets from the frequent pattern tree, and interpret and analyze the possibility of parameter combination faults occurring for the mined frequent item sets. S43: Display the historical fault data in an intuitive chart form through PowerBI, design visualization charts according to the analysis purposes and requirements, and implement interactive visualization functions to allow users to deeply explore the data through interactive operations. S44: Establish a fault prediction model according to machine learning algorithms, evaluate and optimize it, and apply the trained prediction model to real-time monitoring data to predict potential fault risks in real time. S45: Analyze the potential failure risks based on the output results of the failure prediction model, combine with the analysis results of historical failure data to understand the causes and handling methods of failures, and propose specific preventive maintenance suggestions based on the analysis results and in combination with the knowledge base.

9. The method for real-time monitoring based on power-off and fiber-disconnection alarms of broadband user optical network terminals according to claim 8, characterized in that, The failure prediction model is established according to the machine learning algorithm, evaluated and optimized, and the trained prediction model is applied to real-time monitoring data to predict potential failure risks in real time. It includes the following steps: S441: Collect historical failure data and real-time monitoring results as input data for regression analysis, and preprocess the data. S442: Train a regression model using historical failure data through the polynomial regression analysis method. Divide the historical failure data into a training set and a test set, use the training set data to train the regression model, and adjust the model parameters. S443: Use the mean square error, mean absolute error, and coefficient of determination as evaluation indicators, use the test set data to evaluate the trained model, and optimize the regression model according to the evaluation results. S444: Apply the trained regression model to the real-time monitoring results to predict potential failure risks. If the predicted parameter value exceeds the threshold or is similar to the failure mode in the historical data, it is determined that there is a potential failure risk.

10. The method for real-time monitoring of power-off and fiber-disconnection alarms of broadband user optical network terminals according to claim 1, wherein The failure detection and diagnosis model is continuously updated and optimized through the self-learning algorithm model, automatically identifies new failure modes, and uniformly monitors and manages all data collection, analysis, and alarm modules according to the integrated management platform. It includes the following steps: S51: Define the network system state as the environment of deep reinforcement learning. The environmental state is represented by a vector. According to the deep neural network, determine the agent whose input is the environmental state vector and output is a set of action probability distributions, and determine the reward function according to the accuracy and timeliness of failure detection and diagnosis and the impact on network performance. S52: Train the agent through the deep Q-network algorithm. During the training process, the agent interacts with the environment, continuously tries different actions, and adjusts its strategy according to the feedback of the reward function. As the agent interacts with the environment, it learns new failure modes and coping strategies. S53: The integrated management platform is responsible for collecting various network data and preprocessing them. The deep Q-network agent conducts data analysis and decision-making in the integrated management platform. The alarm module of the integrated management platform conducts alarm management according to the decision results of the agent and the user interface of the integrated management platform. S54: Regularly evaluate the performance of the deep Q-network agent and the integrated management platform. According to the evaluation results, optimize and adjust the model parameters of the agent, the reward function, and each module of the integrated management platform.

Images