Method for automatically detecting and analyzing networking equipment and opening rate of wide fixed network user

Abstract

The invention relates to the field of broadband access services, in particular to a method for automatically detecting and analyzing wide fixed network user networking equipment and opening rate, which comprises the following steps of: S1, associating positioning identification information through user optical modem port bandwidth data cached by OLT (Optical Line Terminal) equipment and user networking account and MAC (Media Access Control) address information of BRAS (Broadband Remote Access Server) equipment; s2, determining the matching condition of the user optical modem and the signed bandwidth through a machine learning algorithm and correlation analysis; s3, forming an AI prediction model through a time sequence prediction algorithm and a distributed storage big data analysis technology; s4, providing personalized service suggestions for the user through a clustering analysis algorithm, and integrating data sources of other network devices to perform unified collection and analysis of multi-dimensional data; and S5, carrying out self-learning and optimization through a decision tree and an incremental learning technology. According to the method, the workload and errors of manual analysis are reduced, and the working efficiency and the adaptability to a complex network environment are improved.

Description

Technical Field

[0001] The invention relates to the field of broadband access services, and in particular to a method for automatically detecting and analyzing broadband fixed network user Internet access equipment and activation rates. Background Art

[0002] With the rapid development of Internet technology, broadband access services have become an important way for users to obtain network resources. In order to ensure that users can enjoy stable and high-quality network services, operators need to detect and analyze users' Internet access devices and activation rates when users activate broadband services. However, existing technical means mostly rely on manual or simple automated tools for detection, which is inefficient and not very accurate. In addition, the complexity of the user's Internet environment also puts higher requirements on rate detection. Therefore, how to use existing network equipment and data resources to achieve efficient and accurate automatic detection and analysis of user Internet access devices and activation rates has become a problem that needs to be solved urgently.

[0003] At present, mainstream solutions mainly rely on the combination of network equipment management system (NMS) and simple rate detection tools, or rely on manual regular detection to confirm user equipment and bandwidth conditions. These solutions usually require technicians to manually input or adjust parameters and use historical data in NMS for analysis. However, manual operation steps are many and time-consuming, and it is impossible to reflect the actual usage of users in real time. In addition, the accuracy of the detection results is greatly affected by human factors, lacks the ability to adapt to complex network environments, and cannot fully detect different user devices and diverse access scenarios. Summary of the invention

[0004] To solve the above problems, the present invention provides a method for automatically detecting and analyzing broadband fixed-line user Internet access devices and activation rates, which reduces the workload and errors of manual analysis, improves work efficiency and adaptability to complex network environments.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] A method for automatically detecting and analyzing broadband fixed network user Internet access equipment and activation rate, comprising the following steps:

[0007] S1: Associate the positioning identification information with the user optical modem port bandwidth data cached by the OLT device and the user Internet account and MAC address information of the BRAS device;

[0008] S2: Based on the user's contracted bandwidth data and the actual bandwidth of the user's optical modem port, the matching status of the user's optical modem and the contracted bandwidth is determined through machine learning algorithms and association analysis;

[0009] S3: Extracts meaningful features from raw data and calculates features related to device performance. It uses an AI prediction model formed by time series prediction algorithms and distributed storage big data analysis technology to conduct in-depth mining and trend prediction of user bandwidth usage.

[0010] S4: Provides personalized service recommendations to users through cluster analysis algorithms, integrates data sources from other network devices to collect and analyze multi-dimensional data, and reflects user bandwidth usage and device health status through user-side applications;

[0011] S5: It performs self-learning and optimization through decision trees and incremental learning technology, and automatically generates a detection report on the user's Internet access device and activation rate based on the analysis results.

[0012] Furthermore, the method of associating the user optical modem port bandwidth data cached by the OLT device, the user Internet account and MAC address information of the BRAS device, and the positioning identification information comprises the following steps:

[0013] S11: Configure the data collection function through the OLT device, and collect the user's Internet account and MAC address information parameters set on the BRAS device;

[0014] S12: The bandwidth data of the user's optical modem port is periodically extracted from the cache of the OLT device, and transmitted to the central data processing server through the data transmission channel. At the same time, the user's Internet account and MAC address information are obtained from the BRAS device and transmitted to the same data processing server;

[0015] S13: Integrate the data collected from the OLT and BRAS devices in the data processing server, and associate the positioning identification information with the user's optical modem port bandwidth data and Internet access account through an association algorithm;

[0016] S14: Use a clustering algorithm to divide users into different groups according to their bandwidth usage patterns, detect abnormal bandwidth usage, and further analyze the causes of the abnormalities.

[0017] Furthermore, the method of determining the matching situation between the user's optical modem and the contracted bandwidth through a machine learning algorithm and association analysis based on the user's contracted bandwidth data and the actual bandwidth of the user's optical modem port comprises the following steps:

[0018] S21: Obtain user contracted bandwidth data from the data storage, collect and record actual bandwidth data of the user's optical modem port in different time periods, and clean and pre-process the data;

[0019] S22: extract features from the contracted bandwidth data, calculate statistical features for the actual bandwidth data, introduce time features, and calculate dynamic features of bandwidth growth rate and bandwidth fluctuation rate based on the user's historical usage data;

[0020] S23: By training the support vector machine model and combining it with the FP-Growth algorithm based on the frequent pattern tree, the association rules between the contracted bandwidth characteristics and the actual bandwidth characteristics are mined;

[0021] S24: Evaluate and optimize the model, input new user contracted bandwidth data and actual bandwidth data into the trained model, combine the results of the association analysis and provide detailed matching explanations and suggestions.

[0022] Furthermore, the method of training the support vector machine model and mining the association rules between the contracted bandwidth features and the actual bandwidth features in combination with the FP-Growth algorithm based on the frequent pattern tree comprises the following steps:

[0023] S231: Divide the preprocessed data into a training set, a validation set, and a test set;

[0024] S232: Initialize the parameters of the support vector machine model, use the training set data to train the support vector machine algorithm, optimize the support vector machine model by iteratively updating the parameters, combine the 5-fold cross validation technology in the training process, and use the validation set to evaluate the support vector machine model in the training process;

[0025] S233: Analyze the extracted features through the FP-Growth algorithm based on the frequent pattern tree, explore the potential association between the contracted bandwidth features and the actual bandwidth features, screen and filter the mined association rules, and visualize the screened association rules.

[0026] Furthermore, the method extracts meaningful features from the raw data and calculates features related to device performance, and uses an AI prediction model formed by a time series prediction algorithm and a distributed storage big data analysis technology to perform in-depth mining and trend prediction on user bandwidth usage; the method includes the following steps:

[0027] S31: Remove duplicate data, process missing values ​​and outliers from the original data of users' broadband usage, extract average bandwidth usage, peak bandwidth usage and variance characteristics of bandwidth usage in different time periods, and calculate characteristics related to device performance;

[0028] S32: Based on the long short-term memory network model, use the PyTorch deep learning framework to build the network structure, define the number of neurons in the input layer, hidden layer and output layer, use the Adam optimizer, and use the mean square error as the loss function for training, and adjust the network structure, learning rate and number of iterations to optimize the model performance;

[0029] S33: Store raw data, feature data, and trained models through the HBase distributed database;

[0030] S34: By calculating the mean square error, mean absolute error and coefficient of determination as evaluation indicators, the trained model is evaluated through the validation set. According to the evaluation results, the model is optimized by adjusting the algorithm parameters or increasing the complexity of feature engineering;

[0031] S35: Through the trained long short-term memory network model, the characteristic data of the time period to be predicted is input into the model for prediction. Cluster analysis is used to divide users into different groups according to their bandwidth usage characteristics. Services and optimized resource allocation are provided for different groups. The results are plotted as a time series prediction curve to visualize the results.

[0032] Furthermore, the method uses the PyTorch deep learning framework to build a network structure based on the long short-term memory network model, defines the number of neurons in the input layer, hidden layer, and output layer, selects the Adam optimizer and mean square error as the loss function for training, and optimizes the model performance by adjusting the network structure, learning rate, and number of iterations; and includes the following steps:

[0033] S321: Define the long short-term memory network model structure through the PyTorch deep learning framework, and match the input layer dimension according to the number of features of the input data;

[0034] S322: Set 1 to 2 hidden layers in the initial construction phase, determine that each layer has 64 to 128 neurons, and add a fully connected layer for the final prediction output;

[0035] S323: Set the learning rate according to the Adam optimizer, define the loss function by the mean square error, and determine the batch size and number of training rounds;

[0036] S324: In each round of training, the data is input into the model in batches for forward propagation to calculate the predicted value, and then the error is calculated through the loss function, the error is back-propagated, and the optimizer is used to update the parameters of the model to minimize the loss function;

[0037] S325: Monitor the loss value on the validation set during the training process. If the validation set loss does not decrease for multiple consecutive rounds, stop the training early.

[0038] Furthermore, the method includes inputting the characteristic data of the time period to be predicted into the model through the trained long short-term memory network model for prediction, using cluster analysis to divide users into different groups according to bandwidth usage characteristics, providing services and optimizing resource allocation for different groups, and drawing a time series prediction curve to visualize the results; the method includes the following steps:

[0039] S351: Using the trained long short-term memory network model, input feature data of the time period to be predicted is input into the model, and a predicted value of future bandwidth usage is output;

[0040] S352: Estimate the uncertainty of the prediction results by making multiple predictions and calculating the mean and standard deviation of the prediction values;

[0041] S353: According to the K-Means algorithm, extract the data features for cluster analysis, and standardize the data so that each feature has the same dimension and weight;

[0042] S354: Determine a suitable K value by using the elbow rule, and calculate the sum of squares of clustering errors under different K values. When the reduction in the sum of squares of clustering errors caused by the increase in the K value becomes less obvious, the corresponding K value is a better choice.

[0043] S355: The quality of clustering results is evaluated by the silhouette coefficient evaluation indicator, and the results are plotted as a time series prediction curve to achieve visual display of the results.

[0044] Furthermore, the method provides personalized service suggestions for users through cluster analysis algorithms, integrates data sources of other network devices to uniformly collect and analyze multi-dimensional data, and reflects user bandwidth usage and device health status through user-side applications; including the following steps:

[0045] S41: Density-based cluster analysis to determine the characteristics and dimensions of clusters;

[0046] S42: Calculate various statistics of members for each cluster and analyze the behavior patterns of users in each cluster;

[0047] S43: according to the network device type and specific data indicators that need to collect data, use corresponding data collection interfaces for different devices and data types, collect data from various devices and data sources at set time intervals, and use encrypted TCP / IP connections to transmit the collected data in real time or in batches to a central data processing server. At the data processing server, format conversion and integration of data from different data sources are performed and stored in a distributed database;

[0048] S44: Through the user interface, including the broadband usage display area, the device health status indicator area, the service recommendation area and the notification center, the application interacts with the server through the API provided by the backend, obtains the user's relevant data, and performs real-time calculation and processing on the obtained data locally. When the user's bandwidth usage reaches the set threshold or the device shows abnormal indicators, the application immediately sends a real-time notification to the user.

[0049] Furthermore, the density-based cluster analysis determines the neighborhood radius of the core point and the minimum number of neighbors of the core point, and determines the characteristics and dimensions of the cluster; and includes the following steps:

[0050] S411: Determine the neighborhood radius of the core point and the minimum number of neighbors of the core point;

[0051] S412: Randomly select a point p that has not been visited from the data set, calculate the number of points in the Eps neighborhood of point p, and if there are at least MinPts points in the Eps neighborhood of point p, mark point p as a core point, and mark all points in its Eps neighborhood as directly density reachable;

[0052] S413: Repeat step S412 for the newly marked point, expand the cluster, and if the number of points in the Eps neighborhood of point p is less than MinPts, mark point p as a noise point;

[0053] S414: Continue to select unvisited points and repeat the process of steps S412-S413 until all points have been visited;

[0054] S415: Evaluate the quality of the clustering results by using the Calinski-Harabasz index. If the clustering results are not ideal, adjust the values ​​of Eps and MinPts and perform clustering analysis again.

[0055] Furthermore, the self-learning and optimization through decision tree and incremental learning technology, and automatic generation of a detection report of the user's Internet access device and activation rate according to the analysis results, include the following steps:

[0056] S51: According to the ID3 decision tree algorithm, the target variable of the decision tree is determined by the matching of the user's Internet access device and the activation rate, and an initial decision tree model is constructed using the training set based on the preprocessed data;

[0057] S52: When new data is continuously generated, the model is updated using incremental learning techniques;

[0058] S53: Input the user Internet access device and activation rate data to be detected into the optimized ID3 decision tree model. The model predicts and analyzes the input data, determines the matching situation between the device and the rate, and automatically generates a detection report based on the analysis results;

[0059] S54: Display the generated test report to the user or relevant personnel, and collect feedback from the user or relevant personnel on the report; based on the feedback, further improve the model and report generation process.

[0060] The beneficial effects of the present invention are:

[0061] 1. Accurately detect the match between the device and the activated rate to ensure that users obtain the expected network performance and reduce problems such as slow network speed and lag caused by mismatch, thereby improving the user experience in various network activities such as surfing the Internet, video playback, and online games; provide improvement suggestions for mismatched users to help them optimize network settings or upgrade devices, and further meet users' needs for high-speed and stable networks.

[0062] 2. Help operators understand users' actual needs and equipment conditions, allocate network bandwidth resources more accurately, avoid resource waste or shortage, and upgrade network infrastructure in a targeted manner in areas where users are concentrated but have insufficient equipment performance but high-speed data rates, thereby improving the overall network service quality.

[0063] 3. Automatic detection and analysis reduces the workload and cost of manual detection, improves work efficiency, promptly detects potential network problems and equipment failures, takes measures in advance for maintenance and optimization, and reduces troubleshooting time and costs.

[0064] 4. Provide users with personalized network packages and value-added service recommendations based on the test results, increase user satisfaction and loyalty, and help operators formulate more precise marketing strategies and promote products and services suitable for different user groups.

[0065] 5. A large amount of detection data provides a basis for network technology research and innovation, and promotes the development of more efficient network protocols, smarter bandwidth allocation algorithms, etc. High-quality network services and precise user services can enhance the competitiveness of operators in the market, attract more new users and retain existing users. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 The invention is a flow chart of a method for automatically detecting and analyzing Internet access equipment and activation speed of broadband fixed network users. DETAILED DESCRIPTION

[0067] See also Figure 1 As shown, the present invention relates to a method for automatically detecting and analyzing a broadband fixed network user's Internet access device and an activation rate, comprising the following steps:

[0068] S1: Associate the positioning identification information with the user optical modem port bandwidth data cached by the OLT device and the user Internet account and MAC address information of the BRAS device;

[0069] S2: Based on the user's contracted bandwidth data and the actual bandwidth of the user's optical modem port, the matching status of the user's optical modem and the contracted bandwidth is determined through machine learning algorithms and association analysis;

[0070] S3: Extracts meaningful features from raw data and calculates features related to device performance. It uses an AI prediction model formed by time series prediction algorithms and distributed storage big data analysis technology to conduct in-depth mining and trend prediction of user bandwidth usage.

[0071] S4: Provides personalized service recommendations to users through cluster analysis algorithms, integrates data sources from other network devices to collect and analyze multi-dimensional data, and reflects user bandwidth usage and device health status through user-side applications;

[0072] S5: It performs self-learning and optimization through decision trees and incremental learning technology, and automatically generates a detection report on the user's Internet access device and activation rate based on the analysis results.

[0073] Furthermore, the method of associating the user optical modem port bandwidth data cached by the OLT device, the user Internet account and MAC address information of the BRAS device, and the positioning identification information comprises the following steps:

[0074] S11: Configure the data collection function through the OLT device, and collect the user's Internet account and MAC address information parameters set on the BRAS device;

[0075] S12: The bandwidth data of the user's optical modem port is periodically extracted from the cache of the OLT device, and transmitted to the central data processing server through the data transmission channel. At the same time, the user's Internet account and MAC address information are obtained from the BRAS device and transmitted to the same data processing server;

[0076] S13: Integrate the data collected from the OLT and BRAS devices in the data processing server, and associate the positioning identification information with the user's optical modem port bandwidth data and Internet access account through an association algorithm;

[0077] S14: Use a clustering algorithm to divide users into different groups according to their bandwidth usage patterns, detect abnormal bandwidth usage, and further analyze the causes of the abnormalities.

[0078] Specifically, the collection of user Internet account and MAC address information of BRAS equipment refers to the comprehensive collection of user Internet account, MAC address and other information through the broadband remote access server (BRAS) equipment, and the recording of the network connection status of the user terminal equipment. This process ensures the accuracy and integrity of the data on the connection between the user terminal equipment and the network.

[0079] The positioning identification information refers to the IP address, port number, ONU ID (optical network unit identifier), MAC address and other positioning identification information. Through the clustering algorithm, the collected user optical modem port bandwidth data is accurately matched with the user's Internet account. This step ensures the high relevance and consistency of the data and eliminates the data island phenomenon.

[0080] Furthermore, the method of determining the matching situation between the user's optical modem and the contracted bandwidth through a machine learning algorithm and association analysis based on the user's contracted bandwidth data and the actual bandwidth of the user's optical modem port comprises the following steps:

[0081] S21: Obtain user contracted bandwidth data from the data storage, collect and record actual bandwidth data of the user's optical modem port in different time periods, and clean and pre-process the data;

[0082] S22: extract features from the contracted bandwidth data, calculate statistical features for the actual bandwidth data, introduce time features, and calculate dynamic features of bandwidth growth rate and bandwidth fluctuation rate based on the user's historical usage data;

[0083] S23: By training the support vector machine model and combining it with the FP-Growth algorithm based on the frequent pattern tree, the association rules between the contracted bandwidth characteristics and the actual bandwidth characteristics are mined;

[0084] S24: Evaluate and optimize the model, input new user contracted bandwidth data and actual bandwidth data into the trained model, combine the results of the association analysis and provide detailed matching explanations and suggestions.

[0085] Specifically, obtain the user's contracted bandwidth data, including bandwidth size, package type, validity period and other information; record the actual bandwidth measurement values ​​​​in different time periods (such as daily, weekly, and monthly); clean and preprocess the data, including processing missing values ​​and outliers, and format the data into a format suitable for analysis.

[0086] The extracted features include bandwidth value, package level, etc. The statistical features include mean, median, standard deviation, maximum value, minimum value, etc. The time features include bandwidth characteristics in different time periods and user usage habit characteristics (such as usage time during peak hours, usage time during non-peak hours, etc.).

[0087] In step S24, the trained support vector machine model is finally evaluated using the test set data by calculating the accuracy, precision, recall, F1 value, area under the ROC curve and mean square error. Based on the evaluation results, the performance and shortcomings of the model are analyzed. New user contracted bandwidth data and actual bandwidth data are input into the trained model. The model outputs the matching situation between the user's optical modem and the contracted bandwidth, such as matching, mismatching or partial matching. Combined with the results of the association analysis, a more detailed matching explanation and suggestions are given. New data is continuously monitored to observe the performance of the model in actual applications. When the data distribution changes significantly or the model performance decreases, data collection and model training are performed again to update and optimize the model.

[0088] On the basis of data collection, real-time monitoring and early warning functions are added. When abnormal user bandwidth utilization is detected or the actual bandwidth does not meet the contract requirements, the system can issue an early warning in real time to remind the operator to take timely measures to prevent the user experience from being affected.

[0089] Furthermore, the method of training the support vector machine model and mining the association rules between the contracted bandwidth features and the actual bandwidth features in combination with the FP-Growth algorithm based on the frequent pattern tree comprises the following steps:

[0090] S231: Divide the preprocessed data into a training set, a validation set, and a test set;

[0091] S232: Initialize the parameters of the support vector machine model, use the training set data to train the support vector machine algorithm, optimize the support vector machine model by iteratively updating the parameters, combine the 5-fold cross validation technology in the training process, and use the validation set to evaluate the support vector machine model in the training process;

[0092] S233: Analyze the extracted features through the FP-Growth algorithm based on the frequent pattern tree, explore the potential association between the contracted bandwidth features and the actual bandwidth features, screen and filter the mined association rules, and visualize the screened association rules.

[0093] Specifically, support vector machines are used for machine learning algorithms, and the FP-Growth algorithm based on frequent pattern trees is used for association analysis. The preprocessed data is randomly divided into training sets, validation sets, and test sets in a ratio of 70%, 15%, and 15%. It is ensured that each set contains various types of users and bandwidth conditions to ensure the representativeness of the data.

[0094] Initialize the kernel function type of the support vector machine, use the training set data to train the algorithm, optimize the model by iteratively updating the parameters to minimize the loss function of the model on the training set, combine the 5-fold cross-validation technology in the training process to prevent overfitting, use the validation set to evaluate the model in the training process, and select the model parameters that perform best on the validation set. For the association analysis algorithm, set appropriate support and confidence thresholds to mine meaningful association rules.

[0095] The extracted features are analyzed using the FP-Growth algorithm based on the frequent pattern tree to explore the potential association between the contracted bandwidth features and the actual bandwidth features. The mined association rules are screened and filtered to remove rules that are too common or have no practical significance. The screened association rules are visualized, including using bar charts, line charts, or web charts, so that the content and strength of the rules can be intuitively understood.

[0096] Furthermore, the method extracts meaningful features from the raw data and calculates features related to device performance, and uses an AI prediction model formed by a time series prediction algorithm and a distributed storage big data analysis technology to perform in-depth mining and trend prediction on user bandwidth usage; the method includes the following steps:

[0097] S31: Remove duplicate data, process missing values ​​and outliers from the original data of users' broadband usage, extract average bandwidth usage, peak bandwidth usage and variance characteristics of bandwidth usage in different time periods, and calculate characteristics related to device performance;

[0098] S32: Based on the long short-term memory network model, use the PyTorch deep learning framework to build the network structure, define the number of neurons in the input layer, hidden layer and output layer, use the Adam optimizer, and use the mean square error as the loss function for training, and adjust the network structure, learning rate and number of iterations to optimize the model performance;

[0099] S33: Store raw data, feature data, and trained models through the HBase distributed database;

[0100] S34: By calculating the mean square error, mean absolute error and coefficient of determination as evaluation indicators, the trained model is evaluated through the validation set. According to the evaluation results, the model is optimized by adjusting the algorithm parameters or increasing the complexity of feature engineering;

[0101] S35: Through the trained long short-term memory network model, the characteristic data of the time period to be predicted is input into the model for prediction. Cluster analysis is used to divide users into different groups according to their bandwidth usage characteristics. Services and optimized resource allocation are provided for different groups. The results are plotted as a time series prediction curve to visualize the results.

[0102] Specifically, calculate the characteristics related to device performance, including the device's packet loss rate, latency, network connection stability indicators, etc. In the HBase distributed database, design a reasonable data partitioning and storage strategy to improve data reading and writing efficiency and parallel processing capabilities.

[0103] Calculate the above evaluation indicators on the validation set and compare the model performance under different algorithms and parameter settings. If the model performs poorly, try to add data preprocessing steps, including logarithmic transformation, standardization, etc., to make the data more consistent with the model assumptions, or increase the number and quality of features, introduce more factors related to bandwidth usage, or adjust the model's hyperparameters, or try to use ensemble learning methods, such as weighted averaging of the prediction results of multiple models.

[0104] Furthermore, the method uses the PyTorch deep learning framework to build a network structure based on the long short-term memory network model, defines the number of neurons in the input layer, hidden layer, and output layer, selects the Adam optimizer and mean square error as the loss function for training, and optimizes the model performance by adjusting the network structure, learning rate, and number of iterations; and includes the following steps:

[0105] S321: Define the long short-term memory network model structure through the PyTorch deep learning framework, and match the input layer dimension according to the number of features of the input data;

[0106] S322: Set 1 to 2 hidden layers in the initial construction phase, determine that each layer has 64 to 128 neurons, and add a fully connected layer for the final prediction output;

[0107] S323: Set the learning rate according to the Adam optimizer, define the loss function by the mean square error, and determine the batch size and number of training rounds;

[0108] S324: In each round of training, the data is input into the model in batches for forward propagation to calculate the predicted value, and then the error is calculated through the loss function, the error is back-propagated, and the optimizer is used to update the parameters of the model to minimize the loss function;

[0109] S325: Monitor the loss value on the validation set during the training process. If the validation set loss does not decrease for multiple consecutive rounds, stop the training early.

[0110] Furthermore, the method includes inputting the characteristic data of the time period to be predicted into the model through the trained long short-term memory network model for prediction, using cluster analysis to divide users into different groups according to bandwidth usage characteristics, providing services and optimizing resource allocation for different groups, and drawing a time series prediction curve to visualize the results; the method includes the following steps:

[0111] S351: Using the trained long short-term memory network model, input feature data of the time period to be predicted is input into the model, and a predicted value of future bandwidth usage is output;

[0112] S352: Estimate the uncertainty of the prediction results by making multiple predictions and calculating the mean and standard deviation of the prediction values;

[0113] S353: According to the K-Means algorithm, extract the data features for cluster analysis, and standardize the data so that each feature has the same dimension and weight;

[0114] S354: Determine a suitable K value by using the elbow rule, and calculate the sum of squares of clustering errors under different K values. When the reduction in the sum of squares of clustering errors caused by the increase in the K value becomes less obvious, the corresponding K value is a better choice.

[0115] S355: The quality of clustering results is evaluated by the silhouette coefficient evaluation indicator, and the results are plotted as a time series prediction curve to achieve visual display of the results.

[0116] Specifically, the input feature data includes a future timestamp and related environmental features. In estimating the uncertainty of the prediction result, a larger standard deviation indicates a higher uncertainty. In step S353, the extracted data features include average bandwidth usage, peak bandwidth usage, or volatility of bandwidth usage in different time periods. When determining the K value, the K value can also be preliminarily determined by estimating the number of possible user types based on business understanding and prior knowledge.

[0117] The implementation steps of the K-Means algorithm are as follows: randomly select K data points as the initial cluster centers, calculate the Euclidean distance of each data point to each cluster center, assign the data point to the cluster to which the nearest cluster center belongs, recalculate the center of each cluster, that is, the mean of all data points in the cluster, and repeat the distance calculation, data point assignment, and center calculation until the cluster center no longer changes or the maximum number of iterations is reached.

[0118] Furthermore, the method provides personalized service suggestions for users through cluster analysis algorithms, integrates data sources of other network devices to uniformly collect and analyze multi-dimensional data, and reflects user bandwidth usage and device health status through user-side applications; including the following steps:

[0119] S41: Density-based cluster analysis to determine the characteristics and dimensions of clusters;

[0120] S42: Calculate various statistics of members for each cluster and analyze the behavior patterns of users in each cluster;

[0121] S43: according to the network device type and specific data indicators that need to collect data, use corresponding data collection interfaces for different devices and data types, collect data from various devices and data sources at set time intervals, and use encrypted TCP / IP connections to transmit the collected data in real time or in batches to a central data processing server. At the data processing server, format conversion and integration of data from different data sources are performed and stored in a distributed database;

[0122] S44: Through the user interface, including the broadband usage display area, the device health status indicator area, the service recommendation area and the notification center, the application interacts with the server through the API provided by the backend, obtains the user's relevant data, and performs real-time calculation and processing on the obtained data locally. When the user's bandwidth usage reaches the set threshold or the device shows abnormal indicators, the application immediately sends a real-time notification to the user.

[0123] Specifically, in step S42, various statistics of its members are calculated, including: the average value, median, standard deviation of bandwidth usage, the mean and variance of device performance indicators, the frequency distribution of different application usage, etc. The mode refers to recommending high-speed and stable dedicated line services for clusters with large and stable bandwidth usage and high device performance requirements, launching limited-time high-speed bandwidth packages for clusters that are mainly entertainment-oriented and have high bandwidth requirements in specific time periods, and recommending economical and affordable basic packages for clusters with small bandwidth usage, good device performance, but potential needs.

[0124] In step S43, the corresponding data acquisition interface is used for different devices and data types, including: for devices supporting SNMP protocol, SNMP agent program is written to obtain device performance data; for network flow data, NetF low or sF low protocol is used to collect. The data from different data sources are converted and integrated, including making them have a unified structure and fields. The data is stored in a distributed database and a suitable index is established for fast query and analysis.

[0125] In step S44, the display area displays recent bandwidth usage trends in the form of line graphs, bar graphs, etc., including changes in upload and download speeds; the device health status indicator area displays key performance indicators of the device, such as CPU utilization, memory usage, etc., through color coding or specific numerical values; the service recommendation area displays personalized service recommendations and package suggestions based on the clustering and analysis results to which the user belongs; the notification center displays important notifications sent by the system, such as reminders of bandwidth exhaustion and device failure warnings, etc. The local real-time calculation and processing of the acquired data includes calculating the current bandwidth usage percentage, comparing and analyzing with historical data, etc.

[0126] Furthermore, the density-based cluster analysis determines the neighborhood radius of the core point and the minimum number of neighbors of the core point, and determines the characteristics and dimensions of the cluster; and includes the following steps:

[0127] S411: Determine the neighborhood radius of the core point and the minimum number of neighbors of the core point;

[0128] S412: Randomly select a point p that has not been visited from the data set, calculate the number of points in the Eps neighborhood of point p, and if there are at least MinPts points in the Eps neighborhood of point p, mark point p as a core point, and mark all points in its Eps neighborhood as directly density reachable;

[0129] S413: Repeat step S412 for the newly marked point, expand the cluster, and if the number of points in the Eps neighborhood of point p is less than MinPts, mark point p as a noise point;

[0130] S414: Continue to select unvisited points and repeat the process of steps S412-S413 until all points have been visited;

[0131] S415: Evaluate the quality of the clustering results by using the Calinski-Harabasz index. If the clustering results are not ideal, adjust the values ​​of Eps and MinPts and perform clustering analysis again.

[0132] Specifically, in step S411, Eps is the neighborhood radius of the core point. The selection of these two parameters can be based on the distribution characteristics of the data and prior knowledge, or determined through multiple experiments and visual analysis. For example, the Eps value can be determined by drawing a k-distance graph of the data and observing the inflection point of the curve; MinPts can be preliminarily set according to the density of the data and the expected clustering effect, usually between 5-10.

[0133] Furthermore, the self-learning and optimization through decision tree and incremental learning technology, and automatic generation of a detection report of the user's Internet access device and activation rate according to the analysis results, include the following steps:

[0134] S51: According to the ID3 decision tree algorithm, the target variable of the decision tree is determined by the matching of the user's Internet access device and the activation rate, and an initial decision tree model is constructed using the training set based on the preprocessed data;

[0135] S52: When new data is continuously generated, the model is updated using incremental learning techniques;

[0136] S53: Input the user Internet access device and activation rate data to be detected into the optimized ID3 decision tree model. The model predicts and analyzes the input data, determines the matching situation between the device and the rate, and automatically generates a detection report based on the analysis results;

[0137] S54: Display the generated test report to the user or relevant personnel, and collect feedback from the user or relevant personnel on the report; based on the feedback, further improve the model and report generation process.

[0138] Specifically, in step S52, for incremental learning and model optimization, the preprocessed data is gradually added to the original training set, the ID3 decision tree model is retrained using the updated training set, the structure and parameters of the tree are adjusted, and the performance of the model is regularly evaluated, including using indicators such as accuracy, recall, and F1 value. If the model performance deteriorates, the reason may be changes in data distribution, changes in feature importance, etc. According to the evaluation results and analysis, the model is adjusted and optimized, including reselecting features, adjusting the splitting conditions of the decision tree, etc.

[0139] In step S53, the detection report includes detailed information of the user's Internet access device, the activated rate, and the matching results of the model judgment. If there is a mismatch, possible reasons and improvement suggestions are provided, including mismatches caused by device performance limitations, network congestion, etc., and it is recommended to upgrade the device or adjust the network configuration.

[0140] The above implementation modes are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary engineering and technical personnel in the field shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for automatically detecting and analyzing broadband fixed-line user Internet access equipment and activation rate, characterized in that: The following steps are involved: S1: Associate the positioning identification information with the user optical modem port bandwidth data cached by the OLT device and the user Internet account and MAC address information of the BRAS device; S2: Based on the user's contracted bandwidth data and the actual bandwidth of the user's optical modem port, the matching status of the user's optical modem and the contracted bandwidth is determined through machine learning algorithms and association analysis; S3: Extracts meaningful features from raw data and calculates features related to device performance. It uses an AI prediction model formed by time series prediction algorithms and distributed storage big data analysis technology to conduct in-depth mining and trend prediction of user bandwidth usage. S4: Provides personalized service recommendations to users through cluster analysis algorithms, integrates data sources from other network devices to collect and analyze multi-dimensional data, and reflects user bandwidth usage and device health status through user-side applications; S5: It performs self-learning and optimization through decision trees and incremental learning technology, and automatically generates a detection report on the user's Internet access device and activation rate based on the analysis results.

2. According to claim 1, a method for automatically detecting and analyzing broadband fixed network user Internet access equipment and activation rate, characterized in that: The method of associating the user optical modem port bandwidth data cached by the OLT device, the user Internet account and MAC address information of the BRAS device, and the positioning identification information comprises the following steps: S11: Configure the data collection function through the OLT device, and collect the user's Internet account and MAC address information parameters set on the BRAS device; S12: The bandwidth data of the user's optical modem port is periodically extracted from the cache of the OLT device, and transmitted to the central data processing server through the data transmission channel. At the same time, the user's Internet account and MAC address information are obtained from the BRAS device and transmitted to the same data processing server; S13: Integrate the data collected from the OLT and BRAS devices in the data processing server, and associate the positioning identification information with the user's optical modem port bandwidth data and Internet access account through an association algorithm; S14: Use a clustering algorithm to divide users into different groups according to their bandwidth usage patterns, detect abnormal bandwidth usage, and further analyze the causes of the abnormalities.

3. The method for automatically detecting and analyzing broadband fixed network user Internet access equipment and activation rate according to claim 1, characterized in that: The method of determining the matching situation between the user's optical modem and the contracted bandwidth through a machine learning algorithm and association analysis based on the user's contracted bandwidth data and the actual bandwidth of the user's optical modem port comprises the following steps: S21: Obtain user contracted bandwidth data from the data storage, collect and record actual bandwidth data of the user's optical modem port in different time periods, and clean and pre-process the data; S22: extract features from the contracted bandwidth data, calculate statistical features for the actual bandwidth data, introduce time features, and calculate dynamic features of bandwidth growth rate and bandwidth fluctuation rate based on the user's historical usage data; S23: By training the support vector machine model and combining it with the FP-Growth algorithm based on the frequent pattern tree, the association rules between the contracted bandwidth characteristics and the actual bandwidth characteristics are mined; S24: Evaluate and optimize the model, input new user contracted bandwidth data and actual bandwidth data into the trained model, combine the results of the association analysis and provide detailed matching explanations and suggestions.

4. The method for automatically detecting and analyzing broadband fixed network user Internet access equipment and activation rate according to claim 3 is characterized in that: The method trains the support vector machine model and mines the association rules between the contracted bandwidth features and the actual bandwidth features in combination with the FP-Growth algorithm based on the frequent pattern tree; the method comprises the following steps: S231: Divide the preprocessed data into a training set, a validation set, and a test set; S232: Initialize the parameters of the support vector machine model, use the training set data to train the support vector machine algorithm, optimize the support vector machine model by iteratively updating the parameters, combine the 5-fold cross validation technology in the training process, and use the validation set to evaluate the support vector machine model in the training process; S233: Analyze the extracted features through the FP-Growth algorithm based on the frequent pattern tree, explore the potential association between the contracted bandwidth features and the actual bandwidth features, screen and filter the mined association rules, and visualize the screened association rules.

5. The method for automatically detecting and analyzing broadband fixed network user Internet access equipment and activation rate according to claim 1, characterized in that: The method extracts meaningful features from the raw data and calculates features related to device performance, and uses an AI prediction model formed by a time series prediction algorithm and distributed storage big data analysis technology to perform in-depth mining and trend prediction on user bandwidth usage; the method includes the following steps: S31: Remove duplicate data, process missing values ​​and outliers from the original data of users' broadband usage, extract average bandwidth usage, peak bandwidth usage and variance characteristics of bandwidth usage in different time periods, and calculate characteristics related to device performance; S32: Based on the long short-term memory network model, use the PyTorch deep learning framework to build the network structure, define the number of neurons in the input layer, hidden layer and output layer, use the Adam optimizer, and use the mean square error as the loss function for training, and adjust the network structure, learning rate and number of iterations to optimize the model performance; S33: Store raw data, feature data, and trained models through the HBase distributed database; S34: By calculating the mean square error, mean absolute error and coefficient of determination as evaluation indicators, the trained model is evaluated through the validation set. According to the evaluation results, the model is optimized by adjusting the algorithm parameters or increasing the complexity of feature engineering; S35: Through the trained long short-term memory network model, the characteristic data of the time period to be predicted is input into the model for prediction. Cluster analysis is used to divide users into different groups according to their bandwidth usage characteristics. Services and optimized resource allocation are provided for different groups. The results are plotted as a time series prediction curve to visualize the results.

6. A method for automatically detecting and analyzing broadband fixed network user Internet access equipment and activation rate according to claim 5, characterized in that: The method uses the PyTorch deep learning framework to build a network structure based on the long short-term memory network model, defines the number of neurons in the input layer, hidden layer, and output layer, selects the Adam optimizer and mean square error as the loss function for training, and optimizes the model performance by adjusting the network structure, learning rate, and number of iterations; and includes the following steps: S321: Define the long short-term memory network model structure through the PyTorch deep learning framework, and match the input layer dimension according to the number of features of the input data; S322: Set 1 to 2 hidden layers in the initial construction phase, determine that each layer has 64 to 128 neurons, and add a fully connected layer for the final prediction output; S323: Set the learning rate according to the Adam optimizer, define the loss function by the mean square error, and determine the batch size and number of training rounds; S324: In each round of training, the data is input into the model in batches for forward propagation to calculate the predicted value, and then the error is calculated through the loss function, the error is back-propagated, and the optimizer is used to update the parameters of the model to minimize the loss function; S325: Monitor the loss value on the validation set during the training process. If the validation set loss does not decrease for multiple consecutive rounds, stop the training early.

7. The method for automatically detecting and analyzing broadband fixed network user Internet access equipment and activation rate according to claim 5, characterized in that: The method uses a trained long short-term memory network model to input the characteristic data of the time period to be predicted into the model for prediction, uses cluster analysis to divide users into different groups according to bandwidth usage characteristics, provides services and optimizes resource allocation for different groups, and draws a time series prediction curve to visualize the results; the method includes the following steps: S351: Using the trained long short-term memory network model, input feature data of the time period to be predicted is input into the model, and a predicted value of future bandwidth usage is output; S352: Estimate the uncertainty of the prediction results by making multiple predictions and calculating the mean and standard deviation of the prediction values; S353: According to the K-Means algorithm, extract the data features for cluster analysis, and standardize the data so that each feature has the same dimension and weight; S354: Determine a suitable K value by using the elbow rule, and calculate the sum of squares of clustering errors under different K values. When the reduction in the sum of squares of clustering errors caused by the increase in the K value becomes less obvious, the corresponding K value is a better choice. S355: The quality of clustering results is evaluated by the silhouette coefficient evaluation indicator, and the results are plotted as a time series prediction curve to achieve visual display of the results.

8. The method for automatically detecting and analyzing broadband fixed network user's Internet access equipment and activation rate according to claim 1, characterized in that: The method provides personalized service suggestions for users through cluster analysis algorithms, integrates data sources of other network devices to uniformly collect and analyze multi-dimensional data, and reflects user bandwidth usage and device health status through user-side applications; includes the following steps: S41: Density-based cluster analysis to determine the characteristics and dimensions of clusters; S42: Calculate various statistics of members for each cluster and analyze the behavior patterns of users in each cluster; S43: according to the network device type and specific data indicators that need to collect data, use corresponding data collection interfaces for different devices and data types, collect data from various devices and data sources at set time intervals, and use encrypted TCP / IP connections to transmit the collected data in real time or in batches to a central data processing server. At the data processing server, format conversion and integration of data from different data sources are performed and stored in a distributed database; S44: Through the user interface, including the broadband usage display area, the device health status indicator area, the service recommendation area and the notification center, the application interacts with the server through the API provided by the backend, obtains the user's relevant data, and performs real-time calculation and processing on the obtained data locally. When the user's bandwidth usage reaches the set threshold or the device shows abnormal indicators, the application immediately sends a real-time notification to the user.

9. The method for automatically detecting and analyzing broadband fixed network user Internet access equipment and activation rate according to claim 8, characterized in that: The density-based cluster analysis determines the neighborhood radius of the core point and the minimum number of neighbors of the core point, and determines the characteristics and dimensions of the cluster; and includes the following steps: S411: Determine the neighborhood radius of the core point and the minimum number of neighbors of the core point; S412: Randomly select a point p that has not been visited from the data set, calculate the number of points in the Eps neighborhood of point p, and if there are at least MinPts points in the Eps neighborhood of point p, mark point p as a core point, and mark all points in its Eps neighborhood as directly density reachable; S413: Repeat step S412 for the newly marked point, expand the cluster, and if the number of points in the Eps neighborhood of point p is less than MinPts, mark point p as a noise point; S414: Continue to select unvisited points and repeat the process of steps S412-S413 until all points have been visited; S415: The quality of the clustering results is evaluated by the Calinski-Harabasz index. If the clustering results are not ideal, the values ​​of Eps and MinPts are adjusted and the clustering analysis is repeated.

10. The method for automatically detecting and analyzing broadband fixed network user Internet access equipment and activation rate according to claim 1, characterized in that: The method of performing self-learning and optimization through decision trees and incremental learning technology and automatically generating a detection report of user Internet access devices and activation rates according to the analysis results comprises the following steps: S51: According to the ID3 decision tree algorithm, the target variable of the decision tree is determined by the matching of the user's Internet access device and the activation rate, and an initial decision tree model is constructed using the training set based on the preprocessed data; S52: When new data is continuously generated, the model is updated using incremental learning techniques; S53: Input the user Internet access device and activation rate data to be detected into the optimized ID3 decision tree model. The model predicts and analyzes the input data, determines the matching situation between the device and the rate, and automatically generates a detection report based on the analysis results; S54: Display the generated test report to the user or relevant personnel, and collect feedback from the user or relevant personnel on the report; based on the feedback, further improve the model and report generation process.

Images