A method for automatically detecting and analyzing the opening rate of a wide fixed network user internet access device

Abstract

The present application relates to the field of broadband access service, in particular to a method for automatically detecting and analyzing the broadband network user's online equipment and opening rate, the method comprises the following steps: S1: associating the identification information by using the user optical modem port bandwidth data cached by the OLT device and the user online account and MAC address information of the BRAS device; S2: determining the matching condition of the user optical modem and the subscribed bandwidth by using the machine learning algorithm and the association analysis; S3: forming the AI prediction model by using the time series prediction algorithm and the distributed storage big data analysis technology; S4: providing personalized service suggestions for the user by using the clustering analysis algorithm, and integrating the data sources of other network devices to uniformly collect and analyze multidimensional data; S5: self-learning and optimization by using the decision tree and the incremental learning technology. Through the method, the workload and errors of manual analysis are reduced, the work efficiency is improved, and the adaptability to complex network environment is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of broadband access services, and in particular to a method for automatically detecting and analyzing the broadband access equipment and opening rate of fixed network users. BACKGROUND

[0002] With the rapid development of Internet technology, broadband access services have become an important way for users to access network resources. In order to ensure that users can enjoy stable and high-quality network services, operators need to detect and analyze the user's Internet equipment and opening rate when the user opens the broadband service. However, the existing technical means mostly rely on manual or simple automatic tools for detection, which is low in efficiency and low in accuracy. In addition, the complexity of the user's Internet environment also puts higher requirements on the rate detection. Therefore, how to use the existing network equipment and data resources to realize efficient and accurate automatic detection and analysis of user's Internet equipment and opening rate has become a problem to be solved at present.

[0003] At present, the mainstream scheme mainly relies on the combination of network equipment management system (NMS) and simple rate detection tool, or relies on manual periodic detection to confirm the user equipment and bandwidth. These schemes usually require technicians to manually input or adjust parameters and analyze the historical data in NMS. However, the manual operation steps are many and time-consuming, which cannot reflect the actual use of the user in real time, and the accuracy of the detection result is greatly affected by human factors, lacks the adaptability to complex network environment, and cannot comprehensively detect different user equipment and diversified access scenarios. SUMMARY

[0004] To solve the above problems, the present application provides a method for automatically detecting and analyzing the broadband access equipment and opening rate of fixed network users, which reduces the workload and error of manual analysis, improves the work efficiency and adaptability to complex network environment.

[0005] To achieve the above purpose, the technical scheme adopted by the present application is:

[0006] A method for automatically detecting and analyzing the broadband access equipment and opening rate of fixed network users, comprising the following steps:

[0007] S1: associating and positioning the identification information by using the user optical modem port bandwidth data cached by the OLT device and the user Internet account and MAC address information of the BRAS device;

[0008] S2: determining the matching condition of the user optical modem and the subscribed bandwidth by using machine learning algorithm and association analysis according to the user subscribed bandwidth data and the actual bandwidth of the user optical modem port;

[0009] S3: Extract meaningful features from raw data and calculate features related to device performance, form AI prediction model through time series prediction algorithm and distributed storage big data analysis technology, and deeply mine and predict the trend of user bandwidth usage;

[0010] S4: Provide personalized service suggestions for users through clustering analysis algorithm, integrate data sources of other network devices for unified collection and analysis of multi-dimensional data, and reflect user bandwidth usage and device health status through user end application program;

[0011] S5: Self-learning and optimization through decision tree and incremental learning technology, and automatically generate user online device and opening rate detection report according to analysis results.

[0012] Further, the user optical modem port bandwidth data cached by the OLT device, and the user online account and MAC address information of the BRAS device are associated with the positioning identification information; comprising the following steps:

[0013] S11: Configure data collection function through OLT device, and set user online account and MAC address information parameters through BRAS device;

[0014] S12: Extract user optical modem port bandwidth data from OLT device cache regularly, and transmit it to central data processing server through data transmission channel, and obtain user online account and MAC address information from BRAS device and transmit it to the same data processing server;

[0015] S13: Integrate the data collected from OLT and BRAS devices in the data processing server, and associate the positioning identification information with the user optical modem port bandwidth data and online account through association algorithm;

[0016] S14: Use clustering algorithm to divide users into different groups according to bandwidth usage mode, detect abnormal bandwidth usage, and further analyze the reasons for abnormal situation.

[0017] Further, the matching of user optical modem and subscribed bandwidth is determined according to the user subscribed bandwidth data and the actual bandwidth of user optical modem port through machine learning algorithm and association analysis; comprising the following steps:

[0018] S21: Obtain user subscribed bandwidth data from data storage, collect and record actual bandwidth data of user optical modem port in different time periods, and clean and pretreat the data;

[0019] S22: Extract features from the subscription bandwidth data, calculate statistical features for the actual bandwidth data, and introduce time features, calculate the dynamic features of bandwidth growth rate and bandwidth fluctuation rate based on the user's historical usage data;

[0020] S23: Train the support vector machine model and combine the FP-Growth algorithm based on frequent pattern tree to mine the association rules between the subscription bandwidth features and the actual bandwidth features;

[0021] S24: Evaluate and optimize the model, input new user subscription bandwidth data and actual bandwidth data into the trained model, combine the results of association analysis and give detailed matching explanation and suggestions.

[0022] Further, the support vector machine model is trained, and the FP-Growth algorithm based on the frequent pattern tree is combined to mine the association rules between the subscription bandwidth features and the actual bandwidth features; comprising the following steps:

[0023] S231: Divide the preprocessed data into training set, validation set and test set;

[0024] S232: Initialize the parameters of the support vector machine model, train the support vector machine algorithm using the training set data, update the parameters by iteration to optimize the support vector machine model, and use the validation set to evaluate the support vector machine model during the training process by combining the 5-fold cross-validation technology;

[0025] S233: Analyze the extracted features by the FP-Growth algorithm based on the frequent pattern tree, mine the potential association between the subscription bandwidth features and the actual bandwidth features, filter and filter the mined association rules, and visually display the filtered association rules.

[0026] Further, the AI prediction model formed by the time series prediction algorithm and the distributed storage big data analysis technology extracts meaningful features from the original data and calculates the features related to device performance, and deeply mines and predicts the trend of user bandwidth usage; comprising the following steps:

[0027] S31: Remove duplicate data, process missing values and outliers from the collected user broadband usage raw data, extract average bandwidth usage, peak bandwidth usage and variance features of bandwidth usage in different time periods, and calculate features related to device performance;

[0028] S32: According to the long short-term memory network model, the network structure is built using the PyTorch deep learning framework, the number of neurons in the input layer, hidden layer and output layer is defined, the Adam optimizer is used, and the mean square error is used as the loss function for training, and the network structure, learning rate and iteration number are adjusted to optimize the model performance;

[0029] S33: The original data, feature data and trained model are stored in the HBase distributed database;

[0030] S34: The mean square error, mean absolute error and determination coefficient are calculated as evaluation indicators, the trained model is evaluated through the validation set, and according to the evaluation result, the model is optimized by adjusting the algorithm parameters or increasing the complexity of feature engineering;

[0031] S35: The feature data of the time period to be predicted is input into the trained long short-term memory network model for prediction, the users are divided into different groups according to the bandwidth usage characteristics by cluster analysis, the services are provided and the resource allocation is optimized for different groups, and the result is drawn into a time series prediction curve to realize the result visualization.

[0032] Further, according to the long short-term memory network model, the network structure is built using the PyTorch deep learning framework, the number of neurons in the input layer, hidden layer and output layer is defined, the Adam optimizer is selected and the mean square error is used as the loss function for training, and the network structure, learning rate and iteration number are adjusted to optimize the model performance; including the following steps:

[0033] S321: The long short-term memory network model structure is defined through the PyTorch deep learning framework, and the input layer dimension is matched according to the feature quantity of the input data;

[0034] S322: In the initial construction stage, 1 to 2 hidden layers are set, 64 to 128 neurons are determined for each layer, and a fully connected layer is added for the final prediction output;

[0035] S323: The learning rate is set according to the Adam optimizer, the loss function is defined as the mean square error, the batch size for training and the number of training rounds are determined;

[0036] S324: In each round of training, the data is input into the model in batches for forward propagation to calculate the predicted value, then the error is calculated through the loss function, the error is back propagated, and the optimizer is used to update the parameters of the model to minimize the loss function;

[0037] S325: In the training process, the loss value on the validation set is monitored, and if the loss value on the validation set does not decrease continuously for multiple rounds, the training is stopped in advance.

[0038] Further, the feature data of the time period to be predicted is input into the trained long short-term memory network model for prediction, the users are divided into different groups according to the bandwidth usage features by using clustering analysis, services are provided and resource allocation is optimized for different groups, and the result is drawn into a time series prediction curve to realize result visualization; including the following steps:

[0039] S351: using the trained long short-term memory network model, inputting the input feature data of the time period to be predicted into the model, and outputting the predicted value of the future bandwidth usage;

[0040] S352: estimate the uncertainty of the prediction result by multiple predictions and calculating the mean and standard deviation of the predicted value;

[0041] S353: according to the K-Means algorithm, extracting the data features for clustering analysis, and standardizing the data to make each feature have the same dimension and weight;

[0042] S354: determine the appropriate K value by the elbow rule, calculate the clustering error sum of squares under different K values, when the decreasing amplitude of the clustering error sum of squares caused by the increase of K value becomes not obvious, the corresponding K value is the optimal choice;

[0043] S355: evaluate the quality of the clustering result by the silhouette coefficient evaluation index, draw the time series prediction curve of the result, and realize the result visualization.

[0044] Further, the clustering analysis algorithm is used to provide personalized service suggestions for users, integrate data sources of other network devices for unified collection and analysis of multi-dimensional data, and reflect the user bandwidth usage and device health status through the user end application program; including the following steps:

[0045] S41: determine the features and dimensions of clustering based on density-based clustering analysis;

[0046] S42: calculate various statistics of members for each cluster, and analyze the behavior patterns of users in each cluster;

[0047] S43: according to the types of network devices that need to collect data and specific data indicators, use corresponding data collection interfaces for different devices and data types, collect data from each device and data source at a set time interval, use encrypted TCP / IP connection to transmit the collected data to the central data processing server in real time or batch, and perform format conversion and integration on the data from different data sources at the data processing server end, and store them in the distributed database;

[0048] S44: Through the user interface, including the broadband usage display area, device health status indication area, service recommendation area and notification center, the application program interacts with the server through the API provided by the backend, obtains the relevant data of the user, and performs real-time calculation and processing on the obtained data locally. When the bandwidth usage of the user reaches the set threshold or the device appears abnormal indicators, the application program immediately sends real-time notifications to the user.

[0049] Further, the density-based clustering analysis determines the neighborhood radius of the core point and the minimum number of neighbors of the core point, and determines the characteristics and dimensions of the cluster; comprising the following steps:

[0050] S411: Determine the neighborhood radius of the core point and the minimum number of neighbors of the core point;

[0051] S412: Randomly select an unvisited point p from the data set, calculate the number of points in the Eps neighborhood of point p, if the Eps neighborhood of point p has at least M i nPts points, mark point p as a core point, and mark all points in its Eps neighborhood as directly density reachable;

[0052] S413: Repeat step S412 for newly marked points to expand the cluster, if the number of points in the Eps neighborhood of point p is less than M i nPts, mark point p as a noise point;

[0053] S414: Continue to select unvisited points and repeat steps S412-S413 until all points are visited;

[0054] S415: Evaluate the quality of the clustering result by the Calinski-Harabasz index, if the clustering result is not ideal, adjust the values of Eps and M i nPts and re-cluster the analysis.

[0055] Further, the self-learning and optimization through the decision tree and incremental learning technology automatically generates a detection report of the user's online device and opening rate according to the analysis result; comprising the following steps:

[0056] S51: According to the ID3 decision tree algorithm, determine the target variable of the decision tree through the matching of the user's online device and the opening rate, and based on the preprocessed data, use the training set to build an initial decision tree model;

[0057] S52: When new data is continuously generated, update the model using incremental learning technology;

[0058] S53: input the user's online device to be detected and the open rate data into the optimized ID3 decision tree model, the model makes prediction and analysis according to the input data, judges the matching condition of the device and the rate, and automatically generates a detection report according to the analysis result;

[0059] S54: display the generated detection report to the user or relevant personnel, collect the feedback opinions of the user or relevant personnel on the report, and further improve the model and the report generation process according to the feedback opinions.

[0060] The beneficial effects of the present application are:

[0061] 1. Accurate detection of the matching condition of the device and the open rate ensures that the user obtains the expected network performance, reduces problems such as slow network speed and lag caused by mismatch, and improves the user's experience in various network activities such as online surfing, video playing and online gaming; provides improvement suggestions for users who are not matched, helps them optimize network settings or upgrade devices, and further meets the user's demand for high-speed and stable network.

[0062] 2. Helps operators understand the actual needs and device conditions of users, more accurately allocates network bandwidth resources, avoids resource waste or deficiency, and upgrades network infrastructure in areas where users with insufficient device performance but high-speed open rate are concentrated, to improve the overall service quality of the network.

[0063] 3. Automatic detection and analysis reduces the workload and cost of manual detection, improves work efficiency, discovers potential network problems and device failures in a timely manner, takes measures for maintenance and optimization in advance, and reduces fault handling time and cost.

[0064] 4. Provides personalized network packages and value-added service suggestions for users according to the detection results, increases user satisfaction and loyalty, and helps operators develop more accurate marketing strategies to promote products and services suitable for different user groups.

[0065] 5. A large amount of detection data provides a basis for network technology research and innovation, promotes the development of more efficient network protocols and more intelligent bandwidth allocation algorithms, and high-quality network services and precise user services can improve the competitiveness of operators in the market, attract more new users and retain existing users. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 It is a method flowchart for automatically detecting and analyzing the online device and open rate of a broadband fixed network user. DETAILED DESCRIPTION

[0067] Please refer to Figure 1 The present application relates to a method for automatically detecting and analyzing the online device and open rate of a broadband fixed network user, which comprises the following steps:

[0068] S1: Cache the user optical modem port bandwidth data through the OLT device, and associate the positioning identification information with the user Internet account and MAC address information of the BRAS device;

[0069] S2: According to the user subscription bandwidth data and the actual bandwidth of the user optical modem port, determine the matching condition of the user optical modem and the subscription bandwidth through machine learning algorithm and association analysis;

[0070] S3: Extract meaningful features from the original data and calculate the features related to the performance of the device, and form an AI prediction model through time series prediction algorithm and distributed storage big data analysis technology, to deeply mine and predict the trend of user bandwidth usage;

[0071] S4: Provide personalized service suggestions for users through clustering analysis algorithm, integrate data sources of other network devices for unified collection and analysis of multi-dimensional data, and reflect the user bandwidth usage and device health status through user end application program;

[0072] S5: Through decision tree and incremental learning technology, self-learning and optimization are carried out, and a detection report of user Internet device and opening rate is automatically generated according to the analysis result.

[0073] Further, the user optical modem port bandwidth data cached by the OLT device, and the user Internet account and MAC address information of the BRAS device are associated with the positioning identification information, including the following steps:

[0074] S11: Configure data collection function through OLT device, and set user Internet account and MAC address information parameters through BRAS device;

[0075] S12: Extract user optical modem port bandwidth data from the cache of OLT device regularly, and transmit it to the central data processing server through data transmission channel, and obtain user Internet account and MAC address information from BRAS device and transmit it to the same data processing server;

[0076] S13: Integrate the data collected from OLT and BRAS devices in the data processing server, and associate the positioning identification information with the user optical modem port bandwidth data and Internet account through association algorithm;

[0077] S14: Use clustering algorithm to divide users into different groups according to bandwidth usage mode, detect abnormal bandwidth usage, and further analyze the reasons for abnormal situation.

[0078] Specifically, the collection of the user's online account and MAC address information of the BRAS device refers to collecting the user's online account, MAC address and other information through the broadband remote access server (BRAS) device, and recording the network connection of the user terminal device. This process ensures the accuracy and integrity of the user terminal device and network connection data.

[0079] The positioning identification information refers to information such as IP address, port number, ONU ID (optical network unit identifier), MAC address, etc. for positioning identification. Through clustering algorithm for deep correlation analysis, the collected user optical modem port bandwidth data is accurately matched with the user online account. This step ensures the high correlation and consistency of the data, and eliminates the data island phenomenon.

[0080] Further, the matching of the user optical modem and the subscribed bandwidth is determined according to the user subscribed bandwidth data and the actual bandwidth of the user optical modem port through machine learning algorithm and correlation analysis; including the following steps:

[0081] S21: Obtain user subscribed bandwidth data from data storage, collect and record actual bandwidth data of user optical modem port in different time periods, and clean and preprocess the data;

[0082] S22: Extract features from the subscribed bandwidth data, calculate statistical features of the actual bandwidth data, and introduce time features, calculate bandwidth growth rate and bandwidth fluctuation rate dynamic features based on user's historical use data;

[0083] S23: Train the support vector machine model, and combine the FP-Growth algorithm based on frequent pattern tree to mine the association rules between the subscribed bandwidth features and the actual bandwidth features;

[0084] S24: Evaluate and optimize the model, input new user subscribed bandwidth data and actual bandwidth data into the trained model, combine the correlation analysis results and give detailed matching explanation and suggestion.

[0085] Specifically, the user subscribed bandwidth data includes bandwidth size, package type, validity period and other information; record the actual bandwidth measurement value in different time periods (such as every day, every week, every month), clean and preprocess including processing missing value, abnormal value, and format the data to suitable format for analysis.

[0086] The extracted features include bandwidth value, package level, etc., and the statistical features include mean, median, standard deviation, maximum value, minimum value, etc. For time features, including bandwidth features in different time periods, and user's use habit features (such as peak period use time, non-peak period use time, etc.).

[0087] In step S24, the trained support vector machine model is finally evaluated using the test set data by calculating the accuracy, precision, recall, F1 value, area under the ROC curve, and mean square error. According to the evaluation results, the performance and shortcomings of the model are analyzed. New user subscription bandwidth data and actual bandwidth data are input into the trained model, and the model outputs the matching situation of the user optical modem and the subscription bandwidth, such as matching, non-matching, or partial matching. Combined with the results of the association analysis, more detailed matching explanations and suggestions are given, and new data is continuously monitored to observe the performance of the model in actual application. When the data distribution changes significantly or the model performance decreases, data collection and model training are re-performed to update and optimize the model.

[0088] Based on the collected data, real-time monitoring and early warning functions are added. When abnormal user bandwidth utilization or actual bandwidth that does not meet the subscription requirements is detected, the system can issue real-time warnings to remind operators to take timely measures to prevent user experience from being affected.

[0089] Further, the support vector machine model is trained, and the association rules between the subscription bandwidth features and the actual bandwidth features are mined using the FP-Growth algorithm based on the frequent pattern tree. The steps include:

[0090] S231: The preprocessed data is divided into a training set, a validation set, and a test set;

[0091] S232: The parameters of the support vector machine model are initialized, the support vector machine algorithm is trained using the training set data, the support vector machine model is optimized by iteratively updating the parameters, and the support vector machine model during training is evaluated using the validation set in combination with the 5-fold cross-validation technique;

[0092] S233: The extracted features are analyzed using the FP-Growth algorithm based on the frequent pattern tree to mine potential associations between the subscription bandwidth features and the actual bandwidth features. The mined association rules are filtered and visualized.

[0093] Specifically, the support vector machine is used for machine learning algorithm, and the FP-Growth algorithm based on the frequent pattern tree is used for association analysis. The preprocessed data is randomly divided into a training set, a validation set, and a test set according to the proportions of 70%, 15%, and 15%. Ensure that each set contains various types of users and bandwidth conditions to ensure data representativeness.

[0094] The kernel function type of the support vector machine is initialized, the algorithm is trained using the training set data, the parameters are updated by iteration to optimize the model, the loss function of the model on the training set is minimized, the 5-fold cross-validation technique is combined in the training process to prevent overfitting, the model in the training process is evaluated using the validation set, and the model parameters with the best performance on the validation set are selected. For the association analysis algorithm, set appropriate support and confidence thresholds to mine meaningful association rules.

[0095] The extracted features are analyzed using the FP-Growth algorithm based on the frequent pattern tree, and the potential association between the signed bandwidth features and the actual bandwidth features is mined. For the mined association rules, filtering and filtering are performed to remove rules that are too general or have no actual meaning. The filtered association rules are visually displayed, including using bar charts, line charts, or mesh charts, etc. to intuitively understand the content and strength of the rules.

[0096] Further, the AI prediction model formed by the time series prediction algorithm and the distributed storage big data analysis technology extracts meaningful features from the original data and calculates features related to device performance, and performs deep mining and trend prediction on user bandwidth usage; including the following steps:

[0097] S31: Remove duplicate data, handle missing values and outliers from the collected user broadband usage original data, extract average bandwidth usage, peak bandwidth usage and variance features of bandwidth usage in different time periods, and calculate features related to device performance;

[0098] S32: According to the long short-term memory network model, use the PyTorch deep learning framework to build the network structure, define the number of neurons in the input layer, hidden layer and output layer, use the Adam optimizer, and use the mean square error as the loss function for training, adjust the network structure, learning rate and iteration number to optimize the model performance;

[0099] S33: Store the original data, feature data and trained model in the HBase distributed database;

[0100] S34: Calculate the mean square error, mean absolute error and determination coefficient as evaluation indicators, evaluate the trained model through the validation set, and according to the evaluation result, optimize the model by adjusting the algorithm parameters or increasing the complexity of feature engineering;

[0101] S35: Input the feature data of the time period to be predicted into the trained long short-term memory network model for prediction, use clustering analysis to divide users into different groups according to bandwidth usage features, provide services and optimize resource allocation for different groups, and draw time series prediction curves to realize result visualization.

[0102] Specifically, the features related to device performance are calculated, including the packet loss rate, delay, stability indicators of network connection, etc. of the device. In the HBase distributed database, reasonable data partitioning and storage strategies are designed to improve the read-write efficiency and parallel processing capability of data.

[0103] On the validation set, the above evaluation indicators are calculated to compare the performance of the model under different algorithms and parameter settings. If the model performs poorly, try to increase the data preprocessing steps, including log transformation, standardization, etc. to make the data more consistent with the model assumptions, or increase the number and quality of features, introduce more factors related to bandwidth usage, or adjust the hyperparameters of the model, or try to use ensemble learning methods, such as weighted average of the prediction results of multiple models.

[0104] Further, the long short-term memory network model is constructed using the PyTorch deep learning framework, the number of neurons in the input layer, hidden layer and output layer is defined, the Adam optimizer and mean square error are selected as the loss function for training, and the model performance is optimized by adjusting the network structure, learning rate and iteration number; including the following steps:

[0105] S321: Define the long short-term memory network model structure through the PyTorch deep learning framework, and match the input layer dimension according to the number of features of the input data;

[0106] S322: Set 1 to 2 hidden layers in the initial construction stage, determine that each layer has 64 to 128 neurons, and add a fully connected layer for the final prediction output;

[0107] S323: Set the learning rate according to the Adam optimizer, define the loss function as the mean square error, determine the batch size and the number of training rounds;

[0108] S324: In each round of training, input the data in batches into the model to calculate the predicted value through forward propagation, then calculate the error through the loss function, and update the model parameters using the optimizer to minimize the loss function through back propagation;

[0109] S325: Monitor the loss value on the validation set during the training process, and if the validation set loss does not decrease continuously for multiple rounds, stop training in advance.

[0110] Further, the features of the time period to be predicted are input into the trained long short-term memory network model for prediction, and the users are divided into different groups according to the bandwidth usage characteristics through clustering analysis, and the service and resource allocation are optimized for different groups, and the result is drawn into a time series prediction curve to realize the visualization of the result; including the following steps:

[0111] S351: input the input feature data of the time period to be predicted into the trained long short-term memory network model, and output the predicted value of the future bandwidth usage;

[0112] S352: estimate the uncertainty of the prediction result by multiple predictions and calculation of the mean and standard deviation of the predicted value;

[0113] S353: according to the K-Means algorithm, extract the data features for clustering analysis, and standardize the data to make each feature have the same dimension and weight;

[0114] S354: determine the appropriate K value by the elbow rule, calculate the clustering error sum of squares under different K values, and when the decreasing amplitude of the clustering error sum of squares caused by the increase of K value becomes not obvious, the corresponding K value is the optimal choice;

[0115] S355: evaluate the quality of the clustering result by the silhouette coefficient evaluation index, draw the time series prediction curve of the result, and realize the visual display of the result.

[0116] Specifically, the input feature data includes future time stamp and related environmental features. In estimating the uncertainty of the prediction result, the larger the standard deviation, the higher the uncertainty. In step S353, the extracted data features include average bandwidth usage, peak bandwidth usage or bandwidth usage volatility in different time periods. When determining the K value, the number of possible user types can also be estimated according to business understanding and prior knowledge to preliminarily determine the K value.

[0117] For the implementation steps of K-Means algorithm: randomly select K data points as initial cluster centers, for each data point, calculate its Euclidean distance to each cluster center, assign the data point to the cluster to which the nearest cluster center belongs, recalculate the center of each cluster, i.e. the mean of all data points in the cluster, repeat the calculation of distance, assignment of data points and calculation of center until the cluster center no longer changes or the maximum number of iterations is reached.

[0118] Further, the clustering analysis algorithm is used to provide personalized service suggestions for users, integrate data sources of other network devices for unified collection and analysis of multi-dimensional data, and reflect the bandwidth usage and device health status of users through user-side application programs; including the following steps:

[0119] S41: density-based clustering analysis to determine the characteristics and dimensions of clustering;

[0120] S42: calculate various statistics of members for each cluster, and analyze the behavior patterns of users in each cluster;

[0121] S43: According to the type of network equipment that needs to collect data and the specific data indicators, use the corresponding data collection interface for different equipment and data types, collect data from each device and data source at the set time interval, use encrypted TCP / IP connection to transmit the collected data to the central data processing server in real time or in batches, and perform format conversion and integration on the data from different data sources at the data processing server end, and store them in the distributed database.

[0122] S44: Through the user interface, including the broadband usage display area, the device health status indication area, the service recommendation area and the notification center, the application program interacts with the server through the API provided by the backend to obtain the relevant data of the user, performs real-time calculation and processing on the obtained data locally, and immediately sends real-time notifications to the user when the bandwidth usage of the user reaches the set threshold or the device has abnormal indicators.

[0123] Specifically, in step S42, the various statistical quantities of its members are calculated, including: the mean, median, standard deviation of bandwidth usage, the mean and variance of device performance indicators, and the frequency distribution of different application programs. The mode refers to recommending high-speed and stable dedicated line services for clusters with large and stable bandwidth usage and high device performance requirements, launching limited-time high-speed bandwidth packages for clusters dominated by entertainment and high bandwidth demand in specific time periods, and recommending economical and affordable basic packages for clusters with small bandwidth usage, good device performance, but potential demand.

[0124] In step S43, the corresponding data collection interface is used for different devices and data types, including: for devices that support SNMP protocol, write SNMP agent program to obtain device performance data; for network traffic data, use NetFlow or sFlow protocol to collect. The data from different data sources is converted and integrated, including having a unified structure and fields. It is stored in the distributed database and appropriate indexes are established for fast query and analysis.

[0125] In step S44, the display area displays the recent bandwidth usage trend in the form of line chart, column chart, etc., including the changes of upload and download speed; the device health status indication area displays the key performance indicators of the device, such as CPU utilization, memory usage, etc. through color coding or specific numerical values; the service recommendation area displays personalized service recommendations and package suggestions according to the cluster and analysis results to which the user belongs; the notification center displays important notifications sent by the system, such as bandwidth running out reminder, device failure warning, etc. The real-time calculation and processing of the obtained data locally includes calculating the current bandwidth usage percentage and comparing with historical data for analysis.

[0126] Further, the density-based clustering analysis determines the neighborhood radius of the core point and the minimum number of neighbors of the core point, determines the characteristics and dimensions of the cluster; comprising the following steps:

[0127] S411: Determine the neighborhood radius of the core point and the minimum number of neighbors of the core point;

[0128] S412: Randomly select an unvisited point p from the data set, calculate the number of points in the Eps neighborhood of point p, if there are at least M i nPts points in the Eps neighborhood of point p, mark point p as a core point, and mark all points in its Eps neighborhood as directly density reachable;

[0129] S413: Repeat step S412 for newly marked points to expand the cluster, if the number of points in the Eps neighborhood of point p is less than M i nPts, mark point p as a noise point;

[0130] S414: Continue to select unvisited points and repeat the process of steps S412-S413 until all points are visited;

[0131] S415: Evaluate the quality of the clustering result by the Calinski-Harabasz index, if the clustering result is not ideal, adjust the values of Eps and M i nPts, and perform clustering analysis again.

[0132] Specifically, in step S411, the Eps is the neighborhood radius of the core point, the selection of the two parameters can be based on the distribution characteristics and prior knowledge of the data, or determined through multiple trials and visual analysis. For example, by drawing a k-distance graph of the data to assist in determining the Eps value, and observing the inflection point position of the curve; M i nPts can be initially set according to the density of the data and the expected clustering effect, usually between 5-10.

[0133] Further, the self-learning and optimization through the decision tree and incremental learning technology, automatically generating a detection report of user online equipment and opening rate according to the analysis result; comprising the following steps:

[0134] S51: According to the ID3 decision tree algorithm, determine the target variable of the decision tree through the matching of user online equipment and opening rate, based on the preprocessed data, use the training set to build an initial decision tree model;

[0135] S52: When new data is continuously generated, update the model using incremental learning technology;

[0136] S53: input the user's online device to be detected and the open rate data into the optimized ID3 decision tree model, the model makes prediction and analysis according to the input data, judges the matching condition of the device and the rate, and automatically generates a detection report according to the analysis result;

[0137] S54: display the generated detection report to the user or relevant personnel, collect the feedback opinions of the user or relevant personnel on the report, and further improve the model and the report generation process according to the feedback opinions.

[0138] Specifically, in step S52, for incremental learning and model optimization, the preprocessed data is gradually added to the original training set, the ID3 decision tree model is retrained using the updated training set, the structure and parameters of the tree are adjusted, and the performance of the model is regularly evaluated, including using accuracy, recall rate, F1 value and the like. If the performance of the model decreases, analyze the reason, which may be data distribution change, feature importance change and the like. According to the evaluation result and analysis, the model is adjusted and optimized, including reselecting features, adjusting the splitting condition of the decision tree and the like.

[0139] In step S53, the detection report includes the detailed information of the user's online device, the open rate, the matching result judged by the model, if not matching, provides possible reasons and improvement suggestions, including device performance limitation, network congestion and the like leading to mismatch, and suggests upgrading the device or adjusting the network configuration.

[0140] The above embodiments only describe the preferred embodiments of the present application, and do not limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements of the technical solutions of the present application made by ordinary engineering technical personnel in the art shall fall within the protection scope determined by the claims of the present application.

Claims

1. A method for automatically detecting and analyzing the internet access equipment and activation speed of broadband fixed network users, characterized in that, Includes the following steps: S1: The user's optical modem port bandwidth data cached by the OLT device, as well as the user's Internet account and MAC address information from the BRAS device, are associated with location identification information; S2: Based on the user's contracted bandwidth data and the actual bandwidth of the user's optical modem port, determine the matching status between the user's optical modem and the contracted bandwidth through machine learning algorithms and correlation analysis; S3: Extract meaningful features from raw data and calculate features related to device performance. Through time series prediction algorithms and big data analysis technology of distributed storage, form an AI prediction model to deeply mine and predict the trend of user bandwidth usage. S4: Provide personalized service suggestions to users through clustering analysis algorithms, integrate data sources from other network devices to collect and analyze multi-dimensional data in a unified manner, and reflect user bandwidth usage and device health status through user-end applications; S5: It learns and optimizes itself through decision trees and incremental learning technology, and automatically generates a detection report on the user's Internet access device and activation speed based on the analysis results; In S2, the process of determining the matching status between the user's optical modem and the contracted bandwidth based on the user's subscribed bandwidth data and the actual bandwidth of the user's optical modem port through machine learning algorithms and correlation analysis includes the following steps: S21: Obtain user-subscribed bandwidth data from data storage, collect and record actual bandwidth data of user optical modem ports at different time periods, and clean and preprocess the data; S22: Extract features from contracted bandwidth data, calculate statistical features from actual bandwidth data, and introduce time features to calculate dynamic features of bandwidth growth rate and bandwidth volatility based on users' historical usage data. S23: By training the support vector machine model and combining it with the FP-Growth algorithm based on frequent pattern trees, the association rules between the contracted bandwidth features and the actual bandwidth features are mined. S24: Evaluate and optimize the model by inputting new user subscription bandwidth data and actual bandwidth data into the trained model, combining the results of the correlation analysis, and providing detailed matching explanations and suggestions. In S3, the step of extracting meaningful features from the raw data and calculating features related to device performance, and then using an AI prediction model formed through time series prediction algorithms and distributed storage big data analysis technology to deeply mine and predict trends in user bandwidth usage, includes the following steps: S31: Remove duplicate data, process missing values ​​and outliers from the raw broadband usage data collected from users, extract the average bandwidth usage, peak bandwidth usage and variance features of bandwidth usage in different time periods, and calculate features related to equipment performance. S32: Based on the Long Short-Term Memory network model, construct the network structure using the PyTorch deep learning framework, define the number of neurons in the input layer and hidden layer and the output layer, use the Adam optimizer and train with mean squared error as the loss function, and adjust the network structure, learning rate and number of iterations to optimize model performance. S33: Store raw data, feature data, and trained models using the HBase distributed database; S34: The trained model is evaluated using the mean squared error, mean absolute error and coefficient of determination as evaluation metrics. Based on the evaluation results, the model is optimized by adjusting the algorithm parameters or increasing the complexity of feature engineering. S35: By using a trained long short-term memory network model, the feature data of the time period to be predicted is input into the model for prediction. Cluster analysis is used to divide users into different groups according to their bandwidth usage characteristics. Services are provided and resource allocation is optimized for different groups. The results are then plotted as time series prediction curves to visualize the results.

2. The method for automatic detection and analysis of broadband fixed network user internet access equipment and activation speed according to claim 1, characterized in that, The process of associating user optical modem port bandwidth data cached by the OLT device with user internet account and MAC address information from the BRAS device and location identification information includes the following steps: S11: Configure the data acquisition function through the OLT device, and collect user Internet account and MAC address information parameters set on the BRAS device; S12: The OLT device periodically extracts the user's optical modem port bandwidth data from its cache and transmits it to the central data processing server through the data transmission channel. At the same time, it obtains the user's Internet account and MAC address information from the BRAS device and transmits it to the same data processing server. S13: The data collected from the OLT and BRAS devices is integrated in the data processing server, and the location identification information is associated with the user's optical modem port bandwidth data and Internet account through the association algorithm; S14: Use clustering algorithms to divide users into different groups according to their bandwidth usage patterns, detect abnormal bandwidth usage, and further analyze the causes of the abnormalities.

3. The method for automatic detection and analysis of broadband fixed network user internet access equipment and activation speed according to claim 1, characterized in that, The method of training a support vector machine model and combining it with the FP-Growth algorithm based on frequent pattern trees to mine the association rules between contracted bandwidth features and actual bandwidth features includes the following steps: S231: Divide the preprocessed data into training set, validation set and test set; S232: Initialize the parameters of the support vector machine model, train the support vector machine algorithm using the training set data, optimize the support vector machine model by iteratively updating the parameters, combine the 5-fold cross-validation technique during the training process, and use the validation set to evaluate the support vector machine model during the training process; S233: The extracted features are analyzed using the FP-Growth algorithm based on frequent pattern trees to uncover the potential correlation between contracted bandwidth features and actual bandwidth features. The uncovered correlation rules are then screened and filtered, and the filtered correlation rules are visualized.

4. The method for automatic detection and analysis of broadband fixed network user internet access equipment and activation speed according to claim 1, characterized in that, The method involves constructing a network structure based on a Long Short-Term Memory (LSTM) network model using the PyTorch deep learning framework, defining the number of neurons in the input and hidden layers, and the output layer. The Adam optimizer and mean squared error are selected as the loss function for training. Model performance is optimized by adjusting the network structure, learning rate, and number of iterations. This includes the following steps: S321: Define the Long Short-Term Memory network model structure using the PyTorch deep learning framework, and match the input layer dimension to the number of features in the input data. S322: In the initial construction phase, set 1 to 2 hidden layers, determine that each layer has 64 to 128 neurons, and add fully connected layers for the final prediction output; S323: Set the learning rate based on the Adam optimizer, define the loss function based on the mean squared error, and determine the batch size and number of training epochs; S324: In each round of training, data is input into the model in batches for forward propagation to calculate the predicted value. Then, the error is calculated through the loss function, the error is backpropagated, and the optimizer is used to update the model parameters to minimize the loss function. S325: Monitor the loss value on the validation set during training. If the loss on the validation set no longer decreases after several consecutive rounds, stop training early.

5. The method for automatic detection and analysis of broadband fixed network user internet access equipment and activation speed according to claim 1, characterized in that, The process involves using a trained Long Short-Term Memory (LSTM) network model, inputting feature data of the time period to be predicted into the model for prediction, employing cluster analysis to divide users into different groups based on bandwidth usage characteristics, providing services and optimizing resource allocation for different groups, and visualizing the results by plotting time series prediction curves. This includes the following steps: S351: Using a trained long short-term memory network model, input the input feature data of the time period to be predicted into the model, and output the predicted value of future bandwidth usage. S352: Estimate the uncertainty of the prediction results by making multiple predictions and calculating the mean and standard deviation of the predicted values; S353: Based on the K-Means algorithm, extract data features for cluster analysis, and standardize the data so that each feature has the same dimensions and weights; S354: Determine a suitable K value using the elbow rule, calculate the sum of squared clustering errors under different K values, and when the decrease in the sum of squared clustering errors brought about by increasing the K value becomes insignificant, the corresponding K value is the better choice; S355: The quality of clustering results is evaluated using the silhouette coefficient as an evaluation metric, and the results are plotted as a time series prediction curve to achieve a visual representation of the results.

6. The method for automatic detection and analysis of broadband fixed network user internet access equipment and activation speed according to claim 1, characterized in that, The process of providing personalized service suggestions to users through clustering analysis algorithms, integrating data sources from other network devices for unified collection and analysis of multi-dimensional data, and reflecting user bandwidth usage and device health status through user-end applications includes the following steps: S41: Density-based cluster analysis to determine the characteristics and dimensions of clusters; S42: Calculate various statistics for each member of each cluster and analyze the behavioral patterns of users in each cluster; S43: Based on the network device type and specific data indicators required for data collection, use the corresponding data collection interface for different devices and data types, collect data from various devices and data sources at set time intervals, and use encrypted TCP / IP connections to transmit the collected data to the central data processing server in real time or in batches. On the data processing server, perform format conversion and integration on the data from different data sources, and store it in a distributed database. S44: Through the user interface, including the broadband usage display area, device health status indicator area, service recommendation area and notification center, the application interacts with the server through the API provided by the backend to obtain relevant user data, performs real-time calculation and processing on the obtained data locally, and immediately sends a real-time notification to the user when the user's bandwidth usage reaches the set threshold or the device shows abnormal indicators.

7. The method for automatic detection and analysis of broadband fixed network user internet access equipment and activation speed according to claim 6, characterized in that, The density-based clustering analysis determines the characteristics and dimensions of the clusters, including the following steps: S411: Determine the neighborhood radius of the core point and the minimum number of neighbors of the core point; S412: Randomly select an unvisited point p from the dataset, calculate the number of points in the Eps neighborhood of point p, and if there are at least MinPts points in the Eps neighborhood of point p, mark point p as the core point and mark all points in its Eps neighborhood as directly density reachable. S413: Repeat step S412 for the newly labeled point to expand the cluster. If the number of points in the Eps neighborhood of point p is less than MinPts, then label point p as a noise point. S414: Continue selecting unvisited points and repeat steps S412-S413 until all points have been visited; S415: The quality of clustering results is evaluated using the Calinski-Harabasz index. If the clustering results are not ideal, the values ​​of Eps and MinPts are adjusted, and the clustering analysis is performed again.

8. The method for automatic detection and analysis of broadband fixed network user internet access equipment and activation speed according to claim 1, characterized in that, The process of self-learning and optimization through decision trees and incremental learning techniques, and automatically generating a detection report on the user's internet access device and connection speed based on the analysis results, includes the following steps: S51: Based on the ID3 decision tree algorithm, the target variable of the decision tree is determined by the matching between the user's Internet access device and the activation speed. Based on the preprocessed data, the initial decision tree model is constructed using the training set. S52: As new data is continuously generated, incremental learning techniques are used to update the model; S53: Input the user's internet access device and activation speed data to the optimized ID3 decision tree model. The model makes predictions and analyses based on the input data to determine the matching status of the device and speed. Based on the analysis results, it automatically generates a detection report. S54: Display the generated test report to the user or relevant personnel, collect feedback from the user or relevant personnel on the report, and further improve the model and report generation process based on the feedback.

Images