A method for automated identification of network quality status

Abstract

The application provides a method for automatically identifying network quality state, and relates to the field of network quality state identification, and comprises the following steps: collecting network data, using Lagrange interpolation method on the part of data with missing values to obtain a data set with the same distribution as the original data; using principal component analysis method to reduce the dimension of the data set to obtain a simplified data set without losing the characteristics of the original data set, constructing a weight matrix, a threshold matrix and a final threshold value; constructing a generator based on the weight matrix and the threshold matrix, scoring and comparing the simplified data set, and dividing it into poor quality data and non-poor quality data; constructing a classifier based on a fully connected neural network, calculating the boundary between the poor quality data and the non-poor quality data, obtaining a new threshold matrix, and calculating the Euclidean distance with the previous threshold matrix to determine whether the training is convergent and stable. The application can effectively identify and quantify the network quality state, and provides accurate reference for operation and maintenance personnel to analyze poor quality data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of network quality state recognition, and in particular to a method for automatically recognizing network quality state. BACKGROUND

[0002] In network environment operation and maintenance, operation and maintenance personnel need to maintain and upgrade the communication equipment in the network, and how to accurately find out where the problem is cannot be separated from the judgment of network quality state and the cause analysis of network quality state difference. How to judge the network quality state according to the network state index has become a difficult problem for operation and maintenance personnel.

[0003] Chinese patent CN114648053A "5G network quality automatic evaluation method based on machine learning" discloses an evaluation method for 5G communication network, which judges according to the difference of network demand of different services in the communication network. However, when comparing the error index with the threshold value, the precision of the generative adversarial network used is not enough, and it needs to be improved to reduce the error for more accurate identification of network quality. SUMMARY

[0004] Therefore, the present application provides a method for automatically recognizing network quality state to solve the problems of traditional network quality state.

[0005] The technical scheme of the present application is as follows: The present application provides a method for automatically recognizing network quality state, comprising the following steps:

[0006] S1, collecting network data, using Lagrange interpolation method to fill the missing part of the data based on the adjacent points to obtain a data set with the same distribution as the original data;

[0007] S2, using principal component analysis method to reduce the dimension of the data set to obtain a simplified data set without losing the characteristics of the original data set, and constructing a weight matrix, a threshold matrix and a final threshold value;

[0008] S3, constructing a generator based on the weight matrix and the threshold matrix, scoring and comparing the simplified data set, and dividing it into poor quality data and non-poor quality data;

[0009] S4, constructing a classifier based on a fully connected neural network, calculating the boundary between the poor quality data and the non-poor quality data, obtaining a new threshold matrix, and calculating the Euclidean distance with the previous threshold matrix to determine whether the training is stable;

[0010] S5, when the training is not stable, updating the new threshold matrix obtained in step S4 to the generator, and repeating steps S3-S4; when the training is stable, outputting the weight matrix and the threshold matrix, and outputting the corresponding classifier and generator.

[0011] Preferably, step S1 specifically comprises:

[0012] The collected data is read into the memory through the numpy (Numeric Python) library, and the missing part of the data is filled according to the Lagrange interpolation method.

[0013] Preferably, step S2 specifically comprises:

[0014] The principal component analysis method is used to reduce the dimension of the data set, which is stored in a matrix form to obtain a metadata matrix X,

[0015]

[0016] The X matrix is an m-row n-column matrix, where m is the number of network data records in the data set, and n is the number of network data record indicators. The threshold matrix T = [t1 t2 … t n ] is a 1-row n-column matrix, the weight matrix W T = [ω1 ω2 … ω n ] is a 1-row n-column matrix, and the hyperparameter final threshold T final is constructed.

[0017] Preferably, step S3 specifically comprises the following steps:

[0018] S310, threshold comparison;

[0019] S320, weighted scoring and comparison.

[0020] Preferably, step S310 specifically comprises:

[0021] The threshold matrix T is expanded to m rows and n columns, denoted as T', and is subtracted from the metadata matrix X. According to the positive and negative nature of the data, it is normalized to a 0-1 matrix A.

[0022] Preferably, step S320 specifically comprises:

[0023] The 0-1 matrix A and the weight matrix W T are multiplied to obtain an m-row 1-column final scoring matrix B. Each row parameter of the matrix B is compared with the hyperparameter final threshold T final , and those greater than T final are regarded as poor records, and those less than T final are regarded as non-poor records.

[0024] Preferably, step S4 specifically comprises:

[0025] Add a column after the nth column of the metadata matrix X, denoted as matrix X'. After each row of parameter comparisons in step S320, add 1 after the row corresponding to the poor quality record and add 0 after the row corresponding to the non-poor quality record. Add 0 after the nth column of the threshold matrix T', making it an m-row n+1-column matrix T'". Divide matrix X' and matrix T'' to obtain the difference matrix D, which serves as the interval between the first and second datasets. Input the difference matrix D into the classifier for training, using the cross-entropy function as the loss function. Adjust the threshold in the neural network through gradient descent and backpropagation. The adjustment formula is as follows:

[0026]

[0027] Among them, T n Let η be the threshold and η be the learning rate. Let be the Jacobian matrix corresponding to the threshold.

[0028] Preferably, step S4 further includes:

[0029] The formula for calculating Euclidean distance is:

[0030]

[0031] The threshold vector for the previous step is X = (x1, x2, ..., x...). n The threshold vector for the next step is Y = (y1, y2, ..., y). n When the Euclidean distance approaches 0, the training is considered to be stable and convergent.

[0032] The automated network quality status identification method of the present invention has the following advantages over the prior art:

[0033] (1) By subtracting the input matrix and the threshold matrix, the difference matrix D is obtained and input into the classifier for training, thereby improving the classification ability of the classification algorithm;

[0034] (2) When differentiating the weight matrix, add the distance-related part, and make corrections to the threshold in the neural network through gradient descent and backpropagation to achieve the optimal solution;

[0035] (3) Calculate the Euclidean distance between the threshold vectors before and after the two tests to determine whether the generator and classifier training is stable and converged, thereby improving the model accuracy. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart of an automated network quality status identification method according to the present invention. Detailed Implementation

[0038] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0039] Example 1

[0040] Provide a method for automated network quality status identification, such as Figure 1 As shown, it includes the following steps:

[0041] S1. Collect network data, and use Lagrange interpolation to fill in the missing data based on nearest neighbors to obtain a dataset with the same distribution as the original data.

[0042] S2, Principal component analysis is used to reduce the dimensionality of the dataset to obtain a simplified dataset without losing the features of the original dataset, and the weight matrix, threshold matrix and final threshold matrix are constructed.

[0043] S3, based on the weight matrix and threshold matrix, builds a generator to score and compare the simplified dataset, and splits it into the first dataset and the second dataset;

[0044] S4. A classifier is built based on a fully connected neural network. The interval between the first and second datasets is calculated to obtain a new threshold matrix. The Euclidean distance with the previous threshold matrix is ​​calculated to determine whether the training has converged and stabilized.

[0045] S5. When the training has not converged and stabilized, update the generator with the new threshold matrix obtained in step S4, and repeat steps S3-S4. When the training has converged and stabilized, output the weight matrix and threshold matrix, and output the corresponding classifier and generator.

[0046] Step S1 specifically includes:

[0047] The collected data is read into memory using the NumPy library, and missing data is filled in using the Lagrange interpolation method. The NumPy library is a powerful Python library mainly used to perform calculations on multidimensional arrays. It provides a large number of library functions and operations to help technicians easily perform numerical calculations.

[0048] In the daily operation of the network platform, there may be missing data, so it is necessary to fill it in using the Lagrange interpolation method to form a complete dataset.

[0049] Step S2 specifically includes:

[0050] Principal component analysis was used to reduce the dimensionality of the dataset, and the data was stored in a matrix format to obtain the metadata matrix X.

[0051]

[0052] The X matrix is ​​an m-row, n-column matrix, where m is the number of network data records in the dataset, and n is the number of metrics for each network data record; the threshold matrix T = [t1 t2 … t n ] is a 1xn matrix, and the weight matrix W T =[ω1 ω2 … ω n Let be a 1xn matrix, and construct the final threshold T of the hyperparameters. final .

[0053] Since there is a strong correlation between certain indicators in the network data records, it is necessary to use principal component analysis to "separate" these strongly correlated indicators, that is, to perform dimensionality reduction on the data and divide it into thresholds corresponding to each indicator.

[0054] Step S3 specifically includes the following steps:

[0055] S310, threshold comparison;

[0056] S320, weighted scoring and comparison.

[0057] Step S310 specifically includes:

[0058] Extend the threshold matrix T to m rows and n columns, denoted as T', and subtract it from the metadata matrix X. Then, normalize it into a 0-1 matrix A according to the positive or negative sign of the data.

[0059] Step S320 specifically includes:

[0060] Connect the 0-1 matrix A and the weight matrix W T Perform vector multiplication to obtain the final scoring matrix B with m rows and 1 column. Then, multiply the parameters of each row of matrix B with the final hyperparameter threshold T. final For comparison, greater than Tfinal Records with poor quality are considered to be less than T. final Records that are not of good quality are considered to be poor quality records.

[0061] Step S4 specifically includes:

[0062] Add a column after the nth column of the metadata matrix X, denoted as matrix X'. After each row of parameter comparison in step S320, add 1 after the row corresponding to the poor quality record and add 0 after the row corresponding to the non-poor quality record. Add 0 after the nth column of the threshold matrix T', which becomes an m-row n+1-column matrix T'". Take the difference between matrix X' and matrix T'' to obtain the difference matrix D, which serves as the interval between the first and second datasets. After processing the data in this way, the data that was originally distributed in the positive quadrant of the n-dimensional coordinate axis can be evenly distributed in each quadrant, making the classification algorithm perform better.

[0063] The difference matrix D is input into the classifier for training, and the cross-entropy function is used as the loss function. The threshold is corrected in the neural network through gradient descent and backpropagation. The correction formula is as follows:

[0064]

[0065] Among them, T n Let η be the threshold and η be the learning rate. Let be the Jacobian matrix corresponding to the threshold.

[0066] The threshold obtained from training is input into the generator to further classify the data until the generator and classifier finally reach a convergence state. Here, the convergence state means that the threshold matrix, which serves as a data bridge between the generator and the classifier, will reach a stable state. The stable state described here means that the Euclidean distance between two consecutive data points reaches a certain small number, which can be considered as convergence. At this point, the threshold and weights can be used as model parameters to generate a data analysis model.

[0067] The formula for calculating Euclidean distance is:

[0068]

[0069] The threshold vector for the previous step is X = (x1, x2, ..., x...). n The threshold vector for the next step is Y = (y1, y2, ..., y). n When the Euclidean distance approaches 0, the training is considered to be stable and convergent.

[0070] Example 2

[0071] Network data is collected from the network management platform. Assuming 10,000 data points are collected, each data point consists of seven indicators: Wi-Fi transmit power, LAN port speed, CPU utilization, number of ONUs going online and offline repeatedly, number of connected terminals, Wi-Fi interference ratio, and terminal Wi-Fi receive power.

[0072] First, the data collected by the network administrator is read into memory using the NumPy library. To prevent any missing data, we use Lagrange interpolation to fill in the missing data and form a complete dataset.

[0073] Subsequently, in order to explore the strong correlation of certain indicators in a certain scenario, such as the strong correlation between the number of connected terminals and CPU utilization, we used principal component analysis to retain only one strongly correlated data point. Finally, we obtained the final threshold matrix by scaling the relevant thresholds proportionally in the generated threshold matrix.

[0074] After principal component analysis, the simplified dataset is stored in a matrix to obtain the metadata matrix X, which is a 10,000-row, 7-column matrix. The seven thresholds form a 1-row, 7-column threshold matrix, the random weight matrix W is a 1-row, 7-column matrix, and the final threshold T_final is also obtained.

[0075] We score each row of data in the X matrix according to the pre-set weight matrix and threshold matrix, and compare it with the final threshold T_final. Data greater than the final threshold is marked as 1, and data less than the threshold is marked as 0. At this point, the dataset is split into two parts.

[0076] The segmented dataset is fed into a classifier, which is a fully connected neural network consisting of an input layer, an intermediate hidden layer, and an output layer. The classifier calculates the margin between the segmented data, resulting in a new threshold matrix. Due to the different weights, this threshold matrix differs from the generator's threshold matrix. This newly generated matrix is ​​then fed into the generator, and the process is repeated until the Euclidean distance between two consecutive threshold matrices is less than a very small value. Training is then complete, and we obtain the weight matrix and the threshold matrix. The formula for calculating the Euclidean distance is:

[0077]

[0078] The threshold vector for the previous step is X = (x1, x2, ..., x...). n The threshold vector for the next step is Y = (y1, y2, ..., y). n When the Euclidean distance approaches 0, the training is considered to be stable and convergent.

[0079] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for automatically identifying network quality status, characterized in that: Includes the following steps, S1. Collect network data, and use Lagrange interpolation to fill in the missing data based on nearest neighbors to obtain a dataset with the same distribution as the original data. S2, Principal component analysis is used to reduce the dimensionality of the dataset to obtain a simplified dataset without losing the features of the original dataset, and the weight matrix, threshold matrix and final threshold are constructed. Step S2 specifically includes: Principal component analysis was used to reduce the dimensionality of the dataset, and the data was stored in a matrix format to obtain the metadata matrix. , ； The matrix is ​​an m-row, n-column matrix, where m is the number of network data records in the dataset, and n is the number of metrics for each network data record; the threshold matrix. , is a 1xn matrix, and the weight matrix is... Given a 1xn matrix, construct the final threshold for the hyperparameters. ; S3, based on the weight matrix and threshold matrix, builds a generator to score and compare the simplified dataset, dividing it into poor quality data and non-poor quality data; Step S3 specifically includes the following steps: S310, threshold comparison; Step S310 specifically includes: threshold matrix Extended to m rows and n columns, denoted as , and metadata matrix The difference is calculated, and the result is normalized to a 0-1 matrix based on the sign of the data. ; S320, weighted scoring and comparison; Step S320 specifically includes: 0-1 matrix With weight matrix Perform vector multiplication to obtain the final scoring matrix with m rows and 1 columns. , matrix Each row of parameters and the final threshold of the hyperparameters For comparison, greater than Records smaller than 0 are considered poor quality records. Records that are not of poor quality are considered as non-poor quality records; S4. A classifier is built based on a fully connected neural network. The boundary between poor quality data and non-poor quality data is calculated to obtain a new threshold matrix. The Euclidean distance with the previous threshold matrix is ​​calculated to determine whether the training has converged and stabilized. S5. When the training has not converged and stabilized, update the generator with the new threshold matrix obtained in step S4, and repeat steps S3-S4. When the training has converged and stabilized, output the weight matrix and threshold matrix, and output the corresponding classifier and generator.

2. The method for automatically identifying network quality status as described in claim 1, characterized in that: Step S1 specifically includes: The collected data is read into memory using the NumPy library, and missing data is filled in using the Lagrange interpolation method.

3. The method for automatically identifying network quality status as described in claim 1, characterized in that: Step S4 specifically includes: In the metadata matrix Add a column after the nth column, denoted as matrix After comparing each row of parameters in step S320, add 1 to the row corresponding to the poor quality record and add 0 to the row corresponding to the non-poor quality record; in the threshold matrix Add 0s to the end of the nth column to make it an m x n + 1 column matrix. , matrix With matrix By taking the difference, we obtain the difference matrix. The difference matrix serves as the interval between the first and second datasets. The input is fed into the classifier for training, using the cross-entropy function as the loss function. The threshold is adjusted in the neural network through gradient descent and backpropagation, with the following adjustment formula: ； in, For the threshold, For learning rate, Let be the Jacobian matrix corresponding to the threshold.

4. The method for automatically identifying network quality status as described in claim 3, characterized in that: Step S4 further includes: The formula for calculating Euclidean distance is: ； Wherein, the threshold vector of the previous step is The threshold vector for the next step is When the Euclidean distance approaches 0, the training is considered to be stable and convergent.

Images