Broadband poor quality evaluation method and system, electronic equipment and storage medium

Abstract

The invention discloses a broadband poor-quality evaluation method and system, electronic equipment and a storage medium, and the method comprises the steps: carrying out the data collection of a target broadband according to a preset evaluation factor, and determining a broadband parameter set; performing evaluation processing on the broadband parameter set according to a preset evaluation standard, determining an evaluation score set, performing weighted evaluation according to preset evaluation factors and evaluation scores, and determining a factor evaluation value set; performing calculation according to the factor evaluation value set and the trained first model, determining a poor quality score of the target broadband, and performing matching according to a preset evaluation standard and the poor quality score to determine a poor quality evaluation result of the target broadband; a multi-factor evaluation architecture is set to analyze the network condition of a target broadband, multi-level convergence analysis is adopted, the evaluation conditions of different evaluation factors are subjected to weighted analysis processing, the poor quality of the target broadband under the different evaluation factors is comprehensively analyzed, and the accuracy of broadband poor quality evaluation is improved; the embodiment of the invention can be widely applied to the technical field of broadband networks.

Description

Technical Field

[0001] The present invention relates to the technical field of broadband networks, and in particular, to a method, a system, an electronic device, and a storage medium for evaluating poor broadband quality. Background Art

[0002] With the development and popularization of Internet technology, the scale of the network live broadcast market has been increasing year by year, and the number of live broadcast users has been continuously growing, with higher requirements for the network quality of broadband. Generally, operators use broadband quality difference to evaluate the network quality of broadband, analyze and solve problems occurring in the use of broadband networks, and improve network service instructions. The existing technical solutions only collect the usage status data of the network for analysis and calculation to obtain the broadband quality difference for evaluating the network quality. The evaluation and analysis are too single and prone to redundancy, and the evaluation accuracy is low. Summary of the Invention

[0003] The main purpose of the embodiments of the present invention is to provide a method, a system, an electronic device, and a storage medium for evaluating poor broadband quality, which can improve the evaluation accuracy.

[0004] To achieve the above object, on the one hand, an embodiment of the present invention provides a method for evaluating poor broadband quality, the method including:

[0005] Collect data on a target broadband according to a preset evaluation factor to determine a broadband parameter set;

[0006] Process the broadband parameter set according to a preset evaluation standard to determine an evaluation score set; perform a weighted evaluation according to the preset evaluation factor and the evaluation score set to determine a factor evaluation value set;

[0007] Calculate according to the factor evaluation value set and a trained first model to determine a quality difference score;

[0008] Match according to a preset judgment standard and the quality difference score to determine the quality difference evaluation result of the target broadband.

[0009] In some embodiments, the process of processing the broadband parameter set according to a preset evaluation standard to determine an evaluation score set specifically includes:

[0010] Analyze the broadband parameters in the broadband parameter set to determine the factor type of the preset evaluation factor corresponding to the broadband parameters;

[0011] If the factor type is the first type, match the broadband parameters corresponding to the first type with a preset evaluation standard to determine a first score set; wherein, the first type includes any one of a user intranet network quality factor, a user optical cat terminal quality factor, a user-side network status factor, or a user evaluation factor;

[0012] If the factor type is the second type, determine a first formula according to the preset evaluation criteria; perform calculations according to the broadband parameter set corresponding to the second type and the first formula to obtain a second score set; wherein, the second type includes an end-to-end network quality factor;

[0013] Determine the evaluation score set according to the first score set and the second score set.

[0014] In some embodiments, the weighted evaluation according to the preset evaluation factor and the evaluation score set to determine the factor evaluation value set specifically includes:

[0015] Divide the evaluation score set according to the preset evaluation factor to determine a plurality of score subsets; analyze the plurality of score subsets to determine the factor type;

[0016] If the factor type is the first type, perform weighted calculation according to the preset weight set and the score subset corresponding to the first type to obtain a first evaluation value set; wherein, the first type includes any one of a user intranet network quality factor, a user optical cat terminal quality factor, a user-side network status factor, or a user evaluation factor;

[0017] If the factor type is the second type, perform calculations according to the trained second model and the score subset corresponding to the second type to obtain a second evaluation value set; wherein, the second type includes an end-to-end network quality factor;

[0018] Determine the factor evaluation value set according to the first evaluation value set and the second evaluation value set.

[0019] In some embodiments, the performing calculations according to the trained second model and the corresponding score subset to obtain a second evaluation value set specifically includes:

[0020] Input the score subset into the trained second model for processing to obtain a plurality of first classification results; analyze the plurality of first classification results respectively to determine a plurality of groups of first model parameters;

[0021] Perform calculations on the plurality of groups of first model parameters according to a preset formula to obtain a target weight set; perform weighted calculation according to the target weight set and the score subset to obtain the second evaluation value set.

[0022] In some embodiments, the second model is trained in the following manner:

[0023] Divide the collected sample data according to a preset ratio to obtain a first data set and a second data set, and perform bucketing processing on the first data set to obtain a histogram data set;

[0024] Perform calculations based on the histogram dataset to obtain a first gradient value; construct a decision tree model based on the first gradient value and the histogram dataset;

[0025] Train the decision tree model according to the second dataset and determine the model performance value of the trained decision tree model; compare the model performance value with a preset performance threshold;

[0026] If the model performance value is less than the preset performance threshold, adjust the parameters of the decision tree model, and train the decision tree model with adjusted parameters according to the second dataset until the model performance value is greater than or equal to the preset performance threshold;

[0027] If the model performance value is greater than or equal to the preset performance threshold, use the decision tree model as the second model.

[0028] In some embodiments, the constructing a decision tree model based on the first gradient value and the histogram dataset specifically includes:

[0029] Perform calculations based on the first gradient value and the histogram dataset to determine classification parameters; classify the histogram dataset according to the classification parameters to obtain a first subset and a second subset, and use the first subset and the second subset as the decision tree structure; wherein the classification parameters include the best split feature and the classification threshold;

[0030] Count the number of leaves in the decision tree structure and compare the number of leaves with a preset leaf threshold;

[0031] If the number of leaves is less than the preset leaf threshold, perform calculations on the first subset to obtain a first sub-gradient value; perform calculations on the second subset to obtain a second sub-gradient value; determine a target subset according to the first sub-gradient value and the second sub-gradient value;

[0032] Update the parameters of the histogram dataset according to the target subset, update the first gradient value according to the target subset, and return to execute the calculation based on the first gradient value and the histogram dataset to determine the classification parameters until the number of leaves is greater than or equal to the preset leaf threshold;

[0033] If the number of leaves is greater than or equal to the preset leaf threshold, use the decision tree structure as the decision tree model.

[0034] In some embodiments, the determining a quality difference score by performing calculations based on the factor evaluation value set and the trained first model specifically includes:

[0035] Input the factor evaluation value sets into the trained first model for processing respectively to obtain several second classification results;

[0036] Analyze several second classification results respectively to determine several groups of second model parameters; normalize several groups of second model parameters to determine a factor weight set;

[0037] Calculate according to the factor weight set and the factor evaluation value sets to obtain the quality difference score.

[0038] To achieve the above object, on the other hand, an embodiment of the present application proposes a broadband quality difference evaluation system, and the system includes:

[0039] A first module, configured to collect data of a target broadband according to a preset evaluation factor to determine a broadband parameter set;

[0040] A second module, configured to process the broadband parameter set according to a preset evaluation criterion to determine an evaluation score set; perform weighted evaluation according to the preset evaluation factor and the evaluation score set to determine a factor evaluation value set;

[0041] A third module, configured to calculate according to the factor evaluation value sets and the trained first model to determine a quality difference score; match according to a preset judgment criterion and the quality difference score to determine a quality difference evaluation result of the target broadband.

[0042] To achieve the above object, on the other hand, an embodiment of the present application proposes an electronic device, and the electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the foregoing method is implemented.

[0043] To achieve the above object, on the other hand, an embodiment of the present application proposes a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the foregoing method is implemented.

[0044] Implementing the embodiments of the present invention includes the following beneficial effects: This embodiment provides a broadband quality difference evaluation method, system, electronic device, and storage medium. This solution collects data on the target broadband according to preset evaluation factors to determine a broadband parameter set; evaluates and processes the broadband parameter set according to preset evaluation criteria to determine an evaluation score set, and performs weighted evaluation according to the preset evaluation factors and evaluation scores to determine a factor evaluation value set; then calculates according to the factor evaluation value set and the trained first model to determine the quality difference score of the target broadband, and matches according to the preset judgment criteria and the quality difference score to determine the quality difference evaluation result of the target broadband. By setting up a multi-factor evaluation architecture to analyze the network situation of the target broadband, and at the same time adopting multi-level convergence analysis, the evaluation situations of different evaluation factors are weighted and analyzed, and the quality differences of the target broadband under different evaluation factors are comprehensively analyzed to improve the accuracy of broadband quality difference evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 FIG. is a schematic flowchart of the steps of a broadband quality difference evaluation method provided by an embodiment of the present invention;

[0046] Figure 2 FIG. is a schematic flowchart of the steps of determining an evaluation score set in a broadband quality difference evaluation method provided by an embodiment of the present invention;

[0047] Figure 3 FIG. is a schematic flowchart of the steps of determining a factor evaluation value in a broadband quality difference evaluation method provided by an embodiment of the present invention;

[0048] Figure 4 FIG. is a schematic flowchart of the steps of determining a second evaluation value set in a broadband quality difference evaluation method provided by an embodiment of the present invention;

[0049] Figure 5 FIG. is a schematic flowchart of the steps of determining a trained second model in a broadband quality difference evaluation method provided by an embodiment of the present invention;

[0050] Figure 6 FIG. is a schematic flowchart of the steps of constructing a decision tree model in a broadband quality difference evaluation method provided by an embodiment of the present invention;

[0051] Figure 7 FIG. is a schematic flowchart of the steps of determining a quality difference score in a broadband quality difference evaluation method provided by an embodiment of the present invention;

[0052] Figure 8 FIG. is a schematic flowchart of the steps of a specific embodiment provided by an embodiment of the present invention;

[0053] Figure 9 FIG. is a schematic diagram of multi-factor data collection and analysis in a specific embodiment provided by an embodiment of the present invention;

[0054] Figure 10 It is a structural block diagram of a broadband quality degradation evaluation system provided by an embodiment of the present invention;

[0055] Figure 11 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. Specific Embodiments

[0056] The following further elaborates on the present invention in detail in conjunction with the accompanying drawings and specific embodiments. For the step numbers in the following embodiments, they are only set for the convenience of explanation and illustration, and no limitation is imposed on the order between steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0057] In the following description, "some embodiments" are involved, which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and they can be combined with each other without conflict.

[0058] In the following description, the terms "first / second / third" involved are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when permitted, so that the embodiments of the present invention described here can be implemented in an order other than that illustrated or described here.

[0059] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present invention have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used in the embodiments of the present invention are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0060] Before further elaborating on the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention are described, and the nouns and terms involved in the embodiments of the present invention are applicable to the following explanations.

[0061] DPI, Deep Packet Inspection, is a traffic detection and control technology based on the application layer. When IP data packets, TCP or UDP data streams pass through a bandwidth management system based on DPI technology, the system reorganizes the application layer information in the OSI seven-layer protocol by deeply reading the content of the IP packet payload, so as to obtain the content of the entire application program, and then performs shaping operations on the traffic according to the management policies defined by the system.

[0062] XDR call detail records are data stream call detail records generated by DPI technology. After processing the full volume of Internet data, XDR call detail records are session-level detailed records of the signaling process and service transmission process, containing all the user's Internet access information. Therefore, the call detail records contain very rich value for data analysis and mining.

[0063] OLT, the Optical Line Terminal, is a terminal device used to connect to the optical fiber trunk. The functions of the OLT include sending Ethernet data to the ONU in a broadcast manner; initiating and controlling the ranging process and recording the ranging information; generally deployed in the network access layer as the upstream aggregation device point for the user ONU.

[0064] ONU, the ONU device refers to an optical network device connected to the branch optical fiber of the ODN; taking the optical node as the reference, it is divided into an active optical network unit and a passive optical network unit. The ONU device is usually interpreted as an optical node equipped with devices including an optical receiver, an upstream optical transmitter, and a network monitor for multiple bridging amplifiers, and is generally deployed on the user side as a user terminal.

[0065] In the related technologies, the network quality of the broadband network is usually evaluated by the quality difference. The operator adjusts the broadband according to the quality difference of the broadband network to optimize the network quality; the mainstream network analysis method is mainly to obtain the usage status data of the network from network devices and analyze and calculate the obtained data to evaluate the network quality; however, the mainstream analysis scheme is too single and prone to redundancy, unable to locate the cause of the quality difference, and unable to comprehensively and comprehensively understand the pain points and experiences of users.

[0066] In view of this, the embodiments of the present invention provide a broadband quality difference evaluation method, system, electronic device and storage medium. This solution constructs multiple evaluation factors according to the pain point requirements of users using the broadband, obtains the parameter data of the broadband network based on the constructed evaluation factors, evaluates the broadband network parameters in multiple dimensions according to the parameter evaluation criteria corresponding to each evaluation factor, performs weighted operations on different parameters in each dimension according to the importance of each parameter in each evaluation factor to obtain the evaluation values of the broadband network in different dimensions, and then performs weighted calculations on the evaluation values of different dimensions according to the influence degree of different evaluation dimensions on the broadband network quality to obtain the quality difference score of the broadband network, and further determine the quality difference result of the broadband network; by constructing a multi-factor multi-dimensional evaluation system, evaluating the broadband network parameters in different dimensions, and performing weighting operations according to different dimensions and different parameters, comprehensively and accurately determine the network quality of the broadband network; at the same time, the influencing factors affecting the network quality can also be located according to the evaluation results, improving the efficiency of the operator to optimize the network quality.

[0067] Figure 1It is an optional flowchart of a broadband quality difference evaluation method provided by an embodiment of the present invention. Figure 1 The method in

[0068] Step S101: Collect data on the target broadband according to a preset evaluation factor, and determine a broadband parameter set.

[0069] Step S102: Process the broadband parameter set according to a preset evaluation criterion to determine an evaluation score set; perform weighted evaluation according to the preset evaluation factor and the evaluation score set to determine a factor evaluation value set.

[0070] Step S103: Calculate according to the factor evaluation value set and a trained first model to determine a quality difference score.

[0071] Step S104: Match according to a preset judgment criterion and the quality difference score to determine the quality difference evaluation result of the target broadband.

[0072] In steps S101 to S104 illustrated in the embodiments of the present application, the live broadcast service of the user is affected by the quality of the connected broadband network. According to the factors affecting the broadband network, multiple evaluation factors are set. In this embodiment, the following evaluation factors are set: the user's internal network quality, end-to-end network quality, the user's optical cat terminal situation, the overall situation of the user's network at the user end, user value, and sensitivity; according to the set evaluation factors, relevant parameters of the user's current network device and network status are obtained; based on the set evaluation factors, corresponding evaluation criteria are set, and the obtained network device and network status parameters are evaluated from different dimensions, corresponding scores are given, the evaluation scores of different parameters are aggregated, and different weights are given to the aggregated evaluation scores according to the influence degree of different evaluation factors on the user's network, and the quality difference result corresponding to the user's current network device and network status is comprehensively evaluated.

[0073] In step S101 of some embodiments, several different evaluation parameters are also set for different evaluation factors. Exemplarily, the user's internal network quality includes the WIFI interference situation, the WIFI weak coverage feedback situation, the CPU or memory occupancy situation, the WIFI access to the LAN port situation, and the LAN port negotiation rate situation; these evaluation parameters can be obtained by deploying a soft probe on the optical cat side. The evaluation parameters in other evaluation factors can also be obtained by corresponding methods. For example, the evaluation parameters in the end-to-end network quality can be detected based on the OLT built-in AI intelligent computing board, and the user's various service indicators are extracted and sampled in real time through the probe in the single board, not limited to this.

[0074] Please refer to Figure 2 , in some embodiments, step S102 may include but is not limited to steps S201 to S204:

[0075] Step S201: Analyze the broadband parameters in the broadband parameter set to determine the factor type of the preset evaluation factor corresponding to the broadband parameter.

[0076] Step S202: If the factor type is the first type, match the broadband parameter corresponding to the first type with the preset evaluation criteria to determine the first score set; wherein, the first type includes any one of the user intranet network quality factor, the user optical cat terminal quality factor, the user-side network status factor, or the user evaluation factor.

[0077] Step S203: If the factor type is the second type, determine the first formula according to the preset evaluation criteria; calculate according to the broadband parameter set corresponding to the second type and the first formula to obtain the second score set; wherein, the second type includes the end-to-end network quality factor.

[0078] Step S204: Determine the evaluation score set according to the first score set and the second score set.

[0079] In step S201 of some embodiments, after collecting the relevant evaluation parameter data of the network device and network quality according to the set evaluation factor, it is necessary to evaluate the collected parameter data and assign corresponding evaluation scores; in this embodiment, different evaluation factor settings correspond to evaluation criteria, analyze the collected evaluation parameters to determine the factor type of the evaluation factor corresponding to the evaluation parameter, and evaluate the evaluation parameter according to the analyzed factor type using the corresponding evaluation criteria.

[0080] In step S202 of some embodiments, if the evaluation factor corresponding to the current evaluation parameter is the first type, obtain the evaluation criteria corresponding to the evaluation factor, match the current evaluation parameter data with the evaluation criteria to determine the standard range that the evaluation parameter data conforms to, and assign the corresponding score as the evaluation score; in this embodiment, there are corresponding evaluation criteria for the user intranet network quality, the user optical cat terminal situation, the overall user-side network situation, and the user value and sensitivity. For example, for the WIFI interference situation data in the user intranet network quality, the user intranet network quality is scored according to the number of other SSID signal strengths greater than -85dBm in the same channel or the number of signals in 1-2 adjacent 2.4G channels. The more interference signals in the channel, the lower the score.

[0081] In step S203 of some embodiments, if the evaluation factor corresponding to the current evaluation parameter is of the second type, obtain the corresponding evaluation formula according to the corresponding evaluation factor; calculate the evaluation score corresponding to the current evaluation parameter according to the current evaluation parameter data and the evaluation formula; in this embodiment, calculate the evaluation scores corresponding to the evaluation parameters in the end-to-end network quality through the min-max normalization formula; collect a large amount of data of each evaluation parameter in the end-to-end network quality through probes, clean the collected data, calculate the mean value of the collected data to fill in the missing values, and perform standard normalization processing, and use the standard normalization result as the evaluation score of the current evaluation parameter data. Calculate the evaluation scores corresponding to different evaluation parameters according to the following formula,

[0082]

[0083] where x * is the normalized parameter value, max is the maximum value among the collected parameter values, and min is the minimum value among the collected parameter values.

[0084] In step S204 of some embodiments, according to the evaluation factor types corresponding to the evaluation parameters, use different evaluation methods to evaluate and score the evaluation parameters to obtain the corresponding evaluation scores; summarize the evaluation scores of the evaluation parameters to obtain the total evaluation score set for subsequent convergence analysis.

[0085] Please refer to Figure 3 , in some embodiments, step S102 may include but is not limited to steps S301 to S304:

[0086] Step S301, divide the evaluation score set according to the preset evaluation factor to determine several score subsets; analyze the several score subsets to determine the factor type;

[0087] Step S302, if the factor type is the first type, perform weighted calculation according to the preset weight set and the score subset corresponding to the first type to obtain the first evaluation value set; where the first type includes any one of the user intranet network quality factor, the user optical cat terminal quality factor, the user terminal network status factor, or the user evaluation factor;

[0088] Step S303, if the factor type is the second type, perform calculation according to the trained second model and the score subset corresponding to the second type to obtain the second evaluation value set; where the second type includes the end-to-end network quality factor;

[0089] Step S304, determine the factor evaluation value set according to the first evaluation value set and the second evaluation value set.

[0090] In step S301 of some embodiments, after evaluating and scoring each evaluation parameter in each evaluation factor to obtain corresponding evaluation scores, the evaluation parameter scores are aggregated and analyzed to determine the comprehensive evaluation results of different evaluation factors, and to analyze the performance of the broadband network quality in different dimensions; first, the obtained evaluation scores are divided according to the evaluation factors corresponding to each evaluation parameter, that is, the evaluation scores are classified according to the evaluation factors; then, according to the types of different evaluation factors, corresponding methods are adopted to analyze the classified evaluation scores.

[0091] In step S302 of some embodiments, if the type of the evaluation factor is the third type, a corresponding weight set is obtained according to the factor type of the evaluation factor, and the weight set includes the weights corresponding to each evaluation parameter in the evaluation factor. Based on the weight set, a weighted operation is performed on the evaluation scores of each evaluation parameter in the evaluation factor to obtain the comprehensive evaluation value of the evaluation factor; in this embodiment, when the evaluation factors are the user's internal network quality, the user's optical cat terminal situation, the overall situation of the user-side network, and the user value and sensitivity, a comprehensive evaluation is performed according to the set corresponding weights; for example, the evaluation scores of each evaluation parameter in the user's internal network quality are A, B, C, D, and E respectively, and the weights corresponding to each evaluation parameter are determined to be 15%, 30%, 10%, 30%, and 15% according to the evaluation factor type. A weighted operation is performed according to the evaluation scores and the corresponding weights to obtain the comprehensive evaluation value of the user's internal network quality. The operation formula is S 1 = 15%A + 30%B + 10%C + 30%D + 15%E, where S 1 is the comprehensive evaluation value of the user's internal network quality.

[0092] In step S303 of some embodiments, if the type of the evaluation factor is the fourth type, weights are assigned to each evaluation parameter of the evaluation factor by a trained model, and a weighted calculation is performed according to the assigned weights and the evaluation scores of each evaluation parameter to obtain the comprehensive evaluation value of the evaluation factor; in this embodiment, a decision tree model is used to assign weights to the evaluation parameters, and the decision tree model is trained using the LightGBM algorithm.

[0093] In step S304 of some embodiments, the comprehensive evaluation values obtained by comprehensively analyzing different types of evaluation factors are summarized to obtain a set of comprehensive evaluation values of the total evaluation factors, which is used for subsequent calculation of the quality difference score of the broadband network to determine the quality difference situation of the broadband.

[0094] Please refer to Figure 4 , in some embodiments, step S303 may include but is not limited to steps S401 to S402:

[0095] Step S401: Input the fraction subset into the trained second model for processing to obtain several first classification results; analyze the several first classification results respectively to determine several groups of first model parameters;

[0096] Step S402: Calculate several groups of first model parameters respectively according to a preset formula to obtain a target weight set; perform weighted calculation based on the target weight set and the fraction subset to obtain a second evaluation value set.

[0097] In step S401 of some embodiments, determine an evaluation score set that needs to be weighted by a decision model according to the factor type of the evaluation factor, classify the evaluation score set according to the evaluation parameters in the evaluation factor to obtain an evaluation score subset corresponding to each evaluation parameter, where the evaluation score subset includes the evaluation scores corresponding to different users or different time periods in the evaluation parameter, convert the divided evaluation score subsets corresponding to each evaluation parameter into histogram data, and then input them into the trained decision tree model respectively. The decision tree model processes the input evaluation score subset and outputs corresponding classification results; analyze the classification results to determine the corresponding model parameters, where the model parameters include the data volume and histogram data gradient of the left and right subtrees of the decision tree when outputting the classification results, as well as the histogram data gradient and data volume of the parent node of the decision tree;

[0098] In step S402 of some embodiments, calculate the contribution of the corresponding evaluation parameter to the classification result according to the analyzed model parameters. In this embodiment, calculate the gain of the parameter as the weight of the parameter. The calculation formula of the parameter gain is as follows,

[0099]

[0100] where Gain is the gain value, S L is the sum of the histogram data gradients on the left side of the decision tree, n L is the data volume on the left side of the decision tree, S R is the sum of the histogram data gradients on the right side of the decision tree, n R is the data volume on the right side of the decision tree, S P is the sum of the histogram data gradients of the parent node of the decision tree, n P is the data volume of the parent node of the decision tree; calculate the weights of each evaluation parameter according to the gain formula, and perform weighted operation based on the assigned weights and the corresponding evaluation scores to obtain the comprehensive evaluation value of the evaluation factor.

[0101] Please refer to Figure 5 In some embodiments, the trained second model in step S401 can be obtained according to the following steps S501 to S505:

[0102] Step S501: Divide the collected sample data according to a preset ratio to obtain a first data set and a second data set; perform bucketing on the first data set to obtain a histogram data set.

[0103] Step S502: Calculate a first gradient value based on the histogram data set; construct a decision tree model according to the first gradient value and the histogram data set.

[0104] Step S503: Train the decision tree model according to the second data set, and determine the model performance value of the trained decision tree model; compare the model performance value with a preset performance threshold.

[0105] Step S504: If the model performance value is less than the preset performance threshold, adjust the parameters of the decision tree model, and train the decision tree model with adjusted parameters according to the second data set until the model performance value is greater than or equal to the preset performance threshold.

[0106] Step S505: If the model performance value is greater than or equal to the preset performance threshold, use the decision tree model as the second model.

[0107] In step S501 of some embodiments, weight distribution is performed using a decision tree model, and the construction and training of the decision tree model are carried out using the LightGBM algorithm; first, data collection is performed according to each evaluation parameter in the evaluation factors, and after data cleaning of the collected data, the data is divided into a training set, a test set, and a validation set according to a preset ratio; bucketing is performed on the training set to convert the continuous feature data in the training set into histogram data of discrete values, and different features and corresponding statistical values are counted in the histogram data.

[0108] In step S502 of some embodiments, calculate the mean value of the training set as the initial gradient value of the model, and calculate the gain of each feature according to the data in the histogram data, so as to determine the best splitting feature and splitting threshold, classify the histogram data according to the best splitting feature and splitting threshold, and repeat the above steps for the classified subsets until the leaf number limit or all leaves cannot be further divided, and the decision tree model is constructed.

[0109] In step S503 of some embodiments, after the decision tree model is constructed, the cross-validation method is used to evaluate the model performance of the decision tree model to determine that the decision tree model has good generalization ability; input the test set and the validation set into the decision tree model, and calculate the performance value of the decision tree model; in this embodiment, the root mean square error of the decision tree model is used as the performance value, and the calculation formula of the root mean square error is: where RMSE is the root mean square error value, n is the number of data sets, is the predicted data, y iis real data; compare the calculated performance value with a preset performance threshold; determine whether the currently constructed decision tree model can be deployed and used.

[0110] In step S504 of some embodiments, if the calculated performance value is less than the preset performance threshold, adjust the hyperparameters of the decision tree model to optimize the model performance, such as learning rate, number of trees, maximum depth, minimum sample weight and leaf node weight, sample subsampling ratio, feature subsampling ratio, etc.; after adjusting the model parameters, re-enter the training set data into the model for training again, and calculate the model performance value again after training, and compare the model performance value with the preset performance threshold; until the performance value of the model is greater than or equal to the preset performance threshold, deploy and apply the current decision tree model, and assign weights to the evaluation parameters in the evaluation factor.

[0111] In step S505 of some embodiments, if the performance value of the model is greater than or equal to the preset performance threshold, deploy and apply the current decision tree model, and assign weights to the evaluation parameters in the evaluation factor; after deploying and applying the decision tree model, monitor the performance of the decision tree model in real time. When the performance of the model drops significantly, collect real-time data to optimize the decision tree model and improve the generalization ability and prediction accuracy of the decision tree model.

[0112] Please refer to Figure 6 , in some embodiments, step S502 may include but is not limited to steps S601 to S605:

[0113] Step S601, calculate according to the first gradient value and the histogram data set to determine the classification parameter; classify the histogram data set according to the classification parameter to obtain a first subset and a second subset, and use the first subset and the second subset as the decision tree structure; wherein, the classification parameter includes the best split feature and the classification threshold;

[0114] Step S602, count the number of leaves in the decision tree structure, and compare the number of leaves with a preset leaf threshold;

[0115] Step S603, if the number of leaves is less than the preset leaf threshold, calculate the first sub-gradient value for the first subset; calculate the second sub-gradient value for the second subset; determine the target subset according to the first sub-gradient value and the second sub-gradient value;

[0116] Step S604, update the parameters of the histogram data set according to the target subset, update the first gradient value according to the target subset, and return to execute the calculation according to the first gradient value and the histogram data set to determine the classification parameter until the number of leaves is greater than or equal to the preset leaf threshold;

[0117] Step S605, if the number of leaves is greater than or equal to a preset leaf threshold, use the decision tree structure as the decision tree model.

[0118] In step S601 of some embodiments, in the construction of the decision tree model, convert the training set into a histogram data set, and calculate the mean value of the target parameter in the training set as the initial gradient value of the decision tree model; calculate the gain of each feature according to the histogram data set and the initial gradient value, and determine the best splitting feature and splitting threshold according to the principle of maximizing the gain; based on the determined best splitting feature and splitting feature, divide the histogram data set into two subsets, and construct the nodes of the decision tree model according to the two divided subsets.

[0119] In step S602 of some embodiments, count the number of leaves in the constructed decision tree model, compare the statistical result with the preset leaf number threshold, and determine whether to stop the construction of the decision tree model. Among them, the leaf number threshold can be determined according to the prediction accuracy and prediction speed of the decision tree model; exemplarily, the depth of the decision tree model can also be counted, and it can be determined whether to stop the construction of the decision tree model according to the depth of the decision tree model; in this embodiment, it can also be determined whether all leaf nodes in the current decision tree model can be split, so as to determine whether to stop constructing the decision tree model.

[0120] In step S603 of some embodiments, if the number of leaves in the current decision tree model is less than the preset leaf number threshold, calculate the gradient values of the two divided subsets respectively, and count the data amounts in the two subsets respectively; select the subset with a higher gradient value as the target subset according to the gradient values of the two subsets, and split the target subset to construct the nodes of the decision tree model.

[0121] In step S604 of some embodiments, perform bucketing processing on the determined target subset to convert it into new histogram data, update the histogram data of the previous step according to the new histogram data, and perform the next step of splitting according to the new histogram data; update the first gradient value of the previous step according to the gradient value of the determined target subset for the next step of splitting; according to the updated histogram data and the first gradient value, repeat steps S601 to S602, split the new histogram data, and construct the nodes of the decision tree model until the number of leaves in the decision tree model is greater than or equal to the preset leaf number threshold, stop the construction of the decision tree model, and enter the next step of model performance verification.

[0122] In step S605 of some embodiments, if the number of leaves in the decision tree model is greater than or equal to the preset leaf number threshold, stop the construction of the decision tree model; perform performance verification on the current decision tree model according to the test set and the validation set, and determine whether the current decision tree model can be deployed and applied.

[0123] Please refer toFigure 7 , in some embodiments, step S103 may include but is not limited to steps S701 to S703:

[0124] Step S701, input the factor evaluation value set into the trained first model for processing respectively to obtain a number of second classification results;

[0125] Step S702, analyze the number of second classification results respectively to determine a number of groups of second model parameters; normalize the number of groups of second model parameters to determine the factor weight set;

[0126] Step S703, calculate according to the factor weight set and the factor evaluation value set to obtain the quality difference score.

[0127] In step S701 of some embodiments, after obtaining the comprehensive evaluation values of each evaluation factor, assign weights to each evaluation factor, and perform weighted calculation according to the weights and the corresponding comprehensive evaluation values to obtain the quality difference score of the broadband; in this embodiment, assigning weights to the evaluation factors is similar to assigning weights to each evaluation parameter in the end-to-end network quality, and use the trained decision tree model to assign weights to each evaluation factor; the construction and training process of this decision tree model is similar to steps S601 to S605, and collect the comprehensive evaluation values of each evaluation factor of different users for the construction and training of the decision tree model. Input the comprehensive evaluation values of each evaluation factor into the trained decision tree model respectively, and output the corresponding classification results.

[0128] In step S702 of some embodiments, analyze the classification results output by the trained decision tree model to determine the model parameters of the decision tree model corresponding to the classification results; calculate the gain value of the corresponding evaluation factor according to the model parameters, and perform normalization processing on the calculated gain value to obtain the weight value of the evaluation factor.

[0129] In step S703 of some embodiments, determine the quality difference score formula of the broadband according to the calculated weight value of the evaluation factor and the comprehensive evaluation value of each evaluation factor:

[0130] Y = a * W a + b * W b + c * W c + d * W d + e * W e

[0131] where Y is the quality difference score, a, b, c, d, and e are the comprehensive evaluation values of the evaluation factors, and W a 、W b 、W c 、W d and W eThe weight values of the evaluation factors; in this embodiment, through calculation by the decision tree model, the weights of the user intranet network quality, end-to-end network quality, user optical cat terminal situation, overall user-side network situation, user value, and sensitivity are 14.6%, 28.4%, 25.1%, 19.1%, and 12.8% in sequence.

[0132] Next, in combination with specific application examples, the solutions of the embodiments of the present invention will be introduced and described in detail:

[0133] Please refer to Figure 8 , obtain multiple evaluation factors for broadband quality degradation, and evaluate the broadband quality degradation; the evaluation factors include user internal network quality, end-to-end network quality, user optical cat terminal situation, overall user-side network situation, and user value and sensitivity; for each evaluation factor, multiple evaluation parameters are correspondingly set, as shown in the following table,

[0134] Table 1

[0135]

[0136] Continued Table 1

[0137]

[0138] As Figure 9 shown, the evaluation parameters in each evaluation factor are obtained by collecting data from the corresponding users or network devices for quality degradation analysis and evaluation; specifically, the parameters in the user internal network quality are obtained by deploying a soft probe on the optical cat side; the end-to-end network quality is detected based on the OLT built-in AI intelligent computing board, and the probe and DPI technology in the single board are used to extract and sample the user's various service indicators in real time, and after parsing, they are sent to the background through the XDR call record; the user optical cat terminal situation can be directly queried on the Pon professional network management and ANISS systems; the overall user-side network situation can be detected using the service system; and the user value and sensitivity can be obtained through user surveys and feedback.

[0139] For each evaluation parameter in each evaluation factor, a corresponding evaluation criterion is set, and relevant data on the broadband network quality is collected according to each evaluation parameter; the data collected is matched with the corresponding evaluation criteria to determine the range that the collected data conforms to, and a corresponding score is assigned as the evaluation score corresponding to the evaluation parameter; among them, the evaluation scores of the evaluation parameters in the end-to-end network quality are calculated according to the following formula; after obtaining the evaluation scores of each evaluation parameter, weight assignment is performed on each evaluation parameter to calculate the comprehensive evaluation value of each evaluation factor; among them, the weights of the evaluation parameters in the end-to-end network are classified for the evaluation parameters through the constructed decision tree network, and the gain values of different parameters are calculated as the weights of each evaluation parameter, while weights are correspondingly assigned to the user intranet network quality, user optical cat terminal situation, overall user-side network situation, and user value and sensitivity, and linear weighted operations are performed according to each evaluation parameter and the corresponding weights to obtain the comprehensive evaluation value of each evaluation factor; the quality difference of the broadband is judged according to the comprehensive evaluation value of each evaluation factor. First, weights are assigned to each evaluation factor according to the constructed decision tree network, and a quality difference score formula is determined according to the assigned weights and the comprehensive evaluation value of each evaluation factor, and the quality difference score of the broadband is calculated according to this quality difference score formula; the quality difference result of the broadband is determined by matching with the preset evaluation criteria and the calculated quality difference score. If the quality difference score is higher than 80 points, the broadband quality difference is excellent; if the quality difference score is between 60 and 80 points, the broadband quality difference is medium, and the operator can optimize network devices, adjust network parameters, etc. to optimize the broadband quality difference; if the quality difference score is lower than 60 points, the broadband quality difference is poor, and the operator locates the cause of the network failure according to the quality difference and performs maintenance in a timely manner to improve the network quality.

[0140] Implementing the embodiments of the present invention includes the following beneficial effects: The present embodiment provides a broadband quality difference evaluation method, system, electronic device, and storage medium. This solution collects data on the target broadband according to preset evaluation factors to determine a broadband parameter set; evaluates and processes the broadband parameter set according to preset evaluation criteria to determine an evaluation score set, and performs weighted evaluation according to the preset evaluation factors and evaluation scores to determine a factor evaluation value set; then calculates according to the factor evaluation value set and the trained first model to determine the quality difference score of the target broadband, and matches according to the preset evaluation criteria and the quality difference score to determine the quality difference evaluation result of the target broadband. By setting up a multi-factor evaluation architecture to analyze the network situation of the target broadband, and at the same time adopting multi-level convergence analysis, the evaluation situations of different evaluation factors are weighted and analyzed to comprehensively analyze the quality difference of the target broadband under different evaluation factors, improving the accuracy of broadband quality difference evaluation; it is also possible to locate the cause of the network failure according to the quality difference, and then generate corresponding solutions, improving the network maintenance efficiency of the operator and improving the network quality.

[0141] Such as Figure 10As shown in the figure, an embodiment of the present invention further provides a broadband quality degradation evaluation system, which can implement the above-mentioned broadband quality degradation evaluation method. The system includes:

[0142] A first module, configured to collect data of a target broadband according to a preset evaluation factor and determine a broadband parameter set;

[0143] A second module, configured to process the broadband parameter set and a preset evaluation criterion to determine an evaluation score set; perform weighted evaluation according to the preset evaluation factor and the evaluation score set to determine a factor evaluation value set;

[0144] A third module, configured to calculate according to the factor evaluation value set and a trained first model to determine a quality degradation score;

[0145] A fourth module, configured to match a preset judgment criterion with the quality degradation score to determine a quality degradation evaluation result of the target broadband.

[0146] It can be seen that the content in the above method embodiments is applicable to the system embodiments of the present invention. The functions specifically implemented by the system embodiments of the present invention are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0147] An embodiment of the present application further provides an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the above-mentioned broadband quality degradation evaluation method. The electronic device can be any intelligent terminal including a tablet computer, an in-vehicle computer, etc.

[0148] It can be understood that the content in the above method embodiments is applicable to the device embodiments of the present invention. The functions specifically implemented by the device embodiments of the present invention are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0149] Please refer to Figure 11 , Figure 11 which shows the hardware structure of an electronic device in another embodiment. The electronic device includes:

[0150] A processor 1101, which can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided by the embodiments of the present application;

[0151] The memory 1102 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 1102 can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1102 and are called by the processor 1101 to execute a broadband quality degradation assessment method according to an embodiment of the present application;

[0152] The input / output interface 1103 is used to implement information input and output;

[0153] The communication interface 1104 is used to implement communication interaction between this device and other devices. Communication can be achieved through a wired manner (such as USB, network cable, etc.) or through a wireless manner (such as a mobile network, WIFI, Bluetooth, etc.);

[0154] The bus 1105 transmits information between various components of the device (such as the processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104);

[0155] Among them, the processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104 achieve communication connections with each other inside the device through the bus 1105.

[0156] Among them, the memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. The memory can include a high-speed random access memory, and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory optionally includes a remote memory that is remotely set relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0157] In addition, an embodiment of the present application also discloses a computer program product or a computer program. The computer program product or the computer program is stored in a computer-readable storage medium. The processor of the computer device can read the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above method. Similarly, the content in the above method embodiments is applicable to this storage medium embodiment. The functions specifically implemented by this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0158] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, which when executed by a processor implements the above-mentioned broadband quality degradation assessment method.

[0159] It can be understood that the content in the above method embodiments is applicable to this storage medium embodiment. The functions specifically implemented by this storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0160] It can be understood that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and their appropriate combinations. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, communication media typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium.

[0161] The above is a specific description of the preferred embodiment of the present invention, but the present invention is not limited to the described embodiment. Those skilled in the art can make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A broadband quality assessment method, characterized in that: The method comprises: Collect data on the target broadband according to preset evaluation factors and determine a broadband parameter set; Processing the broadband parameter set according to a preset evaluation standard to determine an evaluation score set; performing a weighted evaluation according to the preset evaluation factor and the evaluation score set to determine a factor evaluation value set; Calculating based on the factor evaluation value set and the trained first model to determine a poor quality score; The quality difference score is matched according to a preset evaluation standard to determine the quality difference evaluation result of the target broadband.

2. The method according to claim 1, characterized in that: The processing of the broadband parameter set according to the preset evaluation criteria to determine the evaluation score set specifically includes: Analyzing the broadband parameters in the broadband parameter set to determine the factor type of the preset evaluation factor corresponding to the broadband parameters; If the factor type is the first type, matching the broadband parameter corresponding to the first type with the preset evaluation criteria to determine a first score set; wherein the first type includes any one of a user intranet network quality factor, a user optical modem terminal quality factor, a user-side network status factor, or a user evaluation factor; If the factor type is the second type, determining a first formula according to the preset evaluation criteria; calculating according to the broadband parameter set corresponding to the second type and the first formula to obtain a second score set; wherein the second type includes an end-to-end network quality factor; The set of evaluation scores is determined based on the first set of scores and the second set of scores.

3. The method according to claim 1, characterized in that The performing weighted evaluation according to the preset evaluation factors and the evaluation score set to determine the factor evaluation value set specifically includes: Dividing the evaluation score set according to the preset evaluation factors to determine a plurality of score subsets; analyzing the plurality of score subsets to determine factor types; If the factor type is the first type, a weighted calculation is performed according to a preset weight set and a score subset corresponding to the first type to obtain a first evaluation value set; wherein the first type includes any one of a user intranet network quality factor, a user optical modem terminal quality factor, a user terminal network status factor, or a user evaluation factor; If the factor type is the second type, calculating according to the trained second model and the score subset corresponding to the second type to obtain a second evaluation value set; wherein the second type includes an end-to-end network quality factor; The factor evaluation value set is determined according to the first evaluation value set and the second evaluation value set.

4. The method according to claim 3, characterized in that: The calculation is performed according to the trained second model and the corresponding score subset to obtain the second evaluation value set, specifically including: Inputting the score subset into the trained second model for processing to obtain a plurality of first classification results; analyzing the plurality of first classification results respectively to determine a plurality of groups of first model parameters; Several groups of the first model parameters are calculated respectively according to a preset formula to obtain a target weight set; and a weighted calculation is performed according to the target weight set and the score subset to obtain the second evaluation value set.

5. The method according to claim 4, characterized in that The second model is trained in the following way: Dividing the collected sample data according to a preset ratio to obtain a first data set and a second data set, and performing bucket processing on the first data set to obtain a histogram data set; Calculating according to the histogram data set to obtain a first gradient value; Building a decision tree model according to the first gradient value and the histogram data set; Training the decision tree model according to the second data set, and determining a model performance value of the trained decision tree model; comparing the model performance value with a preset performance threshold; If the model performance value is less than the preset performance threshold, adjusting the parameters of the decision tree model, and training the decision tree model after the parameter adjustment according to the second data set until the model performance value is greater than or equal to the preset performance threshold; If the model performance value is greater than or equal to the preset performance threshold, the decision tree model is used as the second model.

6. The method according to claim 5, characterized in that The step of constructing a decision tree model according to the first gradient value and the histogram data set specifically includes: Calculating according to the first gradient value and the histogram data set to determine a classification parameter; classifying the histogram data set according to the classification parameter to obtain a first subset and a second subset, and using the first subset and the second subset as a decision tree structure; wherein the classification parameter includes an optimal splitting feature and a classification threshold; Counting the number of leaves in the decision tree structure, and comparing the number of leaves with a preset leaf threshold; If the number of leaves is less than the preset leaf threshold, calculating the first subset to obtain a first sub-gradient value; calculating the second subset to obtain a second sub-gradient value; determining a target subset according to the first sub-gradient value and the second sub-gradient value; Performing parameter updates on the histogram data set according to the target subset, updating the first gradient value according to the target subset, and returning to perform the calculation according to the first gradient value and the histogram data set to determine the classification parameter until the number of leaves is greater than or equal to the preset leaf threshold; If the number of leaves is greater than or equal to the preset leaf threshold, the decision tree structure is used as the decision tree model.

7. The method according to claim 1, characterized in that The step of calculating according to the factor evaluation value set and the trained first model to determine the quality difference score specifically includes: Inputting the factor evaluation value sets into the trained first model for processing respectively to obtain a plurality of second classification results; Analyze several of the second classification results respectively to determine several groups of second model parameters; normalize several groups of the second model parameters to determine factor weight sets; The quality difference score is obtained by performing calculation according to the factor weight set and the factor evaluation value set.

8. A broadband quality assessment system, characterized in that: include: The first module is used to collect data on the target broadband according to the preset evaluation factors and determine the broadband parameter set; The second module is used to process the broadband parameter set according to a preset evaluation standard to determine an evaluation score set; perform a weighted evaluation according to the preset evaluation factor and the evaluation score set to determine a factor evaluation value set; A third module is used to calculate based on the factor evaluation value set and the trained first model to determine the quality difference score; The fourth module is used to match the quality difference score with the preset evaluation criteria to determine the quality difference evaluation result of the target broadband.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to perform the method according to any one of claims 1 to 7 when executed by the processor.

Images