Network application quality evaluation method and device

Abstract

The invention discloses a network application quality evaluation method and device, and relates to the field of network application quality evaluation.The method comprises the steps that monitoring index results of different units are uniformly converted into centesimal scores through an index scoring model, and the centesimal scores corresponding to the monitoring index results of all indexes are obtained; calculating an index weight set corresponding to the index monitoring result set by using the analytic hierarchy model; optimizing and adjusting the weight of each index in the index weight set by using a quadratic programming model to obtain a new weight of each index; for each index, multiplying the centesimal score corresponding to the monitoring index result of the index by the new weight of the index in the index weight set corresponding to the index monitoring result set to obtain the score of each index; and adding the scores of the indexes to obtain a score for evaluating the network application quality. The accuracy of the application quality evaluation result can be improved, so that the application quality evaluation result better conforms to the user experience.

Description

Technical Field

[0001] The present application relates to the technical field of network application quality assessment, and in particular to a network application quality assessment method and device. Background Art

[0002] Today, Internet technology is booming, the scale of Internet users continues to expand, and network application services are becoming more and more abundant. The 52nd "Statistical Report on the Development of Internet in China" released by China Internet Network Information Center (CNNIC) in August 2023 shows that as of June 2023, the scale of Chinese Internet users reached 1.079 billion, an increase of 11.09 million from December 2022, and the Internet penetration rate reached 76.4%, an increase of 0.8 percentage points from December 2022. Among them, the scale of online video users is 1.044 billion, accounting for 95.2% of the total Internet users; the scale of instant messaging users is 1.047 billion, accounting for 97.1% of the total Internet users; the number of websites is 3.83 million. The continuous emergence of these network applications has greatly facilitated people's production and life.

[0003] With the rapid growth of the network application market, competition among network service providers has become increasingly fierce. Differences in network architecture design, policy algorithms, and service providers have led to differences in the performance of the same network service. However, in the face of differentiated service demands, whether the services provided by manufacturers meet the needs of users and what service standards should be achieved have always been a vague issue and need to be quantified. Especially in the face of fierce competition among enterprises, the quality of applications directly determines the success or failure of an application promotion, so accurate evaluation of application quality is extremely important.

[0004] First, the application quality assessment results can help application developers and Internet managers find problems in current applications, thereby improving user satisfaction; secondly, developers and managers can accurately locate the performance bottlenecks of applications, and within the range allowed by application quality fluctuations, adjust resource allocation strategies to optimize in a targeted manner, so as to achieve targeted results and allow users to enjoy better services; finally, application developers can check the quality gap between their own applications and the same type of applications developed by other manufacturers, which will help improve existing services and gain an advantage in the competition. In this user-centric era, only by accurately and objectively grasping the user's service quality experience and responding to user needs in a timely manner can enterprises maintain long-term development. Therefore, it is urgent to establish a comprehensive network application quality assessment system.

[0005] In order to solve the problem of application quality evaluation, Quality of Service (QoS) is usually used as the evaluation standard for application quality. It is a technology-centric concept that includes relevant indicators such as bandwidth, latency, packet loss rate, video frame rate, bit rate, and jam rate. Researchers have invested a lot of money to optimize QoS, but these studies often only focus on a single or partial indicator, without comprehensively considering the impact of the indicators on application quality under the combined effect. For example, in streaming media applications, indicators closely related to user experience include: presentation quality, playback smoothness, quality fluctuation, etc., covering QoS indicators such as video encoding, packet transmission rate, and video bit rate switching. However, for developers, due to actual resource constraints, it is very difficult to maintain high video bit rate, short video jam buffering, and no bit rate switching at the same time. These three indicators are mutually restrained and cannot achieve optimal processing at the same time. Therefore, it is necessary to comprehensively evaluate the quality of network applications under the combined influence of various factors.

[0006] In contrast, Quality of Experience (QoE) is a user-centric concept. The evaluation subject is the user, and the evaluation object is the network application that provides various services. It can better reflect the user's actual experience. Designing a QoE-oriented application service quality evaluation system can provide service providers, network operators and other researchers with a comprehensive description of application services, so as to more reasonably schedule the network and adjust the application, and provide users with a better actual experience under limited resources. Therefore, establishing a network application quality evaluation method based on user QoE feedback is of great significance in improving user experience and optimizing resource allocation.

[0007] Beijing University of Posts and Telecommunications proposed a real-time web page quality assessment method based on deep neural networks. Its characteristics are: obtaining network-level raw data of web page information from edge routers or gateways, converting the format of the raw data, and using the data for training in the WebQMon.ai framework based on deep neural networks, so as to predict the first screen delay when users visit different web pages, and evaluate the quality of web page applications through the first screen delay. However, the invention has fewer evaluation indicators, and only uses the first screen time as the standard for application quality assessment. In practice, there are many factors affecting application quality, such as the maximum content drawing rate, service robustness, etc. The comprehensive impact of these factors on user experience and application quality is not considered, and the evaluation results may be relatively one-sided. In addition, the invention only considers the first screen time as a service quality QoS indicator, while QoE, which is closely related to user experience, is not taken into account. Therefore, the evaluation results cannot well reflect the actual feelings of users.

[0008] In the "Research and Implementation of Cloud Application Service Quality Evaluation Method Based on Objective Perception of QoE" proposed by Inner Mongolia University, the initial cache delay, freeze duration, freeze number, freeze ratio, bitrate switching number and average bitrate are selected as objective evaluation indicators for streaming media services under cloud services. The CRITIC weighting method is used to weighted cluster the QoE indicators according to the importance of each indicator on QoE; the QoE scores are divided into five levels to predict the actual user experience. However, the service quality evaluation results in this technology simply divide the user experience scores into five levels, lacking refined service quality evaluation results. In addition, this technology only replaces the user's subjective scoring with a simple K-Means clustering method, which cannot reflect the user's true feelings.

[0009] In summary, although the existing technology takes into account QoE which is closely related to user experience, there is still a problem that the accuracy of application quality evaluation results is not high and cannot match the user experience. Summary of the invention

[0010] The purpose of this application is to provide a network application quality assessment method and device, which can improve the accuracy of application quality assessment results and make them more in line with user experience.

[0011] To achieve the above objectives, this application provides the following solutions:

[0012] On the one hand, the present application provides a network application quality assessment method, the network application quality assessment method comprising:

[0013] Acquire an indicator set; the indicator set includes multiple indicators related to network application quality;

[0014] Obtaining an indicator historical result set; the indicator historical result set includes a plurality of historical indicator results of each of the indicators; the units of the historical indicator results of different indicators are different;

[0015] Establishing an indicator scoring model according to the indicator historical result set, using the indicator scoring model to uniformly convert the historical indicator results of different units into percentage scores, and obtaining the percentage scores corresponding to the historical indicator results of each indicator;

[0016] According to the indicator set, a weight system among the indicators is constructed using the hierarchical analysis method to obtain a hierarchical analysis model, and the indicator weight set corresponding to the indicator historical result set is calculated using the hierarchical analysis model; the indicator weight set includes the weight of each indicator;

[0017] Obtaining user experience quality scores for network applications of different qualities; the experience quality scores are scores on a percentage basis;

[0018] For each of the indicators, the percentage score corresponding to the historical indicator result of the indicator and the weight of the indicator in the indicator weight set corresponding to the indicator historical result set are multiplied, and then all the multiplication results are added, and the mean square error between the added result and the experience quality score is further transformed and converted into a standard form of quadratic programming problem for solution. The quadratic programming problem optimizes and adjusts the weights of each of the indicators in the indicator weight set corresponding to the indicator historical result set by minimizing the mean square error to obtain a quadratic programming model. The quadratic programming model is used to optimize and adjust the weights of each of the indicators in the indicator weight set corresponding to the indicator historical result set to obtain new weights of each of the indicators in the indicator weight set corresponding to the indicator historical result set;

[0019] Obtaining an indicator monitoring result set; the indicator monitoring result set includes monitoring indicator results of each of the indicators; the units of monitoring indicator results of different indicators are different;

[0020] The monitoring indicator results of different units are uniformly converted into percentage scores using the indicator scoring model to obtain percentage scores corresponding to the monitoring indicator results of each indicator;

[0021] Calculating the indicator weight set corresponding to the indicator monitoring result set using the hierarchical analysis model;

[0022] The quadratic programming model is used to optimize and adjust the weight of each indicator in the indicator weight set corresponding to the indicator monitoring result set to obtain a new weight of each indicator in the indicator weight set corresponding to the indicator monitoring result set;

[0023] For each of the indicators, multiply the percentage score corresponding to the monitoring indicator result of the indicator by the new weight of the indicator in the indicator weight set corresponding to the indicator monitoring result set to obtain the score of each of the indicators;

[0024] The scores of the various indicators are added together to obtain a score for evaluating the quality of the network application.

[0025] Optionally, the network application is a web application, a streaming media application or an instant messaging application;

[0026] When the network application is a web application, the multiple indicators related to the network application quality include connection establishment time, DNS resolution time, redirection time, total transmission time, delay, delay jitter, packet loss rate, maximum content drawing rate and first content drawing rate;

[0027] When the network application is a streaming media application, the multiple indicators related to the network application quality include connection establishment time, DNS resolution time, redirection time, total transmission time, delay, delay jitter, packet loss rate, frame rate, bit rate and freeze rate;

[0028] When the network application is an instant messaging application, the multiple indicators related to the network application quality include connection establishment time, DNS resolution time, redirection time, total transmission time, delay, delay jitter, packet loss rate, message sending delay and file transfer rate.

[0029] Optionally, the indicator scoring model is obtained by sorting all historical indicator results of the indicator from small to large for each indicator, and converting the historical indicator results of the indicator into percentage scores according to the percentage position of the historical indicator results of the indicator in the sorting.

[0030] Optionally, an L2 regularization strategy is introduced into the quadratic programming problem; the quadratic programming problem is solved using an OSQP algorithm; and an augmented Lagrange function with a quadratic penalty term is introduced in the process of solving the quadratic programming problem.

[0031] On the other hand, the present application provides a network application quality assessment device, the network application quality assessment device comprising:

[0032] An indicator set acquisition module, used to acquire an indicator set; the indicator set includes a plurality of indicators related to network application quality;

[0033] An indicator historical result set acquisition module is used to acquire an indicator historical result set; the indicator historical result set includes a plurality of historical indicator results of each of the indicators; the units of the historical indicator results of different indicators are different;

[0034] An indicator scoring module is used to establish an indicator scoring model based on the indicator historical result set, and use the indicator scoring model to uniformly convert the historical indicator results of different units into percentage scores to obtain the percentage scores corresponding to the historical indicator results of each indicator;

[0035] A hierarchical analysis module, used to construct a weight system among the indicators using the hierarchical analysis method according to the indicator set, obtain a hierarchical analysis model, and use the hierarchical analysis model to calculate the indicator weight set corresponding to the indicator historical result set; the indicator weight set includes the weight of each indicator;

[0036] The experience quality scoring module is used to obtain the user's experience quality scores for network applications of different qualities; the experience quality scores are based on a percentage system;

[0037] A quadratic programming module is used to multiply, for each of the indicators, the percentage score corresponding to the historical indicator result of the indicator and the weight of the indicator in the indicator weight set corresponding to the indicator historical result set, then add all the multiplication results, further transform the mean square error between the added result and the experience quality score, and convert it into a standard form of quadratic programming problem for solution. The quadratic programming problem optimizes and adjusts the weights of each indicator in the indicator weight set corresponding to the indicator historical result set by minimizing the mean square error to obtain a quadratic programming model. The quadratic programming model is used to optimize and adjust the weights of each indicator in the indicator weight set corresponding to the indicator historical result set to obtain new weights of each indicator in the indicator weight set corresponding to the indicator historical result set;

[0038] An indicator monitoring result set acquisition module is used to acquire an indicator monitoring result set; the indicator monitoring result set includes monitoring indicator results of each of the indicators; the units of monitoring indicator results of different indicators are different;

[0039] A monitoring indicator result conversion module is used to convert the monitoring indicator results of different units into percentage scores by using the indicator scoring model, and obtain the percentage scores corresponding to the monitoring indicator results of each indicator;

[0040] An indicator weight calculation module, used to calculate the indicator weight set corresponding to the indicator monitoring result set using the hierarchical analysis model;

[0041] An indicator weight optimization and adjustment module is used to optimize and adjust the weights of each indicator in the indicator weight set corresponding to the indicator monitoring result set by using the quadratic programming model to obtain new weights of each indicator in the indicator weight set corresponding to the indicator monitoring result set;

[0042] An indicator score calculation module, for each of the indicators, multiplying the percentage score corresponding to the monitoring indicator result of the indicator by the new weight of the indicator in the indicator weight set corresponding to the indicator monitoring result set, to obtain the score of each of the indicators;

[0043] The evaluation score calculation module is used to add up the scores of the indicators to obtain a score for evaluating the quality of the network application.

[0044] Optionally, the indicator scoring model is obtained by sorting all historical indicator results of the indicator from small to large for each indicator, and converting the historical indicator results of the indicator into percentage scores according to the percentage position of the historical indicator results of the indicator in the sorting.

[0045] Optionally, an L2 regularization strategy is introduced into the quadratic programming problem; the quadratic programming problem is solved using an OSQP algorithm; and an augmented Lagrange function with a quadratic penalty term is introduced in the process of solving the quadratic programming problem.

[0046] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0047] The present application provides a network application quality assessment method and device, which uniformly converts monitoring indicator results of different units into percentage scores through an indicator scoring model, and replaces the existing technology that is only divided into 5 levels in the form of percentage scores. Compared with the existing technology, it is more accurate. Through the quadratic programming model, combined with the user's experience quality scores for network applications of different qualities, the user experience is considered. The quadratic programming problem minimizes the mean square error between the model evaluation score (that is, the percentage score multiplied by the indicator weight and then the result obtained by adding all the multiplication results) and the experience quality score, and optimizes and adjusts the indicator weight calculated by the hierarchical analysis model to obtain a more accurate new indicator weight. By multiplying the more accurate new indicator weight with the percentage score and then adding all the multiplication results, an accurate score for evaluating the quality of the network application is obtained (the higher the score, the better the quality of the network application), thereby achieving the effect of improving the accuracy of the application quality assessment result and making it more in line with the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A flowchart of a network application quality assessment method provided in one embodiment of the present application;

[0050] Figure 2 A functional module diagram provided for an embodiment of the present application;

[0051] Figure 3 An indicator scoring flow chart provided for an embodiment of the present application;

[0052] Figure 4 A flowchart of a hierarchical analysis module provided in one embodiment of the present application;

[0053] Figure 5 A flowchart of a secondary planning module provided in one embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] The purpose of this application is to provide a network application quality assessment method and device, which can improve the accuracy of application quality assessment results and make them more in line with user experience.

[0056] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0057] like Figure 1 As shown, the present application provides a network application quality assessment method, comprising:

[0058] Step 101: Acquire an indicator set; the indicator set includes multiple indicators related to network application quality.

[0059] In step 101, the network application is a web application, a streaming media application or an instant messaging application. When the network application is a web application, the multiple indicators related to the quality of the network application include the time to establish a connection, the DNS resolution time, the redirection time, the total transmission time, the delay, the delay jitter, the packet loss rate, the maximum content drawing rate and the first content drawing rate. When the network application is a streaming media application, the multiple indicators related to the quality of the network application include the time to establish a connection, the DNS resolution time, the redirection time, the total transmission time, the delay, the delay jitter, the packet loss rate, the frame rate, the bit rate and the freeze rate. When the network application is an instant messaging application, the multiple indicators related to the quality of the network application include the time to establish a connection, the DNS resolution time, the redirection time, the total transmission time, the delay, the delay jitter, the packet loss rate, the message sending delay and the file transfer rate.

[0060] Common indicators of network applications include response rate indicators and service robustness indicators, among which response rate indicators include connection establishment time, DNS resolution time, redirection time, total transmission time, etc., and service robustness indicators include latency, latency jitter, packet loss rate, etc. For different network applications, corresponding indicators can be selected in a targeted manner. For example, for web applications, the maximum content drawing rate, first content drawing rate, etc. can be selected in a targeted manner. For streaming media applications, frame rate, bit rate, freeze rate, etc. can be selected in a targeted manner. For instant messaging applications, message sending delay, file transfer rate, etc. can also be selected in a targeted manner.

[0061] Step 102: Obtain an indicator historical result set; the indicator historical result set includes multiple historical indicator results of each indicator; the units of historical indicator results of different indicators are different.

[0062] Step 103: Establish an indicator scoring model based on the indicator historical result set, and use the indicator scoring model to uniformly convert the historical indicator results of different units into percentage scores to obtain the percentage scores corresponding to the historical indicator results of each indicator.

[0063] In step 103, the indicator scoring model is to sort all historical indicator results of each indicator from small to large, and convert the historical indicator results of the indicator into a percentage score according to the percentage position of the historical indicator results of the indicator in the sorting.

[0064] Step 104: Based on the indicator set, use the hierarchical analysis method to construct a weight system between indicators to obtain a hierarchical analysis model, and use the hierarchical analysis model to calculate the indicator weight set corresponding to the indicator historical result set; the indicator weight set includes the weight of each indicator.

[0065] Step 105: Obtain the user's experience quality scores for network applications of different qualities; the experience quality scores are scored on a percentage basis.

[0066] Among them, the user's experience quality score for network applications of different qualities (that is, different network applications have different qualities) is the score given by the user to each network application based on the user's experience of using each network application. For example, the user gives 70 points to network application A based on the user's experience of using network application A, the user gives 80 points to network application B based on the user's experience of using network application B, and the user gives 90 points to network application C based on the user's experience of using network application C, and so on.

[0067] Step 106: For each indicator, multiply the percentage score corresponding to the historical indicator result of the indicator and the weight of the indicator in the indicator weight set corresponding to the indicator historical result set, then add all the multiplication results, further transform the mean square error between the added result and the experience quality score, and convert it into a standard form of quadratic programming problem for solution. The quadratic programming problem optimizes and adjusts the weight of each indicator in the indicator weight set corresponding to the indicator historical result set by minimizing the mean square error, and obtains a quadratic programming model. The quadratic programming model is used to optimize and adjust the weight of each indicator in the indicator weight set corresponding to the indicator historical result set, and obtain new weights of each indicator in the indicator weight set corresponding to the indicator historical result set.

[0068] In step 106, an L2 regularization strategy is introduced into the quadratic programming problem; the quadratic programming problem is solved using the OSQP algorithm; and an augmented Lagrange function with a quadratic penalty term is introduced in the process of solving the quadratic programming problem.

[0069] The following example illustrates the step of "for each indicator, multiplying the percentage score corresponding to the historical indicator result of the indicator and the weight of the indicator in the indicator weight set corresponding to the historical result set of the indicator, and then adding all the multiplication results" in step 106:

[0070] For example, for a certain network application, multiple indicators related to the quality of the network application are connection establishment time indicator, bitrate indicator, and frame rate indicator. For the connection establishment time indicator, the percentage score (for example, 70 points) corresponding to the historical indicator result of the indicator (for example, 50ms) is multiplied by the weight of the indicator in the indicator weight set corresponding to the indicator historical result set (for example, 0.1). For the bitrate indicator, the percentage score (for example, 80 points) corresponding to the historical indicator result of the indicator is multiplied by the weight of the indicator in the indicator weight set corresponding to the indicator historical result set (for example, 0.3). For the frame rate indicator, the percentage score (for example, 90 points) corresponding to the historical indicator result of the indicator is multiplied by the weight of the indicator in the indicator weight set corresponding to the indicator historical result set (for example, 0.5). Then, all the multiplication results are added together, that is, 70×0.1+80×0.3+90×0.5, and the added result is 76 points.

[0071] Step 107: Obtain an indicator monitoring result set; the indicator monitoring result set includes monitoring indicator results of each indicator; and the units of monitoring indicator results of different indicators are different.

[0072] Step 108: The monitoring indicator results of different units are uniformly converted into percentage scores using the indicator scoring model to obtain the percentage scores corresponding to the monitoring indicator results of each indicator.

[0073] Step 109: Calculate the indicator weight set corresponding to the indicator monitoring result set using the hierarchical analysis model.

[0074] Step 110: Utilize a quadratic programming model to optimize and adjust the weight of each indicator in the indicator weight set corresponding to the indicator monitoring result set, and obtain a new weight of each indicator in the indicator weight set corresponding to the indicator monitoring result set.

[0075] Step 111: For each indicator, multiply the percentage score corresponding to the indicator monitoring indicator result of the indicator by the new weight of the indicator in the indicator weight set corresponding to the indicator monitoring result set to obtain the score of each indicator.

[0076] Step 112: Add the scores of the indicators to obtain a score for evaluating the quality of the network application.

[0077] In step 112, the scores of the indicators are added together to obtain a score for evaluating the quality of the network application. Based on this score, the application quality evaluation result can be obtained, that is, the quality of the network application can be known. Generally speaking, the higher the score, the better the quality of the network application, and the lower the score, the worse the quality of the network application.

[0078] The technical solution of this application is described below with a specific embodiment:

[0079] The network application quality assessment method of the present application is a network application quality assessment method based on experience quality feedback, and the application product of the method is an application quality assessment system. Aiming at the problem of network application quality assessment, the present application mainly proposes an assessment modeling method based on experience quality feedback (i.e., a network application quality assessment method based on experience quality feedback). QoE is a user-centric concept, which is closely related to the user's subjective feelings and can more comprehensively demonstrate how various service quality indicators affect the user experience. The present application comprehensively considers various factors that affect the user experience through methods such as hierarchical analysis and quadratic programming, evaluates the application quality from the user's perspective, makes the evaluation results closer to the user's actual usage experience, establishes an application quality assessment model, improves the accuracy of quality assessment, and uses the OSQP algorithm to solve and reduce the complexity of calculation.

[0080] The purpose of this application is to meet the actual business needs for customizable and refined service quality evaluation for different types of network applications, and to establish an accurate and objective service quality evaluation model. In the current evaluation methods, most use QoS indicators such as bandwidth, latency, and packet loss rate as evaluation criteria for application quality, without comprehensively considering the impact of the combined effect of indicators on application quality, and lack a QoE-oriented application service quality evaluation system. Therefore, in order to improve the accuracy of application evaluation and make it more in line with user experience, this application proposes a new evaluation modeling method (i.e., a network application quality evaluation method based on experience quality feedback) to make up for past deficiencies. Figure 2 As shown in the figure, the method consists of three modules: indicator scoring module, hierarchical analysis module and quadratic programming module. In the process of creating the evaluation model, the indicator scoring modeling is first used to formulate the scoring standard of the monitoring indicators, and the results of different indicators are uniformly converted into scores from 0 to 100; then the hierarchical analysis modeling determines the weight of each indicator in the quality evaluation model by comparing the importance of different indicators; finally, the quadratic programming modeling makes further optimization adjustments to the evaluation model by minimizing the mean square error between the user QoE score and the model score, thereby constructing an application quality evaluation model.

[0081] The following is a detailed introduction to the indicator scoring module, hierarchical analysis module and quadratic programming module:

[0082] 1. Index scoring module:

[0083] The function of this module is to establish an indicator scoring model to convert the indicator results of different units into a percentage score. Figure 3 As shown in the figure, this module will divide the indicator score interval into [0-100] points according to the historical data of different application-related indicators collected and the data distribution, and evenly divide it into several levels. Each level corresponds to a score interval. The interval position of the indicator monitoring result is determined according to the percentile of the indicator data in the overall distribution. The indicator data value at each interval endpoint is obtained by taking the value at the corresponding percentile, and the calculation method is the score interval [s i ,s i+1 ] Corresponding index result interval [t i ,t i+1 ], indicator monitoring results t j , the index corresponds to the score s j It can be expressed as the following formula:

[0084]

[0085] Among them, s i 、s i+1 The score intervals [s i ,s i+1 ] left and right endpoints, t i ,t i+1 They are the index result intervals [t i ,t i+1 ]’s left and right endpoints.

[0086] Taking the connection time indicator as an example, the process of converting the indicator result into a percentage score is introduced:

[0087] For the connection establishment time indicator, the historical monitoring results (i.e., historical indicator results) are sorted, where 20.3ms is at the 10% position of all results and 40.5ms is at the 20% position. Then the score interval [100,90] corresponds to the indicator result interval [0,20.3], and the score interval [90,80] corresponds to the indicator result interval [20.3,40.5]. When the newly measured connection establishment time result (i.e., the monitoring indicator result of the connection establishment time) is 18ms, it corresponds to [0,20.3]. Substituting it into the formula The corresponding score calculated is 91.13 points.

[0088] 2. Hierarchical Analysis Module:

[0089] like Figure 4As shown, this module uses the Analytic Hierarchy Process (AHP) to construct a weight system between indicators, and its hierarchical structure includes a target layer, a criterion layer, and a solution layer. The target layer is the top layer of the hierarchy, representing the ultimate goal of the decision. The goal in this application is to build an optimal model that can accurately evaluate the quality of network applications; the criterion layer is located one level below the target layer, including various criteria or standards for evaluating and comparing different solutions. In this application, it is composed of different service quality attribute characteristics, such as response rate attributes, robustness, etc. When the number of evaluation application indicators is large, in order to improve the accuracy of decision-making, the criterion layer can be further subdivided into sub-criterion layers, and different sub-layers represent the specific composition of each item in the previous criterion layer. The bottom layer is the solution layer, including all possible selection options, which refers to the specific weights of different service quality indicators in this application.

[0090] Based on the criterion layer structure, a questionnaire was designed and distributed, and expert users were invited to evaluate the importance of indicators at all levels using the nine-point method. The nine-point method is a scoring system that divides importance into nine levels, from "1" (equal importance) to "9" (extreme importance). The specific definitions of different levels are shown in Table 1.

[0091] Table 1 Nine-point scale

[0092] <![CDATA[Scale X ij > definition 1 Equally important: Factor i is as important as factor j 3 Slightly important: Factor i is slightly more important than factor j 5 Obviously important: factor i is more important than factor j 7 Very important: Factor i is much more important than factor j 9 Absolutely important: Factor i is absolutely more important than factor j. 2，4，6，8 Intermediate value: a state between the above two judgments reciprocal <![CDATA[X ij =1 / X ji ]]>

[0093] First, compare the importance of different attributes in the criterion layer. If the criterion layer is divided into different sub-layers, continue to compare the importance of the indicators of each sub-layer under each criterion layer (that is, continue to compare the importance of the indicators of each sub-layer under each criterion layer) and construct the comparison matrix of each layer. The m-order comparison matrix in the hierarchical analysis method can represent m indicators. The comparison matrix is ​​denoted by A, λ max is the maximum eigenvalue of the contrast matrix, λ max The corresponding original eigenvector is normalized to obtain vector W. According to the meaning of the matrix eigenvalue, it is easy to get that the elements in W represent the degree of influence between indicators at different levels. In order to confirm whether the result is valid, a consistency test is required to confirm whether there are logical contradictions or misjudgments in the judgment matrix. This application calculates the consistency index (Consistency Index, CI) of the comparison matrix, and its calculation formula is:

[0094]

[0095] Where n is the order of the comparison matrix. When CI = 0, it means that the comparison matrix is ​​completely consistent. The larger the CI, the worse the consistency. The consistency ratio (CR) is calculated by referring to the Random Consistency Index (RI) table. The calculation process can be expressed as:

[0096]

[0097] If the CR value is less than 0.1, it means that the matrix passes the consistency test, and the eigenvector can be normalized as a weight vector.

[0098] This module first reads a data set containing several users' comparative evaluations of the importance of different indicators. For each user's evaluation result, the module will first establish a hierarchical structure, build a comparison matrix, and calculate the weight and consistency ratio according to the process described above. Then a consistency test is performed. The consistency test RI value query table is shown in Table 2. If the test is passed, the weight between indicators corresponding to the user's evaluation is recorded. Let the weight vector corresponding to the importance evaluation of the i-th user on the m indicators be recorded as w i , if it fails the consistency check, the evaluation result data is discarded. By reading all user rating data sets, the weight vectors corresponding to all valid user evaluations are finally averaged to obtain the final weight vector x output result. The calculation formula is expressed as:

[0099]

[0100] Table 2 RI value query table for consistency test

[0101]

[0102] 3. Secondary planning module:

[0103] Although the AHP can effectively solve the problem of weight allocation of quality indicators in application quality assessment, there are some defects in the application of the traditional AHP to quality assessment. First, when there are many evaluation factors, contradictory judgments and logical errors are prone to occur, and consistency checks are required, which will generate additional computing overhead and invalid data; secondly, since the comparison matrix is ​​given by experts, it will inevitably be interfered by subjective factors, which may reduce the accuracy of the final evaluation results. In order to make the quality assessment results of the model more consistent with the user's experience, on the basis of the AHP model, the model is further optimized with the collected user QoE data results.

[0104] For an optimal quality assessment model, its assessment result should be close to the actual score of the user on the network application experience. The method of this application adopts a data-driven optimization method based on QoE feedback to adjust the parameters of the model by minimizing the mean square error between the model assessment score and the user's actual experience quality score. Specifically, Figure 5 As shown in the figure, by minimizing the mean square error (MSE) between the model score and the user QoE, the problem is converted into a quadratic programming problem to achieve feedback adjustment of the model weights. The quadratic programming problem is a special type of mathematical optimization problem. It minimizes or maximizes a quadratic function under given linear constraints. It can usually be described by the following mathematical formula:

[0105]

[0106] subject to b≤Ax≤c

[0107] Where x is the decision variable vector, T represents the transpose, P is the coefficient matrix of the quadratic term of the objective function, q is the coefficient vector of the linear term of the objective function, A is the coefficient matrix of the constraint condition, b and c are the left and right vectors of the constraint condition, and x in the objective function is T Px represents the quadratic term, q T x represents a linear term. Specifically for the application quality assessment modeling problem described in this application, let the M×N dimensional matrix Y represent a data set containing N indicator collection data results, and each column vector y j =[a1,a2,…,a M ] T represents an application quality monitoring result covering M indicators. N-dimensional user rating vector Q = [Q1, Q2, ..., Q N ] T , where each element Q j Corresponding to a column vector Y in matrix Y i The M-dimensional weight vector x=[x1,x2,…,x j ,…x M ] T , represents the weights of different indicators, where x i Represents the weight of the i-th indicator, and the weight vector satisfies The mean square error ε between the model prediction result and the user's QoE score MES It is expressed as:

[0108]

[0109] It can be further transformed into a standard quadratic programming problem:

[0110]

[0111] As the weight vector x, it should satisfy the linear constraints that the sum of each item is 1 and each item is greater than or equal to 0:

[0112]

[0113] Among them, W x Represents a set of linear constraints that are satisfied. At the same time, in order to improve the generalization ability of the model in processing unseen data and avoid overfitting, this application introduces the L2 regularization mechanism to balance the model complexity and the degree of fit of the training data. L2 regularization, also known as ridge regression or weight decay, is a technique that reduces model complexity and avoids overfitting by adding a regularization term λ to the loss function. For this application, the regularization term is the L2 norm of the model weight vector x The introduction of L2 regularization can reduce the absolute size of the weight vector, prompting the model to tend to allocate smaller and more uniform weight values, thereby enhancing the generalization of the model. This treatment avoids the model's excessive reliance on a few indicators, prevents the evaluation results from being too biased towards certain indicators, and ensures the overall stability and reliability of the model. In summary, the quadratic programming problem can be finally expressed as:

[0114]

[0115] subject to x∈W x

[0116] Among them, λ represents the regularization term, u represents the constant term, and L represents the optimization objective function. Obviously, the matrix P is a semi-positive definite matrix, and the optimization problem is a convex optimization. There are currently many methods to solve quadratic programming problems for this type of optimization problem. Among them, the OSQP algorithm is an optimization solver specially designed to solve convex quadratic programming problems. It has the advantages of high efficiency and stability and is widely used in control theory, system optimization and other fields. OSQP uses the alternating direction method of multipliers (ADMM) to solve. Its main idea is to decompose the objective function in the original problem and transform it into several equivalent solvable sub-problems, then solve each sub-problem in parallel, and finally coordinate the solutions of the sub-problems to obtain the global solution of the original problem. ADMM is mainly used to solve optimization problems of the following forms:

[0117] minimize F(x,z)=f(x)+g(z)

[0118] subject to Ax+Bz=c

[0119] Among them, the optimization objective function is jointly affected by x and z, F(x,z) represents the optimization objective function jointly affected by the two variables x and z, f(x) represents a convex function about the variable x, g(z) represents a convex function about the variable z, and the matrices A, B and vector c jointly define the constraints of the optimization problem. The augmented Lagrange function with a quadratic penalty term is introduced when solving the problem. The function can be expressed as:

[0120]

[0121] Among them, L ρ (x,z,α) represents the augmented Lagrange function with a quadratic penalty term, α is the Lagrange multiplier, ρ is a penalty parameter greater than zero, and the ADMM iterative solution method is expressed as:

[0122]

[0123] λ k+1 =λ k +ρ(Ax k+1 +Bz k+1 -b)=λ k +Ax k+1 +B k+1 -c

[0124] Where k represents the number of iterations. Each iteration is divided into three steps: first, fix the variables z and α, minimize the augmented Lagrange function to update x; then fix the variables x and α, minimize the augmented Lagrange function to update the variable z; finally, calculate and update the Lagrange multiplier α.

[0125] The OSQP algorithm draws on the idea of ​​the ADMM algorithm. First, the objective optimization function is transformed into:

[0126]

[0127] in, and is the auxiliary variable introduced, I Ax with I C (z) is an indicator function. When the variable satisfies the linear constraints of the quadratic programming problem, its value is 0. Otherwise, its value is infinite, which makes the entire objective function value unbounded, ensuring that x that does not meet the conditions is definitely not a solution to the optimization problem.

[0128]

[0129] After converting it into a binary variable optimization problem with equality constraints, the iterative solution equation is written according to the ADMM iterative solution:

[0130]

[0131] Among them, C represents the Euclidean projection of the constraints in set C, ensuring that in each iteration, the solution remains within the constraint set C. The key to solving the problem is to calculate According to the Karush-Kuhn-Tucker (KKT) condition principle in planning problems, the necessary and sufficient conditions for the optimal solution are:

[0132] Px k+1 +q+σ(x k+1 -x k )+A T v k+1 =0

[0133]

[0134] Among them, P is the coefficient matrix of the quadratic term of the objective function, q is the coefficient vector of the linear term of the objective function, A is the coefficient matrix of the constraint conditions, z is the introduced auxiliary variable, σ and ρ are the step size parameters, and v is the Lagrange multiplier. After matrix row and column transformation, it is converted into a matrix equation for solution.

[0135]

[0136] Where I represents the identity matrix. Thus, the optimal weight parameter vector x of the evaluation model is obtained.

[0137] The key technical means of this application are as follows:

[0138] 1. An application quality assessment method was designed, which includes an indicator scoring module, a hierarchical analysis module, and a quadratic programming module. The indicator scoring module established an indicator scoring model to convert the collection results of different indicators into a percentage score; the hierarchical analysis module collected users' opinions on the importance of different indicator weights and determined the initial weights between each indicator; the quadratic programming module, based on the hierarchical analysis module, according to the user's QoE scores for applications of different qualities, converted the problem into a quadratic programming problem through the mean square error function and the L2 regularization term, so as to further adjust the weights of each indicator and establish an application quality assessment scoring model.

[0139] 2. An indicator scoring model is established to convert the indicator results of different units into a unified percentage score. The indicator scoring module will divide the indicator score interval into several level intervals based on the collected historical data of different application-related indicators and the data distribution, and set the indicator score interval between [0-100] points. Each level corresponds to a score interval. The interval position of the indicator monitoring result is determined according to the percentile of the indicator data in the overall distribution.

[0140] 3. Use the analytic hierarchy process to determine the initial weight of each indicator. The target layer is to build the best model that can accurately evaluate the quality of network applications. The criterion layer is composed of different service quality attribute characteristics, such as response rate attribute, robustness, etc. When the number of criterion layer characteristics is large, it can also be divided into sub-criterion layers. For example, the response time attribute is composed of indicators such as TCP connection time and first packet time. The bottom layer is the solution layer, which refers to the specific weights of different service quality indicators. A questionnaire was designed and distributed, and expert users were invited to use the nine-point method to evaluate the importance of indicators at all levels, build a comparison matrix, and calculate the consistency ratio. The normalized maximum eigenvector of the comparison matrix that passed the consistency test is the weight vector.

[0141] 4. The application quality assessment model is transformed into a quadratic programming problem. Its optimization objective function consists of two parts. One part is the mean square error function between the hierarchical analysis model score and the user QoE score. The other part is the L2 regularization term, that is, the L2 norm of the weight vector, which is used to reduce the degree of overfitting of the model. Finally, the OSQP algorithm is used and the result of the hierarchical analysis method is used as the initial value to solve the optimization problem and obtain the best application quality assessment model.

[0142] This application can be used to establish quality assessment models for different network applications and obtain quantitative scores for quality assessment. Enterprise developers can use this application to evaluate the network applications they develop, promptly discover abnormalities in service quality, find problems with the application based on the total quality score and the scoring results of each indicator, make appropriate adjustments, and improve user experience. Or enterprise developers want to compare the service quality of their own applications with those of other manufacturers, view the service quality gap between different applications based on the evaluation results, and find directions for subsequent application upgrades and optimizations. Or network application users want to find the application with the best service quality and the one that best suits their needs among several similar applications. This application can be used to establish an evaluation model, obtain a quality score, and select the most suitable application.

[0143] Taking streaming media application as an example, the process of using the network application quality assessment method of this application for streaming media application quality assessment is introduced below:

[0144] Network streaming applications are widely used, but users have different experiences with streaming quality. User satisfaction is often reduced due to problems such as lag and low resolution. The application quality assessment model established by this application can analyze problems in the application, compare it with other similar products, and perform subsequent quality optimization.

[0145] A. The user first selects several indicators related to the quality of streaming media applications, such as bit rate, frame rate, freeze rate, connection establishment time, latency, packet loss rate, etc., collects historical result data of these indicators, uses the application method to perform statistical analysis on the data, establishes indicator scoring level intervals, and uniformly maps indicator results of different meanings and units into corresponding indicator scores.

[0146] B. The user then collects comparative evaluations of the importance of the selected indicators from different users, and calculates the initial weight values ​​of the different indicators through the hierarchical analysis module in the method of the present application.

[0147] C. Users can organize a group of test users to rate the application experience in a specific environment. Collect users' subjective feelings and form a QoE data set. Through the quadratic planning module in this application, the mean square error between the model score and the user's QoE score is minimized, and the indicator weights are further adjusted to obtain the best application quality evaluation model.

[0148] Compared with the prior art, the advantages of the present application are as follows: the indicator scoring modeling in the present application formulates the monitoring indicator scoring standard, and uniformly converts the results of different indicators into scores from 0 to 100, so as to better quantify and compare the scores of different indicators; through the method of hierarchical analysis and quadratic programming, the user experience is taken into account, and the optimization function minimizes the mean square error between the user QoE score and the model score, and obtains the weight relationship between the indicators, so that the evaluation score is more in line with the user experience.

[0149] In addition, machine learning-based methods can be used to collect data related to network application quality, including user QoE feedback and various service quality indicators (such as bandwidth, latency, packet loss rate, video frame rate, etc.). Clean and preprocess the data, such as processing missing values, outliers, and standardization. Different neural network algorithms can be selected, and indicators such as accuracy, mean square error, and mean absolute error can be used to measure the performance of the model, thereby determining the indicator weights.

[0150] Based on the same inventive concept, the embodiment of the present application also provides a network application quality assessment device for implementing the network application quality assessment method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more network application quality assessment device embodiments provided below can refer to the limitations of the network application quality assessment method above, and will not be repeated here.

[0151] In an exemplary embodiment, a network application quality assessment device is provided, which is a network application quality assessment device based on experience quality feedback, the application product of the device is an application quality assessment system, and the device includes:

[0152] The indicator set acquisition module is used to acquire the indicator set; the indicator set includes multiple indicators related to the quality of network applications.

[0153] The indicator historical result set acquisition module is used to obtain the indicator historical result set; the indicator historical result set includes multiple historical indicator results of each indicator; the units of the historical indicator results of different indicators are different.

[0154] The indicator scoring module is used to establish an indicator scoring model based on the indicator historical result set, and use the indicator scoring model to uniformly convert the historical indicator results of different units into percentage scores to obtain the percentage scores corresponding to the historical indicator results of each indicator.

[0155] The hierarchical analysis module is used to construct a weight system between indicators based on the indicator set using the hierarchical analysis method to obtain a hierarchical analysis model, and use the hierarchical analysis model to calculate the indicator weight set corresponding to the indicator historical result set; the indicator weight set includes the weight of each indicator.

[0156] The experience quality scoring module is used to obtain the user's experience quality scores for network applications of different qualities; the experience quality scores are scored on a 100-point scale.

[0157] The quadratic programming module is used to multiply the percentage score corresponding to the historical indicator result of the indicator and the weight of the indicator in the indicator weight set corresponding to the historical result set of the indicator for each indicator, then add all the multiplication results, further transform the mean square error between the added result and the experience quality score, and convert it into a standard form of quadratic programming problem for solution. The quadratic programming problem optimizes and adjusts the weight of each indicator in the indicator weight set corresponding to the historical result set of the indicator by minimizing the mean square error to obtain a quadratic programming model. The quadratic programming model is used to optimize and adjust the weight of each indicator in the indicator weight set corresponding to the historical result set of the indicator to obtain a new weight for each indicator in the indicator weight set corresponding to the historical result set of the indicator.

[0158] The indicator monitoring result set acquisition module is used to obtain the indicator monitoring result set; the indicator monitoring result set includes the monitoring indicator results of each indicator; the units of the monitoring indicator results of different indicators are different.

[0159] The monitoring indicator result conversion module is used to uniformly convert the monitoring indicator results of different units into percentage scores using the indicator scoring model, and obtain the percentage scores corresponding to the monitoring indicator results of each indicator.

[0160] The indicator weight calculation module is used to calculate the indicator weight set corresponding to the indicator monitoring result set using the hierarchical analysis model.

[0161] The indicator weight optimization and adjustment module is used to optimize and adjust the weight of each indicator in the indicator weight set corresponding to the indicator monitoring result set using a quadratic programming model to obtain a new weight for each indicator in the indicator weight set corresponding to the indicator monitoring result set.

[0162] The indicator score calculation module is used to multiply the percentage score corresponding to the indicator monitoring indicator result of the indicator and the new weight of the indicator in the indicator weight set corresponding to the indicator monitoring result set for each indicator to obtain the score of each indicator.

[0163] The evaluation score calculation module is used to add up the scores of each indicator to obtain a score for evaluating the quality of the network application.

[0164] The indicator scoring model is to sort all historical indicator results of each indicator from small to large, and convert the historical indicator results of the indicator into a percentage score according to the percentage position of the historical indicator results of the indicator in the sorting.

[0165] The L2 regularization strategy is introduced in the quadratic programming problem; the quadratic programming problem is solved using the OSQP algorithm; the augmented Lagrange function with a quadratic penalty term is introduced in the process of solving the quadratic programming problem.

[0166] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0167] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A network application quality assessment method, characterized in that: The network application quality assessment method comprises: Acquire an indicator set; the indicator set includes multiple indicators related to network application quality; Obtaining an indicator historical result set; the indicator historical result set includes a plurality of historical indicator results of each of the indicators; the units of the historical indicator results of different indicators are different; Establishing an indicator scoring model according to the indicator historical result set, using the indicator scoring model to uniformly convert the historical indicator results of different units into percentage scores, and obtaining the percentage scores corresponding to the historical indicator results of each indicator; According to the indicator set, a weight system among the indicators is constructed using the hierarchical analysis method to obtain a hierarchical analysis model, and the indicator weight set corresponding to the indicator historical result set is calculated using the hierarchical analysis model; the indicator weight set includes the weight of each indicator; Obtaining user experience quality scores for network applications of different qualities; the experience quality scores are scores on a percentage basis; For each of the indicators, the percentage score corresponding to the historical indicator result of the indicator and the weight of the indicator in the indicator weight set corresponding to the indicator historical result set are multiplied, and then all the multiplication results are added, and the mean square error between the added result and the experience quality score is further transformed and converted into a standard form of quadratic programming problem for solution. The quadratic programming problem optimizes and adjusts the weights of each of the indicators in the indicator weight set corresponding to the indicator historical result set by minimizing the mean square error to obtain a quadratic programming model. The quadratic programming model is used to optimize and adjust the weights of each of the indicators in the indicator weight set corresponding to the indicator historical result set to obtain new weights of each of the indicators in the indicator weight set corresponding to the indicator historical result set; Obtaining an indicator monitoring result set; the indicator monitoring result set includes monitoring indicator results of each of the indicators; the units of monitoring indicator results of different indicators are different; The monitoring indicator results of different units are uniformly converted into percentage scores using the indicator scoring model to obtain percentage scores corresponding to the monitoring indicator results of each indicator; Calculating the indicator weight set corresponding to the indicator monitoring result set by using the hierarchical analysis model; The quadratic programming model is used to optimize and adjust the weight of each indicator in the indicator weight set corresponding to the indicator monitoring result set to obtain a new weight of each indicator in the indicator weight set corresponding to the indicator monitoring result set; For each of the indicators, multiply the percentage score corresponding to the monitoring indicator result of the indicator by the new weight of the indicator in the indicator weight set corresponding to the indicator monitoring result set to obtain the score of each of the indicators; The scores of the various indicators are added together to obtain a score for evaluating the quality of the network application.

2. The network application quality assessment method according to claim 1, characterized in that: The network application is a web application, a streaming media application or an instant messaging application; When the network application is a web application, the multiple indicators related to the network application quality include connection establishment time, DNS resolution time, redirection time, total transmission time, delay, delay jitter, packet loss rate, maximum content drawing rate and first content drawing rate; When the network application is a streaming media application, the multiple indicators related to the network application quality include connection establishment time, DNS resolution time, redirection time, total transmission time, delay, delay jitter, packet loss rate, frame rate, bit rate and freeze rate; When the network application is an instant messaging application, the multiple indicators related to the network application quality include connection establishment time, DNS resolution time, redirection time, total transmission time, delay, delay jitter, packet loss rate, message sending delay and file transfer rate.

3. The network application quality assessment method according to claim 1, characterized in that: The indicator scoring model is obtained by sorting all historical indicator results of each indicator from small to large, and converting the historical indicator results of the indicator into a percentage score according to the percentage position of the historical indicator results of the indicator in the sorting.

4. The network application quality assessment method according to claim 1, characterized in that: An L2 regularization strategy is introduced into the quadratic programming problem; the quadratic programming problem is solved using an OSQP algorithm; and an augmented Lagrange function with a quadratic penalty term is introduced in the process of solving the quadratic programming problem.

5. A network application quality assessment device, characterized in that: The network application quality assessment device comprises: An indicator set acquisition module, used to acquire an indicator set; the indicator set includes a plurality of indicators related to network application quality; An indicator historical result set acquisition module is used to acquire an indicator historical result set; the indicator historical result set includes a plurality of historical indicator results of each of the indicators; the units of the historical indicator results of different indicators are different; An indicator scoring module is used to establish an indicator scoring model based on the indicator historical result set, and use the indicator scoring model to uniformly convert the historical indicator results of different units into percentage scores to obtain the percentage scores corresponding to the historical indicator results of each indicator; A hierarchical analysis module, used to construct a weight system among the indicators using the hierarchical analysis method according to the indicator set, obtain a hierarchical analysis model, and use the hierarchical analysis model to calculate the indicator weight set corresponding to the indicator historical result set; the indicator weight set includes the weight of each indicator; The experience quality scoring module is used to obtain the user's experience quality scores for network applications of different qualities; the experience quality scores are based on a percentage system; A quadratic programming module is used to multiply, for each of the indicators, the percentage score corresponding to the historical indicator result of the indicator and the weight of the indicator in the indicator weight set corresponding to the indicator historical result set, then add all the multiplication results, further transform the mean square error between the added result and the experience quality score, and convert it into a standard form of quadratic programming problem for solution. The quadratic programming problem optimizes and adjusts the weights of each indicator in the indicator weight set corresponding to the indicator historical result set by minimizing the mean square error to obtain a quadratic programming model. The quadratic programming model is used to optimize and adjust the weights of each indicator in the indicator weight set corresponding to the indicator historical result set to obtain new weights of each indicator in the indicator weight set corresponding to the indicator historical result set; An indicator monitoring result set acquisition module is used to acquire an indicator monitoring result set; the indicator monitoring result set includes monitoring indicator results of each of the indicators; the units of monitoring indicator results of different indicators are different; A monitoring indicator result conversion module is used to convert the monitoring indicator results of different units into percentage scores by using the indicator scoring model, and obtain the percentage scores corresponding to the monitoring indicator results of each indicator; An indicator weight calculation module, used to calculate the indicator weight set corresponding to the indicator monitoring result set using the hierarchical analysis model; An indicator weight optimization and adjustment module is used to optimize and adjust the weights of each indicator in the indicator weight set corresponding to the indicator monitoring result set by using the quadratic programming model to obtain new weights of each indicator in the indicator weight set corresponding to the indicator monitoring result set; An indicator score calculation module, for each of the indicators, multiplying the percentage score corresponding to the monitoring indicator result of the indicator by the new weight of the indicator in the indicator weight set corresponding to the indicator monitoring result set, to obtain the score of each of the indicators; The evaluation score calculation module is used to add up the scores of the indicators to obtain a score for evaluating the quality of the network application.

6. The network application quality assessment device according to claim 5, characterized in that: The indicator scoring model is obtained by sorting all historical indicator results of each indicator from small to large, and converting the historical indicator results of the indicator into a percentage score according to the percentage position of the historical indicator results of the indicator in the sorting.

7. The network application quality assessment device according to claim 5, characterized in that: An L2 regularization strategy is introduced into the quadratic programming problem; the quadratic programming problem is solved using an OSQP algorithm; and an augmented Lagrange function with a quadratic penalty term is introduced in the process of solving the quadratic programming problem.

Images