Network quality evaluation method and device, electronic equipment and storage medium

Abstract

The application provides a network quality evaluation method and device, electronic equipment and storage medium, and belongs to the technical field of communication, the method comprises the following steps: based on service experience index, the characteristic vector combination of network quality index is carried out; the service experience index is regarded as input data, a preset decision tree algorithm is executed, a decision tree is obtained, and the decision tree is regarded as a prediction model; the residual error corresponding to the service experience index is regarded as input data, a preset decision tree algorithm is executed, a new decision tree is obtained, and the prediction model is updated based on the new decision tree; the step of returning to the residual error corresponding to the service experience index as input data, executing a preset decision tree algorithm, obtaining a new decision tree, and updating the prediction model based on the new decision tree is repeated until a preset stop condition is reached; and the network quality is predicted based on the final prediction model. According to the application, network quality is evaluated according to multi-source data, and the integrated method of multiple decision trees is used, so that the network quality evaluation accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a network quality assessment method, apparatus, electronic device, and storage medium. Background Technology

[0002] Private networks (VNetworks) are dedicated networks customized for specific scenarios such as enterprises, industrial parks, and factories, aiming to provide highly reliable, low-latency, and high-bandwidth communication services. Private Network Quality Assessment (PRA) is a crucial technology for evaluating and optimizing VNet performance. RPA is a key step in ensuring VNet performance and user experience. Regularly assessing network quality allows for timely identification and resolution of problems, optimization of network configuration, and improvement of network reliability and stability.

[0003] Currently, private network quality assessment primarily involves aggregating and analyzing real-time signaling plane and user plane data to construct a three-layer general KPI perception evaluation model, encompassing the network support layer, general performance layer, and service scenario layer. A weighted value calculation method is then used to quantitatively evaluate different service categories and the overall network communication quality. However, this method only considers Deep Packet Inspection (DPI) data, limiting data sources. Furthermore, simply using weighted calculations cannot accurately and effectively assess users' perceived experience with private network services.

[0004] Therefore, there is an urgent need for a network quality assessment method to address the problems of limited data sources and inaccurate business perception assessment in existing private network quality assessment schemes. Summary of the Invention

[0005] This invention provides a network quality assessment method, apparatus, electronic device, and storage medium to address the shortcomings of existing private network quality assessment schemes, such as limited data sources and inaccurate network quality assessment.

[0006] This invention provides a network quality assessment method, comprising the following steps:

[0007] Based on the service experience metrics, the network quality metrics related to the service experience are combined with feature vectors in sequence to obtain the target matrix. The service experience metrics are the metric data that reflects the user experience, and the network quality metrics are the metric data that reflects the network quality that are determined based on deep packet inspection data.

[0008] Using the target matrix as input data, a preset decision tree algorithm is executed to obtain a first decision tree. The preset decision tree algorithm is an algorithm used to generate decision trees.

[0009] The residual corresponding to the target matrix is ​​used as input data. The preset decision tree algorithm is executed to obtain a second decision tree. The first decision tree is then updated based on the second decision tree. The residual is the difference between the business experience index and the loss value. The loss value is calculated based on the prediction result and the actual result of the prediction model.

[0010] After determining the new residual based on the first decision tree, the first decision tree is iteratively updated by the second decision tree until the preset stopping condition is reached.

[0011] Based on the iteratively updated first decision tree, predict business quality.

[0012] According to a network quality assessment method provided by the present invention, the execution steps of the preset decision tree algorithm specifically include:

[0013] Use the first network quality indicator as the current network quality indicator;

[0014] For the input data, it is segmented based on the current network quality index to obtain a first region and a second region, so as to minimize the loss function value. The loss function value is determined based on the sum of the mean squared error of the first region and the mean squared error of the second region.

[0015] The next network quality indicator is taken as the new current network quality indicator, and the process returns to the step of segmenting based on the current network quality indicator to obtain the first region and the second region, until the last network quality indicator is processed to obtain the decision tree.

[0016] According to a network quality assessment method provided by the present invention, the network quality indicators related to service experience are determined based on the following steps:

[0017] For service experience metrics and network quality metrics, the correlation coefficients are calculated pairwise to obtain... The correlation matrix is ​​defined as follows: the service experience index includes service experience sampling data for n periods, and the network quality index includes network quality sampling data for the corresponding n periods, where n is a positive integer.

[0018] Based on the correlation coefficients that have a mapping relationship, the correlation coefficients that do not have a mapping relationship are corrected and calculated. The service experience sampling data of the i-th period has a mapping relationship with the network quality sampling data of the i-th period, where i is a positive integer and 0 < i ≤ n.

[0019] If the corrected correlation coefficient is greater than the preset correlation threshold, then the network quality indicator is determined to be a network quality indicator related to service experience.

[0020] According to a network quality assessment method provided by the present invention, the step of correcting correlation coefficients without mapping relationships based on correlation coefficients with existing mapping relationships includes:

[0021] A correlation prediction model is trained based on business experience sampling data and network quality sampling data that have a mapping relationship.

[0022] Using a trained correlation prediction model, predictions are made for business experience sampling data and network quality sampling data that do not have a mapping relationship;

[0023] Based on the prediction results, the corrected correlation coefficient is calculated.

[0024] According to a network quality assessment method provided by the present invention, the service experience indicators include video clarity, number of pauses within a period of time, and picture quality.

[0025] According to a network quality assessment method provided by the present invention, the service experience indicators include operation success indicators, operation failure indicators, and operation perceived latency.

[0026] The present invention also provides a network quality assessment device, comprising the following modules:

[0027] The matrix generation module is used to: based on the service experience indicators, sequentially combine the feature vectors of the network quality indicators related to the service experience to obtain the target matrix. The service experience indicators are indicator data reflecting user experience, and the network quality indicators are indicator data reflecting network quality determined based on deep packet inspection data.

[0028] The decision tree generation module is used to: take the target matrix as input data, execute a preset decision tree algorithm to obtain a first decision tree, wherein the preset decision tree algorithm is an algorithm used to generate decision trees;

[0029] The model update module is used to: take the residual corresponding to the target matrix as input data, execute the preset decision tree algorithm to obtain a second decision tree, and then update the first decision tree based on the second decision tree. The residual is the difference between the business experience index and the loss value, and the loss value is calculated based on the prediction result and the actual result of the prediction model.

[0030] The iteration module is used to: after determining a new residual based on the first decision tree, iteratively update the first decision tree through the second decision tree until a preset stopping condition is reached;

[0031] The network prediction module is used to predict business quality based on the iteratively updated first decision tree.

[0032] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the network quality assessment method as described above.

[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the network quality assessment method as described above.

[0034] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the network quality assessment method as described above.

[0035] The network quality assessment method, apparatus, electronic device, and storage medium provided by this invention, based on service experience indicators, sequentially combines feature vectors of network quality indicators related to service experience to obtain a target matrix. The service experience indicators are data reflecting user experience, and the network quality indicators are data reflecting network quality determined based on deep packet inspection data. Using the target matrix as input data, a preset decision tree algorithm is executed to obtain a first decision tree. The preset decision tree algorithm is used to generate decision trees. Using the residual corresponding to the target matrix as input data, the preset decision tree algorithm is executed again to obtain a second decision tree. The first decision tree is then updated based on the second decision tree. The residual is the difference between the service experience indicator and the loss value, which is calculated based on the prediction result and the actual result of the prediction model. After determining a new residual based on the first decision tree, the first decision tree is iteratively updated using the second decision tree until a preset stopping condition is reached. Service quality is predicted based on the iteratively updated first decision tree. This solution assesses network quality based on service experience metrics that reflect user experience and network quality data that are strongly correlated with service experience data. The diverse data sources enable a more accurate assessment of private network service perception. By using an ensemble method of multiple decision trees, the solution compensates for the bias of a single model, constructing a stable and accurate model for calculating the relationship between service experience and network quality. This improves the model's accuracy and generalization ability, thereby further enhancing the accuracy of network quality assessment. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1This is one of the flowcharts illustrating the network quality assessment method provided by the present invention;

[0038] Figure 2 This is the second flowchart of the network quality assessment method provided by the present invention;

[0039] Figure 3 This is a schematic diagram of the structure of the prediction model provided by the present invention;

[0040] Figure 4 This is a hierarchical diagram illustrating the relationship between network quality indicators and service experience provided by this invention;

[0041] Figure 5 This is a schematic diagram of the mapping and quantization relationship provided by the present invention;

[0042] Figure 6 This is a schematic diagram of the non-mapping quantization relationship correction provided by the present invention;

[0043] Figure 7 This is a schematic diagram of the corrected quantization relationship provided by the present invention;

[0044] Figure 8 This is a schematic diagram of the network quality assessment device provided by the present invention.

[0045] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0047] It should be noted that in the description of the embodiments of the present invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terms "upper," "lower," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Unless otherwise expressly specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly, for example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two elements. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0048] The terms "first," "second," etc., used in this invention are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0049] The following is combined Figures 1-9 This invention describes the network quality assessment method, apparatus, electronic device, and storage medium provided in embodiments of the present invention.

[0050] Figure 1 This is one of the flowcharts illustrating the network quality assessment method provided by this invention, such as... Figure 1 As shown, this network quality assessment method may include the following steps:

[0051] S110, based on service experience metrics, sequentially combine the feature vectors of network quality metrics related to service experience to obtain the target matrix;

[0052] S120, Using the target matrix as input data, execute a preset decision tree algorithm to obtain a first decision tree;

[0053] S130, using the residual corresponding to the target matrix as input data, execute the preset decision tree algorithm to obtain the second decision tree, and then update the first decision tree based on the second decision tree;

[0054] S140, after determining the new residual based on the first decision tree, the first decision tree is iteratively updated by the second decision tree until the preset stopping condition is reached;

[0055] S150, based on the iteratively updated first decision tree, predicts business quality.

[0056] It should be noted that the execution subject of the network quality assessment method provided in this embodiment of the invention can be a server, computer equipment, such as mobile phone, tablet computer, laptop computer, handheld computer, vehicle electronic equipment, wearable device, ultra-mobile personal computer (UMPC), netbook or personal digital assistant (PDA), etc.

[0057] In this embodiment of the invention, the service experience metrics are indicator data that reflect user experience. For example, observable service experience data can be obtained for MEP (Multi-access Edge Computing Platform), which is a series of performance indicators and user experience feedback generated when users use services provided by the MEP platform. For example, it may include high-definition video stuttering, remote control response, etc.

[0058] In this embodiment of the invention, the network quality indicators are network quality indicators determined based on Deep Packet Inspection (DPI) data; the network quality indicators related to service experience are those strongly correlated with service experience.

[0059] In S110, for service experience metrics and related network quality metrics, high-dimensional feature vectors are combined according to a certain category of service experience metrics. For example, assuming that network quality metrics related to service experience include network layer packet size B, packet interval Z, uplink / downlink RTT delay RD, uplink / downlink retransmission rate R, and uplink / downlink transmission pipeline speed U, high-dimensional feature vectors are combined according to video stuttering conditions to obtain the target matrix L. R This serves as the input data for training the entire model:

[0060] .

[0061] In S120, the preset decision tree algorithm is an algorithm used to generate a decision tree, which includes business experience indicators in the target matrix.

[0062] In S130, the residual is the difference between the business experience metric and the predicted value, and the predicted value is the prediction result of the first decision tree.

[0063] In S140, a new residual is calculated based on the updated first decision tree. The new residual is used as input data to execute the preset decision tree algorithm to obtain a new second decision tree. The first decision tree is then updated again based on the new second decision tree, and so on, until the preset iteration stopping condition is reached (such as reaching the number of iterations or the residual being less than a preset threshold). The resulting first decision tree is the final prediction model.

[0064] Figure 3 This is a schematic diagram of the predictive model provided by the present invention, as shown below. Figure 3 As shown, when a business experience sample is input, the prediction model starts from the root node, selects the corresponding path based on the feature judgment conditions corresponding to each node, and constructs the calculation relationship model between business experience indicators and network quality indicators by using the difference of multiple decision trees to make up for the calculation.

[0065] In S150, based on the first decision tree obtained from the final iteration update, the mapping of network quality indicators to service experience indicators is realized. Based on the network quality data that can be monitored and obtained in the private network, the service quality of the private network is evaluated, and the service perception of users when using the private network is quantified.

[0066] The network quality assessment method provided in this embodiment of the invention, based on service experience indicators, sequentially combines feature vectors of network quality indicators related to service experience to obtain a target matrix. The service experience indicators are data reflecting user experience, and the network quality indicators are data reflecting network quality determined based on deep packet inspection data. Using the target matrix as input data, a preset decision tree algorithm is executed to obtain a first decision tree. The preset decision tree algorithm is used to generate decision trees. Using the residual corresponding to the target matrix as input data, the preset decision tree algorithm is executed again to obtain a second decision tree. The first decision tree is then updated based on the second decision tree. The residual is the difference between the service experience indicator and the loss value, which is calculated based on the prediction result and the actual result of the prediction model. After determining a new residual based on the first decision tree, the first decision tree is iteratively updated using the second decision tree until a preset stopping condition is reached. Service quality is predicted based on the iteratively updated first decision tree. This solution assesses network quality based on service experience metrics that reflect user experience and network quality data that are strongly correlated with service experience data. The diverse data sources enable a more accurate assessment of private network service perception. By using an ensemble method of multiple decision trees, the solution compensates for the bias of a single model, constructing a stable and accurate model for calculating the relationship between service experience and network quality. This improves the model's accuracy and generalization ability, thereby further enhancing the accuracy of network quality assessment.

[0067] In an optional embodiment, the execution steps of the preset decision tree algorithm specifically include:

[0068] Use the first network quality indicator as the current network quality indicator;

[0069] For the input data, it is segmented based on the current network quality index to obtain a first region and a second region, so as to minimize the loss function value. The loss function value is determined based on the sum of the mean squared error of the first region and the mean squared error of the second region.

[0070] The next network quality indicator is taken as the new current network quality indicator, and the process returns to the step of segmenting based on the current network quality indicator to obtain the first region and the second region, until the last network quality indicator is processed to obtain the decision tree.

[0071] Specifically, for a certain service experience metric feature j (the sampled specific value of service experience), the optimal splitting point s is sought for the relevant network quality data. That is, after the network quality metric is split by s, the mean square error of the service experience metric values ​​of the two regions R1 and R2 is minimized.

[0072] ;

[0073] in, It is the i-th network quality metric value and the i-th service experience metric value. It is the mean of the network quality index for region R1. The mean network quality index for region R2;

[0074] Subsequently, the optimal addressing is further performed based on gradient descent hierarchical segmentation. Regions R1 and R2 are segmented based on the next network quality index, and so on, until the last layer, so as to identify the minimum interval relationship between a certain service experience index value and the network quality index value on the gradient decision tree.

[0075] To facilitate understanding, the decision tree algorithm provided in this embodiment of the invention will be explained below with specific examples:

[0076] Suppose that the sample values ​​of the service experience index stuttering frequency are combined into a vector according to the relevant indicators (packet size, packet interval, RTT latency, retransmission rate and transport layer rate).

[0077] Using packet size as the current network quality indicator, the frequency of stuttering is divided into two groups based on the packet size segmentation point. Group R1 consists of packets larger than a certain value [y, +∞] and group R2 consists of packets smaller than the value [-∞, y]. This minimizes the dispersion of stuttering frequency in each group, i.e., minimizes the mean square error.

[0078] The next network quality metric, packet interval, is used as the new current network quality metric. Further, R1 is divided according to packet interval, with regions less than a certain value [y, y1] forming one group R11 and regions greater than that value [y1, +∞] forming another group R12, minimizing the frequency dispersion of stuttering in R11 and R12. Similarly, R2 is divided according to packet interval to obtain regions R21 and R22, minimizing the frequency dispersion of stuttering in R21 and R22.

[0079] By analogy, the data is segmented based on RTT latency, retransmission rate, and transport layer rate until the last layer of processing is completed, i.e., until the segmentation is completed based on the transport layer rate, resulting in a decision tree that realizes the mapping relationship between network quality indicators and video stuttering frequency.

[0080] The network quality assessment method provided in this invention segmentes service experience feature vectors with relevant service quality indicators by region segmentation, and continues to find the optimal address based on gradient descent hierarchical segmentation. This enables the identification of the minimum interval relationship between a service experience indicator value and a network quality indicator value on a gradient decision tree, providing a basis for subsequent network quality assessment.

[0081] In an optional embodiment, the service experience metrics include video clarity, number of pauses within a period of time, and picture quality.

[0082] In this embodiment of the invention, for video surveillance services, which are continuous services, a fixed configurable time period T is set for matrix quantization of the viewing experience L. Viewing experience sampling data is generated based on video resolution M, the number of pauses K within the period, and image quality H (such as screen tearing).

[0083] .

[0084] In an optional embodiment, the service experience metrics include a successful operation indicator, an operation failure indicator, and a perceived operation latency.

[0085] In this embodiment of the invention, for interactive control-type services, which are event-triggered services, the matrix quantization of the interactive experience C is based on the period of abnormal event triggering, and forms interactive experience sampling data according to the control success S flag, operation failure F flag, and control perception delay D:

[0086] .

[0087] In an optional embodiment, the network quality metrics related to service experience are determined based on the following steps:

[0088] For service experience metrics and network quality metrics, the correlation coefficients are calculated pairwise to obtain... The correlation matrix is ​​defined as follows: the service experience index includes service experience sampling data for n periods, and the network quality index includes network quality sampling data for the corresponding n periods, where n is a positive integer.

[0089] Based on the correlation coefficients that have a mapping relationship, the correlation coefficients that do not have a mapping relationship are corrected and calculated. The service experience sampling data of the i-th period has a mapping relationship with the network quality sampling data of the i-th period, where i is a positive integer and 0 < i ≤ n.

[0090] If the corrected correlation coefficient is greater than the preset correlation threshold, then the network quality indicator is determined to be a network quality indicator related to service experience.

[0091] In this embodiment of the invention, for the DPI data collected from the 5G private network user plane (N3), according to time, terminal information, etc., and associated with XDR (eXtended Data Record) call detail records and raw bitstream, the network layer packet number N, packet size B, and packet interval Z corresponding to the service experience data can be obtained. The uplink and downlink TCP connection establishment delay CD, uplink and downlink RTT delay RD, uplink and downlink retransmission rate R, uplink and downlink out-of-order rate O, and the uplink and downlink transmission pipe speed U corresponding to the service experience data can also be obtained, forming the network quality sampling data corresponding to the service.

[0092] .

[0093] The business experience sampling data and network quality sampling data from different dimensions are compared and correlated pairwise using a pre-defined correlation coefficient calculation method (such as the Pearson coefficient), forming a correlation matrix as shown in Table 1.

[0094] Table 1 Correlation Matrix

[0095]

[0096] Here, after calculating the correlation matrix, the correlation quantification results with a mapping relationship (i.e., the correlation coefficient Correl(L) in the diagonal of the matrix) are used. ti C i , P i The correlation coefficients (i.e., the correlation coefficients at non-diagonal positions in the matrix) are corrected and calculated with the correlation results that do not have a mapping relationship. Finally, the network quality indicators that are strongly correlated with the business experience are determined based on the corrected correlation coefficients.

[0097] Optionally, based on the correlation coefficient, the relationship between network quality indicators and service experience can be divided into multiple levels, and the network quality indicators with preset levels are determined to be network quality indicators related to service experience.

[0098] Figure 4 This is a hierarchical diagram illustrating the relationship between network quality indicators and service experience provided by this invention, such as... Figure 4 As shown, the association is divided into 9 levels from Class-4 to Class4. Among them, Class0 is no concern, Class1, Class2, Class-1, and Class-2 are weakly related, and Class3, Class4, Class-3, and Class-4 are strongly related.

[0099] Figure 5 This is a schematic diagram of the mapping and quantization relationship provided by the present invention;

[0100] Figure 6 This is a schematic diagram of the non-mapping quantization relationship correction provided by the present invention;

[0101] Figure 7 This is a schematic diagram of the corrected quantization relationship provided by the present invention;

[0102] like Figures 5-7 As shown, the correlation coefficient between uplink RTT latency and video transmission rate after correction is 0.42, which is greater than the preset correlation threshold (0.4), thus confirming that uplink RTT latency is a network quality indicator related to service experience.

[0103] The network quality assessment method provided in this invention, based on network quality data provided by DPI and service-aware data from MEP (such as lag, screen flickering, control plane protocol success rate, and control protocol response latency), quickly and accurately finds network quality data strongly correlated with service experience data through a correctable Pearson algorithm. This avoids subsequent network quality assessments based on irrelevant or weakly correlated data, thereby reducing the computational load of subsequent network quality assessments and improving the accuracy of network quality assessments.

[0104] In an optional embodiment, the step of performing correction calculations on correlation coefficients without mapping relationships based on correlation coefficients with existing mapping relationships includes:

[0105] A correlation prediction model is trained based on business experience sampling data and network quality sampling data that have a mapping relationship.

[0106] Using a trained correlation prediction model, predictions are made for business experience sampling data and network quality sampling data that do not have a mapping relationship;

[0107] Based on the prediction results, the corrected correlation coefficient is calculated.

[0108] Specifically, data with existing mapping relationships are used as a benchmark, and data without such relationships are corrected. Correction methods can include linear regression, scaling adjustments, etc. Taking linear regression as an example, a linear regression model is trained using business experience sampling data and network quality sampling data with existing mapping relationships. The trained linear regression model is then used to predict data without mapping relationships, and the correlation coefficient is corrected based on the prediction results.

[0109] The network quality assessment method provided in this invention effectively utilizes data with existing mapping relationships to perform correction calculations on data without mapping relationships, thereby improving the accuracy and reliability of the data and providing reliable data basis for subsequently determining network quality indicators that are related to service experience.

[0110] Figure 2 This is the second flowchart of the network quality assessment method provided by this invention. The following is in conjunction with... Figure 2 The preferred network quality assessment method flow of the present invention will be described.

[0111] The service experience data of MEP (Mobile Experience Program) is matrix-quantized to form service experience data labels. Network DPI (Distributed Performance Index) data and network element performance data during service transmission are quantitatively analyzed to form network quality label data. These two types of data are then correlated using a correlation algorithm to obtain network quality data strongly correlated with the service experience data. Based on this correlation, a decision tree algorithm is used to train the mapping relationship between service experience data and network quality data, resulting in a prediction model for both. For the real-time collected and parsed network quality data, combined with the prediction model, the service experience data for the entire network is derived. This, along with the network quality data, supports the quantitative analysis of network quality for service experience.

[0112] The network quality assessment device provided in the embodiments of the present invention will be described below. The network quality assessment device described below can be referred to in correspondence with the network quality assessment method described above.

[0113] Figure 8 This is a schematic diagram of the network quality assessment device provided by the present invention, as shown below. Figure 8 As shown, the network quality assessment device may include, but is not limited to;

[0114] The matrix generation module 810 is used to: based on the service experience index, sequentially combine the feature vectors of the network quality indexes related to the service experience to obtain the target matrix. The service experience index is the index data reflecting the user experience, and the network quality index is the index data reflecting the network quality determined based on deep packet inspection data.

[0115] The decision tree generation module 820 is used to: take the target matrix as input data, execute a preset decision tree algorithm to obtain a first decision tree, wherein the preset decision tree algorithm is an algorithm used to generate decision trees;

[0116] The model update module 830 is used to: take the residual corresponding to the target matrix as input data, execute the preset decision tree algorithm to obtain a second decision tree, and then update the first decision tree based on the second decision tree. The residual is the difference between the business experience index and the loss value, and the loss value is calculated based on the prediction result and the actual result of the prediction model.

[0117] Iteration module 840 is used to: after determining a new residual based on the first decision tree, iteratively update the first decision tree through the second decision tree until a preset stopping condition is reached;

[0118] The network prediction module 850 is used to predict business quality based on the iteratively updated first decision tree.

[0119] It should be noted that the network quality assessment device provided in this embodiment of the invention can execute the network quality assessment method described in any of the above embodiments during specific operation, and this embodiment will not elaborate on this.

[0120] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include a processor 910, a communications interface 920, a memory 930, and a communication bus 940. The processor 910, communications interface 920, and memory 930 communicate with each other via the communication bus 940. The processor 910 can call logical instructions in the memory 930 to execute a network quality assessment method. This method includes: based on service experience indicators, sequentially combining feature vectors of network quality indicators related to service experience to obtain a target matrix. The service experience indicators are indicator data reflecting user experience, and the network quality indicators are indicator data reflecting network quality determined based on deep packet inspection data.

[0121] Using the target matrix as input data, a preset decision tree algorithm is executed to obtain a first decision tree. The preset decision tree algorithm is an algorithm used to generate decision trees.

[0122] The residual corresponding to the target matrix is ​​used as input data. The preset decision tree algorithm is executed to obtain a second decision tree. The first decision tree is then updated based on the second decision tree. The residual is the difference between the business experience index and the loss value. The loss value is calculated based on the prediction result and the actual result of the prediction model.

[0123] After determining the new residual based on the first decision tree, the first decision tree is iteratively updated by the second decision tree until the preset stopping condition is reached.

[0124] Based on the iteratively updated first decision tree, predict business quality.

[0125] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, the computer program being executed by a processor, the computer being able to execute the network quality assessment method provided by the above methods, the method including: based on service experience indicators, sequentially combining feature vectors of network quality indicators related to service experience to obtain a target matrix, the service experience indicators being indicator data reflecting user experience, the network quality indicators being indicator data reflecting network quality determined based on deep packet inspection data;

[0127] Using the target matrix as input data, a preset decision tree algorithm is executed to obtain a first decision tree. The preset decision tree algorithm is an algorithm used to generate decision trees.

[0128] The residual corresponding to the target matrix is ​​used as input data. The preset decision tree algorithm is executed to obtain a second decision tree. The first decision tree is then updated based on the second decision tree. The residual is the difference between the business experience index and the loss value. The loss value is calculated based on the prediction result and the actual result of the prediction model.

[0129] After determining the new residual based on the first decision tree, the first decision tree is iteratively updated by the second decision tree until the preset stopping condition is reached.

[0130] Based on the iteratively updated first decision tree, predict business quality.

[0131] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the network quality assessment method provided by the above methods. The method includes: based on service experience indicators, sequentially combining feature vectors of network quality indicators related to service experience to obtain a target matrix, wherein the service experience indicators are indicator data reflecting user experience, and the network quality indicators are indicator data reflecting network quality determined based on deep packet inspection data.

[0132] Using the target matrix as input data, a preset decision tree algorithm is executed to obtain a first decision tree. The preset decision tree algorithm is an algorithm used to generate decision trees.

[0133] The residual corresponding to the target matrix is ​​used as input data. The preset decision tree algorithm is executed to obtain a second decision tree. The first decision tree is then updated based on the second decision tree. The residual is the difference between the business experience index and the loss value. The loss value is calculated based on the prediction result and the actual result of the prediction model.

[0134] After determining the new residual based on the first decision tree, the first decision tree is iteratively updated by the second decision tree until the preset stopping condition is reached.

[0135] Based on the iteratively updated first decision tree, predict business quality.

[0136] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A network quality assessment method, characterized in that, include: Based on the service experience metrics, the network quality metrics related to the service experience are combined with feature vectors in sequence to obtain the target matrix. The service experience metrics are the metric data that reflects the user experience, and the network quality metrics are the metric data that reflects the network quality that are determined based on deep packet inspection data. Using the target matrix as input data, a preset decision tree algorithm is executed to obtain a first decision tree. The preset decision tree algorithm is an algorithm used to generate decision trees. Using the residual corresponding to the target matrix as input data, the preset decision tree algorithm is executed to obtain a second decision tree. The first decision tree is then updated based on the second decision tree. The residual is the difference between the business experience indicator and the predicted value, and the predicted value is the prediction result of the first decision tree. After determining a new residual based on the first decision tree, the first decision tree is iteratively updated using the second decision tree until a preset stopping condition is reached. Based on the iteratively updated first decision tree, the business quality is predicted. The network quality metrics related to service experience are determined based on the following steps: For service experience metrics and network quality metrics, correlation coefficients are calculated between each pair to obtain a correlation matrix of dimension 1. The service experience metrics include service experience sampling data for n periods, and the network quality metrics include network quality sampling data for the corresponding n periods, where n is a positive integer. Based on the correlation coefficients that have a mapping relationship, the correlation coefficients that do not have a mapping relationship are corrected and calculated. The service experience sampling data of the i-th period has a mapping relationship with the network quality sampling data of the i-th period, where i is a positive integer and 0 < i ≤ n. If the corrected correlation coefficient is greater than the preset correlation threshold, then the network quality indicator is determined to be a network quality indicator related to service experience.

2. The network quality assessment method according to claim 1, characterized in that, The execution steps of the preset decision tree algorithm specifically include: Use the first network quality indicator as the current network quality indicator; For the input data, it is segmented based on the current network quality index to obtain a first region and a second region, so as to minimize the loss function value. The loss function value is determined based on the sum of the mean squared error of the first region and the mean squared error of the second region. The next network quality indicator is taken as the new current network quality indicator, and the process returns to the step of segmenting based on the current network quality indicator to obtain the first region and the second region, until the last network quality indicator is processed to obtain the decision tree.

3. The network quality assessment method according to claim 1, characterized in that, The step of correcting correlation coefficients that do not have a mapping relationship based on correlation coefficients with existing mapping relationships includes: A correlation prediction model is trained based on business experience sampling data and network quality sampling data that have a mapping relationship. Using a trained correlation prediction model, predictions are made for business experience sampling data and network quality sampling data that do not have a mapping relationship; Based on the prediction results, the corrected correlation coefficient is calculated.

4. The network quality assessment method according to any one of claims 1-3, characterized in that, The service experience metrics include video clarity, number of pauses within a given period, and picture quality.

5. The network quality assessment method according to any one of claims 1-3, characterized in that, The business experience metrics include successful operation indicators, failed operation indicators, and perceived operation latency.

6. A network quality assessment device, characterized in that, include: The matrix generation module is used to: based on the service experience indicators, sequentially combine the feature vectors of the network quality indicators related to the service experience to obtain the target matrix. The service experience indicators are indicator data reflecting user experience, and the network quality indicators are indicator data reflecting network quality determined based on deep packet inspection data. The decision tree generation module is used to: take the target matrix as input data, execute a preset decision tree algorithm to obtain a first decision tree, wherein the preset decision tree algorithm is an algorithm used to generate decision trees; The model update module is used to: take the residual corresponding to the target matrix as input data, execute the preset decision tree algorithm to obtain a second decision tree, and then update the first decision tree based on the second decision tree. The residual is the difference between the business experience index and the predicted value, and the predicted value is the prediction result of the first decision tree. The iteration module is used to: after determining a new residual based on the first decision tree, iteratively update the first decision tree through the second decision tree until a preset stopping condition is reached; The network prediction module is used to predict service quality based on the iteratively updated first decision tree. The network quality metrics related to service experience are determined based on the following steps: For service experience metrics and network quality metrics, correlation coefficients are calculated between each pair to obtain a correlation matrix of dimension 1. The service experience metrics include service experience sampling data for n periods, and the network quality metrics include network quality sampling data for the corresponding n periods, where n is a positive integer. Based on the correlation coefficients that have a mapping relationship, the correlation coefficients that do not have a mapping relationship are corrected and calculated. The service experience sampling data of the i-th period has a mapping relationship with the network quality sampling data of the i-th period, where i is a positive integer and 0 < i ≤ n. If the corrected correlation coefficient is greater than the preset correlation threshold, then the network quality indicator is determined to be a network quality indicator related to service experience.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the network quality assessment method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the network quality assessment method as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the network quality assessment method as described in any one of claims 1 to 5.

Images