Detecting network-induced contributions to quality of experience degradation

Abstract

The present disclosure relates to monitoring of network traffic between one or more users and one or more Web applications. In particular, the present disclosure relates to detection and evaluation of network-induced quality of experience (QoE) degradation of the users. Therefore, the present disclosure provides a device for detecting network-induced contributions to the QoE degradation. The device is used to: obtain one or more user metrics, the user metrics indicating the QoE of one or more users of the Web application; obtain one or more content metrics, the content metrics indicating the content of the Web application. The device is also used to: determine the QoE degradation of the QoE of the user based on the one or more user metrics; determine contribution information indicating the network-induced contribution to the determined QoE degradation based on the content metrics.

Description

Technical Field

[0001] The present disclosure generally relates to monitoring of network traffic, and in particular to monitoring of web traffic between users and web applications. The present disclosure specifically relates to detection and evaluation of network-induced quality of experience (QoE) degradation of users. Accordingly, the present disclosure provides an apparatus and method for detecting network-induced contributions to a determined QoE degradation of a user. To this end, an apparatus may be used to identify network-induced contributions to a determined QoE degradation and distinguish them from content-induced contributions caused by content changes of a web application. Background Art

[0002] Inference of the QoE of one or more users and detection of its degradation is an important management tool for Internet service providers (ISPs). There are many objective in-browser metrics, and it is feasible to monitor them based on the encrypted web traffic between users and web applications. Therefore, web traffic corresponds to the amount of data sent or received by users to / from web applications.

[0003] However, QoE degradation may be rooted in different sources. For example, QoE degradation may be rooted in network-induced problems, i.e., problems related to the network that affect Web traffic and thus cause QoE degradation. Alternatively, QoE degradation may simply be caused by changes in the content itself.

[0004] Some related research in this field involves the definition of QoE and its inference. In particular, some conventional techniques define Web QoE metrics related to QoE performance so that Web QoE metrics can be inferred without access to browser metrics. QoE inference can be done based on encrypted network traffic, which is what ISPs typically access.

[0005] Some other studies involve Web QoE monitoring. For example, browser studies often correlate QoE changes with changes in the software infrastructure.

[0006] Some other studies deal more specifically with QoE degradation, especially the detection of QoE degradation for users. The focus on QoE degradation from the ISP perspective was first proposed to handle general applications, then to handle video applications, and more recently to handle Web applications.

[0007] Some other research involves Web content monitoring. Given that the Web has changed a lot in the past three decades, the industry is working hard to monitor its evolution, especially from the perspective of web page characteristics, technology, or content. Academics also regularly perform longitudinal investigations on content changes using large datasets, but QoE is usually not included in these studies. Summary of the invention

[0008] The present disclosure is based on the fact that traditional research focuses on either QoE or content, but rarely considers these two aspects together. In addition, even if these aspects are considered together, only the edge is considered, and the time aspect is not considered. That is, when evaluating QoE, information from content evolution is not considered.

[0009] For example, QoE degradation is traditionally identified by linking network key performance indicator (KPI) characteristics with application-specific metrics representing user QoE (also known as user KQI). However, this ignores the impact of content changes that often occur in web applications. For example, when a web page of a web application changes, the KQI metric may also change (e.g., when the structure of a web page becomes more complex with more objects, it may take longer to download the web page, but this may not be related to any network limitations or reduced web traffic). Therefore, traditional methods that ignore content, especially content changes, are unreliable in the case of web applications and will result in many false alarms.

[0010] In view of the above problems and shortcomings, embodiments of the present invention aim to improve the detection of QoE degradation. In particular, the purpose is to achieve joint monitoring of QoE and Web application content in order to determine the network-induced contribution to the QoE degradation of a user or set of users of the Web application.

[0011] This object is achieved by the embodiments of the invention described in the attached independent claims. Advantageous implementations of the embodiments of the invention are also defined in the dependent claims.

[0012] Embodiments of the present invention provide a device and method for detecting QoE degradation of users of Web applications, in which not only the network level is checked, but also content-related information is considered. To this end, according to the device or method of an embodiment of the present invention, active measurement can be used to access content-related information. Embodiments of the present invention solve two problems. First, the problem of identifying QoE degradation caused by network problems (ISP is responsible) and distinguishing it from obvious QoE degradation caused by content-based changes (ISP is not responsible). Second, the problem of combining different network and content signals to provide clues about the most likely source of the determined QoE degradation.

[0013] A first aspect of the present disclosure provides a device for detecting a network-induced contribution to QoE degradation of one or more Web applications, the device being configured to: obtain one or more user metrics, the user metrics indicating the QoE of one or more users of the Web application; obtain one or more content metrics, the content metrics indicating the content of the Web application; determine the QoE degradation of the QoE of the user according to the one or more user metrics; and determine contribution information indicating the network-induced contribution to the determined QoE degradation according to the content metrics.

[0014] The device of the first aspect is capable of considering content, in particular content changes of web applications, when detecting QoE degradation of a user. For example, the device may passively monitor one or more user metrics (the user metrics may include one or more quality of service (QoS) metrics), and may also perform active measurements separately to track changes in the content of one or more selected web applications, i.e., determine content metrics.

[0015] The device may detect QoE degradation based on user metrics. One or more user metrics may be closely related to the QoE of one or more users. User metrics may be related to one or more users (rather than purely network-related metrics) and may be (passively) accessible even when web traffic is encrypted.

[0016] The device may also determine the network-induced QoE contribution to the QoE degradation. For example, the device may distinguish whether the determined QoE degradation is caused by a network-based problem or change and / or caused by a content-based change. This may improve the detection and analysis of the determined QoE degradation. In particular, this may reduce false alarms. The device may also interpret the determined QoE degradation, for example, determining the contribution of the content, the network, or another source to the QoE degradation. This may provide a better understanding of the problems encountered by the user.

[0017] In an implementation manner of the first aspect, the contribution information indicates one of the following information: the network induced contribution is the maximum contribution to the determined QoE degradation; the network induced contribution is not the maximum contribution to the determined QoE degradation; the network induced contribution is the only contribution to the determined QoE degradation; the network induced contribution to the determined QoE degradation is zero.

[0018] The network induced contribution is the contribution that the network (particularly network problems) has to the determined QoE degradation. The determined QoE degradation reflects the change in QoE (and the corresponding change in user metrics) in a specific direction that is considered to indicate worse QoE. That is, the network induced contribution is the contribution for which the ISP can be held responsible.

[0019] According to this implementation, the network induced contribution can be 100% (i.e., the determined QoE degradation is caused only by network problems) or 0% (i.e., the network does not contribute to the QoE degradation). It can also be distinguished whether the network induced contribution is higher than 50% of the total contribution to the determined QoE degradation or equal to or less than 50%. However, the network induced contribution can be the largest contribution even if its contribution is less than 50% (e.g., among the contributions of the network, content, or other reasons).

[0020] In an implementation of the first aspect, the device is further used to: determine whether the QoE degradation corresponds to a change in the content metric; determine the contribution information, wherein, if the QoE degradation does not correspond to a change in the content metric, the contribution information indicates that the network induced contribution is the maximum contribution to the determined QoE degradation.

[0021] In particular, the device may determine that QoE degradation corresponds to a change in one or more content metrics if the QoE degradation is accompanied by a change in one or more content metrics, or if the QoE degradation occurs within a specific time period of the change in one or more content metrics. In general, the temporal relationship between QoE degradation and the change in one or more content metrics may be determined by the device, for example, if one of the two follows the other very quickly (within a specific time period).

[0022] In an implementation of the first aspect, the device is further used to: determine whether the QoE degradation corresponds to a change in the content metric; determine the contribution information, wherein, if the QoE degradation corresponds to a change in the content metric, the contribution information indicates that the network induced contribution is not the maximum contribution to the determined QoE degradation.

[0023] In an implementation manner of the first aspect, the device is further used to: obtain the content metric by measuring the content of the Web application.

[0024] In particular, changes (over time) in the content of one or more web applications may be obtained by the device.The device may perform active measurements to obtain one or more content metrics, for example, from one or more web pages of the web application.

[0025] In an implementation manner of the first aspect, the device is further used to: obtain the user metrics by monitoring network traffic.

[0026] In particular, changes in one or more user metrics (over time) may be obtained by the device, for example, may be collected by the device. The device may perform passive measurements to obtain these user metrics, for example, from encrypted or unencrypted Web traffic.

[0027] In an implementation of the first aspect, the device is further used to, if the contribution information indicates that the network induced contribution is the maximum contribution to the determined QoE degradation, then: obtain one or more network metrics, wherein the network metrics are related to the performance of the network; and determine whether a change in at least one of the network metrics corresponds to the QoE degradation.

[0028] In particular, changes (over time) in one or more network metrics may be obtained by the device, for example, may be collected by the device. The device may perform passive and / or active measurements to obtain these network metrics, for example, from other network elements.

[0029] In an implementation manner of the first aspect, the device is further used to: report at least one of the network metrics corresponding to the QoE degradation; and / or sort the at least one of the network metrics corresponding to the QoE degradation according to a predefined standard.

[0030] In an implementation manner of the first aspect, the device is further configured to: report an unidentified network problem if at least one of the network metrics corresponding to the QoE degradation does not change.

[0031] In an implementation manner of the first aspect, the device is further used to: estimate the value of the network-induced contribution to the QoE degradation; and / or estimate the value of the content-induced contribution to the QoE degradation, wherein the content-induced contribution is caused by a content change of the content of the Web application.

[0032] In an implementation of the first aspect, the device is also used to: obtain one or more original user metrics and / or one or more original content metrics; obtain the user metrics and / or the content metrics by respectively executing a change point detection (CPD) algorithm and / or an outlier detection (OD) algorithm on the original user metrics and / or the original content metrics.

[0033] The raw user metrics may be a time series of measurements of user metrics on a user or user pool of one or more web applications. The raw content metrics may be a time series of measurements of content metrics on a machine or machine pool. One or more processing operations may be performed on the corresponding raw metrics to obtain the corresponding metrics, the processing operations being, for example, processing operations including CPD and OD operations.

[0034] In an implementation manner of the first aspect, the user metrics include at least one of the following: one or more QoS parameters related to the QoE of the user; one or more QoE indicators indicating the QoE of the user; one or more key quality indicators related to the QoE of the user.

[0035] QoS parameters may be associated with one or more Web applications.

[0036] In an implementation of the first aspect, the user metrics include at least one of the following: page loading time of one or more web pages related to the web application; speed index of one or more web pages related to the web application; byte index of one or more web pages related to the web application.

[0037] In an implementation manner of the first aspect, the content metric includes at least one of the following: the number of objects of one or more web pages related to the Web application; and the page weight of one or more web pages related to the Web application.

[0038] A second aspect of the present disclosure provides a method for detecting a network-induced contribution to QoE degradation of one or more Web applications, the method comprising: obtaining one or more user metrics, the user metrics indicating the QoE of one or more users of the Web application; obtaining one or more content metrics, the content metrics indicating the content of the Web application; determining the QoE degradation of the QoE of the user according to the one or more user metrics; and determining contribution information indicating the network-induced contribution to the determined QoE degradation according to the content metrics.

[0039] In an implementation manner of the second aspect, the contribution information indicates one of the following information: the network induced contribution is the maximum contribution to the determined QoE degradation; the network induced contribution is not the maximum contribution to the determined QoE degradation; the network induced contribution is the only contribution to the determined QoE degradation; the network induced contribution to the determined QoE degradation is zero.

[0040] In an implementation of the second aspect, the method further includes: determining whether the QoE degradation corresponds to a change in the content metric; determining the contribution information, wherein, if the QoE degradation does not correspond to a change in the content metric, the contribution information indicates that the network induced contribution is the maximum contribution to the determined QoE degradation.

[0041] In an implementation of the second aspect, the method further includes: determining whether the QoE degradation corresponds to a change in the content metric; determining the contribution information, wherein, if the QoE degradation corresponds to a change in the content metric, the contribution information indicates that the network induced contribution is not the maximum contribution to the determined QoE degradation.

[0042] In an implementation manner of the second aspect, the method further includes: acquiring the content metric by measuring the content of the Web application.

[0043] In an implementation manner of the second aspect, the method further includes: acquiring the user metrics by monitoring network traffic.

[0044] In an implementation of the second aspect, the method further includes, if the contribution information indicates that the network induced contribution is the maximum contribution to the determined QoE degradation, then: obtaining one or more network metrics, wherein the network metrics are related to the performance of the network; and determining whether a change in at least one of the network metrics corresponds to the QoE degradation.

[0045] In an implementation manner of the second aspect, the method further includes: reporting at least one of the network metrics corresponding to the QoE degradation; and / or sorting the at least one of the network metrics corresponding to the QoE degradation according to a predefined criterion.

[0046] In an implementation manner of the second aspect, the method further includes: if at least one of the network metrics corresponding to the QoE degradation does not change, reporting an unidentified network problem.

[0047] In an implementation manner of the second aspect, the method further includes: estimating a value of the network-induced contribution to the QoE degradation; and / or estimating a value of a content-induced contribution to the QoE degradation, wherein the content-induced contribution is caused by a content change of the content of the Web application.

[0048] In an implementation of the second aspect, the method further includes: obtaining one or more original user metrics and / or one or more original content metrics; obtaining the user metrics and / or the content metrics by respectively executing a CPD algorithm and / or an OD algorithm on the original user metrics and / or the original content metrics.

[0049] In an implementation manner of the second aspect, the user metrics include at least one of the following: one or more QoS parameters related to the QoE of the user; one or more QoE indicators indicating the QoE of the user; one or more key quality indicators related to the QoE of the user.

[0050] In an implementation of the second aspect, the user metrics include at least one of the following: page loading time of one or more web pages related to the Web application; speed index of one or more web pages related to the Web application; byte index of one or more web pages related to the Web application.

[0051] In an implementation manner of the second aspect, the content metric includes at least one of the following: the number of objects of one or more web pages related to the Web application; and the page weight of one or more web pages related to the Web application.

[0052] The method of the second aspect and its implementation provides the same advantages as the device of the first aspect and its corresponding implementation.

[0053] A third aspect of the present disclosure provides a computer program, which includes program code. When executed on a computer, the program code is used to execute the method according to the second aspect or any implementation manner thereof.

[0054] A fourth aspect of the present disclosure provides a non-transitory storage medium storing executable program code, which, when executed by a processor, executes the method according to the second aspect or any implementation manner thereof.

[0055] It should be noted that all devices, elements, units and components described in this application can be implemented in software or hardware elements or any type of combination thereof. All steps performed by various entities described in this application and the functions to be performed by various entities described are intended to refer to the corresponding entities for performing the corresponding steps and functions. Although in the description of the following specific embodiments, the specific functions or steps performed by external entities are not reflected in the description of the specific detailed elements of the entities performing the specific steps or functions, it should be clear to the technician that these methods and functions can be implemented in corresponding software or hardware elements or any type of combination thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In conjunction with the accompanying drawings, the following description of specific embodiments will illustrate the above aspects and their implementation methods, wherein:

[0057] Figure 1 A device according to an embodiment of the invention is shown.

[0058] Figure 2 A device according to an embodiment of the invention is shown.

[0059] Figure 3 The method executed by the device according to the embodiment of the present invention is schematically shown.

[0060] Figure 4 The method executed by the device according to the embodiment of the present invention is schematically shown.

[0061] Figure 5 A method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0062] Figure 1 A device 100 according to an embodiment of the present invention is shown. The device 100 is used to detect QoE degradation of one or more users of one or more Web applications. In particular, the device 100 is used to detect the network-induced contribution to the detected QoE degradation, i.e., to determine to what extent the network (particularly network problems) is responsible for the QoE degradation. The device can be any computer or processing device for performing the steps described below.

[0063] The device 100 is used to obtain one or more user metrics 101, wherein the one or more user metrics 101 indicate the QoE of one or more users of one or more Web applications. There is no limitation on how the device 100 obtains these user metrics 101. For example, the device 100 can monitor network traffic (particularly Web traffic related to one or more Web applications) to obtain the one or more user metrics 101. The network traffic can be encrypted or unencrypted. The device 100 can collect the user metrics 101 from the network traffic.

[0064] In addition, the device 100 is used to obtain one or more content metrics 102, wherein the one or more content metrics 102 indicate the content of one or more Web applications. There is no limitation on how the device 100 obtains these content metrics 102. For example, the device 100 can measure the content of one or more Web applications to obtain the one or more content metrics 102. For example, the device 100 can measure the number of objects of one or more web pages related to the one or more Web applications, and / or the page weight of one or more web pages related to the one or more Web applications, to obtain the one or more content metrics 101. Therefore, the one or more content metrics 102 may include the number of objects and / or may include the page weight.

[0065] Furthermore, the device 100 is configured to determine a QoE degradation 103 of the QoE of one or more users based on the one or more user metrics 101. For example, the device 100 may detect a change in the one or more user metrics 101 and may determine that the change is associated with, or indicates a degradation of the QoE of the users associated with the one or more user metrics 101. The QoE degradation 103 of the one or more users may be temporally correlated with the change (e.g., degradation) of the one or more user metrics 101.

[0066] Then, the device 100 is configured to determine contribution information 104, the contribution information indicating the network induced contribution to the determined QoE degradation 103. The device 100 is configured to determine the contribution information 104 based on the one or more content metrics 102. The contribution information 104 may indicate one of the following information: the network induced contribution is the maximum contribution to the determined QoE degradation 103, such as a contribution of more than 50%; the network induced contribution is not the maximum contribution to the determined QoE degradation 103, such as a contribution of less than 50%; the network induced contribution is the only contribution to the determined QoE degradation 103, i.e., a full contribution of 100%; or the network induced contribution to the determined QoE degradation 103 is zero, i.e., a contribution of 0%.

[0067] The device 100 may include a processor or processing circuit (not shown) for performing, carrying out, or initiating various operations of the device 100 described herein. The processing circuit may include hardware and / or the processing circuit may be controlled by software. The hardware may include analog circuits or digital circuits, or both. The digital circuit may include components such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a multi-purpose processor.

[0068] The device 100 may also include a memory circuit that stores one or more instructions that can be executed by the processor or processing circuit (particularly, executed under the control of software). For example, the memory circuit may include a non-transitory storage medium that stores executable software code that, when executed by the processor or processing circuit, causes the device 100 to perform various operations.

[0069] In one embodiment, the processing circuit includes one or more processors and non-transitory memory connected to the one or more processors. The non-transitory memory may carry executable program code that, when executed by the one or more processors, causes the device 100 to perform, conduct, or initiate the operations or methods described herein.

[0070] Based on the above description, according to an embodiment of the present invention, the device 100 can identify the QoE degradation 103 of one or more users as being partially or completely caused by network problems (i.e., identifying that the degradation is network-induced). For example, the device 100 can use active web measurements to resolve apparent problems caused by changes in content. Figure 2 A possible overview of the device 100 is described.

[0071] Figure 2 A device 100 according to an embodiment of the present invention is shown, which is based on Figure 1 Based on the embodiment shown. Figure 1 and Figure 2 The same elements in FIG. 1 are marked with the same reference symbols and are implemented similarly. Figure 2 In particular, a possible overview of the device 100 is shown.

[0072] The device 100 may obtain the following inputs:

[0073] ● The device 100 obtains one or more content metrics 102. For example, the device 100 may include one or more active content monitor (AM) components 202 (as shown in the "content domain" above the horizontal dashed line in the figure), which can automatically and / or periodically monitor the content of one or more Web applications. For example, the device 100 can monitor a target set 201 of web pages of interest (e.g., a set T of web pages among all web pages W of one or more Web applications; for example, the set T may include the top N (e.g., N=1000) frequently accessed web pages).

[0074] ■ In particular, the device 100 may extract one or more raw content metrics 102r from web pages of interest, where these web pages may produce a steady and smooth stream of metric data.

[0075] ■ The device 100 may perform a CPD and / or OD algorithm method 203 on one or more raw content metrics 102r, which may be used initially to simplify the metric data that needs to be subsequently derived and combined. The CPD and / or OD method 203 may perform a CPD and / or OD algorithm method 203 on one or more raw content metrics 102r (e.g., Figure 2 C) into one or more content metrics 102 (e.g., Figure 2Simplified series C').

[0076] ● The device 100 also obtains one or more user metrics 101. For example, the device 100 may include one or more passive network monitor (PM) components 205 (as shown in the "network domain" below the dotted line in the figure), which can be deployed in the network and may include a measurement application for measuring the QoS and / or QoE and / or KQI indicators of the user's Web application session. For example, the device 100 may receive one or more user metrics 101 based on monitoring network traffic, particularly Web traffic of Web applications, at one or more routers 204 of the network. The router 204 may be associated with one or more users of one or more Web applications.

[0077] ■ Thus, the device 100 may use existing and deployed entities or systems as PM components 205, which are used to extract network and application metrics (where content attributes are not available at the network level).

[0078] ■ In particular, the device 100 may extract one or more raw user metrics 101r associated with one or more users.

[0079] ■ The device 100 may perform CPD and / or OD methods 203 on one or more raw user metrics 101r, which may be used to simplify the metric data that needs to be subsequently derived and combined. The CPD and / or OD methods 203 may perform CPD and / or OD methods 203 on one or more raw user metrics 101r (e.g., Figure 2 The quality series Q in the formula (Q) is converted into one or more user metrics 101 (e.g., Figure 2 The simplified series Q' in .

[0080] The device 100 also processes the input as follows:

[0081] The device 100 may be used to collect, process and / or combine acquired raw user metrics 101r and raw content metrics 102r (eg, Figure 2 ), or the transformed user metrics 101 and content metrics 102 (e.g., Figure 2 For example, the device 100 can be used to apply a combination function f(C',Q') to these metrics, such as Figure 2 As shown in box 206 in .

[0082] ●The device 100 may then be configured to perform a method 207 to process the changes in the content metric 102 and the user metric 101, respectively, to filter the determined QoE degradations 103. Thus, the determined QoE degradations 103 are determined based on one or more user metrics 101. In particular, the device 100 is configured to determine contribution information 104 indicating a network-induced contribution to each determined QoE degradation 103. According to this, the device may also be configured to filter the determined QoE degradations 103, for example, for a subset of the QoE degradations 103 that are only caused by the network (e.g., for reducing false positive detections). Furthermore, the device 100 may interpret the relative importance of these determined QoE degradations 103 as follows.

[0083] ■ The device 100 can extract the largest "cause" of any determined QoE degradation 103 among all possible contributions including {content, network, unknown}, and the device 100 can rank and measure their relative proportions.

[0084] ■ Given the determined “causes” among possible contributions of QoE degradation 103 including {content, network, unknown}, the device 100 may further extract the actual components or indicators that cause changes in one or more user metrics 101 in terms of biggest causes, rankings and / or relative proportions.

[0085] Figure 3 The method executed by the device 100 according to an embodiment of the present invention is schematically shown. The device 100 may be as follows: Figure 1 or Figure 2 shown. Figure 3 and Figure 1 and Figure 2 The same elements in FIG. 1 are marked with the same reference symbols and are implemented similarly.

[0086] The method performed by the device 100 may extract the biggest "cause" of QoE degradation 103 among possible contributions including {content, network, unknown}. The method may include comparing one or more time series of user events with content events, as reflected in one or more user metrics 101 and one or more content metrics 102, respectively. For simplicity, reference is made to Figure 3 , the basic idea of ​​the method performed by the device 100 is explained using a simple high-level heuristic following a tree model, where binary logic can construct a full truth table.

[0087] Device 100 may be used to track QoE metrics (i.e., user metrics 101) of a user or user pool of a Web application. User metrics 101 may, for example, include PLT, such as Figure 3 "U" shown.

[0088] The device 100 may determine QoE degradation 103 based on the user metrics 101. When no QoE degradation 103 is observed, the device 100 may determine that there is no QoE problem and provide a corresponding output 301. When the device 100 determines QoE degradation 103 for this user or user pool, the device 100 may continue to check the content of the web application.

[0089] In particular, the device 100 may track content metrics 102, such as the number of objects of a web page of a web application, such as Figure 3 "C" shown. When the device 100 determines from the content metric 102 that the change in content corresponds to (e.g., is accompanied by) the QoE degradation 103 determined from the user metric 102, the device 100 can determine that there is no QoE problem and the QoE degradation is caused by the content change of the web application. The device 100 can provide a corresponding output 302. When the device 100 determines that no content change is observed at the same time, the device can issue a network problem alarm. In addition, the device 100 can provide contribution information 104, which indicates the network-induced contribution to the determined QoE degradation 103.

[0090] QoE degradation 103 may correspond to a specific moment given by an algorithm (usually a CPD algorithm) on a univariate or multivariate time series. The term "concomitant" may correspond to occurring at approximately the same time (with a certain time error range allowed by the device 100), or to a more complex algorithm for detecting concomitant.

[0091] Figure 4 The method performed by the device 100 according to an embodiment of the present invention is schematically shown. Figure 4 Built on Figure 3 The device 100 can be as follows Figure 1 or Figure 2 shown. Figure 3 and Figure 4 The same elements in FIG. 1 are marked with the same reference symbols and are implemented similarly.

[0092] Figure 4 In particular, it is shown that the device 100 can also refine the identification of network problems, for example, by tracking network QoS metrics on the network. When the device 100 issues a network problem, the device 100 can also be used to check the network metrics 401 (such as those tracked by the network) through the network. Figure 4”N” shown). For example, if the contribution information 104 indicates that the network induced contribution is the largest contribution to the determined QoE degradation 103, the device 100 obtains a network metric 401, wherein the network metric 401 is related to the performance of the network. In addition, the device 100 can determine whether a change in the network metric 401 corresponds to the QoE degradation 103. If the change in the network metric 401 corresponds to the QoE degradation 103, for example, they are concomitant, the device 100 can provide an output 403 indicating the problematic network metric 401. The device 100 can also check multiple network metrics 410 in this manner, and can sort the problematic network metrics 401 by importance. Otherwise, the device 100 can provide an output 402 indicating an unidentified network problem.

[0093] Figure 5 A method 500 according to an embodiment of the present invention is shown. The method 500 may be performed by the device 100 and is used to detect a network-induced contribution to a QoE degradation of one or more web applications. The method 500 comprises: step 501 of obtaining one or more user metrics 101 indicating the QoE of one or more users of the web application; step 502 of obtaining one or more content metrics 102 indicating the content of the web application; step 503 of determining a QoE degradation of the user's QoE according to the one or more user metrics 101; and step 504 of determining contribution information indicating a network-induced contribution to the determined QoE degradation 103 according to the content metrics 103.

[0094] Embodiments of the invention in this disclosure provide several advantages, such as:

[0095] ● In addition to the network metrics used to evaluate QoE degradation 103, user metrics 101 are also integrated. In most traditional solutions, QoE degradation is only identified from the perspective of network degradation, which is not enough to comprehensively evaluate user QoE degradation.

[0096] ● Integrates content metrics 102 for evaluating QoE degradation 103. In most conventional solutions, network and user-based metrics are used exclusively to evaluate QoE degradation, which is not enough to confirm that user QoE degradation is related to network degradation, even for web pages with constantly changing content.

[0097] ● Focusing on the QoE degradation caused by network sources that ISPs are interested in, this reduces two types of false alarms: a. QoE degradation caused by content changes; b. Network degradation that does not affect QoE.

[0098] ● Reduced false alarms compared to network-only solutions, and compared to user-only solutions.

[0099] The invention has been described in conjunction with various embodiments and implementations as examples. However, other variations may be understood and implemented by those skilled in the art in practicing the claimed invention, based on a study of the drawings, the present disclosure and the independent claims. In the claims and in the specification, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" does not exclude a plurality. A single element or other unit may fulfil the functions of several entities or items described in the claims. The listing of certain implementations in mutually different dependent claims does not indicate that a combination of these measures cannot be effectively used.

Claims

1. A device (100) for detecting network-induced contributions to quality of experience (QoE) degradation of one or more web applications, characterized in that The device (100) comprises: Obtaining one or more user metrics (101), the user metrics (101) indicating units of QoE of one or more users of the Web application; Means for obtaining one or more content metrics (102), the content metrics (102) indicating content of the web application; means for determining a QoE degradation (103) of the QoE of the user based on the one or more user metrics (101); After obtaining one or more content metrics (102) and determining a QoE degradation (103) of the QoE of the user, means for determining contribution information (104) indicating a network induced contribution to the determined QoE degradation (103) based on the content metrics (102).

2. The device (100) according to claim 1, characterized in that The contribution information (104) indicates one of the following information: The network induced contribution is the maximum contribution to the determined QoE degradation (103); The network induced contribution is not the maximum contribution to the determined QoE degradation (103); The network induced contribution is the only contribution to the determined QoE degradation (103); The network induced contribution to the determined QoE degradation (103) is zero.

3. The device (100) according to claim 1 or 2, characterized in that Also includes: means for determining whether said QoE degradation (103) corresponds to a change in said content metric (102); Means for determining the contribution information (104), wherein if the QoE degradation (103) does not correspond to a change in the content metric (102), the contribution information (104) indicates that the network induced contribution is the largest contribution to the determined QoE degradation (103).

4. The device (100) according to claim 1 or 2, characterized in that Also includes: means for determining whether said QoE degradation (103) corresponds to a change in said content metric (102); Means for determining the contribution information (104), wherein if the QoE degradation (103) corresponds to a change in the content metric (102), the contribution information (104) indicates that the network induced contribution is not a maximum contribution to the determined QoE degradation (103).

5. The device (100) according to claim 1 or 2, characterized in that Also includes: A unit for obtaining the content metrics (102) by measuring the content of the Web application.

6. The device (100) according to claim 1 or 2, characterized in that Also includes: A unit for obtaining the user metrics (101) by monitoring network traffic.

7. The device (100) according to claim 1 or 2, characterized in that The method further comprises, if the contribution information (104) indicates that the network induced contribution is the largest contribution to the determined QoE degradation (103), then: A unit for obtaining one or more network metrics (401), wherein the network metrics (401) are related to the performance of the network; A determination is made as to whether a change in at least one of the network metrics (401) corresponds to a unit of QoE degradation (103).

8. The device (100) according to claim 7, characterized in that Also includes: a unit for reporting said at least one of said network metrics (401) corresponding to said QoE degradation (103); and / or A unit for ranking said at least one of said network metrics (401) corresponding to said QoE degradation (103) according to a predefined criterion.

9. The device (100) according to claim 7, characterized in that Also includes: A means for reporting an unidentified network problem (402) if at least one of the network metrics (401) corresponding to the QoE degradation (103) has not changed.

10. The device (100) according to claim 1 or 2, characterized in that Also includes: means for estimating a value of said network induced contribution to said QoE degradation (103); and / or A means for estimating a value of a content-induced contribution to the QoE degradation (103), wherein the content-induced contribution is caused by a content change of the content of the web application.

11. The device (100) according to claim 1 or 2, characterized in that Also includes: A means for acquiring one or more raw user metrics (101r) and / or one or more raw content metrics (102r); A unit for obtaining the user metrics (101) and / or the content metrics (102) by executing a change point detection algorithm and / or an outlier detection algorithm (203) on the original user metrics (101r) and / or the original content metrics (102r), respectively.

12. The device (100) according to claim 1 or 2, characterized in that The user metrics (101) include at least one of the following: one or more quality of service parameters related to the QoE of the user; one or more QoE indicators indicative of the QoE of the user; One or more key quality indicators related to the QoE of the user.

13. The device (100) according to claim 1 or 2, characterized in that The user metrics (101) include at least one of the following: a page loading time of one or more web pages (201) associated with the web application; A speed index of one or more web pages (201) associated with the web application; A byte index of one or more web pages (201) associated with the web application.

14. The device (100) according to claim 1 or 2, characterized in that The content metrics (102) include at least one of the following: the number of objects of one or more web pages (201) associated with the web application; The page weight of one or more web pages (201) related to the web application.

15. A method (500) for detecting network-induced contributions to quality of experience (QoE) degradation of one or more web applications, characterized in that: The method (500) comprises: Obtaining (501) one or more user metrics (101) indicating QoE of one or more users of the Web application; Obtaining (502) one or more content metrics (102), the content metrics (102) indicating content of the Web application; determining (503) a QoE degradation (103) of the QoE of the user based on the one or more user metrics (101); After obtaining one or more content metrics (102) and determining a QoE degradation (103) of the QoE of the user, contribution information (104) indicating a network-induced contribution to the determined QoE degradation (103) is determined (504) based on the content metrics (102).

16. A computer program product, characterized in that It comprises a program code for executing the method (500) according to claim 15 when it is executed on a computer.

Images