Learning Quality of Experience with Low Quality of Experience Feedback

Abstract

The present invention relates to a method for determining and / or maximizing Quality of Experience associated with an application deployed on a mobile network, the method being implemented by at least one network function of the mobile network, the method comprising: obtaining information indicative of a distribution of possible levels of quality of service provided by a mobile network for an application over a period of time; - obtaining or generating a model describing the relationship between possible quality of service levels and possible quality of experience states for an application; - estimating the statistical evolution of the quality of experience associated with the application based on the obtained information and the model. The invention further relates to a corresponding device and to a corresponding computer program.

Description

[Technical Field]

[0001] The present disclosure relates to the field of telecommunications.

[0002] The present disclosure relates more particularly to a method for determining quality of experience associated with an application deployed over a mobile network, a corresponding device, and a corresponding computer program. Priority is claimed to European Patent Application No. 23305588.8, filed April 18, 2023, the contents of which are incorporated herein by reference. [Background technology]

[0003] Figure 1 shows a typical deployment of an application over a private 5G network in a smart factory, where the application considered is a video streaming application from clients located on the factory floor to a server (108) located in the edge cloud. The edge server collects data from the clients (102) and provides additional functionality such as image recognition or artificial intelligence (AI).

[0004] A 5G mobile communication network or 5G system comprises a 5G Core Network (5GCN) and a 5G New Radio Access Network (5G-RAN). User Equipment (UE) may connect to the 5GCN via the 5G New Radio Access Network and further to a Data Network (DN), such as the Internet.

[0005] A key component of 5GCN is network functions separated by providing services to manage network behavior. These network functions are: -Access and Mobility Management Function (AMF) that acts as a single entry point for UE connectivity; -A respective Session Management Function (SMF) selected by the AMF to manage a user session for a service requested by the UE; A User Plane Function (UPF) that transports IP data traffic (user plane) between the user equipment (UE) and external networks; and - Policy Control Function (PCF) that provides a policy control framework; -Unified Data Management (UDM) function for managing network user data; A Network Data Analysis Function (NWDAF) that collects data from other 5G network functions to support other 5G network functions, processes this data, and outputs analysis, for example, statistical analysis or machine learning predictions; A Network Exposure Function (NEF) that exposes 3GPP core network capabilities to third parties; - etc., including.

[0006] The 5G network stack comprises an open wireless architecture layer, a network layer, an open transport layer, and an application layer (APP), which can be mapped to the presentation and application layers of the OSI stack.

[0007] The application function (AF) 106 is the control plane of the application layer and provides application services.

[0008] According to the state of the art, the AF may request the 5G Core Network (5GCN) (104) to set up a session with specific Quality of Service (QoS) parameters corresponding to a reference QoS profile and an alternative QoS profile (AQP).

[0009] The state-of-the-art solution for network "admission control" to a request from an AF is to respond to this request by a network acknowledgment of the request, i.e., admission of the AF, and signaling of a possible QoS session from the network side, i.e., reference / alternate QoS parameters are feasible by the network. In the state-of-the-art, the network can only react to AQP requests without long-term optimization of application behavior and / or outages.

[0010] Another patent document [1] filed to the same applicant, entitled "Learning for quality-of-experience optimization in industrial deployment of applications," proposes a proactive technique for learning application quality of experience (QoE) sensitivity to QoS degradation from collected feedback from application clients. This proposed technique further enables orchestration of network QoS across multiple AFs to maximize the overall quality of experience of deployed applications.

[0011] The basic principle of this technique is to use direct feedback from application clients, i.e., the Mean Opinion Score (MoS), and to approximate complex QoS-QoE behavior with a simple exponential model by invoking the exponential hypothesis IQX [Reference IQX].

[0012] However, in some deployment conditions, the 5G-CN may not have access to this QoE feedback from the application layer or may have access to a reduced feedback data set from the application client, and therefore the proactive techniques proposed in [1] may not be applicable in these deployment conditions. Summary of the Invention

[0013] This disclosure improves the situation.

[0014] A method for determining and / or maximizing the Quality of Experience of an application served by a mobile network is proposed, the method being implemented by at least one network function of the mobile network, the method comprising: - obtaining information indicative of a grouped frequency distribution of measurements related to a quality of service provided by a mobile network for an application over a period of time, the grouped frequency distribution having a plurality of grouped time intervals having a one-to-one relationship with a plurality of possible levels of quality of service; - obtaining or generating a model describing the relationship between possible quality of service levels and possible quality of experience states; - estimating the statistical evolution of the quality of experience associated with the application based on the obtained information and the model.

[0015] The method allows for predicting future long-term satisfaction of an application from past and / or current observations.

[0016] The predicted future long-term satisfaction may further be used to optimize the operation of private communication networks, such as 5G private networks, and subsequent industrially deployed applications.

[0017] In an embodiment of the method, the grouped frequency distribution is a probability distribution including a plurality of duration values, each of which is associated with at least one corresponding quality of service level among a plurality of possible levels of quality of service. For example, a measurement period can be considered to include a plurality of measurement cycles during which one or more indicators of quality of service are measured. The indicators of quality of service may include latency, throughput, jitter, etc. The resulting measurements may be classified into several discrete levels, which are interpreted as several possible levels of quality of service. Then, it is possible to count the number of times a given possible level of quality of service is obtained during the total number of measurement cycles. Such a counting procedure is performed for all possible levels of quality of service, allowing for evaluation of duration values ​​as a percentage of the measurement period, and the evaluated duration values ​​are associated with corresponding levels of quality of service among a plurality of possible levels of quality of service. Furthermore, the multiple possible levels of quality of service may be aggregated into groups of possible levels of quality of service (e.g., a first group of "good" QoS levels corresponding to a quality of service that meets or exceeds a target criterion, and a second group of "bad" QoS levels corresponding to a quality of service that does not meet the target criterion).

[0018] In an embodiment of the method, the grouped frequency distribution is a stationary distribution that includes a plurality of mean presence time values, and the mean presence time values ​​are associated with at least one corresponding quality of service level from a plurality of possible quality of service levels.

[0019] In an embodiment, the method comprises: obtaining measurements on the quality of service provided by the mobile network for the application during a measurement period; - determining a grouped frequency distribution by analyzing the measurements.

[0020] In an embodiment of the method, generating the model comprises extracting features from the measurements by performing a dimensionality reduction of the measurements.

[0021] In an embodiment of the method, generating the model further comprises evaluating a relationship between possible quality of service levels and possible quality of experience states based on the extracted features.

[0022] In an embodiment, the method further comprises using quality of experience feedback obtained from a network data analysis function of the mobile network and / or from a network publishing function of the mobile network as input for generating the model.

[0023] In an embodiment of the method, generating the model includes classifying the quality of experience feedback to output a distribution of possible quality of experience states.

[0024] In an embodiment, the method further comprises using one or more quality of experience measurements, e.g. obtained from application functions, as input for generating the model, the one or more quality of experience measurements being indicative of a statistical evolution of the quality of experience over a period of time.

[0025] In an embodiment of the method, the model further comprises an initial distribution of possible quality of experience states and transition relationships between the possible quality of experience states.

[0026] In an embodiment, the method further comprises tuning parameters of the model by running an expectation maximization algorithm.

[0027] In an embodiment of the method, the adjusted parameters of the model include an adjusted distribution of possible quality of experience states, and the method further includes sending the adjusted distribution of possible quality of experience states to an application function corresponding to the application.

[0028] In an embodiment of the method, the adjusted parameters of the model include adjusted transition relationships between possible quality of experience states and adjusted relationships between possible quality of service levels and possible quality of experience states, and the method further includes sending the adjusted transition relationships and the adjusted relationships to a resource orchestrator.

[0029] In an embodiment, the method further includes, in the resource orchestrator, using the coordinated transition relationship and the coordinated relationship to provision network resources of the mobile network such that a quality of experience predicted from the coordinated relationship is maximized.

[0030] A system adapted to perform the above method is further proposed, the system comprising at least one network function entity of a mobile network.

[0031] It is further proposed a computer program comprising instructions that, when the program is executed by a computer, cause the computer to carry out the above method.

[0032] Unless otherwise indicated, and as will be apparent from the description that follows, use of terms such as "computing," "calculating," "generating," and the like throughout this specification will be understood to refer to the acts and / or processes of a computer or computing system, or similar electronic computing device, that manipulate and / or transform data represented as physical quantities, such as electronic quantities, in the registers and / or memory of the computing system into other data that is similarly represented as physical quantities in the memory, registers, or other such information storage, transmission, or display device of the computing system.

[0033] Embodiments of the present invention may include an apparatus for performing the operations herein. This apparatus may be specially constructed for the desired purposes, or it may comprise a general-purpose computer or a digital signal processor ("DSP") selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored on a computer-readable storage medium such as any type of disk, including, but not limited to, a floppy disk, an optical disk, a CD-ROM, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an electrically programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic or optical card, or any other type of medium suitable for storing electronic instructions and which can be coupled to a computer system bus.

[0034] The processes presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct specialized apparatus to perform the desired method. The desired structure for a variety of these systems will appear from the description below. Further, embodiments of the present invention are not described with reference to any particular programming language. It will be understood that a variety of programming languages ​​may be used to implement the teachings of the present invention as described herein.

[0035] Other features, details and advantages are set forth in the following detailed description and drawings. [Brief explanation of the drawings]

[0036] [Figure 1] FIG. 1 illustrates a typical deployment of an application over a private 5G network.

[0037] [Figure 2]FIG. 1 illustrates an exemplary basic workflow for integrating an application with a private 5G network according to an embodiment of the present invention.

[0038] [Figure 3] FIG. 2 is a diagram of a Hidden Markov Model representation of the QoS-QoE relationship according to one embodiment of the present invention.

[0039] [Figure 4] FIG. 4 is an overview of the Baum-Welch algorithm for learning the parameters of the Hidden Markov Model depicted in FIG. 3 according to one embodiment of the present invention.

[0040] [Figure 5] FIG. 2 is a diagram of an exemplary workflow according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0041] The present disclosure proposes integrating applications with private 5G networks through an exemplary basic workflow shown in Figure 2. This exemplary basic workflow provides a solution for statistical prediction of application satisfaction in industrial deployments of applications.

[0042] Specific exemplary embodiments of the various parts of the workflow are further presented as follows: All exemplary embodiments have the same objective of enabling the 5G-CN to determine and / or predict the Quality of Experience provided to an application and to derive metrics for resource orchestration in the 5G system in order to organize resources to maximize at least one aspect of the application's QoE.

[0043] The AF (106) requests the 5GCN (104) to set up a session using specific QoS parameters corresponding to a reference QoS profile and a list of one or more alternative QoS profiles, and the 5GCN (106) then obtains these specific QoS parameters from the AF (104) (202).

[0044] The 5GCN (104) may further obtain or collect (204) QoS measurements from other entities of the 5G system, such as the radio access network (5G-RAN) and the user plane function (UPF).

[0045] The 5GCN (104) may further evaluate probabilities associated with the level of QoS achieved by the network during the application period. These probabilities may be, for example, the average time a particular QoS level is achieved during that period, or a stationary distribution of the mean age of QoS at a particular QoS level obtained from modeling of QoS measurements. For example, the 5GCN may calculate a QoS probability distribution as the average time a QoS profile is measured by a Network Data Analysis Function (NWDAF) during a measurement period across the total number of different available QoS profile measurements. These measurements may be performed by an Access and Mobility Function (AMF) for the 5G-RAN and / or by a Session Management Function (SMF) for the transport network portion of the QoS profile. These measurements performed by the SMF / AMF may be exposed to a trusted third-party network function (NF), such as a multi-access edge computing server (MEC), where the QoS probability distribution is calculated. Alternatively or additionally, the 5GCN may perform modeling of the QoS measurements and thus use the obtained model to calculate the distribution of the mean age of QoS at one or more particular QoS levels. For example, the NWDAF may perform a Markov model fit on the measured QoS. A transition matrix may be derived as the probability of transitioning from a reference QoS to an alternative QoS during the measurement period. This transition matrix may be used to estimate the stationary probability of being in each QoS state. The NWDAF may also perform classification and / or clustering of the measured QoS and use these models to calculate the stationary probability of QoS transitions.

[0046] The 5GCN (104) may further collect (206) QoE feedback from the application client (102) and / or QoE metrics from the application function (106). The QoE feedback or metrics may be collected, for example, in the form of a satisfaction index (IDX), i.e., an index representing the quality of experience (QoE) associated with the application. The QoE feedback from the application client represents a measure of the overall level of satisfaction at the application client. The QoE metrics are derived from QoE feedback from multiple application clients. The QoE feedback may be collected from the UE by the NWDAF through a specific interface. Based on the QoE feedback, the NWDAF may calculate an actual distribution of QoE. The NWDAF may further receive additional assistance information from the AF, such as an expected QoE distribution for the AF's nominal operation. Alternatively or additionally, the QoE metrics may be transmitted by the AF to the 5GCN. For example, the AF may be configured to calculate a statistical distribution of QoE and transmit the calculated statistical distribution to the 5GCN. The 5GCN may further receive additional assistance information from the AF.

[0047] The 5GCN (104) may further predict (208) the statistical and / or long-term behavior of the application's Quality of Experience (QoE). In an exemplary implementation, QoS statistics may be available to the 5GCN, and the QoS statistics may include, for example, a QoS probability distribution at one or more specific QoS levels or a distribution of QoS mean ages. In this implementation, the AF may be configured to send the QoS-QoE relationship to the 5GCN as additional information. The 5GCN may also have access to QoE feedback and / or metrics derived from such feedback, but this is not necessary in this implementation, as the 5GCN may derive, predict, or evaluate QoE feedback from the available QoS statistics and from the QoS-QoE relationship. Alternatively, both the QoS statistics and the QoE feedback and / or metrics may be made available to the 5GCN, thereby enabling the 5GCN to determine a QoS-QoE model from available measurements by performing maximum likelihood model fitting on the available QoS and available QoE feedback and / or metrics.

[0048] The 5GCN may use the QoS-QoE model to derive, predict, estimate and / or evaluate QoE statistics and / or long-term QoE behavior. The 5GCN may further use this QoS-QoE model to determine the maximum possible evaluation value of QoS and / or QoE for one or more application sessions. This solution is described below as a learning from QoS statistics solution based on hidden Markov modeling of the QoS-QoE relationship.

[0049] The 5GCN may further use the QoS-QoE model to orchestrate resources for applications within the 5G-RAN and / or transport network 212. Resource orchestration may be performed to generally optimize the long-term QoE of the application, or specifically optimize a particular quantile of the application's QoE.

[0050] Here we provide details on the Hidden Markov Model (HMM) determination of the QoS-QoE relationship and of the QoE dynamics. The HMM is a probabilistic model of a stochastic process whose state, i.e., the QoE level, is hidden. The network can measure only those QoS variables that are related to the QoE level through the probabilistic relationship.

[0051] 3 shows a basic representation of a simplified HMM, assuming that the QoE has two levels: QoE="good" (302) and QoE="bad" (304). Three QoS variables, QoS(1) (306), QoS(2) (308), and QoS(3) (310), are observed and / or measured by the network. These QoS variables may be, for example, latency, throughput, and coverage level.

[0052] The dashed arrows represent the transition probabilities between possible QoE states, e.g., the dashed arrow from QoE="good" to QoE="bad" represents the probability, denoted Prob(bad|good), that the QoE transitions from the "good" state to the "bad" state.

[0053] The hidden state transition matrix is ​​a matrix “A” that describes the transition probabilities between the possible states of QoE.

number

[0054] The solid arrows represent the probabilistic relationship between possible QoE states and QoS levels. For example, the solid arrow from QoE="good" to QoS1 represents the probabilistic relationship denoted Prob(QoS1|good), which defines the probability distribution of QoS1 when QoE is in the "good" state.

[0055] The observation matrix of the measurements is the matrix "B" that describes the probabilistic relationship between the possible states of QoE and the QoS levels.

number

[0056] The HMM may be initialized using an initial distribution of QoE states corresponding to the probability vector π, where Prob(good) and Prob(bad) are the probabilities that the initial QoE is in state “good” or “bad”, respectively.

number

[0057] One of the key problems in training HMMs is to estimate the parameters so that the joint likelihood of the transition matrix A, the observation matrix B, and the initial distribution of the model π converges. This technique is known as the expectation-maximization (EM) algorithm. In the context of hidden Markov models, this algorithm is called the Baum-Welch algorithm.

[0058] A general description of the Baum-Welch algorithm is shown in Figure 4 with the following notation: t is a time point having a value between 1 and T; pt is the value of the QoE state at time t, qt is the value of the QoS measurement at time t.

[0059] In an initialization step (402), values ​​are assigned to the HMM parameters A, B, and π. These values ​​are, for example, chosen randomly in the literature or set using prior known information, if available.

[0060] After the initialization step, the Baum-Welch algorithm branches into a forward step (404) and a backward step (406). The forward step determines the probability α(i) of reaching QoE state p, at time t, given that the previous QoS measurements followed a subsequence [q,...,q]. This is found recursively. The backward step determines the probability β(i) of observing a subsequence of QoS measurements [q,...,q] given a starting QoE state p at time t.

[0061] The temporary variables γ(i) and ξ(i,j) are determined in an update step (408) according to Bayes' theorem using known procedures. The temporary variables γ(i) and ξ(i,j) are used to estimate new values ​​of A, B, and π in an evaluation step (410).

[0062] In a test step (412), the difference between the new values ​​of A, B, and π and the initial values ​​assigned during the initialization step (402) is determined. The difference is compared to a threshold. If the difference is less than the threshold, the HMM parameter estimates are determined to have converged and the new values ​​are retained and output (414). If the difference is less than the threshold, the HMM parameter estimates are determined to have not converged and a new initialization step (402) is performed with new values ​​assigned to the HMM parameters at A, B, and π.

[0063] The algorithm is offline, ie, the forward-backward steps, update and estimation steps are repeated until convergence of the estimates of the HMM parameters.

[0064] It has been proven that the Baum-Welch (BW) algorithm converges to a local minimum, but the parameters corresponding to this minimum may be far from the expected satisfaction of the application, i.e., the QoE distribution transmitted by the application. Another known drawback of the BW algorithm is its slow convergence.

[0065] To prioritize convergence to a local minimum close to the application's expected satisfaction, it is proposed to adjust the initial step parameters so that the initial QoE distribution is close to the expected satisfaction sent by the application. This may be done, for example, by collecting available reduced feedback from the application and using a classification algorithm to construct an initial estimate close to the initial satisfaction distribution. A Naive Bayes classifier can be used with the initial QoE distribution from the AF as a prior distribution. A Restricted Boltzmann Machine (RBM) can be used to learn a coarse QoE distribution from the available feedback. The initial distribution may be constructed from the output of the RBM and from the AF initial distribution.

[0066] To speed up the convergence of the algorithm, it is further proposed to perform dimensionality reduction of the measured QoS. Principal component analysis may be performed or an autoencoder may be applied to find relevant QoS features from the QoS data. These relevant features may be used to train the HMM and provide an initial guess for the observation probability B.

[0067] In summary, the algorithm workflow is illustrated by Figure 5 and focuses on processing steps (502, 504, 506, 508) that can be performed by functional entities of the 5G core network. These processing steps can be performed in an optimized Network Data Analysis Function (NWDAF) or a Network Publishing Function (NEF).

[0068] QoE feedback, metrics, or indicators - an initial distribution of QoE states corresponding to the probability vector π, and - Initial matrix A describing the transition probabilities between possible states of QoE is used to obtain and generate (502).

[0069] The probability vector π, the initial matrix A, and the initial distributions of QoE states corresponding to the QoS measures are provided to perform 504 dimension reduction of the QoS measures. The output of the dimension reduction is - an evaluation of the initial observation matrix B, which describes the probabilistic relationship between the possible states of QoE and the QoS levels; and -Features extracted from QoS measurements Includes:

[0070] The initial distribution of QoE states corresponding to the probability vector π, the initial matrices A and B, and the QoS features are fed into the Baum-Welch algorithm. After convergence, the Baum-Welch algorithm Evaluation matrices A and B sent to the resource orchestrator, and - Output the evaluation probability vector π to be sent to the AF (506).

[0071] The resource orchestrator orchestrates network resources, such as RAN resources for 5G-RAN (508).

Claims

1. 1. A method for determining and / or maximizing quality of experience of an application served by a mobile network, the method being implemented by at least one network function of the mobile network, the method comprising: - obtaining information indicative of a grouped frequency distribution of measurements related to the quality of service provided by the mobile network for the application over a period of time, the grouped frequency distribution comprising a number of grouped time intervals having a one-to-one relationship with a number of possible levels of quality of service; - obtaining or generating a model describing the relationship between said possible quality of service levels and possible quality of experience states; - estimating the statistical evolution of the quality of experience associated with said application based on said obtained information and said model; A method comprising:

2. 2. The method of claim 1, wherein the grouped frequency distribution is a probability distribution that includes a plurality of presence time values, each presence time value being associated with at least one corresponding quality of service level among the plurality of possible quality of service levels.

3. 2. The method of claim 1, wherein the grouped frequency distribution is a stationary distribution comprising a plurality of mean presence time values, each of which is associated with a corresponding quality of service level of at least one of the plurality of possible quality of service levels.

4. - obtaining measurements on the quality of service provided by said mobile network for said application during a measurement period; - determining said grouped frequency distribution by analyzing said measurements; The method of claim 1 , further comprising:

5. The method of claim 4 , wherein generating the model comprises extracting features from the measurements by performing a dimensionality reduction of the measurements.

6. 6. The method of claim 5, wherein generating the model further comprises evaluating the relationship between the possible quality of service levels and the possible quality of experience states based on the extracted features.

7. 7. The method of claim 1, further comprising using quality of experience feedback obtained from a network data analysis function of the mobile network and / or from a network publishing function of the mobile network as input for generating the model.

8. The method of claim 7 , wherein generating the model comprises classifying the quality of experience feedback to output a distribution of the possible quality of experience states.

9. 9. The method of claim 1, further comprising the step of using one or more quality of experience measurements as input for generating the model, the one or more quality of experience measurements being indicative of a statistical evolution of the quality of experience over the period of time.

10. The method of claim 8 or 9, wherein the model further comprises an initial distribution of the possible quality of experience states and transition relations between the possible quality of experience states.

11. 11. The method of claim 1, further comprising the step of adjusting parameters of the model by running an expectation maximization algorithm.

12. 12. The method of claim 11 , wherein the adjusted parameters of the model include an adjusted distribution of the possible quality of experience states, the method further comprising sending the adjusted distribution of the possible quality of experience states to an application function corresponding to the application.

13. 13. The method of claim 11 or 12, wherein the adjusted parameters of the model include adjusted transition relations between the possible quality of experience states and adjusted relationships between the possible quality of service levels and the possible quality of experience states, the method further comprising the step of sending the adjusted transition relations and the adjusted relationships to a resource orchestrator.

14. 14. The method of claim 13, further comprising: in the resource orchestrator, using the adjusted transition relation and the adjusted relationship to provision network resources of the mobile network such that a quality of experience predicted from the adjusted relationship is maximized.

15. A system comprising at least one network function entity of a mobile network, said system being configured for determining and / or maximizing a Quality of Experience of an application served by the mobile network by means of an application function, - obtaining information indicative of a grouped frequency distribution of measurements related to quality of service provided by the mobile network for the application over a period of time, the grouped frequency distribution comprising a number of grouped time intervals having a one-to-one relationship with a number of possible levels of quality of service; - obtaining or generating a model describing the evolutionary relationship between said possible quality of service levels and possible quality of experience states; - estimating the statistical evolution of the quality of experience associated with the application based on the obtained information and the model; A system that at least implements the following.

16. A non-transitory storage medium storing computer program instructions which, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 14.

Images