Training of end-to-end service quality models

Abstract

Data collection is provided for training of end-to-end service quality models used for real-time and URLLC services in wireless communication networks. In order to train the service quality models for estimating end-to-end WebRTC or QoE data in the downlink (DL) and uplink (UL) data paths, two measurement systems are used to account for degradations in the DL and UL radio interface (IF) and the transport DL and UL data paths. Radio measurements are used for training the service quality models but are not necessary for the operation phase of the service quality model. The two data collection systems; one of the client-side or user side, which includes a radio interface, and one on the server-side, which includes either a radio interface or wired interface. In radio degradation scenarios, radio degradations are introduced by the first test system and transport parameters impacted by the radio degradations are measured by the second test system

Description

[0001] TRAINING OF END-TO-END SERVICE QUALITY MODELS

[0002] TECHNICAL FIELD

[0003] The present disclosure relates generally to end-to-end service quality models for wireless communication networks and, more particularly, to training of end-to-end service quality models.

[0004] BACKGROUND

[0005] Per session, event-based analytics can be provided by a Management Domain Application Function (MDAF) in the management domain or a Network Data Analytic Function (NWDAF) in the core network. One example of a per session, event-based analytics system is the Ericson Expert Analytics (EEA) system. These analytics systems collect and correlate elementary network events from different network domains, such as core network, radio access network (RAN), and transport networks. Generally, eventbased network nodes calculate key performance indicators (KPIs) characterizing radio / environment (i.e. , radio KPIs) or network operation (i.e. , transport KPIs) at the user and session level. These network nodes use service quality models for estimating end user Quality of Service (QoS) and Quality of Experience (QoE). Event-based network nodes are suitable for session-based troubleshooting and analysis of network issues and can be used in closed loop automated network operations for assuring and optimizing service quality.

[0006] Event-based analytics systems are also used in Service Operation Centers (SOC) for monitoring the quality of a wide variety of services at the network level, as well as for monitoring the customer experience at the individual per subscriber level. These tools are widely used in customer care and other business scenarios.

[0007] Event-based analytics require real-time collection and correlation of characteristic node and protocol events from different RAN nodes and core network nodes, probing of signaling interfaces (IFs), and sampling of the user-plane traffic as well. Besides the data collection and correlation functions, the analytics system requires an advanced database, rule engine, and big data analytics platform as well.

[0008] The QoS and transport KPIs are obtained by the analytics system from probe systems or user plane network function measurements. The QoE and, if not available by direct measurements, the end-to-end (e2e) QoS metrics are estimated by service quality models. Service quality models are typically service and application specific, which require appropriate training based on training data collected in either real networks or test networks. The training data is labeled with end user service quality labels. Labeling requires manual work and human test users. Training of the service quality models requires a relatively large number of sessions for wide variation of network conditions, especially those that impact the service quality. For large number of sessions, the data collection can be expensive and time consuming.

[0009] The collection of data for model training can be challenging. In some cases, end- to-end transport parameters are not available. For example, in the case of unreliable data transport using the User Datagram Protocol (UDP), degradations in the downlink radio path are not reflected in the measured transport parameters in the downlink direction. In other cases, radio measurements are not available and only transport data are used for model training. Even when available, the accuracy of service quality indications based on radio measurements is less than service quality indications based on transport parameters. Further, it is more difficult to collect data reflecting radio degradation as compared to transport degradation, which is relatively easy.

[0010] In scenarios where Real Time Protocol (RTP) is used for transport, the Real- Time Transport Control Protocol (RTCP) report contains information about end-to-end packet loss, jitter and Round Trip Time (RTT). However, other significant parameters, such as burst metrics, and interarrival time (IAT) are not reported. Also, the frequency of the RTCP report is relatively low, in the order of 5-10 sec., which is not detailed enough for service quality models. Statistics, such as the Standard Deviation (Stdev), or other quantiles (which are important for service quality of real-time and Ultra Reliable Low Latency Communication (URLLC) traffic) are not reported. In some case, RTCP reports are not implemented (e.g., for voice services in case of Apple terminals). Because RTCP reports are encrypted, it is not possible to obtain the RTCP reports by probing.

[0011] In view of these challenges, there is a need to improved systems for collecting training data for use in training service quality models.

[0012] SUMMARY

[0013] The present disclosure relates generally to the training of end-to-end service quality models used for real-time and URLLC services in wireless communication networks. In order to train the service quality models for estimating end-to-end WebRTC parameters or QoE data in the downlink (DL) and uplink (UL) data paths, two measurement systems are used to account for degradations in the DL and UL radio interface (IF) and the transport DL and UL data paths. Radio measurements are used for training the service quality models but are not necessary for the operation phase of the service quality model. The two data collection systems are disposed on opposite sides of radio interface used by a client to access the server; one of the client-side or user side and one on the server-side. In radio degradation scenarios, radio degradations are introduced by the first test system and transport parameters impacted by the radio degradations are measured by the second test system.

[0014] A first aspect of the disclosure comprises methods of data collection implemented by the measurement system for collecting training data for training a service quality model used in a wireless communication network. The measurement system includes a first test system and a second test system disposed along a communication path including a radio interface. The method comprises, optionally, transmitting data bi-directionally over the communication path between a first end point of the communication path and a second end point of the communication path. The method further comprises degrading, in a first test system on a first side of the radio interface, e.g., client side, a first directional stream and / or a second directional stream. The method further comprises measuring, in a second test system on a second side of the radio interface, e.g., server side, one or more first transport performance metrics associated with the first directional stream and / or one or more second transport performance metrics associated with the second directional stream.

[0015] A second aspect of the disclosure comprises a measurement system for collecting training data for training a service quality model used in a wireless communication network. Optional transmitting units transmit data bi-directionally over a communication path including a radio interface. A first test system disposed on a first side of the radio interface, e.g., client-side, degrades a first directional data stream, a second directional data stream, or both. A second test system disposed on a second side of the radio interface, e.g., server-side, measures one or more first transport performance metrics associated with the first directional stream, second transport performance metrics associated with the second directional stream, or both.

[0016] A third aspect of the disclosure comprises a computer program for a producer network node. The computer program comprises executable instructions that, when executed by processing circuitry in a measurement system, causes the measurement system to perform the method according to the first aspect.

[0017] A fourth aspect of the disclosure comprises a carrier containing a computer program according to the fourth aspect. The carrier is one of an electronic signal, optical signal, radio signal, or a non-transitory computer readable storage medium. A fifth aspect of the disclosure comprises methods of training a service quality model used in a wireless communication network for estimating service quality of data sessions. The method comprises obtaining first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface. The method further comprises obtaining second transport performance metrics associated a second directional data stream transmitted over the communication path, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface. The method further comprises training a service quality model using the first transport performance metrics for both the first and second directional data streams.

[0018] A sixth aspect of the disclosure comprises a network node for training a service quality model used in a wireless communication network to estimate service quality of data sessions. The network node is configured to obtain first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface. The network node is further configured to obtain second transport performance metrics associated a second directional data stream transmitted over the communication path, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface. The network node is further configured to train a service quality model using the first transport performance metrics for both the first and second directional data streams.

[0019] A seventh aspect of the disclosure comprises a network node for training a service quality model used in a wireless communication network to estimate service quality of data sessions. The network node comprises processing circuitry and memory operatively coupled to the processing circuitry. The memory stores program instructions that, when executed by the processing circuitry, causes the network node to obtain first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface. The program instructions, when executed by the processing circuitry, further cause the network node to obtain second transport performance metrics associated a second directional data stream transmitted over the communication path, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface. The program instructions, when executed by the processing circuitry, further cause the network node to train a service quality model using the first transport performance metrics for both the first and second directional data streams.

[0020] An eighth aspect of the disclosure comprises a computer program for a producer network node. The computer program comprises executable instructions that, when executed by processing circuitry in a network node, causes the network node to perform the method according to the fifth aspect.

[0021] A ninth aspect of the disclosure comprises a carrier containing a computer program according to the eighth aspect. The carrier is one of an electronic signal, optical signal, radio signal, or a non-transitory computer readable storage medium.

[0022] A tenth aspect of the disclosure comprises methods of estimating service quality for one or more data sessions in a wireless communication network. The method comprises obtaining first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface. The method further comprises obtaining second transport performance metrics associated a second directional data stream transmitted over the communication path, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface. The method further comprises estimating service quality of one or more data sessions in a service quality model generated from sample training data based on the first and second transport performance metrics.

[0023] An eleventh aspect of the disclosure comprises a network node for estimating service quality for one or more data sessions in a wireless communication network. The network node is configured to obtain first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface. The network node is further configured to obtain second transport performance metrics associated a second directional data stream transmitted over the communication path, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface. The network node is further configured to estimate service quality of one or more data sessions in a service quality model generated from sample training data based on the first and second transport performance metrics.

[0024] A seventh aspect of the disclosure comprises a network node for estimating service quality for one or more data sessions in a wireless communication network. The network node comprises processing circuitry and memory operatively coupled to the processing circuitry, wherein the memory stores program instructions that, when executed by the processing circuitry, causes the network node to obtain first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface. The program instructions, when executed by the processing circuitry, further cause the network node to obtain second transport performance metrics associated a second directional data stream transmitted over the communication path, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface. The program instructions, when executed by the processing circuitry, further cause the network node to estimating service quality of one or more data sessions in a service quality model generated from sample training data based on the first and second transport performance metrics.

[0025] An eighth aspect of the disclosure comprises a computer program for a network node for estimating service quality of data sessions in a wireless communication network. The computer program comprises executable instructions that, when executed by processing circuitry in an network node, causes the network node to perform the method according to the tenth aspect.

[0026] A ninth aspect of the disclosure comprises a carrier containing a computer program according to the eleventh aspect. The carrier is one of an electronic signal, optical signal, radio signal, or a non-transitory computer readable storage medium.

[0027] BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 illustrates an exemplary wireless communication network implementing network data analytics. Figures 2A and 2B illustrate training of service quality models used to generate QoS and QoE estimates.

[0029] Figure 3 illustrates end-to-end data paths for real-time traffic

[0030] Figure 4 illustrates a data collection system to collect training data for training end-to-end service quality models.

[0031] Figure 5 illustrates a measurement setup for measurement of radio degradation.

[0032] Figures 6A and 6B illustrate training of service quality models for radio degradation.

[0033] Figure 7 illustrates a radio degradation scenario in a wireless communication network.

[0034] Figure 8 illustrates a measurement setup for measurement of transport degradation.

[0035] Figures 9A and 9B illustrate training of service quality models for transport degradation.

[0036] Figure 10 illustrates a transport degradation scenario in a wireless communication network.

[0037] Figure 1 1 illustrates a measurement setup for measurement of uplink radio degradation in peer-to-peer communications.

[0038] Figures 12A and 12B illustrate training of service quality models for uplink radio degradation in peer-to-peer communications.

[0039] Figure 13 illustrates an uplink radio degradation scenario in in peer-to-peer communications.

[0040] Figure 14 is a flow chart illustrating an exemplary data collection method.

[0041] Figure 15 is a flow chart illustrating an exemplary method of training a service quality model to estimate service quality of data sessions in the wireless communication network.

[0042] Figure 16 is a flow chart illustrating an exemplary method of estimating service quality of data sessions in the wireless communication network.

[0043] Figure 17 illustrates the main functional components of a measurement system for collection training data.

[0044] Figure 18 illustrates the main functional components of a network node for training a service quality model to estimate service quality of data sessions in the wireless communication network. Figure 19 illustrates the main functional components of a network node for estimating service quality of data sessions in the wireless communication network.

[0045] Figure 20 illustrates a network node according to an embodiment for training and / or implementing a service quality model.

[0046] DETAILED DESCRIPTION

[0047] The present disclosure relates generally to the training of end-to-end service quality models used for real-time and LIRLLC services in wireless communication networks. Data collection relies on network probing. The end-to-end service quality model uses specific transport metrics measured for bidirectional data streams as input during operation. The transport parameters can be measured for encrypted traffic as well and they should not necessarily be available end-to-end.

[0048] The services to which the data collection and training method applies include any traffic types using WebRTC services, e.g., video conference and cloud gaming. One or more machine learning (ML) models are used to estimate end-to-end service quality for WebRTC traffic types. In one approach, a service quality model uses transport measurements to directly estimate QoE. A second approach uses two service quality models to achieve the same result. A first service quality model estimates the end-to- end values of the standard WebRTC statistical parameters using the transport measurements. The estimates generated by the first service quality model are used as input to a second model for estimating QoE.

[0049] In order to train the service quality models for estimating end-to-end WebRTC or QoE data in the downlink (DL) and uplink (UL) data paths, two measurement systems are used to account for degradations in the DL and UL radio interface (IF) and in the transport DL and UL data paths. Radio measurements are used for training the service quality models but are not necessary for the operation phase of the service quality model. The two data collection systems are disposed on opposite sides of the radio interface; one of the client-side or user side and one on the server-side.

[0050] In radio degradation training scenarios, the radio degradation is introduced in the DL radio (i.e. , client-side) in a first measurement system, while the transport data are captured at the UL radio (i.e., server-side) by a second measurement system. Labeling data are captured in the first measurement system. For training the ML models, the bidirectional transport KPIs measured in the second measurement system re used, which takes into account the possible degradation in the DL radio interface. To account for transport degradation on the server-side of the end-to-end path, different types of transport degradations are introduced in the first measurement system and the transport parameters are measured in the first measurement system. Labeling data are also obtained in the first measurement system.

[0051] In scenarios where the services are used in peer-to-peer communication and both users are using mobile access, degradations in the UL radio data paths should also be taken into account. For training the service quality models in these scenarios, radio degradation is introduced in the first measurement system while transport data is captured and used for training in the UL data path in both the first and second measurement systems.

[0052] In all cases, during operation, the DL and UL transport performance metrics (e.g., KPIs) captured in the core network are used as input parameters to the service quality models.

[0053] In the operation phase, the transport metrics are measured by user plane function (UPF) or third party network probes in the core network. The probe reports are sent in real-time to the analytics system, where the data is correlated with other core network events. The analytics system can be implemented in either the core network (e.g., by the NWDAF) or in the network management domain (e.g., by the MDAF). The analytics system implements the ML models and estimates the service quality in realtime, which can be used, for example, in closed loop network function.

[0054] Figure 1 illustrates the network architecture for a wireless communication network implementing data analytics. The present disclosure is described in the context of a Fifth Generation (5G) wireless communication network but the techniques described herein are more generally applicable to other wireless communications implementing data analytics in the core network or management domain.

[0055] The wireless communication network 10 generally comprises a 5G RAN 20 including one or more gNodeBs (gNBs) 25, a core network 30, and a network management domain 70. The core network 30 comprises a collection of network functions (NFs) performing different network tasks. Figure 1 illustrates various NFs relevant to this disclosure including the UPF 35, Access and Mobility Management Function (AMF) 40, Session Management Function (SMF) 45, Policy Control Function (PCF) 50, and NWDAF 60. The NWDAF 60 implements a ML service quality model 65 as hereinafter described. Alternatively, a MDAF 75 in the management domain may perform data analytics and implement the ML service quality model. The NFs shown in Figure 1 comprise logical entities that may be implemented by one or more processors, hardware, firmware, or a combination thereof. In cloud-based networks, the NFs are typically implemented as virtual machines (VMs), or as containers (e.g., Kubernetes). Network 10 may include multiple instances of each NF type. The NFs can also be implemented in stand-alone servers or specialized servers.

[0056] The UPF 35 supports handling of user plane traffic, including packet inspection, packet routing and forwarding, traffic usage reporting, and QoS handling. The UPF 35 connects with external IP networks and serves as an IP anchor point for user equipment (UEs) served by the UPF 35 so that the UEs are reachable even when moving around in the network 10. The UPF 35 processes data being forwarded. Such processing may include packet inspection, classification, and QoS marking of forwarded packets. The UPF 35 generates traffic usage reports, which the SMF 45 includes in charging reports, and is involved in policy enforcement.

[0057] The AMF 40 is a network function that manages access to the 5G network and handles mobility-related functions for the UEs. Its role is similar to that of the Mobility Management Entity (MME) in Fourth Generation (4G) networks. When a UE is not in idle mode, the gNB handovers are exposed to the AMF 40. Additionally, the AMF 40 can use other signaling elements to determine which gNB a UE 15 is attached to. The AMF 40 can expose these events by means of a standardized interface as well.

[0058] The SMF 45 manages Packet Data Unit (PDU) sessions for the UEs, which includes the establishment, modification, and release of PDU sessions. The SMF 45 selects the UPF 35 to handle a PDU session and controls the UPF 35. The SMF 45 receives Policy and Charging Control (PCC) Rules from the PCF 50 and configures the UPF 35 for various data flow tasks, such as shaping, policing to provide bandwidth, and charging functions.

[0059] The PCF 50 supports a unified policy framework to govern the network behavior. Specifically, the PCF 50 provides Policy and Charging Control (PCC) rules to the Policy and Charging Enforcement Function (PCEF), i.e. , the SMF 50 / UPF 35 that enforces policy and charging decisions according to provisioned PCC rules.

[0060] Per flow analytics is implemented in the either the NWDAF 60 or MDAF 75. The NWDAF 60 or MDAF 75receives signaling related events from core network functions (e.g., AMF 40 and SMF 45) and gNB radio events from the RAN including UL and DL radio measurements. The per-flow analytics system correlates the signaling related events from the core network and radio measurements from the RAN with user plane probe reports provided by the UPF or other transport network probe to generate per flow correlated session records, which include all transport KPIs. The per-flow session records serve as input to the ML models. ML model output is the subjective or objective end-to-end QoS and QoE KPIs.

[0061] The NWDAF 60 collects various types of network data and subscriber data, performs an analysis of the data, and may provide analytics reports to other NFs. Consumer NFs within the 5G core network 30 use the Nnwdaf interface to send subscription requests for analytics reports to the NWDAF 60. The requests may specify a target (e.g., group of UEs or a group of NFs) for which data is requested. The NWDAF 60 collects event data from other NFs, such as the AMF 40, SMF 45, and PCF 50, using the event exposure services offered by these NFs. Following data collection, the NWDAF 60 generates analytics reports for the targets identified in the request and sends the analytics reports to the subscribing NF. The analytic reports can be sent periodically or responsive to a triggering event. In some embodiments, the NWDAF 60 can be distributed and the services offered by the NWDAF 60 can be co-located with other NFs, such as AMF 40.

[0062] The event-based analytics system primarily targets real-time operation. For example, during operation, the model output can be used to set or modify policy settings in the PCF 50 and improve service quality in a closed-loop function. The service quality KPIs can also be used for service assurance, e.g., reporting, checking, ensuring SLS target requirements. During operation, the QoS and QoE models require only the server and client-side transport performance metrics. Optionally, radio data may be used if the data source is available.

[0063] Figures 2A and 2B schematically illustrate model training of service quality models. Figure 2A illustrates training for a direct service quality model for downlink QoE estimation and Figure 2B illustrates a two-model solution for downlink QoE estimation. In the direct model, the model input comprises server-side downlink transport KPIs and client-side uplink transport KPIs. The model input may optionally include downlink and uplink radio KPIs is a source for such data is available. The output of the direct model is the estimated downlink QoE. In the two-model solution, the input to the first comprises server-side downlink transport KPIs and client-side uplink transport KPIs. The model input may optionally include downlink and uplink radio KPIs is a source for such data is available. The output of the first model is the estimated end-to-end downlink QoS. The end-to-end downlink QoS is input to the second model, which estimates the downlink QoE.

[0064] Figure 3 illustrates end-to-end paths for real-time traffic. Obtaining end-to-end transport metrics for real-time traffic types using RTP over UDP (UDP / RTP) and Quick UDP Internet Connection (QUIC) over UDP (UDP / QUIC) can be challenging. In case of legacy traffic, using reliable transport, such as the Transport Control protocol (TCP), packets are retransmitted at the transport layer in case of packet loss. The degradation in the downlink traffic in the downlink radio interface can be detected in the UPF by observing the user traffic stream. For real-time traffic types, there is no retransmission at the transport layer (no time for retransmissions). Therefore, if packets are lost or delayed in the data path following the probing, the degradation cannot be observed directly by monitoring the user plane traffic. The data paths where unobserved degradations may occur are indicated by dashed lines in Figure 3. For some protocols the client at the end point of the end-to-end path provides feedback of some transport parameters (packet loss, jitter, RTT by RTCP reports in case of RTP, and RTT based on QUIC spin bit). But end-to-end reports are not available for other parameters important for ML models, such as burst time and interarrival time. Another issue is that RTCP reports are sent less frequently than would be needed for ML models. In some cases, the RTCP reports are not sent or are encrypted.

[0065] Figure 4 illustrates a measurement system 200 used to collect training data for end-to-end service quality models. The measurement system 200 includes transmitting units 210 at the end points of a communication path and two test systems 220, 230. Each test system includes a client measurement unit 222, 232 for collecting labelling data, a transport measurement unit 224, 234 for obtaining transport KPIs, a transport manipulator 225, 235 for simulating degradations in the transport network, a radio measurement unit 226, 236 for collecting radio measurements, and a radio manipulator 228, 238 for simulating degradations (e.g., shielding, interference, path loss, handovers, etc.) in the radio interface affecting both the uplink and downlink. The test systems 220, 230 are controlled by a controller 250. Controller 250 comprises one or more processors, hardware, firmware or combination thereof. Controller 250 includes memory 260 storing program code 270 for implementing the test procedures herein described. The individual components of the test systems are well-known and, for the sake of brevity, are not described in detail herein A first test system 220 is disposed along the communication path on a first side (e.g., client-side) of the radio interface. A second test system 230 is disposed along the communication path on a second side (e.g., server-side) of the radio interface. In some implementations, the radio interface is provided by a RAN in a mobile network that connects to the Internet or other external networks by wireline or optical connections. Typically, the second test system is located in an external network between the mobile network and the server. In some embodiments, the second test system 230 could be located in the mobile network (e.g., in the core network or between the core network and radio access network.)

[0066] Those skilled in the art will appreciate, also, that the communication path may include more than one radio interface. For example, in peer-to-peer communications, two client devices involved in the peer-to-peer communication may access a mobile network using two different RANs, or even two different mobile networks. In these scenarios, a first test system is located at the client side of a first RAN for a first client and the second test system is located anywhere along the communication path between the first RAN and the second client.

[0067] The first and second test system 230 can be used in different configurations to obtain training data for different scenarios. Exemplary scenarios include radio degradation scenarios, transport degradation scenarios, and uplink radio degradation scenarios in peer-to-peer communications. Uplink and downlink data streams are captured in both test systems. The second test on the server-side captures uplink traffic after radio and / or transport degradation.

[0068] For testing, radio parameters including downlink received signal measurements and uplink transmit power measurements are captured at the UE / client. Generally, these radio measurements correspond to values reported in RRC measurement reports. Examples of some radio measurements include:

[0069] • Reference Signal Received Power (RSRP) measures the signal strength in dBm

[0070] • Reference Signal Received Quality (RSRQ) measures the quality of the received reference signal in decibels (dB).

[0071] • Signal to Interference & Noise Ratio (SINR) measures the signal quality: the strength of the wanted signal compared to the unwanted interference and noise

[0072] • The physical layer Cell Identifier (Cell ID) used to detect handover.

[0073] • Downlink Transmission Mode (e.g., 7=Beamforming) • Uplink frequency

[0074] • Downlink frequency

[0075] • List of the uplink Modulation Coding Scheme (MCS) indices.

[0076] • List of the downlink MCS indices.

[0077] • Transmit power on the Physical Uplink Control Channel (PPUCCH), which is used to carry Uplink Control Information (UCI).

[0078] • Transmitted power on the Physical Uplink Shared Channel (PUSCH), which is used to carry signaling messages, control information and application data.

[0079] • Uplink transmit power for Physical Sounding Reference (PSRS), which is used at the gNB to estimate Channel Status Information (CSI) for a range of frequencies.

[0080] • Uplink transmitted power on the Physical Random Access Channel (PRACH), which is used by UEs to request an uplink allocation from the base station.

[0081] The transport manipulator introduces transport errors for different transport degradation scenarios. Transport degradation scenarios include:

[0082] • Packet loss

[0083] • Burst loss

[0084] • Jitter

[0085] • Delay

[0086] • Bandwidth (BW) limitation

[0087] • Bit corruption

[0088] • Background traffic

[0089] The transport measurements are obtained by capturing packets. Transport measurements include:

[0090] • Interarrival time (avg / min / max / std):

[0091] • Bitrate (Kbps)

[0092] • Packets lost percent

[0093] • Round trip time (RTT) (ms)

[0094] • Jitter (ms)

[0095] • Packet rate (packet per sec)

[0096] • Burst rate (bursts per sed)

[0097] • Burst size in packets (avg / min / max / std): The average / min / max / std number of packets of the bursts in the past 1 second. • Burst size in bytes (avg / min / max / std): The average / min / max / std number of packets of the bursts in the past 1 second.

[0098] • Burst length (avg / min / max / std): The average / min / max / std length of the bursts in ms in the past 1 second. The length of one burst is the difference of the timestamp of the first and the last packet in that burst.

[0099] • Burst separation length (avg / min / max / std):The average / min / max / std length of the separation between bursts in ms in the past 1 second. The separation length of two consecutive bursts is the difference between the timestamp of the last packet of the previous burst and the first packet of the next burst.

[0100] Client measurements are captured at the client-side and used for labeling. One example of client measurements are World Wide Web Consortium (W3C) WebRTC statistics. In addition, the audio and video can be recorded and optionally labeled with subjective QoE scores.

[0101] In order to realize the different radio and transport degradation scenarios, the different measurement configurations are used. Figures 5, 8 and 1 1 illustrate measurement configurations for radio degradation scenarios, transport degradation scenarios, and uplink radio degradation scenarios in peer-to-peer communications.

[0102] Figure 5 illustrates a measurement setup for measurement of radio degradation. In this measurement setup, the main goal is to derive end-to-end QoS and QoE observed at the end client for any radio degradations, such as poor radio coverage, handover, and interference. In real networks, radio degradation affects both UL and DL data paths. However, the same shielding, interference, handover etc. affects DL and UL data paths differently, which is taken into account by this measurement setup and training method.

[0103] Radio degradations are introduced both UL and DL by the radio manipulator 226 in the first test system 220. Different types and levels of shielding or other degradation can be applied. The radio measurement unit 226 in the first test system 220 may optionally capture radio measurements for uplink. The client measurements (e.g., WebRTC statistics) can be obtained for both directions. Audio and video streams can be recorded for DL traffic streams. QoE can be obtained, for example, using the techniques described in Gergely Dobreff, Mark Szalay, Marton Molnar, Laszlo Varga, Bence Ladoczki, Attila Bader, Alija Pasic: Predicting QoE for Delay-critical services in Mobile Networks: A video conferencing case study, QMex 2023. The particular method for QoE estimation is not material to the present disclosure and it will be appreciated that other known QoE estimation techniques can be used. The client-side WebRTC statistics and QoE measurements include the radio degradation introduced in the DL radio interface. The transport measurements are captured by the measurement unit 234 of the second system 230 at the server-side. The transport measurements reflect the UL radio degradation in the UL data path, but does not reflect the radio degradation in the DL radio path. The transport measurement unit 234 may, for example, capture data packets using a protocol analyzer such as TSHARK®, which can be processed to derive the transport KPIs. WebRTC statistic labels are captured by the client measurement unit 222 in the first test system 220. Labels for the QoE model are recorded in the first test system 220. The transport data and service quality labels are used as input to the ML model. Figures 6A and 6B illustrate training of service quality models for radio degradation using training data obtained with the test configuration shown in Figure 5. Both the WebRTC and QoE models are trained with the UL and DL transport data measured in the second test system 230. The effect of radio degradation is included in the UL transport data while the DL data without degradation serves as a kind of baseline input data to the models. Optionally, the DL and UL radio measurements, obtained in the first test system 220 can be used as input to the ML models, which improves little the accuracy of the ML model.

[0104] Figure 7 illustrates the radio degradation scenario covered by the test configuration shown in Figure 5. The transport measurements captured by the test configuration of Figure 5 includes both UL and DL radio degradation.

[0105] For training the ML models for different transport degradation scenarios, which occur at the server-side, a different measurement setup is used. Figure 8 illustrates an exemplary measurement setup for measurement of transport degradation. The transport degradations for the test scenarios are introduced in the DL path by the transport manipulator 225 in the first test system 220. Transport performance metrics are captured by the transport measurement unit 224 in the first test system 220 for both directions. QoE and WebRTC statistic labels are captured in the first test system 220 as previously described. The transport data and service quality labels are used as input to the ML models.

[0106] Figure 9A and 9B illustrate training of service quality models for transport degradation. Figure 9A illustrates a training for a direct service quality model for downlink QoE estimation and Figure 9B illustrates training for the WebRTC parameter model used for end-to-end QoS estimation. Both models use DL transport measurements and UL transport measurements captured by the second test system 230. The QoE ML model receives end user QoE labels recorded in the first test system 220. The WebRTC parameter model uses WebRTC statistics measured in the first test system 220.

[0107] Figure 10 illustrates a transport degradation scenario in a wireless communication network covered by the test configuration shown in Figure 8. The transport measurements captured by the test configuration of Figure 5 includes degradations in the server-side downlink transport.

[0108] It can also be important to train ML service quality models for peer-to-peer communication (e.g., web conferencing, gaming, voice over LTE / 5G) where both end users use RAN access. Figure 1 1 illustrates a measurement setup for measurement of uplink radio degradation in peer-to-peer communications.

[0109] Radio degradations are introduced by the radio manipulator 228 in the first test system 220. The radio measurement unit 226 in first test system 220 may optionally capture radio measurements for uplink. The client-side labels (WebRTC stat params) and QoE (based on the recorded media) are also captured at the client-side in the second test system 230. The transport measurements, which are used as input for the ML model training, are captured by the transport measurement unit 224 in the UL data path in the first test system 220 before the radio degradation and by the transport measurement unit 234 in the second test system 230 after the degradation. Note that this measurement setup assumes symmetric traffic types in UL and DL direction.

[0110] Figures 12A and 12B illustrate training of service quality models for uplink radio degradation in peer-to-peer communications. Figure 12A illustrates a training for a direct service quality model for downlink QoE estimation and Figure 12B illustrates training for the WebRTC parameter model used for end-to-end QoS estimation. The direct service quality model in Figure 12A uses UL transport measurements captured in both the first and second test system 230s. The model input may optionally include downlink and uplink radio KPIs if a source for such data is available. The QoE ML model also receives end user QoE labels recorded in the second test system 230 as input. The WebRTC parameter model in Figure 12B receives as input the uplink transport measurements captured in the second test system 230 and UL transport measurements capture in the first test system 220. The model input may optionally include downlink and uplink radio KPIs if a source for such data is available. The WebRTC parameter model uses WebRTC statistics measured in the second test system 230.

[0111] Figure 13 illustrates an uplink radio degradation scenario in peer-to-peer communications covered by the measurement setup in Figure 1 1 . Uplink radio degradations are captured on the source client-side.

[0112] Figure 14 illustrates an exemplary method 100 of data collection implemented by the measurement system including a first test system 220 and a second test system 230 disposed along a communication path having a radio interface. The measurement system optionally transmits test data bi-directionally over a communication path between a first end point of the communication path and a second end point of the communication path (block 110). The communication path includes a radio interface. The first test system 220 on a first side of the radio interface degrades a first directional stream and / or a second directional stream (block 120). The second test system 230 on a second side of the radio interface measures one or more first transport performance metrics associated with the first directional stream and / or second transport performance metrics associated with second directional stream (block 130). The method optionally comprises training a service quality model using the first and / or second transport performance metrics (block 140).

[0113] In some embodiments of method 100, degrading the first and / or directional streams of the bi-directional data comprises introducing simulated radio degradation in the first directional stream.

[0114] In some embodiments of method 100, measuring one or more first transport performance metrics comprises measuring first transport performance metrics for the first directional stream in the second test system 230, wherein the first transport performance metrics associated with the first directional stream are affected by the radio degradation.

[0115] In some embodiments of method 100, degrading the first and / or directional streams of the test data comprises introducing simulated radio degradation in the second directional stream.

[0116] In some embodiments of method 100, measuring one or more first transport performance metrics associated with the first and / or second directional streams comprises measuring second transport performance metrics for the second directional stream in the second test system 230, wherein the second transport performance metrics associated with the second directional stream are not affected by the radio degradation.

[0117] In some embodiments of method 100, measuring one or more first transport performance metrics associated with the first and / or second directional streams comprises measuring first transport performance metrics for the first directional stream and second transport performance metrics for the second directional stream in the second test system 230.

[0118] Some embodiments of method 100, further comprise measuring, in the first test system 220, one or more third transport performance metrics associated with the first directional stream, wherein the third transport performance metrics are not affected by the radio degradation.

[0119] In some embodiments of method 100, the radio interface is provided by a radio access network (RAN).

[0120] In some embodiments of method 100, the first test system is located between the RAN and a first end point of the communication path.

[0121] In some embodiments of method 100, the RAN is part of a mobile network and the second test system is located between the RAN and a core network in the mobile network.

[0122] In some embodiments of method 100, the RAN is part of a mobile network and the second test system is located in a path segment of the communication path between the mobile network and the second end point of the communication path.

[0123] Some embodiments of method 100, further comprise correlating the transport performance metrics with service quality labels.

[0124] Some embodiments of method 100, further comprise training a service quality model with the transport performance metrics and correlated service quality labels.

[0125] Some embodiments of method 100, further comprise correlating radio performance metrics with the service quality labels.

[0126] Some embodiments of method 100, further comprise training a service quality model with the radio performance metrics and correlated service quality labels.

[0127] Figure 15 is a flow chart illustrating an exemplary method 300 of training a service quality model to estimate service quality of data sessions in the wireless communication network. The method 300 may be carried out in a network node 350 (Fig. 18) in the wireless communication network or external network. The network node 350 obtains first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface (block 310). The network node 350 further obtains second transport performance metrics associated a second directional data stream transmitted over a communication path having a radio interface, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface (block 320). The network node 350 trains a service quality model using the first transport performance metrics for both the first and second directional data streams (block 330).

[0128] Some embodiments of method 300 further comprise obtaining service quality labels for the first and second directional data streams, wherein the service quality labels for the second directional data stream are affected by radio degradation. Training the service quality model further comprises using the service quality labels for the first and second directional data streams for training of the service quality model.

[0129] Some embodiments of method 300 further comprise obtaining radio performance metrics for the first and second directional data streams. Training the service quality model further comprises using the radio performance metrics for the first and second directional streams for training of the service quality model.

[0130] Figure 16 is a flow chart illustrating an exemplary method 400 of estimating service quality of data sessions in the wireless communication network. Method 400 may be carried out in a network node 450 (Fig. 19) in the wireless communication network or external network. The network node 450 obtains first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface (block 410). The network node 450 further obtains second transport performance metrics associated a second directional data stream transmitted over a communication path having a radio interface, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface (block 420). The network node 450 further estimates service quality of one or more data sessions in a service quality model generated from sample training data based on the first and second transport performance metrics (block 430). Some embodiments of method 350 further comprise obtaining radio performance metrics for the first and second directional data streams. Estimating service quality is further based on the radio performance metrics for the first and second directional streams.

[0131] An apparatus can perform any of the methods described herein by implementing any functional means, modules, units, or circuitry. In one embodiment, for example, the apparatuses comprise respective circuits or circuitry configured to perform the steps dedicated to performing certain functional processing and / or one or more microprocessors in conjunction with memory. For instance, the circuitry may include one or more microprocessors or microcontrollers, as well as other digital hardware, which may include Digital Signal Processors (DSPs), special-purpose digital logic, and the like. The processing circuitry may be configured to execute program code stored in memory, which may include one or several types of memory such as read-only memory (ROM), random-access memory, cache memory, flash memory devices, optical storage devices, etc. Program code stored in memory may include program instructions for executing one or more telecommunications and / or data communications protocols as well as instructions for carrying out one or more of the techniques described herein, in several embodiments. In embodiments that employ memory, the memory stores program code that, when executed by one or more processors, carries out the techniques described herein.

[0132] Figure 17 the main functional components of a measurement system for collection training data. The measurement system includes optional transmitting units 210 for transmitting data bi-directionally over a communication path having a radio interface, a first test system 220, a second test system 230, and a processing unit 250. The transmitting units 210 may comprise test data sources at the end points of the communication path. In some embodiments, the transmitting units may comprise a client and server communicating over a real network. The first test system 220 is disposed on a first side of the radio interface, e.g., client-side, and is configured to degrade a first directional data stream, a second directional data stream, or both. The second test system 230 is disposed on a second side of the radio interface, e.g., serverside, and is configured to measure one or more first transport performance metrics associated with the first directional stream, one or more second transport performance metrics associated with the second directional stream, or both. Figure 18 illustrates the main functional components of a network node 350 for training a service quality model to estimate service quality of data sessions in the wireless communication network. The network node comprises a first obtaining unit 360, a second obtaining unit 370, and a training unit 380. The various units may be implemented by one or more processors, hardware, software, firmware or some combination thereof. The first obtaining unit 360 is configured to obtain first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface. The second obtaining unit 370 is configured to obtain second transport performance metrics associated a second directional data stream transmitted over a communication path having a radio interface, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface. Training unit 380 is configured to train a service quality model using the first transport performance metrics for both the first and second directional data streams.

[0133] Figure 19 illustrates the main functional components of a network node 450 for estimating service quality of data sessions in the wireless communication network. The network node 450 comprises a first obtaining unit 460, a second obtaining unit 470, and a training unit 480. The various units may be implemented by one or more processors, hardware, software, firmware or some combination thereof. The first obtaining unit 460 is configured to obtain first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface. The second obtaining unit 470 is configured to obtain second transport performance metrics associated a second directional data stream transmitted over a communication path having a radio interface, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface. Estimating unit 480 is configured to train a service quality model using the first transport performance metrics for both the first and second directional data streams.

[0134] Figure 20 illustrates a network node 500 according to an embodiment for training and / or implementing a service quality model. The network node 500 comprises communication circuitry 520, processing circuitry 530, and memory 540. Communication circuitry 520 comprises network interface circuitry for communicating with other core network nodes over a communication network, such as an Internet Protocol (IP) network.

[0135] Processing circuitry 530 controls the overall operation of the network node 300 and is configured to perform one or more of the methods as herein described. The processing circuitry 530 may comprise one or more microprocessors, hardware, firmware, or a combination thereof. The processing circuitry 530 can be configured by software to perform all or part of the methods described herein.

[0136] Memory 540 comprises both volatile and non-volatile memory for storing computer program code and data needed by the processing circuitry 530 for operation. Memory 540 may comprise any tangible, non-transitory computer-readable storage medium for storing data including electronic, magnetic, optical, electromagnetic, or semiconductor data storage. Memory 540 stores a computer program 550 comprising executable instructions that configure the processing circuitry 530 to implement one or more of the methods described herein. A computer program in this regard may comprise one or more code modules corresponding to the means or units described above. In general, computer program instructions and configuration information are stored in a non-volatile memory, such as a ROM, erasable programmable read only memory (EPROM) or flash memory. Temporary data generated during operation may be stored in a volatile memory, such as a random access memory (RAM). In some embodiments, computer program 550 for configuring the processing circuitry 530 as herein described may be stored in a removable memory, such as a portable compact disc, portable digital video disc, or other removable media. The computer program 540 may also be embodied in a carrier such as an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0137] Those skilled in the art will also appreciate that embodiments herein further include corresponding computer programs. A computer program comprises instructions which, when executed on at least one processor of an apparatus, cause the apparatus to carry out any of the respective processing described above. A computer program in this regard may comprise one or more code modules corresponding to the means or units described above.

[0138] Embodiments further include a carrier containing such a computer program. This carrier may comprise one of an electronic signal, optical signal, radio signal, or computer readable storage medium. In this regard, embodiments herein also include a computer program product stored on a non-transitory computer readable (storage or recording) medium and comprising instructions that, when executed by a processor of an apparatus, cause the apparatus to perform as described above.

[0139] Embodiments further include a computer program product comprising program code portions for performing the steps of any of the embodiments herein when the computer program product is executed by a computing device. This computer program product may be stored on a computer readable recording medium.

[0140] The data collection methods herein described account for service quality degradations for real-time traffic types in the downlink radio path, which is a problem for rea-time traffic types using unreliable transport protocols. In the case of RTP transport, the methods estimate the service quality degradations in the downlink radio path even if RTCP is or other transport feedback is missing or encrypted.

[0141] The service quality models are based on transport performance metrics that are measurable for encrypted traffic as well. The method enables use of transport parameters, which may not be available in end-to-end probing.

[0142] Radio data are needed only for training the ML models but are not necessarily needed for the network-wide operating system. In this way, the analytics system is simpler, has a lower footprint, and can be used for radio network parts where radio data are not available, e.g., when radio is provided by another vendor.

[0143] The described training method accounts for differences in the way that the UL radio effects on the user plane traffic and the end-to-end service quality compared to the DL radio, even if the degradation (e.g., shielding or interference, handover) is the same.

[0144] The method can be applied when the user traffic types in UL and DL direction, i.e., in client-server and server-client direction is different. In the case of server-client communication, the observed traffic types should not necessarily be the same.

[0145] It exploits, however, that in peer-to-peer communication there are similar traffic types, which is used in case of estimating the degradation effect in the UL radio IF.

Claims

CLAIMSWhat is claimed is:1 . A method (100) of data collection for training of a service quality model used in a wireless communication network, wherein the method (100) is implemented by a measurement system including a first test system and a second test system disposed along a communication path carrying bi-directional data and having a radio interface, the method (100) comprising degrading (1 10), in the first test system disposed along the communication path on a first side of the radio interface, a first directional data stream, a second directional stream of the test data, or both; and measuring (120), in the second test system disposed along the communication path on a second side of the radio interface, one or more first transport performance metrics associated with the first directional stream, one or more second transport performance metrics associated with the second directional data stream, or both.

2. The method (100) of claim 1 , wherein degrading the first and / or directional streams of the test data comprises introducing simulated radio degradation in the first directional stream.

3. The method (100) of claim 2 or 3, wherein measuring one or more first transport performance metrics comprises measuring first transport performance metrics for the first directional stream in the second test system, wherein the first transport performance metrics associated with the first directional stream are affected by the radio degradation.

4. The method (100) of any one of claims 1 - 3, wherein degrading the first and / or directional streams of the test data comprises introducing simulated radio degradation in the second directional stream.

5. The method (100) of claim 4, wherein measuring one or more first transport parameters associated with the first and / or second directional streams comprises measuring second transport performance metrics for the second directional stream in the second test system, wherein the second transport performance metrics associated with the second directional stream are not affected by the radio degradation.

6. The method (100) of claim 5, wherein measuring one or more first transport parameters associated with the first and / or second directional streams comprisesmeasuring first transport performance metrics for the first directional stream and second transport performance metrics for the second directional streams in the second test system.

7. The method (100) of claim 2, further comprising measuring, in the first test system, one or more third transport performance metrics associated with the first directional stream, wherein the third transport performance metrics are not affected by the radio degradation.

8. The method (100) of any one of claims 1 - 7, wherein: the radio interface is provided by a radio access network (RAN); the first test system is located between the RAN and a first end point of the communication path; and the second test system is located between the RAN and a second end point of the communication path.

9. The method (100) of claim 8, wherein the RAN is part of a mobile network and wherein the second test system is located between the RAN and a core network in the mobile network.

10. The method (100) of claim 8, wherein the RAN is part of a mobile network and wherein the second test system is located in a path segment of the communication path between the mobile network and the second end point of the communication path.11 . The method (100) of claim 10, wherein the path segment includes a second radio interface.

12. The method (100) of any one of claims 1 - 11 , further comprising correlating the transport performance metrics with service quality labels.

13. The method (100) of claim 12, further comprising training a service quality model with the transport performance metrics and correlated service quality labels.

14. The method (100) of claim 12 or 1213 further comprising further comprising correlating radio performance metrics with the service quality labels.

15. The method (100) of claim 14, further comprising training a service quality model with the radio performance metrics and correlated service quality labels.

16. An measurement system (200) for collecting training data used for training a service quality model used in a wireless communication network, the measurement system comprising: a first test system (220) disposed along a communication path on a first side of the radio interface and configured to degrade a first directional data stream, a second directional data stream, or both; a second test system (230) disposed along the communication path on a second side of the radio interface and configured to measure one or more first transport performance metrics associated with the first directional stream, one or more second transport performance metrics associated with the second directional stream, or both.

17. The measurement system (200) of claim 16, wherein the first test system is configured to introduce simulated radio degradation in the first directional stream.

18. The measurement system (200) of claim 16 or 17, wherein the second test system is configured to measure first transport performance metrics for the first directional data stream, wherein the first transport performance metrics associated with the first directional data stream are affected by the radio degradation.

19. The measurement system (200) of any one of claims 16 - 18, wherein the first test system is configured to introduce simulated radio degradation in the second directional stream.

20. The measurement system (200) of claim 19, wherein the second test system is configured to measure second transport performance metrics associated with the second directional stream, wherein the second transport performance metrics associated with the second directional stream are not affected by the radio degradation.21 . The measurement system (200) of claim 20, wherein the second test system is configured to measure first transport performance metrics for the first directional data stream and second transport performance metrics for the second data directional stream.

22. The measurement system (200) of claim 16, further comprising measuring, in the first test system, one or more third transport performance metrics associated with thefirst directional stream, wherein the third transport performance metrics are not affected by the radio degradation.

23. The measurement system (200) of any one of claims 16 - 22, wherein: the radio interface is provided by a radio access network (RAN); the first test system is located between the RAN and a first end point of the communication path; and the second test system is located between the RAN and a second end point of the communication path.

24. The measurement system (200) of claim 23, wherein the RAN is part of a mobile network and wherein the second test system is located between the RAN and a core network in the mobile network.

25. The measurement system (200) of claim 23, wherein the RAN is part of a mobile network and wherein the second test system is located in a path segment of the communication path between the mobile network and the second end point of the communication path.

26. The measurement system(200) of claim 25, wherein the path segment includes a second radio interface.

27. The measurement system (200) of any one of claims 16 - 26, further comprising correlating the transport performance metrics with service quality labels.

28. The measurement system (200) of claim 27, further comprising training a service quality model with the transport performance metrics and correlated service quality labels.

29. The measurement system (200) of claim 27 or 28, further comprising further comprising correlating radio performance metrics with service quality labels.

30. The measurement system (200) of claim 29, further comprising training a service quality model with the radio performance metrics and correlated service quality labels.31 . The measurement system (200) of any one of claims 16 - 30, further comprising one or more transmitting units for transmitting the first and second directional streams over the communication path.

32. A computer program (270) comprising executable instructions that, when executed by processing circuitry (250) in a measurement system (200) causes the measurement system (200) to perform the method of any one of claims 1 - 15.

33. A carrier containing a computer program (270) of claim 32, wherein the carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

34. A non-transitory computer-readable storage medium (260) containing a computer program comprising executable instructions that, when executed by processing circuitry (250) in a measurement system (200) causes the measurement system to perform the method of any one of claims 1 - 15.

35. A method (300) of training of a service quality model used in a wireless communication network to estimate service quality of data sessions, the method (300) comprising: obtaining (310) first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface; and obtaining (320) second transport performance metrics associated a second directional data stream transmitted over a communication path having a radio interface, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface; and training (330) a service quality model using the first transport performance metrics for both the first and second directional data streams.

36. The method (300) of claim 35, further comprising: obtaining service quality labels for the first and second directional data streams, wherein the service quality labels for the second directional data stream are affected by radio degradation; andwherein training the service quality model further comprises using the service quality labels for the first and second directional data streams for training of the service quality model.

37. The method (300) of claim 35 or 36, further comprising: obtaining radio performance metrics for the first and second directional data streams; and wherein training the service quality model further comprises using the radio performance metrics for the first and second directional streams for training of the service quality model.

38. A network node (350, 500) for training a service quality model used in a wireless communication network to estimate service quality of data sessions, the training system being configured to: obtain first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface; and obtain second transport performance metrics associated a second directional data stream transmitted over a communication path having a radio interface, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface; and train a service quality model using the first transport performance metrics for both the first and second directional data streams.

39. The network node (350, 500) of claim 38, further configured to perform the method of any one of claims 36 - 37.

40. A network node (350, 500) for training of a service quality model used in a wireless communication network to estimate service quality of data sessions, the training system comprising processing circuitry (530) and memory (540) operatively coupled to the processing circuitry (530), wherein the memory (540) stores program instructions that, when executed by the processing circuitry (530), causes the training system to:obtain first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface; and obtain second transport performance metrics associated a second directional data stream transmitted over a communication path having a radio interface, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface; and train a service quality model using the first transport performance metrics for both the first and second directional data streams.41 . The network node (350, 500) of claim 40, further configured to perform the method of any one of claims 36 - 37.

42. A computer program (550) comprising executable instructions that, when executed by processing circuitry (530) in a network node causes the network node to perform the method any one of claims 35 - 37.

43. A carrier containing a computer program (550) of claim 42, wherein the carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

44. A non-transitory computer-readable storage medium (540) containing a computer program (550) comprising executable instructions that, w when executed by processing circuitry (530) in a network node causes the network node to perform the method of any one of claims 35 - 37.

45. A method (400) of estimating service quality for one or more data sessions in a wireless communication network, the method (400) comprising: obtaining (410) first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface; andobtaining (420) second transport performance metrics associated a second directional data stream transmitted over a communication path having a radio interface, wherein the second transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface; and estimating (430) service quality of one or more data sessions in a service quality model generated from sample training data based on the first and second transport performance metrics.

46. The method (400) of claim 45, further comprising: obtaining radio performance metrics for the first and second directional data streams; and wherein estimating service quality is further based on the radio performance metrics for the first and second directional streams.

47. A network node (450, 500) for estimating service quality for one or more data sessions in a wireless communication network, the apparatus being configured to: obtain first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface; and obtain second transport performance metrics associated a second directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface; and estimate service quality of one or more data sessions in a service quality model generated from sample training data based on the first and second transport performance metrics.

48. The network node (450, 500) of claim 47, further configured to perform the method of claim 46.

49. A network node (450, 500) for training of a service quality model used in a wireless communication network for detection of irregularities, the training system comprising processing circuitry (530) and memory (540) operatively coupled to the processing circuitry (530), wherein the memory (540) stores program instructions that, when executed by the processing circuitry (530), caused the training system to: obtain first transport performance metrics associated with a first directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the first directional data stream are affected by radio degradation introduced by the radio interface; and obtain second transport performance metrics associated a second directional data stream transmitted over a communication path having a radio interface, wherein the first transport performance metrics for the second directional data stream are not affected by the radio degradation introduced by the radio interface; and estimate service quality of one or more data sessions in a service quality model generated from sample training data based on the first and second transport performance metrics.

50. The network node (450, 500) of claim 49, further configured to perform the method of claim 46.51 . A computer program (550) comprising executable instructions that, when executed by processing circuitry (530) in a network node causes the network node to perform the method of claim 45 or 46.

52. A carrier containing a computer program (550) of claim 51 , wherein the carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

53. A non-transitory computer-readable storage medium (540) containing a computer program (550) comprising executable instructions that, when executed by processing circuitry (530) in a network node causes the network node to perform the method of any one of claims 45 or 46.

Images