Quality of experience metrics, reporting and configuration for real-time media communication services

Abstract

Some embodiments of the method may include configuring a device to process a set of one or more metrics; collecting the set of one or more metrics; generating a metric report including the collected set of one or more metrics; and reporting the metric report including the collected set of one or more metrics to a server. In some embodiments, the method may further include receiving a metric configuration message, wherein the metric configuration message may include a metric scheme, and wherein the metric scheme includes at least one rule or step corresponding to at least one metric in the set of one or more metrics. In some embodiments, the method may further include establishing a media session with an Application Function (AF) server for 5G media streaming services; and determining one or more metrics to be collected for the media session.

Description

Cross-reference to related applications

[0001] This application claims the benefit of U.S. Provisional Patent Application 63 / 547,528, filed November 6, 2023, entitled “QUALITY OF EXPERIENCE METRICS, REPORTING AND CONFIGURATION FOR REAL-TIME MEDIA COMMUNICATION SERVICES”, which is incorporated herein by reference in its entirety. Background Technology

[0002] Long Term Evolution (LTE), 5G New Radio (5GNR), and other emerging communication technologies enable wireless devices to transmit information at data rates several orders of magnitude higher (e.g., in gigabits per second) than those available just a few years ago. Today’s communication networks are also more secure, more resistant to multipath fading, allow for lower latency in network traffic, and provide better communication efficiency (e.g., in bits per second per unit of bandwidth used). Summary of the Invention

[0003] A first exemplary method according to some embodiments may include: establishing a media session with an application function (AF) server for 5G media streaming services; in response to establishing the media session, determining one or more metrics to be collected for the media session, wherein determining the one or more metrics to be collected for the media session is at least in part based on one or more metric measurement requirements, collection requirements, and reporting requirements at the start of the media session; receiving a metric configuration message corresponding to the one or more metrics, wherein the metric configuration message includes a metric scheme, and wherein the metric scheme includes at least one rule or step corresponding to at least one of the one or more metrics; configuring a device to process the one or more metrics; collecting the one or more metrics; generating a metric report including the collected one or more metrics; and reporting the metric report including the collected one or more metrics to a server.

[0004] In some embodiments of the first exemplary method, at least one of the one or more metrics is a Quality of Experience (QoE) metric.

[0005] In some embodiments of the first exemplary method, the QoE metric is associated with an application or content service provided from outside the 5GS network.

[0006] Some embodiments of the first exemplary method may also include requesting the QoE metric from a device outside the 5GS network.

[0007] In some embodiments of the first exemplary method, at least one of the one or more metrics is a non-3GPP (non-3rd Generation Partnership Project) metric.

[0008] In some embodiments of the first exemplary method, the metric configuration message is received from the application function (AF) server of the 5GS network.

[0009] In some embodiments of the first exemplary method, the metric configuration message is received via the application layer control plane.

[0010] For some embodiments of the first exemplary method, generating a metric report includes at least one of filtering, aggregating, and reformatting the collected one or more metrics.

[0011] Some embodiments of the first exemplary method may also include receiving information indicating one or more rules for processing at least one of one or more metrics.

[0012] Some embodiments of the first exemplary method may also include measuring at least one of the one or more metrics.

[0013] For some embodiments of the first exemplary method, the reporting metrics report includes: sending an HTTP POST message.

[0014] In some embodiments of the first exemplary method, at least one of the one or more metrics is associated with a streaming service.

[0015] For some embodiments of the first exemplary method, the reporting of metrics includes sending the metrics report to at least one of the following: an application server (AS) for the 5G media streaming service, an operator of the 5GS network, an operations management and maintenance (OAS) server, and a third-party server.

[0016] Some embodiments of the first exemplary method may further include: updating the metric report; and reporting the updated metric report to a server.

[0017] For some embodiments of the first exemplary method, the report metrics report includes: using the Real-Time Media Communication (RTC) interface in a 5G system.

[0018] For some embodiments of the first exemplary method, generating a metric report includes using a format corresponding to 5G system requirements.

[0019] For some embodiments of the first exemplary method, generating a metric report includes using a format corresponding to the DASH streaming service.

[0020] For some embodiments of the first exemplary method, generating a metric report includes using an XML format.

[0021] For some embodiments of the first exemplary method, collecting one or more metrics includes: receiving at least one measurement corresponding to one of the one or more metrics.

[0022] Some embodiments of the first exemplary method may also include triggering the reporting of one or more metrics by a 5G-RTC client, wherein the triggering includes using an Open Mobile Alliance Device Management (OMA-DM) Quality of Experience (QoE) management object.

[0023] Some embodiments of the first exemplary method may also include triggering the reporting of one or more metrics by a 5G-RTC client, wherein the triggering includes using the QMC function.

[0024] For some embodiments of the first exemplary method, reporting the metric report includes sending the metric report via an RRC message.

[0025] For some embodiments of the first exemplary method, reporting the metric report includes sending an HTTP POST request carrying metadata in XML format.

[0026] For some embodiments of the first exemplary method, the reported metrics report includes: using a 5G-RTC quality reporting scheme.

[0027] In some embodiments of the first exemplary method, at least one of the one or more metrics is a damage duration metric.

[0028] In some embodiments of the first exemplary method, at least one of the one or more metrics is the continuous loss of RTP packets.

[0029] In some embodiments of the first exemplary method, at least one of the one or more metrics is the frame rate.

[0030] In some embodiments of the first exemplary method, at least one of the one or more metrics is jitter duration.

[0031] In some embodiments of the first exemplary method, at least one of the said one or more metrics is the duration of synchronization loss.

[0032] In some embodiments of the first exemplary method, at least one of the one or more metrics is round-trip time.

[0033] In some embodiments of the first exemplary method, at least one of the one or more metrics is the average codec bit rate.

[0034] A first exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: configure the apparatus to process a set of one or more metrics; collect the set of one or more metrics; generate a metric report including the collected set of one or more metrics; and report the metric report including the collected set of one or more metrics to a server.

[0035] A second exemplary method according to some embodiments may include: configuring a server to process a set of one or more metrics; collecting the set of one or more metrics; and sending the set of one or more metrics to a user equipment (UE) device.

[0036] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is a Quality of Experience (QoE) metric.

[0037] In some embodiments of the second exemplary method, the QoE metric is associated with an application or content service provided from outside the 5GS network.

[0038] Some embodiments of the second exemplary method may also include requesting the QoE metric from a device outside the 5GS network.

[0039] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is a non-3rd Generation Partnership Project (Non-3GPP) metric.

[0040] Some embodiments of the second exemplary method may further include sending a metric configuration message corresponding to at least one of the set of one or more metrics.

[0041] In some embodiments of the second exemplary method, the metric configuration message is sent via the application function (AF) server of the 5GS network.

[0042] In some embodiments of the second exemplary method, the metric configuration message is sent via the application layer control plane.

[0043] In some embodiments of the second exemplary method, the metric configuration message includes a metric scheme, and the metric scheme includes at least one rule or step corresponding to at least one of the set of one or more metrics.

[0044] Some embodiments of the second exemplary method may further include determining one or more metrics to be collected.

[0045] In some embodiments of the second exemplary method, determining one or more metrics to be collected for a media session is based at least in part on one or more metric measurement requirements, collection requirements, and reporting requirements at the start of the media session.

[0046] Some embodiments of the second exemplary method may also include at least one of filtering, aggregating, and reformatting the collected set of one or more metrics.

[0047] Some embodiments of the second exemplary method may also include sending information indicating one or more rules for processing at least one metric in the set of one or more metrics.

[0048] Some embodiments of the second exemplary method may also include measuring at least one of the set of one or more measures.

[0049] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is associated with a streaming service.

[0050] In some embodiments of the second exemplary method, sending the set of one or more metrics to the user equipment (UE) device includes using a real-time media communication (RTC) interface in a 5G system.

[0051] In some embodiments of the second exemplary method, collecting the set of one or more metrics includes performing at least one measurement corresponding to one of the set of one or more metrics.

[0052] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is a damage duration metric.

[0053] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is the continuous loss of RTP packets.

[0054] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is the frame rate.

[0055] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is jitter duration.

[0056] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is the duration of synchronization loss.

[0057] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is round-trip time.

[0058] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is the average codec bit rate.

[0059] A second exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: configure a server to process a set of one or more metrics; collect the set of one or more metrics; and send the set of one or more metrics to a user equipment (UE) apparatus.

[0060] A third exemplary method according to some embodiments may include: configuring a quality reporting scheme by a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint for measuring and reporting a set of Quality of Experience (QoE) metrics; and, according to the QoE metric reporting protocol and the quality reporting scheme, having the 5G-RTC user equipment or endpoint report information related to a set of QoE metrics to a 5G-RTC metric server by means of transmission.

[0061] A third exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: configure a quality reporting scheme by a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint for measuring and reporting a set of Quality of Experience (QoE) metrics; and, according to the QoE metric reporting protocol and the quality reporting scheme, report information related to a set of QoE metrics to a 5G-RTC metric server by means of transmission.

[0062] A fourth exemplary method according to some embodiments may include: configuring a quality reporting scheme by a first network device for measuring and reporting a set of quality of experience (QoE) metrics; and sending information related to the set of QoE metrics to a second network device by the first network device in accordance with the QoE metric reporting protocol and the quality reporting scheme.

[0063] A fourth exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: be configured by a first network device to measure and report a set of quality of experience (QoE) metrics using a quality reporting scheme; and, according to a QoE metric reporting protocol and the quality reporting scheme, be transmitted by the first network device to a second network device containing information related to the set of QoE metrics.

[0064] A fifth exemplary method according to some embodiments may include: a first network device sending information related to a set of Quality of Experience (QoE) metrics to a second network device according to a QoE metric reporting protocol and the quality reporting scheme, wherein the quality reporting scheme is configured to measure and report the information related to the set of QoE metrics.

[0065] In some embodiments of the fifth exemplary method, the set of QoE metrics includes a damage duration metric.

[0066] In some embodiments of the fifth exemplary method, the set of QoE metrics includes the successive loss of RTP packets.

[0067] In some embodiments of the fifth exemplary method, the set of QoE metrics includes frame rate.

[0068] In some embodiments of the fifth exemplary method, the set of QoE metrics includes jitter duration.

[0069] In some embodiments of the fifth exemplary method, the set of QoE metrics includes synchronization loss duration.

[0070] In some embodiments of the fifth exemplary method, the set of QoE metrics includes round-trip time.

[0071] In some embodiments of the fifth exemplary method, the set of QoE metrics includes the average codec bit rate.

[0072] A fifth exemplary device according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: transmit information related to a set of Quality of Experience (QoE) metrics to a second network device, according to a QoE metric reporting protocol and the quality reporting scheme, wherein the quality reporting scheme is configured to measure and report the information related to the set of QoE metrics.

[0073] A sixth exemplary method according to some embodiments may include: configuring an XML metrics scheme by a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint for measuring and reporting a set of Quality of Experience (QoE) metrics; and, according to the XML metrics scheme, reporting information related to the set of QoE metrics to a 5G-RTC metrics server by the 5G-RTC user equipment or endpoint.

[0074] A sixth exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: configure an XML metrics scheme for measuring and reporting a set of quality of experience (QoE) metrics by a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint; and, according to the XML metrics scheme, report information related to the set of QoE metrics to a 5G-RTC metrics server by the 5G-RTC user equipment or endpoint.

[0075] A seventh exemplary method according to some embodiments may include: configuring a quality metric reporting scheme by a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint for measuring and reporting a set of quality of experience (QoE) metrics; and reporting information related to the set of QoE metrics to a 5G-RTC metric server according to the QoE metric reporting protocol and the quality reporting scheme.

[0076] A seventh exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: configure a quality metric reporting scheme by a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint for measuring and reporting a set of Quality of Experience (QoE) metrics; and, according to the QoE metric reporting protocol and the quality reporting scheme, report information related to the set of QoE metrics to a 5G-RTC metric server.

[0077] An eighth exemplary method according to some embodiments may include: configuring a 5G real-time media communication (5G-RTC) user equipment or endpoint to report a set of quality of experience (QoE) metrics to a metrics server; collecting information related to the set of QoE metrics; and having the 5G-RTC user equipment or endpoint report the information related to the set of QoE metrics to the metrics server.

[0078] An eighth exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: configure a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint to report a set of Quality of Experience (QoE) metrics to a metrics server; collect information related to the set of QoE metrics; and have the 5G-RTC user equipment or endpoint report the information related to the set of QoE metrics to the metrics server.

[0079] A ninth exemplary method according to some embodiments may include: configuring a 5G Real-Time Media Communication (5G-RTC) user equipment to measure and report a set of quality of experience (QoE) metrics; and having the 5G-RTC user equipment report information related to the set of QoE metrics via an RTC-1 interface.

[0080] A ninth exemplary device according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: configure a 5G Real-Time Media Communication (5G-RTC) user equipment to measure and report a set of quality of experience (QoE) metrics; and have the 5G-RTC user equipment report information related to the set of QoE metrics via an RTC-1 interface.

[0081] A tenth exemplary method according to some embodiments may include: receiving a metric configuration message corresponding to a set of one or more metrics; configuring a device to process the set of one or more metrics; collecting the set of one or more metrics; generating a metric report including the collected set of one or more metrics; and reporting the metric report including the collected set of one or more metrics to a server.

[0082] A tenth exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: receive a metric configuration message corresponding to a set of one or more metrics; configure the apparatus to process the set of one or more metrics; collect the set of one or more metrics; generate a metric report including the collected set of one or more metrics; and report the metric report including the collected set of one or more metrics to a server.

[0083] An eleventh exemplary device according to some embodiments may include at least one processor configured to perform any of the methods listed above.

[0084] A twelfth exemplary apparatus according to some embodiments may include a computer-readable medium storing instructions for causing one or more processors to perform any of the methods listed above.

[0085] A thirteenth exemplary apparatus according to some embodiments may include at least one processor and at least one non-transitory computer-readable medium storing instructions for causing the at least one processor to perform any of the methods listed above.

[0086] Exemplary signals according to some embodiments may include a bitstream generated according to any of the methods listed above.

[0087] One or more embodiments of the present invention also provide a computer-readable storage medium storing instructions for performing bidirectional optical flow, encoding, or decoding video data according to any of the methods described above. This embodiment also provides a computer-readable storage medium storing a bitstream generated according to the methods described above. This embodiment also provides a method and apparatus for transmitting a bitstream generated according to the methods described above. This embodiment also provides a computer program product including instructions for performing any of the described methods. Attached Figure Description

[0088] Figure 1A This is a system diagram illustrating an exemplary communication system according to some embodiments.

[0089] Figure 1B This illustrates that, according to some embodiments, it is possible to Figure 1A The system diagram shows an exemplary wireless transmit / receive unit (WTRU) used in the communication system.

[0090] Figure 1C This is a system diagram illustrating a set of exemplary interfaces for a system according to some embodiments.

[0091] Figure 2A This is a schematic side view illustrating an exemplary waveguide display that can be used with extended reality (XR) applications according to some embodiments.

[0092] Figure 2B This is a schematic side view illustrating an exemplary alternative display type that can be used with extended reality applications according to some embodiments.

[0093] Figure 2C This is a schematic side view illustrating an exemplary alternative display type that can be used with extended reality applications according to some embodiments.

[0094] Figure 3 This is a system diagram illustrating a set of exemplary interfaces for 5G media streaming within a 5G system, according to some embodiments.

[0095] Figure 4 This is a system diagram illustrating a set of exemplary interfaces for a 5G media streaming architecture according to some embodiments.

[0096] Figure 5 This is a system diagram illustrating a set of exemplary interfaces for real-time media communication in a 5G system according to some embodiments.

[0097] Figure 6 This is a system diagram illustrating a set of exemplary interfaces for a real-time media communication architecture according to some embodiments.

[0098] Figure 7 This is a flowchart illustrating an exemplary process for reporting quality metrics according to some embodiments.

[0099] Figure 8 This is a flowchart illustrating an exemplary process of a server sending quality metrics according to some embodiments.

[0100] The entities, connections, arrangements, etc., depicted and described in connection with the various figures are presented by way of example rather than limitation. Therefore, any and all statements or other indications regarding what a particular figure “depicts,” that a particular element or entity in a particular figure “is” or “has,” and any and all similar statements (which, if isolated and taken out of context, might be interpreted as absolute and therefore limiting) are only to be properly interpreted as being preceded by an implicit statement such as “in at least one embodiment.” For the sake of brevity and clarity, this implicit preamble is not repeated in the detailed description. Detailed Implementation

[0101] Figure 1A This diagram illustrates an example communication system 100 in which one or more of the disclosed embodiments may be implemented. The communication system 100 may be a multiple access system that provides content such as voice, data, video, messaging, and broadcasting to multiple wireless users. The communication system 100 enables multiple wireless users to access such content by sharing system resources, including wireless bandwidth. For example, the communication system 100 may employ one or more channel access methods, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Zero-Tail Unique Word DFT Extended OFDM (ZT-UW DTS-s OFDM), Unique Word OFDM (UW-OFDM), Resource Block Filtered OFDM, Filter Bank Multicarrier (FBMC), etc.

[0102] like Figure 1AAs shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106, Public Switched Telephone Network (PSTN) 108, Internet 110, and other networks 112. However, it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Any of the WTRUs 102a, 102b, 102c, and 102d may be any type of device configured to operate and / or communicate in a wireless environment. For example, WTRUs 102a, 102b, 102c, and 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or MiFi devices, Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain environments), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. Any of WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0103] The communication system 100 may also include base station 114a and / or base station 114b. Each of base stations 114a and 114b can be any type of device configured to wirelessly connect to at least one of WTRUs 102a, 102b, 102c, and 102d to facilitate access to one or more communication networks, such as CN 106, the Internet 110, and / or other networks 112. As an example, base stations 114a and 114b may be base transceiver stations (BTS), NodeBs, eNodeBs, home NodeBs, home eNodeBs, gNBs, NRNodeBs, site controllers, access points (APs), wireless routers, etc. Although base stations 114a and 114b are each depicted as a single element, it will be understood that base stations 114a and 114b may include any number of interconnected base stations and / or network elements.

[0104] Base station 114a may be part of RAN 104 / 113, and may also include other base stations and / or network elements (not shown), such as base station controllers (BSCs), radio network controllers (RNCs), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage of a specific geographic area, which may be relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, the cell associated with base station 114a may be divided into three sectors. Thus, in one embodiment, base station 114a may include three transceivers, i.e., one transceiver per sector of the cell. In one embodiment, base station 114a may employ multiple-input multiple-output (MIMO) technology and may use multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0105] Base stations 114a and 114b can communicate with one or more of WTRUs 102a, 102b, 102c, and 102d via air interface 116, which can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). Air interface 116 can be established using any suitable radio access technology (RAT).

[0106] More specifically, as described above, the communication system 100 can be a multiple access system and can employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base stations 114a and WTRUs 102a, 102b, and 102c in RAN 104 / 113 can implement radio technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which can use Wideband CDMA (WCDMA) to establish the air interface 116. WCDMA can include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA can include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0107] In one embodiment, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which can use Long Term Evolution (LTE) and / or LTE-A Advanced (LTE-A) and / or LTE-A Pro Advanced (LTE-A Pro) to establish air interface 116.

[0108] In one embodiment, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as NR radio access, which can use New Radio (NR) to establish air interface 116.

[0109] In one embodiment, base station 114a and WTRUs 102a, 102b, and 102c can implement multiple radio access technologies. For example, base station 114a and WTRUs 102a, 102b, and 102c can, for instance, use a dual connectivity (DC) principle to implement both LTE and NR radio access. Therefore, the air interface used by WTRUs 102a, 102b, and 102c can be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0110] In other embodiments, base station 114a and WTRUs 102a, 102b, 102c may implement wireless technologies such as IEEE 802.11 (i.e., Wi-Fi), IEEE 802.16 (i.e., Global Microwave Access Interoperability (WiMAX)), CDMA2000, CDMA 2000 1X, CDMA 2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rate GSM Evolution (EDGE), GSMEDGE (GERAN), etc.

[0111] Figure 1ABase station 114b can be, for example, a wireless router, a home NodeB, a home eNodeB, or an access point, and can utilize any suitable RAT to facilitate wireless connectivity in a local area such as a business premises, home, vehicle, campus, industrial facility, air corridor (e.g., for drone use), road, etc. In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, base station 114b and WTRUs 102c, 102d can utilize cellular-based RATs (e.g., WCDMA, CDMA 2000, GSM, LTE-A, LTE-A Pro, NR, etc.) to establish picocells or femtocells. Figure 1A As shown, base station 114b can have a direct connection to Internet 110. Therefore, base station 114b does not need to access Internet 110 via CN 106.

[0112] RAN 104 / 113 can communicate with CN 106, which can be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRUs 102a, 102b, 102c, and 102d. Data may have varying Quality of Service (QoS) requirements, such as different throughput requirements, latency requirements, fault tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN 106 can provide call control, billing services, location-based services, prepaid calling, internet connectivity, video distribution, etc., and / or perform advanced security functions (such as user authentication). Although in Figure 1A Although not shown, it should be understood that RAN 104 / 113 and / or CN 106 can communicate directly or indirectly with other RANs that use the same RAT as RAN 104 / 113 or a different RAT. For example, in addition to being connected to RAN 104 / 113, which can utilize NR radio technology, CN 106 can also communicate with another RAN (not shown) that uses GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0113] CN 106 can also serve as a gateway for WTRUs 102a, 102b, 102c, and 102d to access PSTN 108, the Internet 110, and / or other networks 112. PSTN 108 may include a circuit-switched telephone network providing Common Old-Style Telephone Service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices using common communication protocols, such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) from the TCP / IP Internet Protocol suite. Network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs, which may use the same RAT as RAN 104 / 113 or a different RAT.

[0114] Some or all of the WTRUs 102a, 102b, 102c, and 102d in the communication system 100 may include multi-mode capability (e.g., WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers to communicate with different wireless networks via different wireless links). For example, Figure 1A The WTRU 102c shown can be configured to communicate with base station 114a, which can use cellular-based radio technology, and with base station 114b, which can use IEEE 802 radio technology.

[0115] Figure 1B This is a system diagram illustrating example WTRU 102. (See diagram below.) Figure 1B As shown, WTRU 102 may include a processor 118, a transceiver 120, a transmitting / receiving element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a Global Positioning System (GPS) chipset 136, and / or other peripheral devices 138, etc. It is understood that WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with the embodiments.

[0116] Processor 118 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. Processor 118 may perform signal encoding, data processing, power control, input / output processing, and / or any other functions that enable WTRU 102 to operate in a wireless environment. Processor 118 may be coupled to transceiver 120, and transceiver 120 may be coupled to transmitting / receiving element 122. Although Figure 1B While the processor 118 and transceiver 120 are depicted as separate components, it should be understood that the processor 118 and transceiver 120 may be integrated together in an electronic package or chip.

[0117] Transmitting / receiving element 122 can be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via air interface 116. For example, in one embodiment, transmitting / receiving element 122 can be an antenna configured to transmit and / or receive RF signals. In one embodiment, transmitting / receiving element 122 can be a transmitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another embodiment, transmitting / receiving element 122 can be configured to transmit and / or receive both RF and optical signals. It should be understood that transmitting / receiving element 122 can be configured to transmit and / or receive any combination of wireless signals.

[0118] Although the transmitting / receiving element 122 is in Figure 1B While described as a single element, WTRU 102 may include any number of transmitting / receiving elements 122. More specifically, WTRU 102 may use MIMO technology. Therefore, in one embodiment, WTRU 102 may include two or more transmitting / receiving elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals via air interface 116.

[0119] Transceiver 120 can be configured to modulate signals transmitted by transmitting / receiving element 122 and demodulate signals received by transmitting / receiving element 122. As described above, WTRU 102 can have multimode capability. Therefore, transceiver 120 can include multiple transceivers to enable WTRU 102 to communicate via various RATs, such as NR and IEEE 802.11.

[0120] The processor 118 of WTRU 102 may be coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) unit or an organic light-emitting diode (OLED) display unit) and may receive user input data therefrom. The processor 118 may also output user data to the speaker / microphone 124, keypad 126, and / or display / touchpad 128. Additionally, the processor 118 may access information from any suitable type of memory and store data in said memory, such as non-removable memory 130 and / or removable memory 132. Non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 132 may include a user identification module (SIM) card, memory stick, secure digital storage (SD) card, etc. In other embodiments, the processor 118 may access information from memory that is not physically located on WTRU 102 (e.g., located on a server or home computer (not shown)) and store data in said memory.

[0121] The processor 118 can receive power from the power supply 134 and can be configured to distribute and / or control power to other components in the WTRU 102. The power supply 134 can be any suitable device for powering the WTRU 102. For example, the power supply 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0122] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) about the current location of the WTRU 102. In addition to, or alternatively to, the information from the GPS chipset 136, the WTRU 102 may receive location information from base stations (e.g., base stations 114a, 114b) via the air interface 116, and / or determine its location based on the timing of signals received from two or more neighboring base stations. It should be understood that the WTRU 102 may acquire location information using any suitable location determination method, while remaining consistent with the embodiments.

[0123] The processor 118 may also be coupled to other peripheral devices 138, which may include one or more software and / or hardware modules providing additional features, functions, and / or wired or wireless connectivity. For example, peripheral devices 138 may include accelerometers, electronic compasses, satellite transceivers, digital cameras (for photos and / or video), Universal Serial Bus (USB) ports, vibration devices, television transceivers, hands-free headsets, Bluetooth® modules, FM radio units, digital music players, media players, video game player modules, internet browsers, virtual reality and / or augmented reality (VR / AR) devices, activity trackers, etc. Peripheral devices 138 may include one or more sensors, which may be one or more of the following: gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors; geolocation sensors; altimeters, light sensors, touch sensors, magnetometers, barometers, gesture sensors, biometric sensors, and / or humidity sensors.

[0124] WTRU 102 may include a full-duplex radio, for which the transmission and reception of some or all of the signals (e.g., signals associated with specific subframes for uplink (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and / or substantially eliminate self-interference through hardware (e.g., chokes) or through signal processing by a processor (e.g., a separate processor (not shown) or processor 118). In one embodiment, WTRU 102 may include a half-duplex radio, for which the transmission and reception of some or all of the signals (e.g., signals associated with specific subframes for uplink (e.g., for transmission) or downlink (e.g., for reception)) are separate.

[0125] Although WTRU is Figure 1A-1B While described as a wireless terminal, it is conceivable that in some representative embodiments, such a terminal may use (e.g., temporarily or permanently) a wired communication interface with a communication network.

[0126] In a representative embodiment, the other network 112 may be a WLAN.

[0127] Given Figure 1A-1B As described herein, one or more of the functions described herein may be performed by one or more emulation devices (not shown). An emulation device may be one or more devices configured to emulate one or more of the functions described herein. For example, an emulation device may be used to test other devices and / or simulate network and / or WTRU functions.

[0128] Simulation devices can be designed to perform one or more tests on other devices in laboratory and / or carrier network environments. For example, one or more simulation devices may perform one or more functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices within the communication network. The one or more simulation devices may perform one or more or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. Simulation devices may be directly coupled to another device for testing purposes and / or may use over-the-air wireless communication to perform tests.

[0129] One or more simulation devices can perform one or more functions (including all functions) without being implemented / deployed as part of a wired and / or wireless communication network. For example, simulation devices can be used in test scenarios within a test laboratory and / or an undeployed (e.g., tested) wired and / or wireless communication network to perform testing of one or more components. One or more simulation devices can be test equipment. Simulation devices can transmit and / or receive data using direct RF coupling and / or wireless communication via RF circuitry (e.g., which may include one or more antennas).

[0130] Figure 1C This is a system diagram illustrating a set of exemplary interfaces for a system according to some embodiments. In some embodiments, an extended reality display device and its control electronics can be implemented. System 150 can be implemented as a device including the various components described below and configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 150 can be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing elements and encoder / decoder elements of system 150 are distributed across multiple ICs and / or discrete components. In various embodiments, system 150 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 150 is configured to implement one or more aspects described herein.

[0131] System 150 includes at least one processor 152 configured to execute instructions loaded therein for implementing various aspects, such as those described herein. Processor 152 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 150 includes at least one memory 154 (e.g., a volatile memory device and / or a non-volatile memory device). System 150 may include a storage device 158, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 158 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0132] System 150 includes an encoder / decoder module 156 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 156 may include its own processor and memory. The encoder / decoder module 156 represents a module that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Alternatively, the encoder / decoder module 156 may be implemented as a separate element of system 150, or may be incorporated into processor 152 as a combination of hardware and software as known to those skilled in the art.

[0133] Program code to be loaded onto processor 152 or encoder / decoder 156 to execute the various aspects described herein may be stored in storage device 158 and subsequently loaded onto memory 154 for execution by processor 152. According to various embodiments, one or more of processor 152, memory 154, storage device 158, and encoder / decoder module 156 may store one or more of various items during the execution of the processes described herein. These stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0134] In some embodiments, the memory within processor 152 and / or encoder / decoder module 156 is used to store instructions and provides working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be processor 152 or encoder / decoder module 152) is used for one or more of these functions. External memory may be memory 154 and / or storage device 158, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory, such as RAM, is used as working memory for video encoding and decoding operations, such as working memory for MPEG-2 (MPEG stands for Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Universal Video Coding, a new standard developed by the Joint Video Experts Group JVET).

[0135] As shown in box 172, input to the components of system 150 can be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section that receives radio frequency (RF) signals transmitted over the air, for example by a broadcaster, (ii) a component (COMP) input terminal (or a set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Figure 1C Other examples not shown include composite video.

[0136] In various embodiments, the input device of block 172 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) down-converting the selected signal, (iii) re-band-limiting the signal to a narrower band to select, for example, a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the aforementioned (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0137] Additionally, USB and / or HDMI terminals may include corresponding interface processors for connecting system 150 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) can be implemented as needed, for example, in a separate input processing IC or processor 152. Similarly, various aspects of USB or HDMI interface processing can be implemented as needed within a separate interface IC or within processor 152. The demodulated, error-corrected, and demultiplexed stream is provided to various processing elements (including, for example, processor 152 and encoder / decoder 156 operating in conjunction with memory and storage elements) to process the data stream as needed for presentation on the output device.

[0138] Various components of system 150 can be housed within an integrated housing. Within the integrated housing, the various components can be interconnected and data can be transmitted therebetween using suitable connection means 174 (e.g., internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards).

[0139] System 150 includes a communication interface 160 that enables communication with other devices via a communication channel 162. The communication interface 160 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 162. The communication interface 160 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 162 may be implemented, for example, within a wired and / or wireless medium.

[0140] In various embodiments, a wireless network (e.g., a Wi-Fi network, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers)) is used to stream or otherwise provide data to system 150. In these embodiments, the Wi-Fi signal is received via a communication channel 162 and a communication interface 160 suitable for Wi-Fi communication. The communication channel 162 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box to provide streaming data to system 150, delivering data via an HDMI connection in input box 172. Still other embodiments use an RF connection in input box 172 to provide streaming data to system 150. As described above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0141] System 150 can provide output signals to various output devices, including a display 176, a speaker 178, and other peripheral devices 180. The display 176 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 176 can be used in a television, tablet computer, laptop computer, cellular phone (mobile phone), or other device. The display 176 can also be integrated with other components (e.g., as in a smartphone) or stand alone (e.g., as an external monitor for a laptop computer). In various examples of embodiments, other peripheral devices 180 include one or more of a standalone digital video disc (or digital multifunction disc, used for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 180 that provide functionality based on the output of system 150. For example, a disc player performs the function of playing the output of system 150.

[0142] In various embodiments, signaling (such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention) is used to deliver control signals between system 150 and display 176, speaker 178, or other peripheral devices 180. Output devices may be communicatively coupled to system 150 via dedicated connections through corresponding interfaces 164, 166, and 168. Alternatively, output devices may be connected to system 150 via communication interface 160 using communication channel 162. Display 176 and speaker 178 may be integrated into a single unit within an electronic device (e.g., a television set) along with other components of system 150. In various embodiments, display interface 164 includes a display driver, such as a timing controller (TCon) chip.

[0143] For example, if the RF portion of input 172 is part of a separate set-top box, then display 176 and speaker 178 can alternatively be separated from one or more of the other components. In various embodiments where display 176 and speaker 178 are external components, output signals can be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.

[0144] System 150 may include one or more sensor devices 168. Examples of sensor devices that may be used include one or more GPS sensors, gyroscope sensors, accelerometers, light sensors, cameras, depth cameras, microphones, and / or magnetometers. Such sensors can be used to determine information such as the user's position and orientation. Where system 150 is used as a control module (such as control modules 124, 132) for an extended reality display, the user's position and orientation can be used to determine how image data is rendered so that the user perceives the correct portion of a virtual object or scene from the correct viewpoint. In the case of a head-mounted display device, the position and orientation of the device itself can be used to determine the user's position and orientation for the purpose of rendering virtual content. In the case of other display devices such as telephones, tablets, computer monitors, or televisions, other inputs can be used to determine the user's position and orientation for rendering content. For example, the user can use a touchscreen, keypad or keyboard, trackball, joystick, or other inputs to select and / or adjust the desired viewpoint and / or viewing direction. Where the display device has sensors such as accelerometers and / or gyroscopes, the viewpoint and orientation used for rendering content can be selected and / or adjusted based on the movement of the display device.

[0145] These embodiments can be executed by processor 152 or computer software implemented by a combination of hardware and software. As a non-limiting example, embodiments can be implemented by one or more integrated circuits. As a non-limiting example, memory 154 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. As a non-limiting example, processor 152 can be of any type suitable for the technical environment and can comprise one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0146] Figure 2A This is a schematic side view illustrating an exemplary waveguide display that can be used with extended reality (XR) applications according to some embodiments. The image is projected by an image generator 202. The image generator 202 can project the image using one or more of a variety of techniques. For example, the image generator 202 can be a laser beam scanning (LBS) projector, a liquid crystal display (LCD), a light-emitting diode (LED) display (including organic LED (OLED) or micro LED (μLED) displays), a digital light processor (DLP), a liquid crystal on silicon (LCoS) display, or other types of image generators or light engines.

[0147] Light representing image 212 generated by image generator 202 is coupled into waveguide 204 via diffraction inner coupler 206. Inner coupler 206 diffracts the light representing image 212 into one or more diffraction orders. For example, ray 208 (one of the rays representing a portion of the bottom of the image) is diffracted by inner coupler 206, and one of the diffraction orders 210 (e.g., the second order) is at an angle capable of propagating through waveguide 204 via total internal reflection. Image generator 202 displays the image according to instructions from control module 224, which operates to render image data, video data, point cloud data, or other displayable data.

[0148] At least a portion of the light 210, already coupled into waveguide 204 by the inner diffraction coupler 206, is coupled out of the waveguide by the outer diffraction coupler 214. At least some of the light coupled out of waveguide 204 replicates the angle of incidence of the light coupled into the waveguide. For example, in the illustration, the outer coupled rays 216a, 216b, and 216c replicate the angle of the inner coupled ray 208. Because the light leaving the outer coupler replicates the direction of the light entering the inner coupler, the waveguide essentially replicates the original image 212. The user's eye 218 can focus on the replicated image.

[0149] exist Figure 2AIn the example, external coupler 214 couples only a portion of the light with each reflected output, allowing a single input beam (such as beam 208) to generate multiple parallel output beams (such as beams 216A, 216b, and 216c). Thus, even if the eye is not perfectly aligned with the center of the external coupler, at least some of the light originating from each part of the image may reach the user's eye. For example, if eye 218 moves downwards, beam 216c may enter the eye even if beams 216a and 216b do not, so the user can still perceive the bottom of image 212 despite the positional shift. Therefore, external coupler 214 partially functions as an exit pupil expander in the vertical direction. The waveguide may also include one or more additional exit pupil expanders (…). Figure 2A (not shown in the image) to expand the exit pupil in the horizontal direction.

[0150] In some embodiments, waveguide 204 is at least partially transparent to light originating from outside the waveguide display. For example, at least some of the light 220 from a real-world object (such as object 222) passes through waveguide 204, allowing a user to see the real-world object when using the waveguide display. Since the light 220 from the real-world object also passes through diffraction grating 214, multiple diffraction orders will exist, resulting in multiple images. To minimize the visibility of multiple images, it is desirable that the zeroth-order diffraction (without the bias caused by 214) has high diffraction efficiency for both light 220 and the zeroth order, while higher-order diffraction has lower energy. Therefore, in addition to extending and externally coupling virtual images, external coupler 214 is preferably configured to allow the zeroth order of the real image to pass through. In such embodiments, the image displayed by the waveguide display may appear superimposed on the real world.

[0151] Figure 2B This is a schematic side view illustrating an exemplary alternative display type that can be used with extended reality applications according to some embodiments. In the XR head-mounted display device 230, a control module 232 controls a display 234 (which may be an LCD) to display images. The head-mounted display includes a partially reflective surface 236 that reflects (and in some embodiments, both reflects and focuses) the image displayed on the LCD so that the image is visible to the user. The partially reflective surface 236 also allows at least some external light to pass through, thereby allowing the user to see their surroundings.

[0152] Figure 2C This is a schematic side view illustrating exemplary alternative display types that can be used with extended reality applications according to some embodiments. In the XR head-mounted display device 240, a control module 242 controls a display 244 (which may be an LCD) to display an image. The image is focused by one or more lenses of a display optics 246 so that the image is visible to the user. Figure 2C In some examples, external light does not reach the user's eyes directly. However, in some such embodiments, an external camera 248 can be used to capture images of the external environment and display these images on a display 244 along with any virtual content that may also be displayed.

[0153] The embodiments described herein are not limited to any particular type or structure of XR display device.

[0154] As discussed below, the API and protocols are used to transmit Quality of Experience (QoE) metric reports using the Real-Time Media Communication (RTC) interface, which is based on the General Architecture Technical Specification for Real-Time AR and MR (GA4RTAR). 5G Real-time Media Communication 5G Real-Time Media Communication Architecture (Stage 2) As defined in TS26.506, this technical specification can be found on the portal. <dot>3gpp <dot>org / desktopmodules / Specifications / SpecificationDetails <dot>The URL aspx?specificationId=4102 (referred to as "TS 26.506" in this document) is used to retrieve the RTC user equipment (UE) / endpoint, which reports a set of QoE metrics via a quality metric reporting protocol and format. The quality reporting scheme is used in the metric configuration API to configure the RTC UE to send the collected metrics to the metric server.

[0155] According to some embodiments, the RTC UE / endpoint reports a set of QoE metrics as defined below to the metrics server. According to some embodiments, an XML schema for these metrics is also defined. According to some embodiments, a quality metric format and reporting protocol for reporting the aforementioned metrics by the RTC UE / endpoint are defined. According to some embodiments, a quality reporting scheme for configuring the 5G-RTC UE / endpoint to report the collected metrics to the metrics server is also defined. Furthermore, according to some embodiments, a definition of metric reporting configuration for the metric reporting configuration API defined in the RTC-1 interface is given.

[0156] Long Term Evolution (LTE), 5G New Radio (5GNR), and other emerging communication technologies enable wireless devices to transmit information at data rates several orders of magnitude higher (e.g., in gigabits per second) than those available just a few years ago. Today's communication networks are also more secure, more resistant to multipath fading, allow for lower latency in network traffic, and offer better communication efficiency (e.g., in bits per second per unit of bandwidth used). These improvements, along with other recent advancements, have spurred the emergence of new methods for delivering media and content to mobile wireless devices, including broadcast, multicast, unicast, and real-time media delivery technologies.

[0157] Overall architecture of 5G media streaming (5GMS) Figure 3 This is a system diagram illustrating a set of exemplary interfaces for 5G media streaming within a 5G system, according to some embodiments. Figure 3 The overall 5G media streaming architecture is shown in the diagram.

[0158] In the context of the TS 26.501 specification, streaming is defined as the delivery of time-continuous media as the primary medium. For streaming, the media is primarily sent in a single direction and consumed as it is received. Alternatively, media content can be streamed as it is generated; this is called live streaming. If streaming content has already been generated, the media is called video-on-demand (VOD) streaming.

[0159] exist Figure 3 In System 300, functions indicated by diagonally filled style boxes fall within the scope of the 5GMS Phase 3 specification. Functions indicated by white / transparent boxes are defined in the 5G system specification. Functions indicated by crosshair filled style boxes fall neither within the scope of 5G media streaming nor within the scope of the 5G system specification. Therefore, functions indicated by boxes such as UE's 5GMS client 318, trusted DN's 5GMS AF 324, trusted DN's 5GMS AS 326, 5GMS application provider's external DN's 5GMS AF 320, and 5GMS application provider's external DN's 5GMS AS 322 fall within the scope of the 5GMS Phase 3 specification. Functions indicated by UE 302, RAN 306, NEF 314, PCF 312, UPF 308, and trusted DN 310 are defined in the 5G system specification. The functions indicated by 5GMS Sensing Application 316 and 5GMS Application Provider External DN 304 are neither within the scope of 5G media streaming nor within the scope of the 5G system specification.

[0160] 5GMS application providers (APs) use 5GMS for streaming services. They provide 5GMS-aware applications on the UE to leverage 5GMS client and network functions using the interfaces and APIs defined in 5GMS.

[0161] Figure 3 The architecture in the document represents the specified 5GMS functions within a 5G system (5GS) as defined in TS 23.501. The three main functions are: 5GMS Application Function (AF), 5GMS Application Server (AS), and 5GMS Client.

[0162] 5GMS Application Function (AF) is similar to the application function defined in Clause 6.2.10 of TS 23.501 and is dedicated to 5G media streaming. 5GMS Application Server (AS) is an application server dedicated to 5G media streaming. 5GMS Client is an internal UE function dedicated to 5G media streaming. The 5GMS Client is a logical function, and its sub-functions can be selectively distributed within the UE depending on the implementation.

[0163] 5GMS AF and 5GMS AS are data network (DN) functions, and they communicate with the UE via N6 as defined in TS 23.501. Figure 4 As shown in .2.3-5, functions in a trusted DN (e.g., a 5GMS AF in a trusted DN) are trusted by the operator's network. Such an AF can communicate directly with the relevant 5G core functions. Functions in an external DN (e.g., a 5GMS AF in an external DN) can use N33 to communicate only with the 5G core functions via Network Open Function (NEF).

[0164] Figure 4 This is a system diagram illustrating a set of exemplary interfaces for a 5G media streaming architecture according to some embodiments. The 5G media service architecture will... Figure 3 The overall high-level architecture shown is mapped to Figure 4 The (general) architecture shown.

[0165] exist Figure 4 In system 400, with Figure 3 Similarly, functions indicated by diagonally filled style boxes fall within the scope of the 5GMS Phase 3 specification. Functions indicated by white / transparent boxes are defined in the 5G system specification. Functions indicated by crosshair filled style boxes fall neither within the scope of 5G media streaming nor within the scope of the 5G system specification. Therefore, functions indicated by boxes such as UE's 5GMS client 416, UE's media session processor 412, UE's media stream processor 414, trusted DN's 5GMS AF 418, and trusted DN's 5GMS AS 420 fall within the scope of the 5GMS Phase 3 specification. Functions indicated by boxes such as UE 402, NEF 406, PCF 408, and DN 404 are defined in the 5G system specification. Functions indicated by 5GMS aware applications 410 and 5GMS application providers 422 fall neither within the scope of 5G media streaming nor within the scope of the 5G system specification.

[0166] The 5GMS client in the exemplary UE is described as consisting of a media session processor and a media stream processor, which expose their APIs to each other in the same way that APIs are exposed to 5GMS-aware applications. This UE architecture is merely an example and is not universally applicable.

[0167] The architecture specification TS 26.501 addresses two scenarios for each individual media streaming operation: downlink streaming and uplink streaming. For downlink streaming, the network is the source of the media, and the UE acts as the consumer. For uplink streaming, the UE is the source of the media, and the network acts as the consumer.

[0168] The functional entities and interfaces of the general architecture for media streaming describe the characteristics of downlink and uplink streaming in TS 26.501. A Media AF is an application function dedicated to 5G media delivery, similar to the one defined in Clause 6.2.10 of TS 23.501

[11] . A Media AS is an application server dedicated to 5G media delivery. A Media Client is an internal UE function dedicated to 5G media delivery. A Media Session Processor is a function on the UE that communicates with the Media AF to establish, control, and support the delivery of media sessions. A Media Access Function is a function on the UE that communicates with the Media AS to access and deliver media content. For example, a Media Access Function can be further subdivided into content delivery protocols, codecs, media types, and metadata representations.

[0169] The following interfaces and APIs are defined for 5G media delivery.

[0170] M1 (Configuration API) is an external API opened by Media AF that enables media application providers to configure the use of 5G media delivery and obtain feedback.

[0171] M2 (User Plane Interface) is an external interface provided by the Media AS. When the Media AS is in a Trusted DN, it is used to exchange data media data with the application service provider.

[0172] M3 (Server Configuration API) is an API used to exchange information between Media AF and Media AS for configuration purposes.

[0173] M4 (Media Delivery Interface) is the interface and reference point between the Media Access Function and the Media AS, enabling the exchange of media content.

[0174] M5 (Session Processing API) is a set of APIs that Media AF exposes to the Media Session Processor for media session processing, control, reporting, and assistance. This set of APIs also includes appropriate security mechanisms (e.g., authorization and authentication).

[0175] M6 (Client Configuration API) is a set of APIs that the Media Session Processor exposes to applications and media access functions for internal client communication.

[0176] M7 (Media Access API) is a set of APIs exposed by the Media Access Function for configuring and communicating with the Media Access Function.

[0177] M8 (Application Reference Point) is an application interface used for information exchange between media applications and media application providers.

[0178] TS 26.522 defines all procedures for downlink media stream transmission using the different 5G media stream reference points defined in Clause 4 of TS 26.512, and defines procedures for uplink media stream transmission using the 5GMS reference point in Clause 5 of TS 26.512.

[0179] Section 4.3.9 of TS 26.512 defines the M1 interface configuration procedures for measurement reporting, which are used by 5GMS application providers (APs) to configure QoE measurement reporting functionality associated with downlink or uplink media streaming. Section 4.3.9 of TS 26.512 defines procedures such as creating, reading, updating, and corrupting measurement report configurations. Further details are provided in Section 7.8.3.1.

[0180] The M5 API is used by the media session processor within the 5GMS client to invoke services related to downlink or uplink media streaming at the 5GMS AF.

[0181] The M5 process for QoE metric reporting involves a combination of metric collection and reporting in the Media Session Processor using relevant service access information, and the Media Session Processor sending the collected metrics to the 5 GMSAF according to a configured metric scheme. The metric scheme can be 3GPP-defined or non-3GPP-defined.

[0182] When metric collection and reporting is activated for a downlink media streaming session, a set of one or more metric configurations can be provided to the 5GMS client, each set of metric configurations being associated with a metric scheme. A given set of metric configurations contains information such as: the 5GMSAF address to which the media session processor will send metrics, the metric reporting interval, the target percentage of media streaming sessions to which reports should be sent, and a set of metrics to be collected and reported.

[0183] For Dynamic Adaptive Streaming (DASH) services via HTTP, the metrics listed in a set of metric configurations are associated with the 3GPP metric scheme and correspond to one or more of the metrics specified in Clauses 10.3 and 10.4 of TS 26.247, respectively.

[0184] To submit a measurement report, the media session processor sends an HTTP POST message to the 5GMS AF.

[0185] In 5G media streaming, some embodiments may include methods for supporting metric measurement, collection, and reporting in a 5G System (5GS) network. Details of the APIs and processes used for these aspects are described in the technical specification 5G Media Streaming Protocol TS 26.512. Some embodiments may use the processor of an Application Function (AF) server in the 5GS network to perform a portion of the metric measurement, collection, and / or reporting. Some embodiments may use a media session processor running on the processor of a radio device connected to the Radio Access Network (RAN) of the System (5GS) network to perform a portion of the metric measurement, collection, and / or reporting. Some embodiments may include, for a downlink 5G media streaming service, sending a metric configuration message to the radio device via an M5d interface based on the application layer control plane included in the 5GS network, wherein the metric configuration message indicates that the 5G media streaming service is associated with the 5G media streaming service. One or more metrics for measurement requests, collection requests, and reporting requests are associated with the streaming service. Some embodiments may include sending a metric configuration message to a wireless device via an M5u interface based on the application layer control plane included in a 5GS network for uplink 5G media streaming services, wherein the metric configuration message indicates one or more metrics associated with 5G media streaming for measurement requirements, collection requirements, and reporting requirements.

[0186] Some embodiments may include sending metric configuration messages to wireless devices via a control plane-based M5 interface included in a 5GS network, wherein the metric configuration messages indicate one or more metric measurements associated with 5G media streaming services, collection and reporting requirements.

[0187] Some embodiments may further include: establishing a media session with the wireless device for the 5G media streaming service; receiving a measurement report associated with the media session from the wireless device via a control plane-based M5d interface, wherein the measurement report includes collected data and one or more measurement requirements, collection requirements, and reporting requirements associated with the measurement; processing the measurement report to generate a processed measurement report; and providing the processed measurement report to an application server of the 5G media streaming service, an operator of the 5GS network, an Operations Management and Maintenance (OAM) server, and / or a third-party server.

[0188] In some embodiments, processing metric reports to generate a processed metric report may include filtering, aggregating, and / or reformatting the collected metrics. In some embodiments, processing metric reports to generate a processed metric report may include one or more rules, such as those indicated by an application server of a 5G media streaming service, an OAM server of a 5GS network, and / or a third-party server, to process the metric reports to generate a processed metric report. In some embodiments, one or more metric measurement requirements, collection requirements, and reporting requirements may include QoE metrics associated with: application services or content services provided to the 5GS network from outside the 5GS network; and / or metrics requested from outside the 5GS network. In some embodiments, the metric configuration message also indicates a metric scheme. In some embodiments, a metric scheme may be set based on the settings of the 5GS network and / or a metric scheme received from outside the 5GS network. In some embodiments, a metric scheme defines metric measurement rules and procedures, collection rules and procedures, and reporting rules and procedures. In some embodiments, one or more metric measurement requirements, collection requirements, and reporting requirements include non-3GPP (non-3rd Generation Partnership Project) metrics. In some embodiments, a 5G media streaming service may be an uplink 5G media streaming service or a downlink 5G media streaming service.

[0189] Some embodiments may include: receiving a metric configuration message from an AF server in a 5GS network via an M5 interface based on the application layer control plane, wherein the metric configuration message indicates one or more metric measurement requirements, collection requirements, and reporting requirements associated with the 5G media streaming service; establishing a media session with the AF server for the 5G media streaming service; determining one or more metrics to be collected for the media session in response to establishing the media session, at least in part based on the one or more metric measurement requirements, collection requirements, and reporting requirements for initiating the media session; collecting metrics from a media stream processor after the media session has started; aggregating the collected metrics according to a metric scheme to generate a metric report associated with the media session; and sending the metric report associated with the media session to the AF server via the M5 interface based on the application layer control plane.

[0190] Some embodiments may further include: querying the media stream processor of the wireless device selected for the media session to determine whether the media stream processor has the capability to measure and record one or more determined metrics; and, in response to determining that the media stream processor does not have the capability to measure and record one or more determined metrics, sending an error message to the AF server via the M5 interface based on the application layer control plane. In some embodiments, starting a media session may include starting the media session in response to determining that the media stream processor has the capability to measure and record one or more determined metrics.

[0191] 5G-based real-time media communication architecture Figure 5 This is a system diagram illustrating a set of exemplary interfaces for real-time media communication in a 5G system according to some embodiments. Figure 5 The overall RTC architecture is displayed in the middle.

[0192] In the context of this specification, Real-time Media Communication (RTC) based on 5G systems is defined, for example, as the delivery of delay-sensitive media from one peer to another in the presence of a 5G network. The AR Dialogue Service described in Technical Report TR 26.998 is a typical use case for RTC, which enables end users to directly transmit real-time media including AR / MR media content as specified in TS 26.119.

[0193] exist Figure 5 In System 500, functions indicated by diagonally filled style boxes are within the scope of this document and the 5G RTC specification. Functions indicated by white / transparent boxes are defined in the 5G System Specification. Functions indicated by crosshair filled style boxes are neither within the scope of the 5G RTC nor the 5G System Specification. Therefore, functions indicated by boxes such as UE's RTC endpoint 520, Trusted DN's RTC AF 522, and Trusted DN's RTC AS 524 are within the scope of this document and the 5G RTC specification. Functions indicated by boxes such as UE1 (502), RAN 504, UPF 506, NEF 512, SMF 514, PCF 516, and Trusted DN 508 are defined in the 5G System Specification. Functions indicated by RTC-aware applications 518 and application provider external DN 510 are neither within the scope of the 5G RTC nor the 5G System Specification.

[0194] Media data is exchanged between two or more RTC endpoints via a 5G system. An RTC endpoint is an endpoint configured by the RTC architecture. An RTC endpoint is typically a UE, but for some embodiments, an edge computing server can also be an RTC endpoint. Application providers offer RTC-aware applications on the UE to leverage RTC endpoints and network functions using interfaces and APIs. The RTC architecture provides the core functions and entities for supporting WebRTC-based services on 5G systems. Two main functions are defined in the Trusted DN: RTC AF and RTC AS. RTC AF is an application function dedicated to real-time media communications, similar to that defined in TS 26.501 [6]. RTC AS is an application server dedicated to real-time media communications.

[0195] Figure 6 This is a system diagram illustrating a set of exemplary interfaces for a real-time media communication architecture according to some embodiments. Figure 6 Shows mapping to Figure 5 The detailed RTC architecture of the overall high-level architecture. Figure 6 Only show Figure 5 One half (the link from one RTC endpoint to the RTC AF and RTC AS), because the other half is symmetrical. Sub-functions within the RTC AF, RTC AS, and RTC endpoints are defined in Section 4.2, while... Figure 6 The interface shown (corresponding to TS 26.506) Figure 4 .1-2) are defined in Clause 4.3 of TS 26.506.

[0196] exist Figure 6 In system 600, such as Figure 5 As described in the document, functions indicated by a diagonal fill style box fall within the scope of this document and the 5G RTC specification. Functions indicated by white / transparent boxes are defined in the 5G System Specification. Functions indicated by a crosshair fill style box are neither within the scope of the 5G RTC nor the 5G System Specification. Therefore, the functions indicated by the boxes RTC AS 606, RTC AF 608, UE's RTC endpoint 620, RTC Media Session Processor (RTC MSH) of the RTC endpoint 622, RTC endpoint's web-based RTC framework 624, RTC AF's network support function (NS-AF) 626, RTC AF's configuration function 628, RTC AF's provisioning function 630, RTC AS's ICE function 632, RTC AS's media function 634, RTC AS's transport gateway function 636, RTC AS's web-based RTC signaling function 638, RTC AS's application support web function 640, and RTC AS's interoperability function 642 are within the scope of this document and belong to the 5G RTC specification. The functions indicated by the boxes UE 602, NEF 612, SMF 614, and PCF 610 are defined in the 5G system specification. The functions indicated by the UE’s local web RTC application 616, the UE’s web application 618, and the RTC application provider 604 are neither within the scope of 5G RTC nor within the scope of the 5G system specification.

[0197] Technical specification TS 26.506 details the Phase 2 architecture supporting real-time media communication for web-based RTC. In the specified RTC architecture, various interfaces and their functions are identified as Phase 2 results. While RTC primarily involves bidirectional services, where media is delivered in both directions, it shares commonalities with 5GMS, where media is primarily transmitted in only one direction and consumed upon reception. Therefore, a generic Phase 3 media delivery framework needs to be developed for RTC interfaces and functions.

[0198] Therefore, technical specification TS 26.113 begins to define the Phase 3 protocols, procedures, and APIs of TS 26.506 for media session processing and media content delivery (referred to as both C- / U-planes).

[0199] 5GMS protocol specification TS 26.512 discusses the procedures and APIs for measuring, collecting, and reporting metrics in 5G system networks used for 5GMS services. TS 26.512 does not cover the procedures and APIs for measuring, collecting, and reporting Quality of Experience (QoE) metrics in 5G system networks used for real-time media communication services as defined in TS 26.506.

[0200] Technical specification TS 26.234 describes a list of QoE metrics for progressive download and DASH streaming services. The specification also provides a quality reporting scheme for configuring a UE to send QoE metric reports and a quality reporting protocol for the UE to report QoE metrics to a metric server.

[0201] As discussed below, QoE metrics are reported to the metrics server by the 5G Real-Time Media Communication (5G-RTC) UE or endpoint. The QoE metric reporting protocol for sending metrics from the 5G-RTC UE to the metric collection server is also discussed below. A quality reporting scheme is used by the 5G-RTC UE to configure the system for QoE metric measurement and reporting.

[0202] QoE for Real-time Media Communication (RTC) 5G-RTC clients that support Quality of Experience (QoE) report QoE metrics according to their QoE configuration. QoE reporting is optional, but if a 5G-RTC client reports RTC metrics, the client reports all requested metrics.

[0203] The following specifies the quality metrics used for RTC streaming. In this case, QoE reporting can be triggered using an Open Mobile Alliance Device Management (OMA-DM) QoE management object as specified in Annex F of TS 26.522, or via a QMC function as specified in Annex L of TS 26.522. If QoE reporting is triggered via either of the above methods, the 5G-RTC client is expected to collect quality metrics according to the QoE configuration. If using the OMA DM or QMC function, the quality reporting scheme defined below can be used.

[0204] QoE configuration should only be evaluated by the client at the start of a QoE measurement and reporting session ("QoE session") associated with a streaming session. This includes evaluations of any filtering criteria, such as by geographic region. Client evaluations of all measurement and reporting criteria in an ongoing QoE session are unaffected by any QoE configuration changes received during that session. Any changes to the QoE configuration only affect QoE sessions that begin after these configuration changes have been received.

[0205] The quality metric reporting protocol is defined below. This protocol is used when a QoE report is triggered via the OMA DM QoE management object. If the QoE report is triggered via the QMC function, the QoE configuration is received via UMTS-specific RRC messages, LTE-specific RRC messages, and NR-specific RRC messages on the control plane, and the QoE report is also sent back via RRC messages on the control plane. QoE metrics from the 5G-RTC client are formatted in XML according to the definitions given below.

[0206] QoE metric definition This section provides general QoE metric definitions and a measurement framework. 5G-RTC clients supporting QoE metric functionality support metric reporting as described in this section. These metrics are valid for voice, video, and text media and are calculated for each measurement resolution interval, "measureinterval". They are reported to the server according to the measurement reporting interval, "reportinginterval", and after the session ends.

[0207] Semantics are defined using an abstract syntax. A mapping to an XML schema is provided. Items in this abstract syntax have one of the following primitive types (UnsignedInteger, Integer, double, String) or one of the following composite types: object, list, or collection. An object is an unordered sequence of (key, value) pairs, where the "key" part of the pair is a string type and unique within the sequence. A list is a sorted collection of items. A collection is an unordered collection of items. Additionally, two types of timestamps are defined: real time (wall clock time) and Media Time .

[0208] The optional "measureinterval" field (if used) indicates the time taken to compute each metric. The "measureinterval" field divides the session duration into multiple equally sized periods, each with a length specified by the "measureinterval" field. The "measureinterval" field specifies the amount of time preceding the start of QoE parameter computation. If the "measureinterval" field is not present, the metric resolution overrides the periods specified by the "measureinterval" field. If the "measureinterval" field is not present, the metric resolution is set to use the entire session duration.

[0209] The optional "measurerange" field (if used) indicates the time range in the stream from which QoE metrics will be reported. Only one range is allowed per measurement specification. The range format can be any format permitted by the media. If the "measurerange" field is not present, the measurement range is set to use the entire call duration.

[0210] Damage duration measurement The corruption duration M is the time period from the NPT time of the last good frame before corruption (because the NPT time of the first corrupted frame is not always certain) to the NPT time of the first subsequent good frame. A corrupted frame can be a completely lost frame or a media frame whose quality has degraded and whose decoded frame differs from the result in error-free decoding.

[0211] A good frame is a fully received frame where: (1) all parts of the image contain the correct content; or (2) the frame is a refresh frame that does not reference any previously decoded frames; or (3) the frame references only previously decoded good frames. The term "fully received" means that all bits have been received and no bit errors have occurred.

[0212] The duration of corruption, M, in milliseconds, can be calculated as discussed in this paper. The value M can be derived by the client using the codec layer, in which case the codec layer signals the client to decode a good frame. Good frames can also be derived using error tracking methods, but without using decoding quality assessment methods. Alternatively, corruption can be considered complete after N milliseconds of continuously received frames, or when refresh frames have been fully received, whichever occurs first.

[0213] The optional configuration parameter N can be used to indicate the average characteristics of the codec. If N is not configured, it defaults to the length of a measurement interval for video media and to the duration of a frame for non-video media. The N parameter is specified in milliseconds and is used in conjunction with the "CorruptionDuration" parameter. The value of N can be set by the server.

[0214] Sum the durations of all damages that occur within each measurement cycle and store the summation results in a vector. Total Corruption Duration The metric is expressed in milliseconds. Within each measurement period, the number of individual damage events is summed, and the summation result is stored in a vector number CorruptionEvents.

[0215] Table 1 specifies the syntax for the "CurruptionDuration" metric object used in quality reports.

[0216] Table 1

[0217] Continuous loss of RTP packets The metric "SuccessiveLoss" indicates the number of Real-Time Protocol (RTP) packets lost consecutively on each media channel.

[0218] Within each measurement resolution period of the stream, the total number of consecutively lost RTP packets is summed, and the summation result is stored in a vector. TotalNumberofSuccessivePacketLoss In this context, the metric is represented as an integer equal to or greater than 0. The number of consecutive packet loss events within each measurement resolution period is summed, and the summation is stored in a vector. NumberOfAccessiveLossEvents In each measurement resolution period, the number of received packets is also summed, and the summation result is stored in a vector. NumberOfReceivedPackets These three vectors are reported by the 5G-RTC UE / endpoint as part of the QoE report.

[0219] Table 2 specifies the syntax for the RTP packet successive loss metric object "SuccessiveLoss", which provides information for quality reporting.

[0220] Table 2

[0221] Frame rate Frame rate refers to the playback frame rate. The playback frame rate is equal to the number of frames displayed during the measurement resolution period divided by the duration of the measurement resolution period (in seconds).

[0222] The metric titled "Frame Rate" indicates the frame rate value. This metric is expressed in frames per second and can be a fractional value. The frame rate per resolution period is stored in a vector. framerate In addition, it is reported by 5G-RTC UE / endpoint as part of the QoE report.

[0223] Table 3 specifies the syntax for the frame rate metric "frame rate" used in quality reports.

[0224] Table 3

[0225] jitter duration When the absolute difference between the actual playback time and the expected playback time is greater than Jitterthreshold Jitter occurs at milliseconds. The expected time of a frame is equal to the actual playback time of the last frame plus the difference between the NPT time of that frame and the NPT time of the last frame.

[0226] Optional configuration parameters can be set. Jitterthreshold This controls the amount of jitter allowed. If the parameter is not set, it defaults to 100 ms. Specify the parameter in milliseconds. Jitterthreshold It is used in conjunction with the parameter "JitterDuration". This can be set by the server. Jitterthreshold The value of .

[0227] The duration of all jitter is summed within each measurement resolution period, and the summation result is stored in a vector. Total Jitter Duration The metric is expressed in seconds, but can also be in minutes. The number of individual events within the measurement resolution period is summed, and the summation is stored in a vector. NumberOfSyncLossEvents These two vectors are reported by the 5G-RTC UE / endpoint as part of the QoE report.

[0228] Table 4 specifies the syntax for the "JitterDuration" metric object used in quality reports.

[0229] Table 4

[0230] Synchronization loss duration When the absolute difference between value A and value B is greater than SyncThreshold Synchronization loss occurs at milliseconds. Value A represents the difference between the playback time of the last frame of the video stream and the playback time of the last frame of the audio / video stream. Value B represents the difference between the expected playback time of the last frame of the video stream and the expected playback time of the last frame of the audio / video stream.

[0231] Optional configuration parameters can be set. syncthreshold This controls the allowable amount of synchronization mismatch. If the parameter is not set, it defaults to 100 ms. Specify the parameter in milliseconds. syncthreshold And it is used in conjunction with the "SyncLossDuration" parameter. syncthreshold The value can be set by the server.

[0232] The duration of all synchronization loss within each measurement resolution period is summed, and the summation result is stored in a vector. TotalSyncLossDuration The unit of measurement is seconds, but can also be minutes. The number of events within the measurement resolution period is summed, and the summation result is stored in a vector. NumberOfSyncLossEvents These two vectors are reported by the 5G-RTC UE / endpoint as part of the QoE report.

[0233] Table 5 specifies the syntax for the "SynclossDuration" metric object used in quality reporting.

[0234] Table 5

[0235] Round trip time Round-trip time (RTT) is the RTP-level round-trip time plus the bidirectional latency due to buffering and other processing in each UE / endpoint. The last RTCP round-trip time value estimated within each measurement resolution period is stored in the... NetworkRTT The metric is expressed in milliseconds. The effective bidirectional additional internal client latency at the end of each measurement resolution cycle is stored in a vector. InternalRTT The metric is expressed in milliseconds. Both vectors are reported by the 5G-RTC UE / endpoint as part of the QoE report.

[0236] Table 6 specifies the syntax for the round-trip time metric "RoundTripDuration" used in quality reports. Table 6

[0237] Average codec bit rate The average codec bit rate is the bit rate used to encode "active" media information during a measurement resolution period.

[0238] For speech media, the average codec bit rate can be calculated as the number of "active" speech bits received for "active" frames divided by the total time, in seconds, covered by these frames. The total time covered is calculated as the number of "active" frames multiplied by the length of each speech frame.

[0239] For non-voice media, the average codec bit rate is the total number of received RTP payload bits divided by the length of the measurement resolution period.

[0240] The average codec bit rate value for each measurement resolution period will be stored in a vector. Average CodecBitrate The metric is expressed in kbit / s and can be a fractional value. This vector is reported by the 5G-RTC UE / endpoint as part of the QoE report.

[0241] Table 7 specifies the syntax for the "AverageBitrate" object used in quality reports for the average codec bitrate metric.

[0242] Table 7

[0243] Quality Measurement Reporting Agreement The Quality Metrics Reporting Protocol includes the following XML-based report format and the following reporting protocol. The MIME type of the XML-formatted QoE report is "application / 3gprtc-qoe-report+xml".

[0244] Report Format Use the XML schema shown in Table 8 to format the QoE report as an XML document.

[0245] Table 8

[0246]

[0247]

[0248]

[0249]

[0250] Reporting Agreement For configurations completed via the QMC function, the client also sends a QoE report via the QMC function. For Open Mobile Alliance Device Management (OMA-DM) configurations, if a specific metrics server has been configured, the client sends a QoE report using an HTTP (RFC 2616) POST request carrying metadata in XML format in its body. Table 9 shows an example QoE report based on HTTP POST request signaling. In the code list shown in Table 9, all occurrences of the term "www" may be prefixed with "HTTP", a colon, and two forward slashes. Periods appearing in the URL have been replaced with the term "www". <dot>"replace.

[0251] Table 9

[0252]

[0253] Quality Reporting Scheme for Real-Time Media Communications (RTC) This section specifies the 5G-RTC quality reporting scheme. The quality reporting scheme is signaled using the reporting element from the metric elements defined in Clause 10.4 of TS 26.247. The Uniform Resource Name (URN) used for Reporting@schemeIdUri is "urn:3GPP:ns:PSS:RTC:QM1". The semantics for the 5G-RTC quality reporting scheme are specified in Table 10.

[0254] Table 10

[0255] Table 11 specifies the XML syntax for the 5G-RTC quality reporting scheme. In the code list shown in Table 11, all occurrences of the term "www" can be prefixed with "http", a colon, and two forward slashes. Periods appearing in URLs have been replaced with the term "http". <dot>"replace.

[0256] Table 11

[0257]

[0258]

[0259]

[0260] Metrics Report Configuration The data model for the metric reporting configuration API defined in Clause 7.8.3 of TS 26.512 can be extended for 5G-RTC media services. Table 12 specifies the extended... MetricsReportingConfiguration resource.

[0261] Table 12

[0262] Figure 7 This is a flowchart illustrating an exemplary process for reporting quality metrics according to some embodiments. In some embodiments, exemplary process 700 may include configuring 702 a device to process a set of one or more metrics. In some embodiments, exemplary process 700 may also include collecting 704 the set of one or more metrics. In some embodiments, exemplary process 700 may also include generating 706 a metric report including the collected set of one or more metrics. In some embodiments, exemplary process 700 may also include reporting 708 the metric report including the collected set of one or more metrics to a server.

[0263] Figure 8 This is a flowchart illustrating an exemplary process of a server sending quality metrics according to some embodiments. In some embodiments, exemplary process 800 may include configuring 802 the server to process a set of one or more metrics. In some embodiments, exemplary process 800 may also include collecting 804 the set of one or more metrics. In some embodiments, exemplary process 800 may also include sending 806 the set of one or more metrics to a user equipment (UE) device.

[0264] Although the methods and systems according to some embodiments are generally discussed in the context of extended reality (XR), some embodiments can be applied to any XR context (e.g., virtual reality (VR) context / mixed reality (MR) context / augmented reality (AR) context). Furthermore, although the term "head-mounted display (HMD)" is used herein according to some embodiments, for some embodiments, some embodiments can be applied to wearable devices (which may or may not be attached to the head) possessing, for example, XR, VR, AR, and / or MR capabilities.

[0265] A first exemplary method according to some embodiments may include: establishing a media session with an application function (AF) server for 5G media streaming services; in response to establishing the media session, determining one or more metrics to be collected for the media session, wherein determining the one or more metrics to be collected for the media session is at least in part based on one or more metric measurement requirements, collection requirements, and reporting requirements at the start of the media session; receiving a metric configuration message corresponding to the one or more metrics, wherein the metric configuration message includes a metric scheme, and wherein the metric scheme includes at least one rule or step corresponding to at least one of the one or more metrics; configuring a device to process the one or more metrics; collecting the one or more metrics; generating a metric report including the collected one or more metrics; and reporting the metric report including the collected one or more metrics to a server.

[0266] In some embodiments of the first exemplary method, at least one of the one or more metrics is a Quality of Experience (QoE) metric.

[0267] In some embodiments of the first exemplary method, QoE metrics are associated with applications or content services provided from outside the 5GS network.

[0268] Some embodiments of the first exemplary method may also include requesting QoE metrics from devices outside the 5GS network.

[0269] In some embodiments of the first exemplary method, at least one of the one or more metrics is a non-3GPP (non-3rd Generation Partnership Project) metric.

[0270] In some embodiments of the first exemplary method, a metric configuration message is received from the application function (AF) server of the 5GS network.

[0271] In some embodiments of the first exemplary method, metric configuration messages are received via the application layer control plane.

[0272] For some embodiments of the first exemplary method, generating a metric report includes at least one of filtering, aggregating, and reformatting the collected one or more metrics.

[0273] Some embodiments of the first exemplary method may also include receiving information indicating one or more rules for processing at least one of the one or more metrics.

[0274] Some embodiments of the first exemplary method may also include measuring at least one of the one or more metrics.

[0275] For some embodiments of the first exemplary method, the reporting metrics report includes: sending an HTTP POST message.

[0276] In some embodiments of the first exemplary method, at least one of the one or more metrics is associated with a streaming service.

[0277] For some embodiments of the first exemplary method, the reporting of metrics includes sending the metrics report to at least one of the following: an application server (AS) for the 5G media streaming service, an operator of the 5GS network, an operations management and maintenance (OAS) server, and a third-party server.

[0278] Some embodiments of the first exemplary method may further include: updating the metric report; and reporting the updated metric report to a server.

[0279] For some embodiments of the first exemplary method, the report metrics report includes: using the Real-Time Media Communication (RTC) interface in a 5G system.

[0280] For some embodiments of the first exemplary method, generating a metric report includes using a format corresponding to 5G system requirements.

[0281] For some embodiments of the first exemplary method, generating a metric report includes using a format corresponding to the DASH streaming service.

[0282] For some embodiments of the first exemplary method, generating a metric report includes using an XML format.

[0283] For some embodiments of the first exemplary method, collecting one or more metrics includes: receiving at least one measurement corresponding to one of the one or more metrics.

[0284] Some embodiments of the first exemplary method may also include triggering the reporting of one or more metrics by a 5G-RTC client, wherein the triggering includes using an Open Mobile Alliance Device Management (OMA-DM) Quality of Experience (QoE) management object.

[0285] Some embodiments of the first exemplary method may also include triggering the reporting of one or more metrics by a 5G-RTC client, wherein the triggering includes using the QMC function.

[0286] For some embodiments of the first exemplary method, the reporting of metric reports includes sending metric reports via RRC messages.

[0287] For some embodiments of the first exemplary method, the reporting metrics report includes sending an HTTP POST request carrying metadata in XML format.

[0288] For some embodiments of the first exemplary method, the report metrics report includes: using a 5G-RTC quality reporting scheme.

[0289] In some embodiments of the first exemplary method, at least one of the one or more metrics is a damage duration metric.

[0290] In some embodiments of the first exemplary method, at least one of the one or more metrics is the continuous loss of RTP packets.

[0291] In some embodiments of the first exemplary method, at least one of the one or more metrics is the frame rate.

[0292] In some embodiments of the first exemplary method, at least one of the one or more metrics is jitter duration.

[0293] In some embodiments of the first exemplary method, at least one of the one or more metrics is the duration of synchronization loss.

[0294] In some embodiments of the first exemplary method, at least one of the one or more metrics is round-trip time.

[0295] In some embodiments of the first exemplary method, at least one of the one or more metrics is the average codec bit rate.

[0296] A first exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: configure the apparatus to process a set of one or more metrics; collect the set of one or more metrics; generate a metric report including the collected set of one or more metrics; and report the metric report including the collected set of one or more metrics to a server.

[0297] A second exemplary method according to some embodiments may include: configuring a server to process a set of one or more metrics; collecting the set of one or more metrics; and sending the set of one or more metrics to a user equipment (UE) device.

[0298] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is a Quality of Experience (QoE) metric.

[0299] In some embodiments of the second exemplary method, the QoE metric is associated with an application or content service provided from outside the 5GS network.

[0300] Some embodiments of the second exemplary method may also include requesting QoE metrics from devices outside the 5GS network.

[0301] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is a non-3rd Generation Partnership Project (Non-3GPP) metric.

[0302] Some embodiments of the second exemplary method may further include sending a metric configuration message corresponding to at least one of the set of one or more metrics.

[0303] In some embodiments of the second exemplary method, the metric configuration message is sent via the application function (AF) server of the 5GS network.

[0304] In some embodiments of the second exemplary method, the metric configuration message is sent via the application layer control plane.

[0305] In some embodiments of the second exemplary method, the metric configuration message includes a metric scheme, and the metric scheme includes at least one rule or step corresponding to at least one of the set of one or more metrics.

[0306] Some embodiments of the second exemplary method may also include determining one or more metrics to be collected.

[0307] In some embodiments of the second exemplary method, determining one or more metrics to be collected for the media session is based at least in part on one or more metric measurement requirements, collection requirements, and reporting requirements at the start of the media session.

[0308] Some embodiments of the second exemplary method may also include at least one of filtering, aggregating, and reformatting the collected set of one or more metrics.

[0309] Some embodiments of the second exemplary method may also include sending information indicating one or more rules for processing at least one metric in a set of one or more metrics.

[0310] Some embodiments of the second exemplary method may also include measuring at least one of the set of one or more measures.

[0311] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is associated with a streaming service.

[0312] In some embodiments of the second exemplary method, sending the set of one or more metrics to the user equipment (UE) device includes using a real-time media communication (RTC) interface in a 5G system.

[0313] In some embodiments of the second exemplary method, collecting the set of one or more metrics includes performing at least one measurement corresponding to one of the set of one or more metrics.

[0314] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is a damage duration metric.

[0315] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is the continuous loss of RTP packets.

[0316] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is the frame rate.

[0317] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is jitter duration.

[0318] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is the duration of synchronization loss.

[0319] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is round-trip time.

[0320] In some embodiments of the second exemplary method, at least one of the set of one or more metrics is the average codec bit rate.

[0321] A second exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: configure a server to process a set of one or more metrics; collect the set of one or more metrics; and send the set of one or more metrics to a user equipment (UE) apparatus.

[0322] A third exemplary method according to some embodiments may include: configuring a quality reporting scheme by a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint for measuring and reporting a set of Quality of Experience (QoE) metrics; and, according to the QoE metric reporting protocol and the quality reporting scheme, having the 5G-RTC user equipment or endpoint report information related to the set of QoE metrics to a 5G-RTC metric server by means of transmission.

[0323] A third exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: configure a quality reporting scheme by a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint for measuring and reporting a set of Quality of Experience (QoE) metrics; and, according to the QoE metric reporting protocol and the quality reporting scheme, report information related to the set of QoE metrics to a 5G-RTC metric server by means of transmission.

[0324] A fourth exemplary method according to some embodiments may include: configuring a quality reporting scheme by a first network device for measuring and reporting a set of quality of experience (QoE) metrics; and sending information related to the set of QoE metrics to a second network device by the first network device in accordance with the QoE metric reporting protocol and the quality reporting scheme.

[0325] A fourth exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: be configured by a first network device to measure and report a set of quality of experience (QoE) metrics using a quality reporting scheme; and, according to a QoE metric reporting protocol and the quality reporting scheme, be transmitted by the first network device to a second network device containing information related to the set of QoE metrics.

[0326] A fifth exemplary method according to some embodiments may include: a first network device sending information related to a set of Quality of Experience (QoE) metrics to a second network device according to a QoE metric reporting protocol and the quality reporting scheme, wherein the quality reporting scheme is configured to measure and report the information related to the set of QoE metrics.

[0327] In some embodiments of the fifth exemplary method, the set of QoE metrics includes a damage duration metric.

[0328] In some embodiments of the fifth exemplary method, the set of QoE metrics includes the successive loss of RTP packets.

[0329] In some embodiments of the fifth exemplary method, the set of QoE metrics includes frame rate.

[0330] In some embodiments of the fifth exemplary method, the set of QoE metrics includes jitter duration.

[0331] In some embodiments of the fifth exemplary method, the set of QoE metrics includes synchronization loss duration.

[0332] In some embodiments of the fifth exemplary method, the set of QoE metrics includes round-trip time.

[0333] In some embodiments of the fifth exemplary method, the set of QoE metrics includes the average codec bit rate.

[0334] A fifth exemplary device according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: transmit information related to a set of Quality of Experience (QoE) metrics to a second network device according to a QoE metric reporting protocol and the quality reporting scheme, wherein the quality reporting scheme is configured to measure and report the information related to the set of QoE metrics.

[0335] A sixth exemplary method according to some embodiments may include: configuring an XML metrics scheme by a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint for measuring and reporting a set of Quality of Experience (QoE) metrics; and, according to the XML metrics scheme, reporting information related to the set of QoE metrics to a 5G-RTC metrics server by the 5G-RTC user equipment or endpoint.

[0336] A sixth exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: configure an XML metrics scheme for measuring and reporting a set of quality of experience (QoE) metrics by a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint; and, according to the XML metrics scheme, report information related to the set of QoE metrics to a 5G-RTC metrics server by the 5G-RTC user equipment or endpoint.

[0337] A seventh exemplary method according to some embodiments may include: configuring a quality metric reporting scheme by a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint for measuring and reporting a set of quality of experience (QoE) metrics; and reporting information related to the set of QoE metrics to a 5G-RTC metric server according to the QoE metric reporting protocol and the quality reporting scheme.

[0338] A seventh exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: configure a quality metric reporting scheme by a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint for measuring and reporting a set of Quality of Experience (QoE) metrics; and, according to the QoE metric reporting protocol and the quality reporting scheme, report information related to the set of QoE metrics to a 5G-RTC metric server.

[0339] An eighth exemplary method according to some embodiments may include: configuring a 5G real-time media communication (5G-RTC) user equipment or endpoint to report a set of quality of experience (QoE) metrics to a metrics server; collecting information related to the set of QoE metrics; and having the 5G-RTC user equipment or endpoint report the information related to the set of QoE metrics to the metrics server.

[0340] An eighth exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: configure a 5G Real-Time Media Communication (5G-RTC) user equipment or endpoint to report a set of Quality of Experience (QoE) metrics to a metrics server; collect information related to the set of QoE metrics; and have the 5G-RTC user equipment or endpoint report the information related to the set of QoE metrics to the metrics server.

[0341] A ninth exemplary method according to some embodiments may include: configuring a 5G Real-Time Media Communication (5G-RTC) user equipment to measure and report a set of quality of experience (QoE) metrics; and having the 5G-RTC user equipment report information related to the set of QoE metrics via an RTC-1 interface.

[0342] A ninth exemplary device according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: configure a 5G Real-Time Media Communication (5G-RTC) user equipment to measure and report a set of quality of experience (QoE) metrics; and have the 5G-RTC user equipment report information related to the set of QoE metrics via an RTC-1 interface.

[0343] A tenth exemplary method according to some embodiments may include: receiving a metric configuration message corresponding to a set of one or more metrics; configuring a device to process the set of one or more metrics; collecting a collection of the one or more metrics; generating a metric report including the collected set of one or more metrics; and reporting the metric report including the collected set of one or more metrics to a server.

[0344] A tenth exemplary apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to: receive a metric configuration message corresponding to a set of one or more metrics; configure the apparatus to process the set of one or more metrics; collect the set of one or more metrics; generate a metric report including the collected set of one or more metrics; and report the metric report including the collected set of one or more metrics to a server.

[0345] An eleventh exemplary device according to some embodiments may include at least one processor configured to perform any of the methods listed above.

[0346] A twelfth exemplary apparatus according to some embodiments may include a computer-readable medium storing instructions for causing one or more processors to perform any of the methods listed above.

[0347] A thirteenth exemplary apparatus according to some embodiments may include at least one processor and at least one non-transitory computer-readable medium storing instructions for causing the at least one processor to perform any of the methods listed above.

[0348] Exemplary signals according to some embodiments may include a bitstream generated according to any of the methods listed above.

[0349] This disclosure describes various aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are specifically described and are generally described in a manner that may sound restrictive, at least to show individual characteristics. However, this is for clarity and does not limit the disclosure or scope of these aspects. In fact, all the different aspects can be combined and interchanged to provide more aspects. Furthermore, these aspects can also be combined and interchanged with aspects described in earlier applications.

[0350] The aspects described and contemplated in this disclosure can be implemented in many different forms. While some embodiments are specifically shown, other embodiments are also contemplated, and the discussion of particular embodiments does not limit the breadth of implementation. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to the transmission of generated or encoded bitstreams. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions thereon stored for encoding or decoding video data according to any of the methods, and / or computer-readable recording media having a bitstream generated according to any of the methods stored thereon.

[0351] In this invention, the terms "reconstruction" and "decoding" are used interchangeably, as are the terms "pixel" and "sample," and the terms "image," "picture," and "frame." Typically, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0352] The terms "HDR (High Dynamic Range)" and "SDR (Standard Dynamic Range)" generally convey specific values ​​of dynamic range to those skilled in the art. However, additional embodiments are also contemplated, wherein a reference to HDR can be understood as meaning "higher dynamic range," and a reference to SDR can be understood as meaning "lower dynamic range." These additional embodiments are not bound by any specific value of dynamic range that may often be associated with the terms "high dynamic range" and "standard dynamic range."

[0353] This document describes various methods, each comprising one or more steps or actions to implement the method. Unless a specific order of steps or actions is required for the method to operate correctly, the order and / or use of specific steps and / or actions can be modified or combined. Furthermore, in various embodiments, terms such as "first," "second," etc., may be used to define elements, components, steps, operations, etc., e.g., "first decoding" and "second decoding." Unless specifically required, the use of such terms does not imply an ordering of modified operations. Therefore, in this example, the first decoding does not need to be performed before the second decoding and can be performed, for example, before, during, or within a time period overlapping with the second decoding.

[0354] For example, various numerical values ​​may be used in this disclosure. Specific values ​​are for illustrative purposes only, and the aspects described are not limited to these specific values.

[0355] The embodiments described herein can be implemented by computer software or other hardware implemented by a processor, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. As a non-limiting example, the processor can be of any type suitable for the technical environment and can include one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0356] Multiple implementations involve decoding. As used herein, "decoding" can encompass all or part of the processes performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, this process may include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, differential decoding, etc. In various embodiments, this process also includes, or alternatively includes, processes performed by a decoder of the various implementations described herein, such as extracting a frame from a segmented (encapsulated) frame, determining the upsampling filter to use, then upsampling the frame, and flipping the frame to its intended orientation.

[0357] As a further example, in one embodiment, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; and in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Based on the specific description above, it becomes clear whether the expression "decoding process" is intended to specifically refer to a subset of operations or to refer to a more generalized decoding process.

[0358] Various implementations involve encoding. Similar to the discussion above regarding "decoding," as used in this disclosure, "encoding" can encompass all or part of a process performed, for example, on an input video sequence, to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as segmentation, differential coding, transform, quantization, and entropy coding. In various embodiments, such processes also include, or alternatively include, processes performed by an encoder of the various implementations described in this disclosure.

[0359] As a further example, in one embodiment, "encoding" refers only to entropy encoding; in another embodiment, "encoding" refers only to differential encoding; and in yet another embodiment, "encoding" refers to a combination of differential and entropy encoding. Based on the specific description above, it will become clear whether the expression "encoding process" is intended to specifically refer to a subset of operations or to refer to a broader encoding process.

[0360] When a diagram is presented as a flowchart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0361] Various implementations involve rate-distortion optimization. Specifically, during the encoding process, a balance or trade-off between bit rate and distortion is typically considered, often constrained by computational complexity. Rate-distortion optimization is generally formulated as minimizing a rate-distortion function, which is a weighted sum of bit rate and distortion. Different approaches exist to address the rate-distortion optimization problem. For example, these methods can be based on extensive testing of all encoding options (including all considered modes or encoding parameter values) while fully evaluating their encoding costs and the associated distortion of the reconstructed signal after encoding and decoding. Faster methods can also be used to save encoding complexity, particularly by calculating approximate distortion based on prediction or prediction of the residual signal (rather than the reconstructed signal). A hybrid of these two approaches can also be used, such as using approximate distortion for only a subset of the possible encoding options and full distortion for others. Other methods evaluate only a subset of the possible encoding options. More generally, many methods employ any of a variety of techniques to perform optimization, but this optimization is not necessarily a complete evaluation of both encoding costs and associated distortion.

[0362] The implementations and aspects described herein can be implemented, for example, in methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), the implementation of the features in question can be implemented in other forms (e.g., apparatuses or programs). For example, an apparatus can be implemented with appropriate hardware, software, and firmware. These methods can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate information communication between end users.

[0363] The references to "an embodiment" or "an embodiment," "a method of implementation" or "an implementation," and other variations thereof, mean that a particular feature, structure, characteristic, etc., associated with that embodiment is included in at least one embodiment. Therefore, the phrases "in an embodiment" or "in a method of implementation" or "an implementation" appearing in various places in this disclosure, as well as any other variations, do not necessarily refer to the same embodiment.

[0364] Furthermore, this disclosure may involve "determining" various types of information. Determining information may include, for example, one or more of estimation information, calculation information, prediction information, and information retrieved from memory.

[0365] Furthermore, this disclosure may involve "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, and estimating information, or more of these.

[0366] Furthermore, this disclosure may involve "receiving" various kinds of information. Receiving, like "accessing," is a broad term. Receiving information may include, for example, accessing information and retrieving information (e.g., from memory). Moreover, "receiving" is generally involved in some way during operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0367] It should be understood that the use of any of the following " / ", "and / or" (e.g., in the cases of "A / B", "A and / or B", and "at least one of A and B") is intended to include selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As another example, in the cases of "A, B, and / or C" and "at least one of A, B, and C", this wording is intended to include selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and second listed options (A and C), or only the second and third listed options (B and C). This can be extended to any item listed.

[0368] Furthermore, as used herein, the term "signaling" refers, among other things, to instructing the corresponding decoder to provide certain information. For example, in some embodiments, the encoder signals a specific one of several parameters for use in region-based filter parameter selection for artifact removal filtering. Thus, in one embodiment, the same parameter is used on both the encoder and decoder sides. Therefore, for example, the encoder can send a specific parameter to the decoder (explicit signaling) so that the decoder can use the same specific parameter. Conversely, if the decoder already has the specific parameter as well as other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functionality. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the term "signaling" is mentioned above, "signaling" can also be used as a noun here.

[0369] Implementations can generate various signals that are formatted to carry information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, the signal may be formatted to carry a bit stream of the embodiments described. Such a signal may be formatted as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or a baseband signal. Formatting may include, for example, encoding the data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. It is well known that signals can be transmitted over various wired or wireless links. The signal may be stored on a processor-readable medium.

[0370] We have described several embodiments. Features of these embodiments may be provided individually or in any combination across various claim classes and types. Furthermore, embodiments may include one or more of the following features, devices, or aspects, individually or in any combination across various claim classes and types: • Adapt the residual at the encoder according to any of the embodiments discussed.

[0371] • A bitstream or signal containing one or more of the said syntax elements or their variations.

[0372] • A bitstream or signal containing a syntax that conveys information generated according to any of the described embodiments.

[0373] • Insert syntax elements into the signaling that enable the decoder to adapt the residuals in a manner corresponding to that used by the encoder.

[0374] • Creating and / or sending and / or receiving and / or decoding bitstreams or signals that include one or more of the syntax elements or variations thereof.

[0375] • Create and / or send and / or receive and / or decode according to any of the embodiments described.

[0376] • Methods, processes, apparatus, media storing instructions, media storing data or signals according to any of the embodiments described.

[0377] • A TV, set-top box, cellular phone, tablet computer, or other electronic device that performs filter parameter adaptation according to any of the described embodiments.

[0378] • A TV, set-top box, cellular phone, tablet computer, or other electronic device that performs filter parameter adaptation according to any of the described embodiments and displays the resulting image (e.g., using a monitor, screen, or other type of display).

[0379] • A TV, set-top box, cellular phone, tablet computer, or other electronic device that (e.g., using a tuner) selects a channel to receive a signal including an encoded image and performs filter parameter adaptation according to any of the described embodiments.

[0380] • A TV, set-top box, cellular phone, or other electronic device that receives over the air (e.g., using an antenna) a signal including an encoded image and performs filter parameter adaptation according to any of the described embodiments.

[0381] Note that the various hardware elements of one or more of the embodiments described herein may be referred to as "modules," which, in combination with the modules, perform (i.e., execute, implement, etc.) the various functions described herein. As used herein, a module includes hardware (e.g., one or more processors, one or more microprocessors, one or more microcontrollers, one or more microchips, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more storage devices) that a person skilled in the art would consider suitable for a given implementation. Each described module may also include executable instructions for performing one or more functions described as being performed by the corresponding module, and it is worth noting that these instructions may take the form of hardware (i.e., hardwired) instructions, firmware instructions, software instructions, and / or similar instructions, or include hardware (i.e., hardwired) instructions, firmware instructions, software instructions, and / or similar instructions, and may be stored in any suitable non-transitory computer-readable medium, such as commonly referred to as RAM, ROM, etc.

[0382] Although the features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in combination with other features and elements. Furthermore, the methods described herein can be implemented in a computer program, software, or firmware contained in a computer-readable medium for execution by a computer or processor. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor storage devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital multifunction disks (DVDs). A processor associated with the software can be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host.< / dot> < / dot> < / dot> < / dot> < / dot>

Claims

1. A method comprising: Establish a media session with the Application Function (AF) server used for 5G media streaming services; In response to establishing the media session, determine one or more metrics to be collected for the media session. Specifically, the determination of one or more metrics to be collected for the media session is based, at least in part, on one or more metric measurement requirements, collection requirements, and reporting requirements at the start of the media session; Receive a metric configuration message corresponding to the one or more metrics. The metric configuration message includes a metric scheme, and The metric scheme includes at least one rule or step corresponding to at least one of the one or more metrics; Configure the device to process the one or more metrics; Collect one or more of the aforementioned metrics; Generate a measurement report that includes one or more of the collected metrics; and The metric report, which includes one or more of the collected metrics, is sent to the server.

2. The method according to claim 1, wherein, At least one of the one or more metrics is a Quality of Experience (QoE) metric.

3. The method according to claim 2, wherein, The QoE metric is associated with applications or content services provided from outside the 5GS network.

4. The method of claim 2 further includes requesting the QoE metric from a device outside the 5GS network.

5. The method according to any one of claims 1 to 4, wherein, At least one of the one or more metrics is a non-3GPP (non-3rd Generation Partnership Project) metric.

6. The method according to any one of claims 1 to 5, wherein, The metric configuration message is received from the Application Function (AF) server of the 5GS network.

7. The method according to any one of claims 1 to 5, wherein, The metric configuration message is received via the application layer control plane.

8. The method according to any one of claims 1 to 7, wherein, Generating the metric report includes at least one of filtering, aggregating, and reformatting one or more of the collected metrics.

9. The method according to any one of claims 1 to 8, further comprising receiving information indicating one or more rules for processing at least one of the one or more metrics.

10. The method according to any one of claims 1 to 9, further comprising measuring at least one of the one or more measures.

11. The method according to any one of claims 1 to 10, wherein, The metrics reported in the report include: sending HTTP POST messages.

12. The method according to any one of claims 1 to 11, wherein, At least one of the one or more metrics is associated with a streaming service.

13. The method according to any one of claims 1 to 12, wherein, The metrics report mentioned in the report includes at least one of the following: the application server (AS) of the 5G media streaming service, the operator of the 5GS network, the operation, management and maintenance (OAS) server, and a third-party server.

14. The method according to any one of claims 1 to 13, further comprising: Update the metric report; as well as Report the updated metrics report to the server.

15. The method according to any one of claims 1 to 14, wherein, The metrics reported include: Real-time Media Communication (RTC) interface in 5G systems.

16. The method according to any one of claims 1 to 15, wherein, Generating the metric report involves using a format that corresponds to the requirements of 5G systems.

17. The method according to any one of claims 1 to 15, wherein, Generating the metric report involves using a format corresponding to the DASH streaming service.

18. The method according to any one of claims 1 to 15, wherein, Generating the metric report includes using XML format.

19. The method according to any one of claims 1 to 18, wherein, Collecting the one or more metrics includes receiving at least one measurement corresponding to one of the one or more metrics.

20. The method according to any one of claims 1 to 19, further comprising: Triggered by the 5G-RTC client reporting one or more of the aforementioned metrics, The trigger includes using the Open Mobile Alliance Device Management (OMA-DM) Quality of Experience (QoE) management object.

21. The method according to any one of claims 1 to 19, further comprising: Triggered by the 5G-RTC client reporting one or more of the aforementioned metrics, The triggering mentioned above includes using the QMC function.

22. The method according to claim 21, wherein, The metric report mentioned in the report includes: sending the metric report via RRC message.

23. The method according to any one of claims 1 to 22, wherein, The metrics report described in the report includes sending an HTTP POST request carrying metadata in XML format.

24. The method according to any one of claims 1 to 22, wherein, The metrics reported in the report include: using the 5G-RTC quality reporting scheme.

25. The method according to any one of claims 1 to 24, wherein, At least one of the one or more metrics is a damage duration metric.

26. The method according to any one of claims 1 to 24, wherein, At least one of the one or more metrics is the continuous loss of RTP packets.

27. The method according to any one of claims 1 to 24, wherein, At least one of the one or more metrics is the frame rate.

28. The method according to any one of claims 1 to 24, wherein, At least one of the one or more metrics is jitter duration.

29. The method according to any one of claims 1 to 24, wherein, At least one of the one or more metrics is the duration of synchronization loss.

30. The method according to any one of claims 1 to 24, wherein, At least one of the one or more metrics is round-trip time.

31. The method according to any one of claims 1 to 24, wherein, At least one of the one or more metrics is the average codec bit rate.

32. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: Configure the device to process a set of one or more metrics; Collect the set of one or more metrics; Generate a metrics report that includes one or more of the collected metrics; as well as The metric report, which includes one or more sets of metrics collected, is sent to the server.

33. A method comprising: Configure the server to process a set of one or more metrics; Collect the set of one or more metrics; as well as Send the set of one or more metrics to the user equipment (UE) device.

34. The method according to claim 33, wherein, At least one of the set of one or more metrics is a Quality of Experience (QoE) metric.

35. The method according to claim 34, wherein, The QoE metric is associated with applications or content services provided from outside the 5GS network.

36. The method of claim 34, further comprising requesting the QoE metric from a device outside the 5GS network.

37. The method according to any one of claims 33 to 36, wherein, At least one of the set of one or more metrics is a non-3GPP (non-3rd Generation Partnership Project) metric.

38. The method according to any one of claims 33 to 37, further comprising sending a metric configuration message corresponding to at least one of the set of one or more metrics.

39. The method according to claim 38, wherein, The metric configuration message is sent via the Application Function (AF) server of the 5GS network.

40. The method of claim 38, wherein, The metric configuration message is sent via the application layer control plane.

41. The method according to claim 38, in, The metric configuration message includes a metric scheme, and The metric scheme includes at least one rule or step corresponding to at least one of the one or more metrics in the set.

42. The method according to any one of claims 33 to 41, further comprising determining one or more metrics to be collected.

43. The method according to any one of claims 33 to 42, wherein, The determination of one or more metrics to be collected for a media session is based, at least in part, on one or more metric measurement requirements, collection requirements, and reporting requirements at the start of the media session.

44. The method according to any one of claims 33 to 43, further comprising at least one of filtering, aggregating, and reformatting the collected set of one or more metrics.

45. The method according to any one of claims 33 to 44, further comprising sending information indicating one or more rules for processing at least one of the set of one or more metrics.

46. ​​The method according to any one of claims 33 to 45, further comprising measuring at least one of the group of one or more measures.

47. The method according to any one of claims 33 to 46, wherein, At least one of the set of one or more metrics is associated with a streaming service.

48. The method according to any one of claims 33 to 47, wherein, Sending the set of one or more metrics to the user equipment (UE) device includes using a real-time media communication (RTC) interface in a 5G system.

49. The method according to any one of claims 33 to 48, wherein, Collecting the set of one or more metrics includes: performing at least one measurement corresponding to one of the set of one or more metrics.

50. The method according to any one of claims 33 to 49, wherein, At least one of the set of one or more metrics is a damage duration metric.

51. The method according to any one of claims 33 to 49, wherein, At least one of the set of one or more metrics is the continuous loss of RTP packets.

52. The method according to any one of claims 37 to 53, wherein, At least one of the set of one or more metrics is the frame rate.

53. The method according to any one of claims 33 to 49, wherein, At least one of the set of one or more metrics is jitter duration.

54. The method according to any one of claims 33 to 49, wherein, At least one of the set of one or more metrics is the duration of synchronization loss.

55. The method according to any one of claims 33 to 49, wherein, At least one of the set of one or more metrics is round-trip time.

56. The method according to any one of claims 33 to 49, wherein, At least one of the set of one or more metrics is the average codec bit rate.

57. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: Configure the server to process a set of one or more metrics; Collect the set of one or more metrics; as well as Send the set of one or more metrics to the user equipment (UE) device.

58. A method comprising: A quality reporting scheme configured by 5G Real-Time Media Communication (5G-RTC) user equipment or endpoints for measuring and reporting a set of Quality of Experience (QoE) metrics; as well as According to the QoE metric reporting protocol and the quality reporting scheme, the 5G-RTC user equipment or endpoint reports information related to the set of QoE metrics to the 5G-RTC metric server by sending the information.

59. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: A quality reporting scheme configured by 5G Real-Time Media Communication (5G-RTC) user equipment or endpoints for measuring and reporting a set of Quality of Experience (QoE) metrics; as well as According to the QoE metric reporting protocol and the quality reporting scheme, the 5G-RTC user equipment or endpoint reports information related to the set of QoE metrics to the 5G-RTC metric server by sending the information.

60. A method comprising: A quality reporting scheme is configured by the first network device to measure and report a set of Quality of Experience (QoE) metrics; as well as According to the QoE metric reporting protocol and the quality reporting scheme, the first network device sends information related to the set of QoE metrics to the second network device.

61. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: A quality reporting scheme is configured by the first network device to measure and report a set of Quality of Experience (QoE) metrics; as well as According to the QoE metric reporting protocol and the quality reporting scheme, the first network device sends information related to the set of QoE metrics to the second network device.

62. A method comprising: According to the QoE metric reporting protocol and the quality reporting scheme, the first network device sends information related to a set of Quality of Experience (QoE) metrics to the second network device, wherein the quality reporting scheme is configured to measure and report the information related to the set of QoE metrics.

63. The method according to claim 62, wherein, The set of QoE metrics includes a damage duration metric.

64. The method according to claim 62, wherein, The set of QoE metrics includes the consecutive loss of RTP packets.

65. The method according to claim 62, wherein, The set of QoE metrics includes frame rate.

66. The method according to claim 62, wherein, The set of QoE metrics includes jitter duration.

67. The method according to claim 62, wherein, The set of QoE metrics includes the duration of synchronization loss.

68. The method according to claim 62, wherein, The set of QoE metrics includes round-trip time.

69. The method according to claim 62, wherein, The set of QoE metrics includes the average codec bit rate.

70. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: According to the QoE metric reporting protocol and the quality reporting scheme, the first network device sends information related to a set of Quality of Experience (QoE) metrics to the second network device, wherein the quality reporting scheme is configured to measure and report the information related to the set of QoE metrics.

71. A method comprising: An XML metrics scheme configured by 5G Real-Time Media Communication (5G-RTC) user equipment or endpoints for measuring and reporting a set of Quality of Experience (QoE) metrics; as well as According to the XML metrics scheme, the 5G-RTC user equipment or endpoint reports information related to the set of QoE metrics to the 5G-RTC metrics server.

72. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: An XML metrics scheme configured by 5G Real-Time Media Communication (5G-RTC) user equipment or endpoints for measuring and reporting a set of Quality of Experience (QoE) metrics; and According to the XML metrics scheme, the 5G-RTC user equipment or endpoint reports information related to the set of QoE metrics to the 5G-RTC metrics server.

73. A method comprising: A quality metric reporting scheme configured by 5G Real-Time Media Communication (5G-RTC) user equipment or endpoints for measuring and reporting a set of Quality of Experience (QoE) metrics; as well as According to the QoE metric reporting protocol and the quality reporting scheme, the 5G-RTC user equipment or endpoint reports information related to the set of QoE metrics to the 5G-RTC metric server.

74. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: A quality metric reporting scheme configured by 5G Real-Time Media Communication (5G-RTC) user equipment or endpoints for measuring and reporting a set of Quality of Experience (QoE) metrics; as well as According to the QoE metric reporting protocol and the quality reporting scheme, the 5G-RTC user equipment or endpoint reports information related to the set of QoE metrics to the 5G-RTC metric server.

75. A method comprising: Configure 5G Real-Time Media Communication (5G-RTC) user equipment or endpoints to report a set of Quality of Experience (QoE) metrics to the metrics server; Collect information related to the set of QoE metrics; as well as The 5G-RTC user equipment or endpoint reports information related to the set of QoE metrics to the metrics server.

76. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: Configure 5G Real-Time Media Communication (5G-RTC) user equipment or endpoints to report a set of Quality of Experience (QoE) metrics to the metrics server; Collect information related to the set of QoE metrics; as well as The 5G-RTC user equipment or endpoint reports information related to the set of QoE metrics to the metrics server.

77. A method comprising: Configure 5G Real-Time Media Communication (5G-RTC) user equipment to measure and report a set of Quality of Experience (QoE) metrics; as well as The 5G-RTC user equipment reports information related to the set of QoE metrics via the RTC-1 interface.

78. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: Configure 5G Real-Time Media Communication (5G-RTC) user equipment to measure and report a set of Quality of Experience (QoE) metrics; and The 5G-RTC user equipment reports information related to the set of QoE metrics via the RTC-1 interface.

79. A method comprising: Receive metric configuration messages corresponding to a set of one or more metrics; Configure the device to process a set of one or more metrics; Collect the set of one or more metrics; Generate a metrics report that includes one or more of the collected metrics; as well as The metric report, which includes one or more sets of metrics collected, is sent to the server.

80. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the device to: Receive metric configuration messages corresponding to a set of one or more metrics; Configure the device to process a set of one or more metrics; Collect the set of one or more metrics; Generate a metrics report that includes one or more of the collected metrics; as well as The metric report, which includes one or more sets of metrics collected, is sent to the server.

81. An apparatus comprising at least one processor configured to perform the method according to any one of claims 1 to 31, 33 to 56, 58, 60, 62 to 69, 71, 73, 75, 77 and 79.

82. An apparatus comprising a computer-readable medium storing instructions for causing one or more processors to perform the method according to any one of claims 1 to 31, 33 to 56, 58, 60, 62 to 69, 71, 73, 75, 77 and 79.

83. An apparatus comprising at least one processor and at least one non-transitory computer-readable medium storing instructions for causing the at least one processor to perform the method according to any one of claims 1 to 31, 33 to 56, 58, 60, 62 to 69, 71, 73, 75, 77 and 79.

84. A signal comprising a scene description file generated according to any one of claims 1 to 31, 33 to 56, 58, 60, 62 to 69, 71, 73, 75, 77 and 79.

Images