Techniques for PDU set aware applications and associated signaling

Abstract

Aspects of the present disclosure relate to PDU set aware multimedia applications and associated signaling. An NE (1800) includes at least one memory (1804) and at least one processor (1802) configured to cause the NE (1800) to encode an applied media into a plurality of ADUs, group each ADU into one or more PDU sets by an application RTP transmitter routine as a logically independent information unit, determine PDU set information for each of the PDU sets, and transmit the PDU set information to the at least one processor (1802). The method includes the steps of: packaging PDU set information of each PDU in the PDU set in an extension header element, the extension header element being packaged within an RTP extension header of the RTP PDU, the RTP extension header corresponding to an RTP PDU payload of each PDU in one or more PDUs in the PDU set, and transmitting the RTP PDU including the PDU set information.

Description

Technical Field

[0001] The subject matter disclosed herein relates generally to wireless communications and, more particularly, to techniques for packet data unit (PDU) set aware applications and associated signaling. Background Art

[0002] A wireless communication system may include one or more network communication devices, such as a base station, which may support wireless communication of one or more user communication devices, which may also be referred to as user equipment (UE), or other suitable terms. A wireless communication system may support wireless communication with one or more user communication devices by utilizing resources of the wireless communication system (e.g., time resources (e.g., symbols, time slots, subframes, frames, etc.) or frequency resources (e.g., subcarriers, carriers, etc.). In addition, a wireless communication system may support wireless communication across various radio access technologies, including third generation (3G) radio access technology, fourth generation (4G) radio access technology, fifth generation (5G) radio access technology, and other suitable radio access technologies other than 5G (e.g., sixth generation (6G)). Summary of the invention

[0003] The word "one" before an element is not limited and is understood to refer to "at least one" of these elements or "one or more" of these elements. The terms "one", "at least one", "one or more" and "at least one of one or more" can be interchangeable. As used herein, including in the claims, the "or" used in a list of items (e.g., a list of items ending with "at least one of" or "one or more of" or "one or two of") indicates an inclusive list, so that, for example, a list of at least one of A, B or C means A or B or C or AB or AC or BC or ABC (i.e. A and B and C). In addition, as used herein, the phrase "based on" should not be interpreted as a reference to a closed set of conditions. For example, the example steps described as "based on condition A" can be based on both condition A and condition B without departing from the scope of the present disclosure. In other words, as used herein, the phrase "based on" should be interpreted in the same manner as the phrase "at least partially based on". In addition, as used herein, including in the claims, a "set" may include one or more elements.

[0004] Some implementations of the methods and apparatus described herein may also encode the media of an application into multiple application data units (ADUs), as logically independent information units, and group each ADU into one or more PDU sets by an application real-time transport protocol (RTP) transmitter routine, determine PDU set information for each PDU set in the PDU set, encapsulate the PDU set information of each PDU in the PDU set in an extended header element, the extended header element is packaged in an RTP extended header of the RTP PDU, the RTP extended header corresponds to the RTP PDU payload of each PDU in the one or more PDUs in the PDU set, and send the RTP PDU including the PDU set information.

[0005] Some implementations of the methods and apparatus described herein receive an RTP PDU set, the RTP PDU including an extended header having an extended header element, the extended header element including PDU set information and a PDU payload, extract the PDU set information from the extended header element of each RTP PDU in the PDU set, the PDU set information including: an identifier of the PDU set and the PDUs within the PDU set, and a set of PDU set attributes of the PDU set, package each RTP PDU and the corresponding PDU set information in the RTP PDU of the PDU set in a general packet radio service (GPRS) user plane tunneling protocol (GTP-U) PDU, the GTP-U PDU including: each RTP PDU in the RTP PDU in the PDU set encapsulated as a GTP-U payload, and each PDU set information in the corresponding PDU set information encapsulated as a GTP-U header field, and send the GTP-U encapsulated PDU set and the corresponding PDU set information. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 An example of a wireless communication system according to aspects of the present disclosure is shown.

[0007] Figure 2 is a diagram illustrating one embodiment of a New Radio (“NR”) protocol stack in accordance with aspects of the present disclosure.

[0008] Figure 3 is a block diagram illustrating one embodiment of a sidelink (SL) protocol stack according to aspects of the present disclosure.

[0009] Figure 4 is a diagram illustrating one embodiment of an extended reality media (XRM) architecture and processing of PDU sets in accordance with aspects of the present disclosure.

[0010] Figure 5is a diagram of an embodiment of a 5GS PDU set-aware Quality of Service (QoS) processing framework description showing PDU set to QoS flow to data radio bearer (DRB) mapping according to various aspects of the present disclosure.

[0011] Figure 6 is a diagram illustrating one embodiment of an RTP and Real-Time Control Protocol (RTCP) stack over an Internet Protocol (IP) network in accordance with aspects of the present disclosure.

[0012] Figure 7 is a diagram illustrating one embodiment of a WebRTC Secure Real-time Transport Protocol (SRTP) stack over an IP network in accordance with aspects of the present disclosure.

[0013] Figure 8 is a diagram illustrating one embodiment of an RTP / SRTP packet format and header information according to aspects of the present disclosure.

[0014] Fig. 9 is a diagram illustrating one embodiment of an RTP / SRTP header extension format and syntax according to aspects of the present disclosure.

[0015] Fig.10 is a diagram illustrating one embodiment of a representation of PDU sets and PDU set information determination and signaling for both downlink (DL) and uplink (UL) transmissions through the user plane of 5GS in accordance with various aspects of the present disclosure.

[0016] Fig.11 is a diagram illustrating one embodiment of an overview of RTP PDU packing without PDU set awareness and RTP PDU packing with PDU set awareness according to aspects of the present disclosure.

[0017] Fig.12 is a diagram illustrating one embodiment of an example of a general PDU set information extension header payload according to aspects of the present disclosure.

[0018] Fig.13 is a diagram of one embodiment showing an example of an RTP header extension format of general PDU set information according to aspects of the present disclosure.

[0019] Fig.14 is a diagram illustrating one embodiment of an example of a single-byte RTP header extension format for a PDU Set Information extension header element in accordance with aspects of the present disclosure.

[0020] Fig.15 is a diagram illustrating one embodiment of an example of multiplexing a single-byte RTP header extension element containing PDU set information with a double-byte RTP header extension element according to aspects of the present disclosure.

[0021] Fig.16 An example of a UE according to aspects of the present disclosure is shown.

[0022] Fig.17 An example of a processor according to aspects of the present disclosure is shown.

[0023] Fig.18 An example of a network equipment (NE) according to aspects of the present disclosure is shown.

[0024] Fig.19 A flowchart illustrating a method performed by a UE, a NE, or a processor according to various aspects of the present disclosure is shown.

[0025] Fig. 20 A flowchart illustrating a method performed by a UE, a NE, or a processor according to various aspects of the present disclosure is shown. DETAILED DESCRIPTION

[0026] In general, the present disclosure describes systems, methods, and devices for PDU set-aware multimedia applications and associated signaling. In some embodiments, these methods may be performed using computer code embedded on a computer-readable medium. In some embodiments, a device or system may include a computer-readable medium containing computer-readable code, which, when executed by a processor, causes the device or system to perform at least a portion of the technical solutions described below.

[0027] In the context of XR media traffic, the 3GPP SA2 WG recently introduced the concept of PDU sets to group a series of PDUs carrying information units at the application level. PDU sets can therefore be processed according to the same set of QoS requirements and associated delay budget and error rate constraints, while providing support for the radio access network (RAN) to perform differentiated QoS treatment at the PDU set level. This improves the granularity of the traditional 5G QoS flow framework, allowing the RAN to optimize the mapping between QoS flows and DRBs to meet stringent XR media requirements (e.g., high-rate transmission with a short delay budget).

[0028] One issue related to the latter activity is how to communicate PDU set information (e.g., PDU set boundaries, PDU set importance) between the application and the 5GS. The current disclosure approaches this issue from an application-centric perspective, where the application, i.e., the application logic on the UE device or the application provider's application server, determines the PDU set information and signals the PDU set information to the 5GS core network. This signaling is provided in real time with low signaling / control overhead to achieve QoS performance benefits based on the PDU set concept.

[0029] In the following, XR is used as an umbrella term for different types of reality, including. For example, VR is a rendered version of a delivered visual and audio scene. In this case, the rendering is designed to mimic the visual and audio sensory stimulation of the real world as naturally as possible to the observer or user as the observer or user moves within the limitations defined by the application. Virtual reality typically, but not necessarily, requires the user to wear a head-mounted display ("HMD") that completely replaces the user's field of view with simulated visual components, and wear headphones that provide the user with accompanying audio. Some form of head and motion tracking is also typically required in VR to allow the simulated visual and audio components to be updated to ensure that items and sound sources remain consistent with the user's movements from the user's perspective. In some implementations, additional components for interacting with the virtual reality simulation may be provided, but this is not strictly required.

[0030] In another example, AR refers to providing the user with additional information or artificially generated items, or content overlaid on the current environment. Such additional information or content will typically be visual and / or auditory, and their observation of their current environment may be direct, without intervening sensing, processing, and rendering, or indirect, where their perception of their environment is relayed via sensors and possibly augmented or processed.

[0031] MR is an advanced form of AR in which some virtual elements are inserted into the physical scene, aiming to provide the illusion that these elements are part of the real scene.

[0032] XR refers to the combined real and virtual environment and human-computer interaction generated by computer technology and wearable devices. It includes representative forms such as AR, MR and VR and the areas inserted between them. The level of virtuality ranges from partial sensory input to fully immersive VR. A key aspect of XR is the expansion of human experience, especially related to presence (represented by VR) and cognitive acquisition (represented by AR).

[0033] Aspects of the disclosure are described in the context of a wireless communication system.

[0034] Figure 1An example of a wireless communication system 100 according to various aspects of the present disclosure is shown. The wireless communication system 100 may include one or more NEs 102, one or more UEs 104, and a core network (CN) 106. The wireless communication system 100 may support various radio access technologies. In some implementations, the wireless communication system 100 may be a 4G network, such as an LTE network or an LTE-Advanced (LTE-A) network. In some other implementations, the wireless communication system 100 may be an NR network, such as a 5G network, a 5G-Advanced (5G-A) network, or a 5G ultra-wideband (5G-UWB) network. In other implementations, the wireless communication system 100 may be a combination of a 4G network and a 5G network, or other suitable radio access technologies, including the Institute of Electrical and Electronics Engineers (IEEE) 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20. The wireless communication system 100 may support radio access technologies other than 5G, such as 6G. Additionally, the wireless communication system 100 may support technologies such as time division multiple access (TDMA), frequency division multiple access (FDMA), or code division multiple access (CDMA).

[0035] One or more NEs 102 may be dispersed throughout a geographic area to form a wireless communication system 100. One or more NEs 102 described herein may be or include or may be referred to as network nodes, base stations, network elements, network functions, network entities, radio access networks (RAN), NodeBs, eNodeBs (eNBs), next generation NodeBs (gNBs), or other suitable terms. NE 102 and UE 104 may communicate via a communication link, which may be a wireless or wired connection. For example, NE 102 and UE 104 may perform wireless communication (e.g., receive signaling, send signaling) via a Uu interface.

[0036] NE 102 may provide a geographic coverage area within which NE 102 may support services for one or more UE 104. For example, NE 102 and UE 104 may support wireless communications of signals associated with services (e.g., voice, video, packet data, messaging, broadcast, etc.) according to one or more radio access technologies. In some implementations, NE 102 may be mobile, such as a satellite associated with a non-terrestrial network (NTN). In some implementations, different geographic coverage areas 112 associated with the same or different radio access technologies may overlap, but different geographic coverage areas may be associated with different NE 102.

[0037] One or more UEs 104 may be dispersed throughout the geographic area of ​​the wireless communication system 100. UE 104 may include or may be referred to as a remote unit, a mobile device, a wireless device, a remote device, a subscriber device, a transmitter device, a receiver device, or some other suitable terminology. In some implementations, UE 104 may be referred to as a unit, a station, a terminal, or a client, etc. Additionally or alternatively, UE 104 may be referred to as an Internet of Things (IoT) device, an Internet of Everything (IoE) device, or a Machine Type Communication (MTC) device, etc.

[0038] The UE 104 may be capable of supporting wireless communications directly with other UEs 104 via a communication link. For example, the UE 104 may support wireless communications directly with another UE 104 via a device-to-device (D2D) communication link. In some implementations such as vehicle-to-vehicle (V2V) deployments, vehicle-to-everything (V2X) deployments, or cellular-V2X deployments, the communication link 114 may be referred to as a side link. For example, the UE 104 may support wireless communications directly with another UE 104 via a PC5 interface.

[0039] NE 102 can support communication with CN 106 or with another NE 102, or both. For example, NE 102 can be docked with other NE 102 or CN 106 via one or more backhaul links (e.g., S1, N2, N2 or network interface). In some implementations, NE 102 can communicate directly with each other. In some other implementations, NE 102 can communicate with each other or communicate indirectly (e.g., via CN 106). In some implementations, one or more NE 102 may include subcomponents, such as access network entities, which may be examples of access node controllers (ANCs). ANC can communicate with one or more UE 104 via one or more other access network transmission entities, which may be referred to as radio heads, smart radio heads, or transmission-reception points (TRPs).

[0040] The CN 106 may support user authentication, access authorization, tracking, connectivity, and other access, routing, or mobility functions. The CN 106 may be an evolved packet core (EPC) or a 5G core (5GC), which may include control plane entities that manage access and mobility (e.g., mobility management entity (MME), access and mobility management function (AMF)) and user plane entities that route or interconnect packets to external networks (e.g., serving gateway (S-GW), packet data network (PDN) gateway (P-GW), or user plane function (UPF)). In some implementations, the control plane entities may manage non-access stratum (NAS) functions, such as mobility, authentication, and bearer management (e.g., data bearers, signal bearers, etc.) of one or more UEs 104 served by one or more NEs 102 associated with the CN 106.

[0041] The CN 106 may communicate with a packet data network via one or more backhaul links (e.g., via S1, N2, N2 or another network interface). The packet data network may include an application server. In some implementations, one or more UEs 104 may communicate with the application server. The UE 104 may establish a session (e.g., a protocol data unit (PDU) session, etc.) with the CN 106 via the NE 102. The CN 106 may use the established session (e.g., an established PDU session) to route traffic (e.g., control information, data, etc.) between the UE 104 and the application server. A PDU session may be an example of a logical connection between a UE 104 and a CN 106 (e.g., one or more network functions of the CN 106).

[0042] In the wireless communication system 100, the NE 102 and the UE 104 may use the resources of the wireless communication system 100 (e.g., time resources (e.g., symbols, time slots, subframes, frames, etc.) or frequency resources (e.g., subcarriers, carriers)) to perform various operations (e.g., wireless communication). In some implementations, the NE 102 and the UE 104 may support different resource structures. For example, the NE 102 and the UE 104 may support different frame structures. In some implementations, such as in 4G, the NE 102 and the UE 104 may support a single frame structure. In some other implementations, such as in 5G and other suitable radio access technologies, the NE 102 and the UE 104 may support various frame structures (i.e., multiple frame structures). The NE 102 and the UE 104 may support various frame structures based on one or more parameter sets.

[0043] One or more parameter sets may be supported in the wireless communication system 100, and the parameter sets may include subcarrier spacing and cyclic prefixes. A first parameter set (e.g., μ=0) may be associated with a first subcarrier spacing (e.g., 15kHz) and a normal cyclic prefix. In some implementations, a first parameter set (e.g., μ=0) associated with a first subcarrier spacing (e.g., 15kHz) may utilize one slot per subframe. A second parameter set (e.g., μ=1) may be associated with a second subcarrier spacing (e.g., 30kHz) and a normal cyclic prefix. A third parameter set (e.g., μ=2) may be associated with a third subcarrier spacing (e.g., 60kHz) and a normal cyclic prefix or an extended cyclic prefix. A fourth parameter set (e.g., μ=3) may be associated with a fourth subcarrier spacing (e.g., 120kHz) and a normal cyclic prefix. A fifth parameter set (e.g., μ=4) may be associated with a fifth subcarrier spacing (e.g., 240kHz) and a normal cyclic prefix.

[0044] The time intervals of resources (e.g., communication resources) can be organized according to frames (also referred to as radio frames). Each frame can have a duration, for example, a duration of 10 milliseconds (ms). In some implementations, each frame can include multiple subframes. For example, each frame can include 10 subframes, and each subframe can have a duration, for example, a duration of 1 ms. In some implementations, each frame can have the same duration. In some implementations, each subframe of a frame can have the same duration.

[0045] Additionally or alternatively, the time intervals of resources (e.g., communication resources) can be organized according to time slots. For example, a subframe may include multiple (e.g., number) time slots. The number of time slots in each subframe may also depend on one or more parameter sets supported in the wireless communication system 100. For example, the first parameter set, the second parameter set, the third parameter set, the fourth parameter set, and the fifth parameter set (i.e., μ=0, μ=1, μ=2, μ=3, μ=4) associated with the corresponding subcarrier spacings of 15kHz, 30kHz, 60kHz, 120kHz, and 240kHz may utilize a single time slot per subframe, two time slots per subframe, four time slots per subframe, eight time slots per subframe, and 16 time slots per subframe, respectively. Each time slot may include multiple (e.g., number) symbols (e.g., OFDM symbols). In some implementations, the number of time slots (e.g., number) of a subframe may depend on the parameter set. For a normal cyclic prefix, a time slot may include 14 symbols. For an extended cyclic prefix (e.g., for a 60 kHz subcarrier spacing), a slot may include 12 symbols. For a normal cyclic prefix and an extended cyclic prefix, the relationship between the number of symbols per slot, the number of slots per subframe, and the number of slots per frame may depend on the parameter set. It should be understood that references to a first parameter set (e.g., μ=0) associated with a first subcarrier spacing (e.g., 15 kHz) may be used interchangeably between subframes and slots.

[0046] In the wireless communication system 100, the electromagnetic (EM) spectrum can be divided into various categories, frequency bands, channels, etc. based on frequency or wavelength. For example, the wireless communication system 100 can support one or more operating frequency bands, such as the frequency range designations FR1 (410 MHz–7.125 GHz), FR2 (24.25 GHz–52.6 GHz), FR3 (7.125 GHz–24.25 GHz), FR4 (52.6 GHz–114.25 GHz), FR4a or FR4-1 (52.6 GHz–71 GHz), and FR5 (114.25 GHz–300 GHz). In some implementations, the NE 102 and the UE 104 can perform wireless communications on the one or more operating frequency bands. In some implementations, FR1 can be used by the NE 102 and the UE 104 and other devices or apparatuses for cellular communication traffic (e.g., control information, data). In some implementations, FR2 can be used by the NE 102 and the UE 104 and other devices or apparatuses for short-range, high data rate capabilities.

[0047] FR1 may be associated with one or more parameter sets (e.g., at least three parameter sets). For example, FR1 may be associated with a first parameter set (e.g., μ=0) including a 15kHz subcarrier spacing, a second parameter set (e.g., μ=1) including a 30kHz subcarrier spacing, and a third parameter set (e.g., μ=2) including a 60kHz subcarrier spacing. FR2 may be associated with one or more parameter sets (e.g., at least 2 parameter sets). For example, FR2 may be associated with a third parameter set (e.g., μ=2) including a 60kHz subcarrier spacing, and a fourth parameter set (e.g., μ=3) including a 120kHz subcarrier spacing.

[0048] Figure 2 An NR protocol stack 200 is depicted according to an embodiment of the present disclosure. Figure 2 UE 205, RAN node 210, and 5GC 220 are shown, but these represent a collection of remote units 105 that interact with base station unit 121 (or gNB) and mobile core network 140. As depicted, NR protocol stack 200 includes user plane protocol stack 201 and control plane protocol stack 203. User plane protocol stack 201 includes physical ("PHY") layer 207, medium access control ("MAC") sublayer 209, radio link control ("RLC") sublayer 211, packet data convergence protocol ("PDCP") sublayer 213, and service data adaptation protocol ("SDAP") layer 215. Control plane protocol stack 203 includes PHY layer 207, MAC sublayer 209, RLC sublayer 211, and PDCP sublayer 213. Control plane protocol stack 203 also includes radio resource control ("RRC") layer 217 and NAS layer 219.

[0049] The AS layer 221 (also referred to as the "AS protocol stack") of the user plane protocol stack 201 consists of at least SDAP, PDCP, RLC and MAC sublayers and a physical layer. The AS layer 223 of the control plane protocol stack 203 consists of at least RRC, PDCP, RLC and MAC sublayers and a physical layer. Layer 1 ("L1") consists of the PHY layer 207. Layer 2 ("L2") is divided into SDAP, PDCP, RLC and MAC sublayers. Layer 3 ("L3") includes the RRC sublayer 217 and NAS layer 219 of the control plane, and includes, for example, an Internet Protocol ("IP") layer or a PDU layer of the user plane (note what is depicted). L1 and L2 are referred to as "lower layers", while L3 and above (e.g., transport layer, application layer) are referred to as "higher layers" or "upper layers".

[0050] The PHY layer 207 provides transport channels to the MAC sublayer 209. The MAC sublayer 209 provides logical channels to the RLC sublayer 211. The RLC sublayer 211 provides RLC channels to the PDCP sublayer 213. The PDCP sublayer 213 provides radio bearers to the SDAP sublayer 215 and / or the RRC layer 217. The SDAP sublayer 215 provides QoS flows to the core network (e.g., 5GC 220). The RRC layer 217 provides the addition, modification, and release of carrier aggregation and / or dual connectivity. The RRC layer 217 also manages the establishment, configuration, maintenance, and release of signaling radio bearers ("SRBs") and data radio bearers ("DRBs").

[0051] The NAS layer 219 is located between the UE 205 and the AMF in the 5GC 220. NAS messages are passed transparently through the RAN. The NAS layer 219 is used to manage the establishment of communication sessions and maintain continuous communication with the UE 205 as the UE 205 moves between different cells in the RAN. In contrast, the AS layers 221 and 223 are located between the UE 205 and the RAN (i.e., the RAN node 210) and carry information through the wireless part of the network. Although Figure 2 Not depicted, but the IP layer is located above the NAS layer 219, the transport layer is located above the IP layer, and the application layer is located above the transport layer.

[0052] The MAC sublayer 209 is the lowest sublayer in the L2 architecture of the NR protocol stack. It is connected to the PHY layer 207 below through a transport channel and to the RLC sublayer 211 above through a logical channel. Therefore, the MAC sublayer 209 performs multiplexing and demultiplexing between logical channels and transport channels: the MAC sublayer 209 in the transmitting side constructs a MAC PDU (also called a transport block ("TB")) from a MAC service data unit ("SDU") received through a logical channel, while the MAC sublayer 209 in the receiving side recovers a MAC SDU from a MAC PDU received through a transport channel.

[0053] The MAC sublayer 209 provides data transmission services to the RLC sublayer 211 through logical channels, which are either control logical channels carrying control data (e.g., RRC signaling) or traffic logical channels carrying user plane data. On the other hand, data from the MAC sublayer 209 is exchanged with the PHY layer 207 through transport channels, which are classified as UL or DL. Data is multiplexed into transport channels according to the way it is transmitted over the air.

[0054] The PHY layer 207 is responsible for the actual transmission of data and control information via the air interface, i.e., the PHY layer 207 carries all information from the MAC transport channel over the air interface on the transmission side. Some important functions performed by the PHY layer 207 include coding and modulation, link adaptation (e.g., adaptive modulation and coding (“AMC”)), power control, cell search and random access (for initial synchronization and handover purposes), and other measurements of the RRC layer 217 (within a 3GPP system (i.e., NR and / or LTE system) and between systems). The PHY layer 207 performs transmission based on transmission parameters such as modulation scheme, coding rate (i.e., modulation and coding scheme (“MCS”)), number of physical resource blocks, etc.

[0055] Figure 3 SL protocol stack 300 according to an embodiment of the present disclosure is depicted. Figure 3 A transmitting SL UE 301 (denoted as "Tx UE") and a receiving SL UE 303 (denoted as "Rx UE") are shown, but these represent a collection of UEs communicating peer to peer via a PC5 interface, and other embodiments may involve different UEs. As depicted, the protocol stack 300 includes a physical layer 305, a MAC sublayer 307, an RLC sublayer 309, a PDCP sublayer 311, and RRC and SDAP layers (depicted as a combined element "RRC / SDAP" 313) for the control plane and the user plane, respectively. The physical layer 305, the MAC sublayer 307, the RLC sublayer 309, the PDCP sublayer 311, and the RRC / SDAP layer 313 can perform substantially the same functions as described above with reference to the NR protocol stack 200, but support UE-to-UE communication between the Tx UE 301 and the Rx UE 303.

[0056] The AS protocol stack for the control plane in the SL protocol stack 300 consists of at least RRC, PDCP, RLC and MAC sublayers and a physical layer. The AS protocol stack for the user plane in the SL protocol stack 300 consists of at least SDAP, PDCP, RLC and MAC sublayers and a physical layer. L2 is divided into SDAP, PDCP, RLC and MAC sublayers. L3 includes an RRC sublayer for the control plane and includes, for example, an IP layer for the user plane. L1 and L2 are referred to as "lower layers", while L3 and above (e.g., transport layer, V2X layer, application layer) are referred to as "higher layers" or "upper layers".

[0057] Regarding PDU sets within the core network, the study of XRM at the CN level in Release 18 introduced the concept of PDU sets to handle the QoS requirements of XRM applications and flows at a finer granularity than is possible with QoS flows. Therefore, according to "3GPP Technical Report TR23.700-60 (v0.0.3–May 2022). Study on XR (Extended Reality) and media services" (hereinafter referred to as "Study on XR", which is incorporated herein by reference), a PDU set consists of one or more PDUs that carry the payload of an information unit generated at the application level (e.g., a frame or video slice of an XRM service). In some implementations, all PDUs in a PDU set are required by the application layer to use the corresponding information unit. In other implementations, when some PDUs are lost, the application layer can still recover part or all of the information unit.

[0058] In addition, the PDU set is associated with QoS requirements in terms of delay budget and error rate, see "Study on XR", "3GPP Technical Specification TS23.501 (v17.5.0–June 2022). System architecture for the 5G System (5GS)" (hereinafter referred to as "the system architecture of the 5G system", which is incorporated herein by reference).

[0059] For example, the PDU Set Delay Budget (PSDB) defines an upper limit on the time that a PDU set may be delayed between the UE and the N6 termination point at the UPF. The PSDB applies to DL PDU sets received by the UPF over the N6 interface, as well as UL PDU sets sent by the UE.

[0060] In addition, the PDU set error rate (PSER) defines an upper limit on the ratio of PDU sets (e.g., a collection of IP packets constituting a PDU set) that have been processed by a transmitter of a link layer protocol (e.g., RLC in a RAN accessed by 3GPP), but all PDUs in the PDU set have not been successfully delivered to an upper layer (e.g., PDCP in a RAN accessed by 3GPP) by a corresponding receiver, while the PSER is used to determine an upper limit on the ratio of non-congestion related packet losses.

[0061] exist Figure 4 An overview of the CN XRM architecture processing of PDU sets is depicted in FIG.

[0062] The general processing steps for DL ​​traffic include an application function ("AF") 402, which provides the PCF 404 with the QoS requirements (e.g., PSDB and PSER) of the packets of the PDU set and information for identifying the application (i.e., a 5-tuple or application id). The AF 402 may also include an importance parameter for the PDU set and information for the core network to identify the packets belonging to the PDU set. In one embodiment, the PDU set importance field / parameter linearly encodes l=16 importance values, where a "0" encoded value indicates the highest PDU set importance and a "15" encoded value indicates the lowest PDU set importance. As used herein, linear means: in terms of PDU set importance, 0>1>2>....>15, where 0 is the most important and 15 is the least important.

[0063] The PCF 404 derives QoS rules for XR applications (i.e., using 5QI for XR media traffic) and specific QoS requirements for PDU sets, and configures the SMF 406. The PCF 404 may include PCC rules for each importance of a PDU set (based on information received from the AF or based on operator configuration).

[0064] SMF 406 establishes QoS flows according to the QoS rules of PCF 404 and configures UPF 408 to route packets of XR applications to QoS flows and, in addition, enables PDU set processing. SMF 406 also provides QoS profiles containing PDU set QoS requirements to RAN 410 via AMF 412.

[0065] The UPF 408 inspects the packets and determines the packets that belong to the PDU set (e.g., by inspecting the RTP packets). When the UPF 408 detects the packets of the PDU set, the UPF 408 marks the packets that belong to the PDU set in the GTP-U header. The GTP-U header information includes the PDU set sequence number and the size of the PDU set. The UPF 408 can also determine the importance of the PDU set based on the UPF 408 implementation component, the information provided by the AF 402, or the information provided as metadata from the application server. Based on the importance of the PDU set, the UPF 408 can route the traffic to the corresponding QoS flow (according to the rules received from the SMF) or include the importance of the PDU set in the GTP-U header.

[0066] The RAN 410 identifies packets belonging to a PDU set (based on the GTP-U marking) and processes the packets of the PDU set according to the QoS requirements of the PDU set provided by the SMF 406. In one implementation, the RAN node may use different radio bearers with higher QoS requirements (according to the PDU set PSDB / PSER) to guarantee the delivery of packets of the PDU set while using different radio bearers for non-PDU set packets according to the 5QI of the QoS flow.

[0067] The mutual processing is applicable to the UL, while the role of UPF packet inspection is assumed by the UE 414, which is expected to inspect the packets, determine the packets belonging to the PDU set, and signal the PDU set to the RAN accordingly for scheduling and resource allocation corresponding to the associated DRBs that can meet the PDU set QoS requirements (i.e. PSDB and PSER). The low-level signaling mechanisms associated with the UL UE to RAN information transfer depend on the specification and implementation of the RAN signaling procedures.

[0068] Depending on the QoS flow mapping and RAN procedures, several alternative PDU set to QoS flow to DRB mappings may be achieved given two different PDU sets with different PDU set properties (such as PDU set importance).

[0069] Figure 5 It is shown that two PDU sets 510, 512 with different importance and characteristics are mapped to QoS flows 506, 508 and DRBs 502, 504, respectively. In this example, consider that PDU set 1 510 is of high importance and has strict QoS requirements (i.e., PSDB, PSER, etc.), while PDU set 2 512 is of low importance and has lower QoS requirements (i.e., PSDB, PSER, etc.) than PDU set 1 510. Figure 5 As shown in , according to the QoS flow policy and layer 2 RAN procedures, the PDU aggregation to QoS flow to DRB can adopt the following examples.

[0070] 1-to-1-to-1 514: wherein QoS flows 506, 508 and DRBs 502, 504 are completely separated between high importance PDU sets 510 and low importance PDU sets 512, thereby finely optimizing radio and network resources on a per PDU set basis.

[0071] M to M to 1 516: wherein the separation between the high importance PDU set 510 and the low importance PDU set 512 is performed only at the QoS flow level 506, 508, and the same DRB 502 is used for air transmission of both PDU sets 510, 512, which may result in over-configuration of radio resources for the low importance PDU set, but requires lower RAN complexity and management overhead.

[0072] M to 1 to 1 518: wherein there is no separation between the QoS flows 506 and DRBs 502 of different importance PDU sets 510, 512, and the QoS requirements of higher importance PDU sets have priority when handling QoS management across both CN and RAN; this may result in over-provisioning of resources for low importance PDU sets in both CN and RAN implementations, but requires lower overhead and control within the 5GS QoS framework.

[0073] M to 1 to M 520: wherein there is no separation across QoS flows 506 between PDU set importance levels, but different DRBs 502, 504 are used to meet individual requirements of different importance levels; this reduces the QoS flow management complexity, and the PDU set information is used to filter the PDU sets 510, 512 on different DRBs 502, 504 to better match the QoS requirements at the RAN level and optimize resource allocation based on individual PDU set requirements.

[0074] Regarding the media transport protocol, the XR traffic model was analyzed in Release 17SA4, see "3GPP Technical Report TR26.926 (v1.1.0–February 2022). Traffic Models and Quality Evaluation Methods for Media and XR Services in 5G Systems" (hereinafter referred to as "the traffic model", which is incorporated herein by reference), and the QoS requirements in terms of delay budget, data rate and error rate necessary to obtain a satisfactory experience at the application level were summarized. This leads to 4 additional 5QIs for 5GS XR QoS flows as part of Table 5.7.4-1 of "System architecture for the 5G System", with delay-critical GBR 5QI values ​​of 87-90. The latter applies to XR video streams and control metadata necessary to provide immersive and interactive XR experiences.

[0075] XR video traffic consists primarily of multiple high-resolution (e.g., typically at least 1080p binocular buffers), frames per second (e.g., 60+fps), and high-bandwidth (e.g., typically at least 20-30Mbps) DL / UL video streams that need to be sent across the network with minimal latency (typically capped at 15-20ms) to maintain reduced end-to-end application round-trip interaction latency. The latter requirement is critical given that XR applications rely on cloud / edge processing (e.g., content download, viewport generation and configuration, viewport updates, viewport rendering, media encoding / transcoding, etc.).

[0076] Traffic for immersive and interactive XR applications as described above typically requires real-time applicable transport architectures and protocols. As part of the latter, the state-of-the-art technologies are represented by the Real-time Transport Protocol (RTP) (according to "RFC 3550-RTP: Transport Protocol for Real-time Applications (ietf.org)" (hereinafter referred to as "RFC 3550", which is incorporated herein by reference)), its secure configuration Secure Real-time Transport Protocol (SRTP) [Ref-6] and its stack for the Web Web Real-time Communication WebRTC (according to "WebRTC 1.0: Real-time Communication between Browsers (w3.org)" (hereinafter referred to as "WebRTC 1.0", which is incorporated herein by reference)).

[0077] like Figure 6 As shown in , RTP 602 is a media codec-independent network protocol with an application layer framework for real-time delivery of multimedia (e.g., audio, video, etc.) data over an IP network, see RFC 3550. It is used in conjunction with a sister protocol for control, namely the Real-time Transport Control Protocol (RTCP) 604, to provide end-to-end features such as jitter compensation, packet loss and out-of-order delivery detection, synchronization, and source stream multiplexing.

[0078] SRTP is a secure version of RTP, see "RFC 3711 - Secure Real-time Transport Protocol (SRTP) (ietf.org)" (hereinafter referred to as "RFC 3711", which is incorporated herein by reference), providing encryption (mainly by means of payload confidentiality), message authentication and integrity protection (by means of PDU, i.e. header and payload, signature), and replay attack protection. Similar to RTP, the SRTP sister protocol is SRTCP. This provides the same functionality as its RTCP counterpart. Therefore, in the original SRTP version, the RTP header information remains accessible but cannot be modified, while the payload is encrypted. These security provisions are in Figure 6 The right part of is shown. In addition, the key exchange and additional security parameters required for using SRTP are based on the Datagram Transport Layer Security (DTLS) key exchange process. For these reasons, SRTP is used as the transport protocol for media in the WebRTC stack, which ensures secure RTC multimedia communication through the web browser interface.

[0079] exist Figure 7An overview of the WebRTC stack is provided in . As shown, the IP layer carries signaling from the data plane 702 and the control plane 704. The data plane stack includes the functions of User Datagram Protocol (UDP) 706, Interactive Connectivity Establishment (ICE) 708, Datagram Transport Layer Security (DTLS) 710, SRTP 712, SRTCP 714, Media Codecs 716, Quality Control 718, and SCTP 720. ICE 708 can use Session Traversal Utilities for NAT (STUN) protocol and Traversal Using Relays around NAT (TURN) to solve the real-time media content delivery across heterogeneous networks and NAT rules and firewalls. The SCTP 720 data plane is primarily used as an application data channel and may not be time-critical, while the SRTP 712-based stack including control elements (i.e., SRTCP 714), coding elements (i.e., media codecs 716), and Quality of Service (QoS) (i.e., Quality Control 718) is dedicated to time-critical transmissions.

[0080] Figure 8 The RTP 802 and SRTP 804 header information are shown sharing the same format. Fig. 9 The individual fixed header information and complete header information (including header extension) of the RTP / SRTP packet are briefly summarized and shown as follows:

[0081] Fixed header information:

[0082] V 806 – 2 bits, which indicate the protocol version used.

[0083] P 808 – A 1-bit field that indicates the presence of one or more zero-padding octets at the end of the payload, so that the padding may be necessary, among other things, for fixed-size encryption blocks or for carrying multiple RTP / SRTP packets over lower layer protocols.

[0084] X 810 – 1 bit, which indicates that the standard fixed RTP / SRTP header will be followed by an RTP header extension, typically associated with a specific data / profile, which will carry more information about the data (e.g., a frame marking RTP header extension for video data, see “RFC 3711”, or a generic RTP header extension such as the RTP / SRTP Extension Protocol, see “WebRTC 1.0”).

[0085] CC 812 - 4 bits, which indicates the number of contributing media sources (CSRC) following the fixed header.

[0086] M 814 – 1 bit, which is intended to mark information frame boundaries in a packet stream, and its behavior is exactly specified by the RTP profile (e.g. H.264, H.265, H.266, AV1, etc.).

[0087] PT 816 – 7 bits, which indicate the payload type, which in case of video profile is dynamic and negotiated with the help of SDP (e.g., 96 for H.264, 97 for H.265, 98 for AV1, etc.).

[0088] Sequence Number 818 - 16 bits indicating a sequence number that increments by one with each RTP data packet sent over the session.

[0089] Timestamp 820 - 32 bits indicating a timestamp in ticks of the payload type clock, reflecting the sampling instant of the first octet of the RTP data packet (associated with a video frame for a video stream), where the first timestamp of the first RTP packet is randomly selected.

[0090] Synchronization Source (SSRC) Identifier 822 - A 32-bit field that indicates a random identifier of the source of a stream of RTP packets that form part of the same timing and sequence number space, so that a receiver can group packets based on a synchronization source for playback.

[0091] Contributing Source (CSRC) Identifier 824 - A list of up to 16 32-bit CSRC entries, each entry given the amount of CSRC mixed by the RTP mixer within the current payload as signaled by the CC bits; this list identifies the contributing sources of the payload contained in this packet given their SSRC identifiers.

[0092] Complete header information (including header extensions):

[0093] RTP Header Extension 826 - If the X bit is marked, a variable length field is present; if present, the header extension is appended to the RTP fixed header information following the CSRC list; the RTP header extension is 32-bit aligned and is formed by the following fields:

[0094] A 16-bit extension identifier defined by a profile and typically negotiated and determined via the Session Description Protocol (SDP) signaling mechanism.

[0095] A 16-bit length field describing the length of the extended header in multiples of 32 bits, excluding the first 32 bits corresponding to the 16-bit extension identifier and the 16-bit length field itself.

[0096] A 32-bit aligned header extension raw data field formatted according to the format specified by some RTP header extension identifier.

[0097] The RTP header extension format and syntax are similar to those of SRTP. In addition, in both RTP and SRTP, only one RTP extension header can be appended to the fixed header information, see "RFC 3550". However, according to "RFC8285: Generic Mechanism for RTP Header Extensions (rfc-editor.org)" (hereinafter referred to as "RFC 8285", which is incorporated herein by reference), for both RTP and SRTP, there are extensions to the basic protocol to allow multiple predetermined types of RTP header extensions to be appended to the fixed header information of the protocol.

[0098] In some embodiments, an RTP header extension generated at a source may be ignored by a destination endpoint that does not have the knowledge to interpret and process the RTP header extension sent by the source endpoint.

[0099] Regarding PSB identification and importance, according to Figure 4 , determine PDU sets and control QoS The PDU set to QoS flow to DRB mapping of the 5GS framework means determining the PDU set boundary (PSB), determining the PDU set importance (PSI) and additional PDU set characteristics (e.g., PDU set size, PDU set discard flag, PDU set forward error correction (FEC) information, etc.), and / or making the PDU set information available to CN / RAN and 5GS.

[0100] These elements can be addressed in some technical solutions by the UPF for DL, where the UPF acts as a PDU filter with embedded intelligence to determine the PSB, PSI and PDU set information. In addition, as described in the previous article, the UPF will also set, configure and map PDU sets to appropriate QoS rules with the help of QoS flow. The disadvantage of this UPF-centric processing of PDU set information and the determination of PDU sets is the increase in processing on the UPF side, which increases the complexity of UPF implementation linearly with the increase in the number of connections that attempt to utilize the concept of PDU sets. This situation occurs because the UPF technical solutions that can perform PDU sets and PDU set information determination rely on deep packet inspection and RTP header processing, which increases the buffering and computational requirements for UPF implementation. Technical solutions along these lines have been proposed in the 3GPP SA2 WG study phase "Study on XR", and these technical solutions use the RTP packet format to determine the video frame boundary using a combination of one or more of the RTP timestamp, sequence number and M-bit mark. In some technical solutions, this information is supplemented by additional information extracted from application-specific and / or profile-specific RTP header extensions (e.g., draft-ietf-avtext-framemarking-13, "RTP Frame Marking RTP Header Extension (November 2021) - draft-ietf-avtext-framemarking-13" (hereinafter referred to as "RTP Frame Marking", which is incorporated herein by reference) or from parsing RTP payload headers (e.g., NAL units for video coding in H.26x codecs). This last collected information is then used to extract some classification / estimation of the importance of the detected PDU set and determine the PSB.

[0101] However, due to the shortcomings of these technical solutions and the fact that they are asymmetric with respect to UL processing, they are not easy to handle in practice. In the UL, the determination of PDU sets and PDU set information falls within the UE responsibility, as the UE will use the PDU set information to map the PDU sets to the appropriate DRBs. With the help of SDAP processing, the DRBs are mapped to QoS flows over the N3 interface based on the existing Layer 2 configured on the SMF. The UPF will then route the UL to the application server based on the QoS rules configured by the PCF and the QoS flows set by the SMF.

[0102] The present disclosure proposes to use an application-centric design to determine PDU sets, PDU set information, and signal such information to 5GS. The advanced technical solution relies on determining PDU set boundaries at application runtime and grouping PDUs within a PDU set when encapsulating XR media data into RTP / SRTP / WebRTC real-time transport protocol, determining PDU set characteristics at application runtime (e.g., PDU set size, PDU set importance, PDU discardability, i.e., the necessity of all PDUs for using PDU sets at the application layer, etc.) and extracting this PDU set information, including a PDU set identifier (e.g., as a PDU sequence number), a PDU identifier within a PDU set (e.g., as a PDU sequence number within a PDU set), and / or encapsulating the PDU set information within an RTP extension header element for in-band signaling of the PDU set information, and fully packaging all PDUs in the PDU set as RTP / SRTP / WebRTC real-time transport protocol PDUs.

[0103] The above processing steps are driven by the source, which is the original source for the XR media stream. The technical solution is application-driven and applies to both UE and AS as an over-the-top process to assist the 5GS QoS framework, which therefore benefits PDU sets and PDU set information determination and signaling. In addition, the proposed in-band signaling allows interoperability within 5GS by separating the additional complexity and implementation details of PDU sets and PDU set information determination from 5GS.

[0104] Referring to the main steps of the UL and DL user plane processing of 5GS above, Fig.10 A high-level technical solution is shown in .

[0105] In DL, PCF 1002 decides QoS flow and PDU set awareness (i.e., RTP header extension configuration that captures RTP header extension containing PDU set information) PCC rules based on requirements provided by AF 1004 through NEF / PCF interface for specific types of applications (e.g., XR, CG, etc.) or based on operator configuration for specific types of applications (e.g., XR, CG, etc.). PCC rules include information that enables PDU set filtering based on application in-band signaling of PDU set information via RTP header extension. Filters are associated with service data flows (identified by 5-tuples) of the AS at the UPF.

[0106] The PCC rules with PDU set aware policy are sent to the SMF 1006 over the N7 interface, which in turn establishes a QoS flow and provides an additional N4 rule to the UPF 1008, indicating that the UPF 1008 enables PDU set filtering and QoS mapping to a specific QoS flow in view of the information obtained from the RTP header extension as signaled by the AS 1010 through the N6 user plane interface. Given the PDU set information and the QoS rules with PDU set awareness, the UPF 1008 extracts the PDU set awareness and encapsulates the latter along with the PDU set payload in the GTP-U header of the selected QoS flow. The user plane data is transmitted to the RAN 1012 over the N3 reference point. The RAN 1012 applies the QoS profile information signaled by the AMF 1014 via the SM container, which includes the new PDU set awareness obtained through the GTP-U header. The RAN 1012 handles the mapping of QoS flows to specific DRBs based on QoS rules and available PDU set information to meet the QoS requirements of the application, including 5QI, PSDB, and PSER constraints.

[0107] In the UL, the UE 1016 negotiates the PDU set and PDU set information determination and associated signaling via RTP header extension signaling within the session management (i.e., establishment, update operations) process. This may include direct signaling of PDU set filtering policies and QoS rules associated with PDU sets from the SMF 1006 via the AMF 1014. The PDU set aware UE 1016 applies QoS rules and PDU set policies to process incoming media (including encoding) and determines PDU sets and PDU set information at runtime. After packaging the PDU sets and PDU set information into RTP PDUs and RTP header extensions, the UE 1016 will send the data to the RAN 1012 in the UL over the Uu interface. In doing so, the UE 1016 will further signal its PDU set awareness by selecting DRBs in view of the SMF QoS rules, PDU set policies and the PDU set awareness it obtains. The RAN 1012 will then remap the selected DRB to the QoS flow based on the SDAP processing of the UL data from the UE 1016 and based on the QoS rules and PDU set policy received from the SMF 1006 via the AMF 1014 over the N2 reference point. Therefore, the RAN 1012 will send the uplink data to the UPF 1008 over the N3 interface through the optimized QoS flow, and the UPF 1008 will then communicate the data with the remote data network over the N6 interface.

[0108] In summary, the proposed approach provides application-centric PDU sets and PDU set information determination with unified processing and logic on both server and client sides; an interoperable in-band signaling process between the AS and 5GS UPF entities through N6 over well-established RTP and WebRTC transport protocols; and reduced processing complexity achieved by the UPF / 5GS for DL / UL PDU set-aware QoS integrated processing, wherein, in the DL, the UPF needs to identify the PDU sets based on the in-band indication on the RTP header extension from the AS by applying a PDU set-aware packet filter associated with a 5-tuple of the application's service data flow; and wherein, in the UL, given that the PDU set-aware QoS rules are negotiated and established by the AF on the control plane, a traditional SDAP-based DRB selection mechanism can be used by the PDU set-aware UE.

[0109] In a first embodiment involving determining a PDU set and PDU set information, the determination of the PDU set and the corresponding PDU set information is performed by an application at runtime. In an embodiment, an application runtime that processes an input media stream (e.g., a captured video stream, a rendered video stream, a binocular stream buffer, etc.) first encodes the raw available information. This information may originate from an embedded capture device (i.e., one or more RGB / RGBD cameras), from a remote capture device (i.e., a remote camera monitoring system), or from a rendering engine (e.g., Unity wait).

[0110] The encoding process involves compressing the original picture data format (e.g. R8G8B8A8 or alternatively R10G10B10) of the video stream into a bitstream by applying a video codec (e.g. H.264, H.265, H.266, VP8, VP9, ​​AV1, etc.). The resulting bitstream contains video coded frames, video coded frame partitions (i.e. video coded slices, video coded tiles) or alternatively video coded layers (i.e. video coded temporal and / or spatial layers), which are packaged into information units ready for transmission over a network or storage medium device. These packets contain video coded information units or alternatively application data units (ADUs), which are constructed according to the syntax and semantic rules of the codec specification, i.e. NALU in H.264, H.265, H.266, open bitstream unit (OBU) in AV1, etc. Therefore, ADU represents an atomic unit of video coded information generated by the encoder and output (i.e. copied / written) to the output buffer at one time. Thus, in some embodiments, an ADU is parsed independently by a decoder, but it may need other ADUs to decode the encoded information (e.g., a P frame / P slice may need another I / P / B frame or I / P / B segment to reference). In some embodiments, depending on the encoding configuration, the encoder may output ADUs representing encoded video frames, encoded video frame partitions (e.g., encoded video slices, encoded video tiles), encoded video frames / slices / tiles within an encoded video enhancement layer (e.g., as part of a temporal or spatial enhancement layer).

[0111] In embodiments involving real-time media communication over a network, each ADU (e.g., video frame, video slice, video tile, video slice corresponding to a temporal layer, video slice corresponding to a spatial layer, etc.) in the encoded bitstream resulting from the encoding process is buffered and formatted as an RTP / SRTP payload at runtime. The payload format is determined by an IETF RTP / SRTP AV profile such as RFC 6184 for H.264, RFC 7798 for H.265, and draft-ietf-avcore-rtp-vvc-18 for H.266 that matches the encoder used to encode the media.

[0112] In an embodiment, the encoded ADU is packaged within a single RTP PDU payload according to the maximum transmission unit (MTU) along the network path (e.g., typically 1500 bytes for conventional Ethernet-based packet switches). In another embodiment, the encoded ADU may be aggregated with one or more other ADUs within a single RTP PDU payload. For example, this applies to H.264, H.265, H.266 RTP payload formats for non-video coding layer NALUs (e.g., video parameter sets, sequence parameter sets, picture parameter sets). In many other embodiments, given that the ADU compressed size may exceed the MTU size minus the overhead of the UDP / IP header (i.e., 1460 bytes), the ADU may be segmented over multiple RTP PDUs.

[0113] Fig.11 Part a of shows an overview of the RTP media packaging process applicable to video media streams at runtime. The media source 1102 block outputs the video input original source stream according to a certain original picture format (e.g., R8G8B8A8, R10G10B10, YUV420, etc.). The original source stream is directly or after a certain conversion (e.g., RGB format to YUV format) is input to the media encoder 1104 block, where modern hybrid video codec encoding (e.g., H.264, H.265, H.266, VP8, VP9, ​​AV1, etc.) is applied to compress the original source stream into a coded bit stream containing multiple ADUs. This process is usually periodic, and its frequency is determined by the frames per second acquisition and encoding configuration of the real-time processing chain. The encoded ADUs are each output by the encoder to the input buffer of the media packager 1106. The media packager 1106 is configured by the application to use the media codec payload format according to the encoder to package each ADU into an RTP PDU. The media packager 1106 is also aware of and further configured with the MTU size of the transmission path, and partially applies these configurations to determine the final packaging of the encoded ADU into RTP PDUs. In one embodiment, the latter MTU size perception is provided by hard-coded fixed values ​​corresponding to typical Internet MTUs that can be transmitted over both IPv4 and IPv6 networks (e.g., as in the case of the Chromium WebRTC RTP packager, which is limited to a maximum RTP PDU of 1280 bytes). In another embodiment, the MTU size perception corresponds to the MTU size determined based on, for example, probing the transmission path through the MTU path discovery procedure according to RFC 1191, RFC 8201. The packager performs ADU or alternatively PDU payload, segmentation and packaging into one or more RTP PDUs, which are buffered in the RTP media stream buffer and released to the media transmission 1108 block, which implements a network library specific to network transmission, such as UDP / IP.

[0114] Depend on Fig.11 The process shown in part a is in some examples performed by a mobile, PC or embedded platform (e.g. Android Windows The native RTP library included with the network stack of Unix / Linux, etc., the WebRTC runtime within third-party browser implementations (such as Chrome, Edge, Mozilla, etc.), or the WebRTC runtime implemented by mobile, PC or embedded platforms (such as Android Windows Chrome etc.) to implement it.

[0115] Fig.11 Part b of on the other hand shows the same RTP media packetization applicable to video media streams at runtime, with additional PDU set awareness. In some embodiments, PDU set awareness means determining PDU sets, i.e., grouping PDUs into PDU sets, and signaling PDU set information, such as a PDU set identifier (e.g., a PDU set sequence number), a PDU identifier within a PDU set (e.g., a sequence of PDUs within a PDU set), a PDU set size (e.g., in bytes or in number of PDUs), a PDU set start, a PDU set end, and a PDU set importance.

[0116] like Fig.11As shown in part b of , in some embodiments, the encoded ADU is mapped to a PDU set, where the RTP PDU representing the ADU is grouped into a PDU set 1110. The grouping is the result of the RTP packetizer, provided by the upstream processing of the ADU (actually the RTP payload) by the media encoder. The RTP payload is available in the RTP packetizer input buffer, where the continuous operations of PDU set determination, packet segmentation and RTP packaging are subsequently performed. The segmentation and packaging process of the RTP packetizer takes into account the MTU size configuration of the packetizer to ensure that the resulting RTP PDU does not exceed the MTU threshold, while taking into account further UDP / IP protocol overhead. In an embodiment, the PDU set information 1112 is obtained after this grouping, and the PDU set-aware packetizer 1114 can access the entire encoded ADU size (in bytes or in bits), namely the beginning of the ADU, the PDU set, the end of the ADU, and the PDU set. This information can be obtained by directly mapping the ADU to the PDU set concept and grouping the resulting PDUs into PDU sets. In another embodiment, the PDU set aware packetizer 1114 for a specific payload format (e.g., H.264, H.265, H.266, AV1, VP8, VP9) may also have access to the ADU payload and ADU header information, which in some embodiments contains further information about the ADU type, its presence in the temporal or spatial layer of the video coding (in case of layered coding), its importance, and some references to other ADUs.

[0117] In one embodiment, the PDU set aware packetizer 1114 may output a set of ADUs grouped within one RTP PDU, given that the ADUs are released together to the RTP protocol stack and the RTP media packetizer 1106 for processing by the media encoder. In an example involving an H.26x MPEG codec, this may apply to one or more parameter sets and metadata elements (e.g., video parameter sets, sequence parameter sets, picture parameter sets, MPEG SEI metadata) that are packaged together within one RTP PDU. In this case, for one embodiment, the PDU set concept is not applied to aggregated ADUs within one RTP PDU, and by default or alternatively, conventional QoS processing is applied to the RTP PDU containing the aggregated ADUs.

[0118] In an embodiment, the running PDU set aware media application acts as an RTP sender. The RTP sender sends the RTP data according to the above details and Fig.11Determine the RTP payload size, i.e., the size of one or more ADUs generated by the media encoder. The RTP payload available in the RTP sender input buffer is also passed to the RTP protocol stack implemented in the user space of the operating system. The application also enables the PDU set feature based on the application or network configuration, and notifies the RTP stack of additional PDU set information (e.g., PDU set sequence number, PDU sequence within the PDU set, PDU set size, PDU set end marker, PDU set importance, etc.) of its request RTP payload. In some sender configurations, one RTP payload (i.e., ADU) is therefore mapped 1:1 to a PDU set, while in other configurations, one or more RTP payloads are mapped to a PDU set. When the application also enables the determination of the PDU set size as part of the PDU set information, the RTP stack determines the PDU set size based at least in part on the available RTP payload size, the MTU size, and at least one of the UDP transport socket configurations (e.g., based on the service configuration or programmatically available through an operating system or similar application-level API (such as, for example, getsockopt)).

[0119] Therefore, for the identified RTP payload mapped to the PDU set, the RTP stack uses the MTU knowledge for RTP packet segmentation based on the link MTU size. The MTU size can be statically configured, for example, 1280 bytes for the RTP SDU in WebRTC, or dynamically determined based on ICMP and the MTU path discovery process. Once the RTP segmentation is determined, the RTP stack can perform packaging and also release the RTP packets corresponding to the RTP payload (i.e., the PDU set) to the corresponding UDP socket agreed upon during the session establishment as via the SDP offer / answer process. In this process, the application also provides RTP stack control to the UDP socket and UDP socket configuration. Therefore, the RTP stack can further determine the necessary socket options (i.e., the IP version of the transport layer) to determine the protocol header overhead on the PDU set size. Therefore, the RTP stack with PDU set marking capability can understand the UDP / IP header overhead of each packet and take it into account when performing RTP segmentation and packaging to determine the PDU set size. Therefore, the RTP stack uses this information together with the RTP payload size during RTP packet segmentation and RTP packaging to determine the PDU set size field (in bytes), which corresponds to the size of the RTP payload and the overhead of the UDP / IP header of the corresponding RTP packet that forms the PDU set. In one example, if the RTP packets of the application RTP sender are transmitted using UDP / IPv4, the RTP stack can determine that the overhead of each RTP packet is 20 bytes (IPv4) + 8 bytes (UDP) = 28 bytes. In another example, if the RTP packets are transmitted using UDP / IPv6, the RTP stack can determine that the overhead of each RTP packet is 40 bytes (IPv6) + 8 bytes (UDP) = 48 bytes. Therefore, the RTP stack can calculate the PDU set size (in bytes), which corresponds to the PDU set size (including the RTP payload size and the additional UDP / IP protocol header overhead). This information, in turn, can be signaled further downstream to other media-aware network nodes, or sent to one or more receivers in-band per PDU set as part of a PDU set RTP header extension, as described in detail in the following.

[0120] In the example of H.264, the first byte of the ADU or NALU consists of the following fields: a field "F" marked with a 1-bit "forbidden_ero_bit" indicating the absence or presence of a bit error or syntax violation; a field "NRI" of a 2-bit "nal_ref_idc" indicating the importance of the NALU relative to the reference picture, retaining the semantics of the value 00 and non-zero in the H.264 specification (i.e., a value of 00 indicates that the content of the NALU is not a reference for inter-frame prediction of any picture and can be deleted without losing quality and reducing the reference, while a value greater than 00 indicates that the NALU needs to be decoded to maintain the integrity of the inter-frame prediction reference; and a 5-bit field "type" indicating the exact type of the NALU according to the H.264 specification, such as a coded slice of a non-IDR picture, a coded slice of an IDR picture, a sequence / video / picture parameter set, etc.

[0121] In another example of H.265, the first two bytes of the ADU or NALU consist of the following fields: a field "F" marked with a 1-bit "forbidden_ero_bit", which indicates the absence or presence of a bit error or syntax violation; a 6-bit field "type", which indicates the type of the NALU, such as TRAIL_R, RADL_R, CRA_NUT, sequence / video / parameter set, etc., and therefore, in some examples, the importance of the NALU can be determined based on the type of the NALU; a 6-bit field "layer_id", which indicates the identifier to which the NALU belongs (e.g., base layer, enhancement layer, etc.); and a 3-bit field "temporal_id", which indicates the temporal identification layer to which the NALU belongs minus 1.

[0122] In some embodiments, the extracted PDU set information is signaled together with the RTP PDU that encapsulates the PDUs in the PDU set. In such examples, the PDU set information is encapsulated in a fixed-size RTP extension header element that is injected into the final RTP PDU stream by the media packager, such as by Figure 8 In these examples, the RTP extension header is enabled by means of the SDP offer / answer procedure and the configuration is available to the media packager. In addition, when mapping the ADU to the PDUs in the PDU Set, the packager considers the length of the RTP extension header element and the length of the RTP extension headers of other SDP-enabled ones in view of the MTU size constraints.

[0123] In an embodiment, Fig.11The PDU set determination process described and illustrated in and the corresponding associated PDU set information determine the application encoder configuration to group the ADU into PDU sets. In one example, a video encoder configured to encode 4 slices per video frame, the media packager will generate 4 PDU sets for the video frame slice ADU, each PDU set corresponding to a slice of the video frame. In another embodiment, the encoder generates encoding information metadata (e.g., MPEG supplemental enhancement information (SEI), intra-codec messaging, and associated implementation-specific interfaces) that can be used to identify regions of interest (e.g., included within tiles / slices), video temporal layers, or spatial layers to group and restrict PDU sets. In some embodiments, the application can have greater control over the PDU set grouping through application programming interfaces (APIs) exposed within the RTP / WebRTC stack and library implementations, so that the application can provide configurations and policies regarding the level of granularity (i.e., per frame, per slice / tile) of performing PDU set grouping on the encoded video stream at runtime.

[0124] In some embodiments, where FEC is applied to RTP transmission of media, the application runtime also determines the PDU set and PDU set information based on the FEC configuration. In one embodiment, where a maximum distance separable FEC (e.g., Reed-Solomon) or an approximation thereof (e.g., Raptor / RaptorQ) is being used, the minimum number of PDUs required to recover the PDU set, or ADU information, can be determined based on the FEC configuration (i.e., coding block size, FEC code rate / redundancy, number of source blocks, or source block size). In such examples, the configuration of the RTP FEC is dynamically determined by negotiation between the source and the receiver through the SDP offer / answer process. In such examples, the minimum necessary number of PDUs required to recover the PDU set indicates the recovery information of the PDU set for network and transmission errors that cause PDU deletion. In some embodiments, after enabling RTP FEC on the SDP offer / answer process when establishing an RTP session for the media stream, this information is embedded in the PDU set information via an RTP extension header.

[0125] In an embodiment, a media packager with PDU set awareness and caching capabilities can indicate references of a PDU set to other previous or other layer PDU sets. The media packager therefore records and tracks inter-PDU set references and decoding dependencies. This is possible given that the media packager has access to the encoded ADU and its attached reference information related to the inter-prediction encoding and decoding process. In one example, the media packager can encapsulate this information as a list of references to other previous PDU sets in the RTP extension header PDU set information. This allows a PDU set aware RAN / UPF implementation to delete PDU sets that could not be decoded given that some of their PDU set references had previously failed to be transmitted to the receiver endpoint.

[0126] In a reduced complexity embodiment, the discard information may be limited to a discard bit indicating a PDU within a PDU set, indicating that if the transmission of one or more PDUs within the PDU set fails to meet the PDB / PSDB, the remaining PDUs in the PDU set may be deleted at the UE / RAN or UPF level. This is beneficial, for example, in the case of non-reference NALUs, such as in H.264 encoded PDU sets where the NALU NRI field is "00". Another example where it can be applied is an IDR frame / segment, where errors within an intra-predicted frame / segment with no other references make it impossible for the decoder to decode the information. In such an example, the decoder will apply error concealment and the receiver endpoint can trigger a full intra-frame request RTCP feedback (as defined in RFC 5104), requesting a decoder refresh point, such as, for example, a new IDR frame / segment.

[0127] In an embodiment, the PDU set information may therefore be formed by a combination of a PDU set identifier field, a PDU sequence number field, a PDU set start flag field, a PDU set end flag field, a PDU set size field, a PDU set importance flag field, a PDU set discard flag bit field, a PDU set error recovery information field, and a PDU set reference field containing a list of references to one or more PDU set sequence numbers / identifiers.

[0128] In one example, a PDU set sequence number generated sequentially between PDU sets is used to identify the PDU set, while in other examples, the PDU set is identified based on a random identifier (e.g., a hash code, a random ID, etc.). In one example, a PDU sequence number is generated to identify and order the PDUs grouped within a PDU set, while the PDU set start and end markers are intended to mark the first and last PDUs in the PDU set. In some examples, the PDU set size can be expressed in absolute units (such as bits, bytes, words, etc.), or can be indicated relative to the PDU concept within the PDU set as the number of PDUs that form the PDU set. The importance tag of the PDU set indicates the importance value of the PDU set in a predetermined set of importance values ​​(e.g., HIGH, MEDIUM, LOW, DEFAULT). These importance values ​​indicate in some examples the preferred QoS treatment at each PDU set level. The number of bits encoding the importance tag information is determined by the cardinality I of the set, i.e., log2(|I|). The discard flag in some examples indicates whether a complete PDU set or a subset of a PDU set can be safely discarded without disrupting the application logic or degrading the user quality of experience ("QoE") below a predetermined minimum threshold. In some examples, the PDU set error recovery information marks the fact that the PDU set is forward error corrected. In such examples, the FEC ratio and configuration may determine the minimum number of PDUs of the PDU set that need to be sent in order to correctly recover the ADU information within the PDU set. In some examples, the PDU set reference field contains a list of one or more PDU set sequence numbers that are referenced by the PDU set information and without which the PDU set information cannot be perfectly recovered. Given that some of the information referenced by the PDU / PDU set has previously been lost during transmission, this indication is intended to inform the CN or RAN of the possibility of deleting certain PDU sets that the application cannot recover. In some examples, the PDU set importance and discard corresponding to a bit field may be the same, with a "HIGH" importance value mapping to "do not discard" and a "LOW" importance value mapping to "free discard".

[0129] In a second embodiment involving signaling of PDU sets and PDU set information, Fig.12 An example embodiment of the PDU Set Information data format is presented in , where an extended combination of PDU Set Information fields is shown to provide a complete example. To this end, the following fields that may be redundant from a semantic point of view are highlighted together with some reference implementations:

[0130] A "PDU Set Identifier" field 1202 contains a PDU set sequence number intended to identify a PDU set. In one example, the PDU set sequence number is associated with an RTP media stream, and the first value in the PDU set sequence number is "1", with "0" being reserved. The value is incremented by "1" for each subsequent PDU set generated by the media encoder associated with the RTP media stream, regardless of the temporal or spatial layer of the encoded ADU. In one example, this field can be encoded within 16 bits. The PDU set sequence should wrap around 65536 so that the valid value area always increases from 1 to 65535 and then returns to 1.

[0131] "PDU Set Size" 1204, which indicates the size of the PDU set, expressed as the number of PDUs grouped within the PDU set, for example spanning an 8-bit encoding. In another example, this field may indicate the PDU set size in bytes, spanning a width of 32 bits in one example. The latter example may include the PDU set size (in bytes) in some implementations, including the RTP / UDP / IP encapsulation, i.e., taking into account the protocol header overhead, or, in other implementations, the PDU set size (in bytes) as the ADU size, or similarly, corresponding to the RTP payload of the PDU set.

[0132] "PDU Sequence Number" 1206, which identifies the PDUs that form a PDU set and assists in delivering them in sequence. In one example, this field can span a width of 8 bits, limiting the maximum number of PDUs contained in a PDU set to 256, which is consistent with the "PDU Set Size" value, which encodes the PDU set size as the number of PDUs contained.

[0133] The "PDU Set Start" bit field 1208, which in one example indicates that the PDU set start (ie, the first PDU in a PDU set) has a value of "1", while any other PDU in the PDU set has a value of "0".

[0134] The "PDU set end" bit field 1210, which in one example indicates the end of a PDU set (ie, the last PDU in a PDU set) has a value of "1", while any other PDU in the PDU set has a value of "0".

[0135] A "PDU set importance" bit field 1212 indicates a binary importance value of the PDU set, for example, a value of "1" indicates a HIGH importance PDU set (i.e., corresponding to intra-decodable frames / slices, intra-frames, key frames of VP8 / 9, IDR of H.264, IDR / CRA / BLA / RAP of H.265, etc.), while a value of "0" indicates a LOW / DEFAULT importance PDU set (i.e., corresponding to inter-decodable frames / slices, inter-frames, non-key frames of V8 / 9, non-IDR of H.264, etc.).

[0136] A “PDU set discarded” bit field 1214 indicates a binary flag whose value is “1” in the example where a PDU set can be discarded by the UE / RAN and / or CN without disrupting the decoding capability of future PDU sets (for example, if there is no inter-frame prediction reference for the PDU set (i.e., frame / slice / tile)), and whose value is “0” otherwise.

[0137] A "PDU Set Reference List Flag" bit field 1216 indicates a binary flag, the value of which is "1" in the example where at least one reference dependency is signaled within the PDU Set Information, and the value of which is "0" if no reference dependency is signaled within the PDU Set Information.

[0138] The "PDU Set Temporal Layer ID" field 1218 identifies the temporal enhancement layer of the video coding to which the PDU set (ie, frame / slice / tile) belongs, whereby the base coding layer is coded as temporal layer ID "0." In one example, the width of this field is 3 bits.

[0139] The "PDU Set Spatial Layer ID" field 1220 identifies the spatial enhancement layer of the video coding to which the PDU set (ie, frame / slice / tile) belongs, where the base coding layer is coded as the spatial layer with ID "0". In one example, the width of this field is 8 bits.

[0140] The "PDU Set Reference List of PDU Sets" field 1222, which is included in the example of up to 5 PDU Set Identifier fields, marks the reference dependency of the current PDU Set with the previous layer / other layer PDU. In the same example, the reference list of PDU Sets should be provided in coding order, i.e., the wraparound of the PDU Set Identifier is reserved, and if less than 5 PDU Sets are referenced, a reserved PDU Set Identifier value of "0" should be used. In this example, a total of 80 bits can be used to reference PDU Set dependencies.

[0141] exist Fig.12In the example, a 16-byte fixed RTP extension header payload is therefore provided to encode the PDU set information. In an embodiment, the PDU set information is formed by common information shared by all PDUs in the PDU set (e.g., PDU set identifier 1202, PDU set size 1204, PDU set reference list flag 1216, PDU set time / space layer ID 1218, 1220, etc.) and PDU specific information (e.g., PDU sequence number 1206, PDU set start flag 1208, PDU set stop flag 1210), and therefore, in some embodiments, the PDU set information includes: identification of the PDU set and the PDUs in the PDU set, and a set of attributes of the PDU set. Therefore, the PDU set identification data and the set of attributes of the PDU set are the PDU common information of the PDU set, while the identification data of the PDUs in the PDU set are the PDU-specific PDU set information.

[0142] also, Fig.12 It is shown that each RTP PDU in a PDU set contains an RTP header extension including an RTP header extension element composed of corresponding PDU set information data according to a fixed PDU set information data format (ie, both PDU-common PDU and PDU-specific PDU set information of the RTP PDU).

[0143] In some embodiments, only Fig.12 . For example, the combination of the PDU set size (expressed in number of PDUs), the start of the PDU set, and the end of the PDU set is sufficient to define the PDU set boundaries and identify the individual PDUs within the PDU set. In another example, only the PDU set size (expressed in number of PDUs) and the PDU sequence number, which starts at "0" and increases by "1" for each PDU grouped within the PDU set, are sufficient to identify the PDU set boundaries and the PDUs it contains. Both of these examples will in turn rely on a PDU set identifier, such as a PDU set sequence number, as an identification component for the PDU set itself.

[0144] In other embodiments, the PDU set information can be an extension of the draft-ietf-avtext-framemarking-13 RFC, where an additional subset of PDU set specific information, such as a PDU set identifier, PDU set size, and PDU sequence number, is added to the front of the short or long draft-ietf-avtext-framemarking-13 RTP header extension format, see "RTP Frame Marking".

[0145] In many embodiments, described above and Fig.12The basic PDU set information field generally shown in FIG. 1 is byte aligned. Therefore, in some embodiments, padding 1302 may be appended at the end of the payload to ensure 32-bit alignment corresponding to the RTP header extension, see "RFC 8285", such as Fig.13 As presented in Fig.13 A complete RTP header extension 1304 representation is provided in, which includes an identifier and length determined by the RTP header extension profile, see "RFC 8285" as a reference for some embodiments.

[0146] Although in one embodiment, Fig.13 The RTP header extension format of may be sufficient, but in other embodiments, it may be necessary to send other RTP header extensions simultaneously with the PDU set information. In these latter embodiments, the general RTP header extension format (see "RFC 8285") is used to allow additional extension elements to be signaled together with the PDU set information data as RTP header extension elements within the RTP header extension.

[0147] In one embodiment, the length of the PDU set information is limited to a maximum of 16 bytes. In such an embodiment, the PDU set information can be multiplexed with other RTP extended header elements using the single-byte extension format of the general RTP header extension mechanism "RFC 8285". In one example, such multiplexed RTP extended header elements can be composed of a 3GPP extension header for coordinating video orientation (i.e., "urn:3gpp:video-orientation"), a transmission time offset (i.e., RFC 5450), "urn:ietf:params:rtp-hdrext:toffset"), a client-to-mixer audio level indication (e.g., RFC 6464 or RFC6465), etc. Thus, this mechanism allows other header extension elements that are commonly available in WebRTC browsers / natives (e.g., Chrome, Edge, MozillaFirefox) and RTP native implementations to be multiplexed with the PDU set information header extension element on the same RTP / SRTP PDU of the PDU set. For example, in Fig.14 An implementation is shown that implements the single-byte extended format of "RFC 8285" and an embodiment of the previously described generic PDU set information payload as one of the RTP header extension elements.

[0148] exist Fig.14 In the Fig.12The 16-byte PDU set information 1402 payload of the previous example shown is transmitted via a single-byte generic RTP header extension mechanism (identified by profile ID 0xBEDE). The entire RTP extension header element is 12 words. The PDU set information data is the first RTP header extension element, preceding the other two extension elements. This reduces the complexity of media-aware network devices or similarly PDU set-aware UPFs to process PDU set information without buffering and additional processing of unnecessary RTP header extension elements. The RTP header extension is also padded with 3 bytes to ensure that the overall RTP header extension is 32-bit aligned. Therefore, in some embodiments, the PDU set information can be placed first to facilitate UPF filtering and processing of PDU set information without the need to increase buffering of additional unnecessary RTP header extension elements. In other embodiments, the placement of the PDU set information depends on the packetizer implementation in view of the extended mapping negotiated via SDP and the active and enabled RTP extensions.

[0149] In other embodiments, where multiplexing with longer RTP extension header elements is performed, the PDU set information can still maintain a fixed size, such as a fixed size of 16 bytes, thereby allowing it to be encapsulated into a single-byte RTP header extension element or a double-byte RTP header extension element. However, in such an embodiment, a subset of other multiplexed RTP extension header elements exceeding the 16-byte payload limit of the single-byte header format is encapsulated in a two-byte header format according to the general RTP header extension mechanism of "RFC 8285". This allows multiplexing with up to 256 bytes of RTP extension header elements, thereby meeting most of the requirements of the RTP extension headers available in the latest IETF specifications, WebRTC native / browser stacks, and RTP native library implementations. This mechanism of mixing single-byte and two-byte general RTP extension header formats across RTP sessions or media streams is possible based on "RFC8285", and is negotiated during the SDP provision / answer process via the attribute "a=extmap-allow-mixed" used at the session or media level of the RTP session. When both the SDP offer and answer from the client provider and answerer respectively contain the attribute extmap-allow-mixed, mixing of format types will be activated.

[0150] In another embodiment, the PDU set information payload can therefore be flexibly encapsulated in a single-byte general RTP extension header payload format or a two-byte general RTP extension header payload format. In one example, this is performed when the answerer does not support a mix of formats and some RTP extension header elements exceed the 16-byte size limit of the single-byte format. In such an example, all RTP extension header elements are encapsulated in a two-byte RTP extension header format and multiplexed as an RTP extension header.

[0151] exist Fig.15 An example of multiplexing involving a two-byte PDU Set Information RTP extension header element and a second two-byte RTP extension header element is provided in. Fig.14 The same generic PDU set information 1402 of is multiplexed with a second generic RTP extension 1502 encapsulated within a two-byte generic RTP extension format. The second extension includes padding with null bytes to ensure that the resulting multiplexed RTP header extension achieves 32-bit alignment.

[0152] In some embodiments, the signaling of the local ID and extension mapping corresponding to the RTP header extension element of the PDU set information data follows SDP signaling according to "RFC 8285". In such embodiments, during the SDP offer / answer process, the client sender and the provider negotiate and determine the RTP extension mapping, local extension element ID, uniform resource identifier (URI), direction (e.g., "sendrecv", "sendonly", "recvonly", "inactive"), and namespace (i.e., RTP session scope or RTP media scope) of the PDU set information extension header element transmitted by the RTP PDU through the session / media stream. Given that the PDU set information is source generated and intended to be consumed within the 5GS to optimize QoS processing, in one embodiment, the PDU set information RTP header extension mapping can be modified with the attribute "sendonly". This indicates that the PDU set information is only populated in one direction from the source to the receiver of the RTP session or media stream.

[0153] An example of an SDP answer snapshot is listed below for the case of a single-byte generic RTP header extension mechanism that maps local ID 42 to a URI corresponding to a 3GPP PDU set information payload (e.g., "urn:3gpp:pdu-set-info" at the media level for an H.264 media stream). The SDP response (e.g., UPF, UE) indicates that for a video media stream, the answerer accepts receiving PDU set information via the attribute "recvonly". This will map to the provider marking the PDU set information RTP extension header element as "sendonly" for the video media stream in accordance with RFC 8285. In addition, the URI is obtained from the central registry that hosts the OpenXR data model and syntax associated with and mapped to local ID 1. The SDP answer example also contains 2 other extension elements, namely the 3GPP Coordinated Video Orientation (CVO) with a precision of 6 bits and a local ID of 3, and the Transmission Time Offset extension header element with a local ID of 2, see RFC5450.

[0154] Examples of SDP answers include:

[0155] …

[0156] m=video 50000RTP / AVP 96

[0157] a=rtpmap:96H264 / 90000

[0158] a=fmtp:96media=video; clock-rate=90000; encoding-name=H264

[0159] a=extmap:1 / recvonly urn:3gpp:pdu-set-info

[0160] a=extmap:2urn:ietf:params:rtp-hdrext:toffset

[0161] a=extmap:3urn:3gpp:video-orientation:6

[0162] …

[0163] In some embodiments, the RTP header extension element of the PDU set information generated at the source may be ignored by the destination endpoint (e.g., a traditional UE) or network waypoint (e.g., a traditional UPF implementation, a TURN server, etc.) that does not understand the concept of PDU sets and is unable to interpret and process the PDU set information.

[0164] In some examples, known set values ​​for reserved fields may include all "0" or "1" fields, or similarly, one or more "0x00" or "0x11" byte fields, etc.

[0165] The PDU set sequence number is used to identify the PDU set, and in some examples, may be generated sequentially between PDU sets, while in other examples, it may be generated based on a random identifier process (e.g., hashing, randomization, etc.). The PDU sequence number is generated to identify and order the PDUs grouped within the PDU set, while the PDU set start and end markers are intended to indicate the first and last PDUs in the PDU set. The PDU set size may be expressed in an absolute unit of measure of digital information (such as bits, bytes, words, etc.), or may be indicated relative to a conceptual indication of the PDUs within the PDU set as the number of PDUs forming the PDU set. The importance flag of the PDU set indicates the importance value of the PDU set in a predetermined set of importance values ​​(e.g., HIGH, MEDIUM, LOW, DEFAULT) to indicate the preferred QoS treatment at the level of each PDU set. The number of bits encoding the importance flag information is determined by the set cardinality I of the importance value, i.e., log2(|I|). The discard flag indicates whether a complete PDU set or a subset of a PDU set can be safely discarded without disrupting the application logic or reducing the user quality of experience (QoE) below a predetermined minimum threshold. In some examples, the PDU set error recovery information may mark the fact that the PDU set is forward error corrected (FEC), and the FEC ratio indicates the minimum number of PDUs of the PDU set that need to be sent in order to correctly recover the ADU information within the PDU set. In some examples, the PDU set reference field contains a list of one or more PDU set sequence numbers that are referenced by the PDU set information and without which the PDU set information cannot be successfully recovered. Given that some of the information referenced by the PDU set has previously been lost during transmission, this indication is intended to notify the network, or alternatively the radio access network, of the possibility of deleting certain PDU sets that the application cannot recover. In some examples, the PDU set importance and discard corresponding to a bit field may be the same, with a "HIGH" importance value mapping to "do not discard" and a "LOW" importance value mapping to "free discard".

[0166] In some example implementations, the determination of PDU set grouping and the generation of PDU set information for populating the PDU set extended header element can be based on the video encoded network abstraction layer unit (NALU) and its header information, which indicates the type of video encoded frame / slice / layer information included by the NALU payload. In another example, the encoder configuration can be used to group the ADU into PDU sets, so that for a video encoder configured to encode 4 slices per video frame, 4 PDU sets will be generated for the video frame ADU, each PDU set corresponding to a slice of the video frame. In some examples, the encoder can generate encoding information metadata as MPEG supplemental enhancement information (SEI), which can be used to identify the video time or space layer to group the PDU sets. In some implementations, the application can have greater control over the PDU set grouping through the application programming interface (API) disclosed within the RTP / WebRTC library and implementation, so that the application can provide configuration and instructions on how the PDU set grouping should be performed for the video source at runtime.

[0167] Fig.16 An example of a UE 1600 according to various aspects of the present disclosure is shown. The UE 1600 may include a processor 1602, a memory 1604, a controller 1606, and a transceiver 1608. The processor 1602, the memory 1604, the controller 1606, or the transceiver 1608, or various combinations thereof or various components thereof may be examples of means for performing various aspects of the present disclosure as described herein. These components may be coupled (e.g., operatively, communicatively, functionally, electronically, electrically) via one or more interfaces.

[0168] The processor 1602, memory 1604, controller 1606, or transceiver 1608, or various combinations or components thereof may be implemented in hardware (e.g., circuits). The hardware may include a processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or other programmable logic device, or any combination thereof, which is configured to or otherwise supports components for performing the functions described in the present disclosure.

[0169] The processor 1602 may include an intelligent hardware device (e.g., a general-purpose processor, a DSP, a CPU, an ASIC, an FPGA, or any combination thereof). In some implementations, the processor 1602 may be configured to operate the memory 1604. In some other implementations, the memory 1604 may be integrated into the processor 1602. The processor 1602 may be configured to execute computer-readable instructions stored in the memory 1604 so that the UE 1600 performs various functions of the present disclosure.

[0170] The memory 1604 may include volatile or non-volatile memory. The memory 1604 may store computer-readable, computer-executable code including instructions that, when executed by the processor 1602, cause the UE 1600 to perform various functions described herein. The code may be stored in a non-transitory computer-readable medium, such as the memory 1604 or another type of memory. Computer-readable media include both non-transitory computer storage media and communication media, including any media that facilitates the transfer of computer programs from one place to another. Non-transitory storage media may be any available media that can be accessed by a general or special purpose computer.

[0171] In some implementations, the processor 1602 and the memory 1604 coupled to the processor 1602 may be configured to cause the UE 1600 to perform one or more functions described herein (e.g., the processor 1602 executes instructions stored in the memory 1604). For example, the processor 1602 may support wireless communication at the UE 1600 according to the examples disclosed herein. The UE 1600 may be configured to support components for performing the following operations: encoding the media of the application into a plurality of ADUs as logically independent information units, grouping each ADU into one or more PDU sets by an application RTP sender routine, determining PDU set information for each PDU set in the PDU set, encapsulating the PDU set information of each PDU in the PDU set in an extended header element, the extended header element being packaged within an RTP extended header of the RTP PDU, the RTP extended header corresponding to the RTP PDU payload of each PDU in the one or more PDUs in the PDU set, and sending the RTP PDU including the PDU set information.

[0172] In one embodiment, the PDU set information optimizes media transmission in view of QoS requirements of the application. In one embodiment, optimizing media transmission based on the PDU set information in view of QoS requirements includes: mapping PDU sets to data radio bearers, group scheduling of radio resources for air transmission of PDUs in the PDU set, dynamic adjustment of radio modulation and coding scheme strategy for air transmission of PDU sets, dynamic decision to delete one or more PDUs in one or more PDU sets from air transmission queues, determining that the deleted PDUs are unnecessary for the application based on at least one of signaled PDU set discard flags and their dependencies on other PDUs or PDU sets that failed to be successfully transmitted, or a combination thereof.

[0173] In one embodiment, the extended header element encapsulating the PDU set information has a fixed size and includes a plurality of information fields, a plurality of reserved fields, or a combination thereof.

[0174] In one embodiment, the set of PDU set attributes of the PDU set information includes a PDU set sequence number / identifier field, a PDU sequence number field, a PDU set start marker field, a PDU set end marker field, a PDU set size field indicating the PDU set size in bytes, a PDU set size field indicating the PDU set size in the number of included PDUs, a PDU set importance field, a PDU set discard flag bit field, a PDU set coding layer identifier field, a PDU set error recovery information field, a PDU set reference field containing a reference list to one or more PDU set sequence numbers / identifiers, or a combination thereof.

[0175] In one embodiment, the PDU set importance field consists of log2|I| bits of information indicating a PDU set importance value in a set of I importance values ​​comprising at least two elements, where at least one value corresponds to "high" importance and at least one value corresponds to "low" importance.

[0176] In one embodiment, the PDU set importance field linearly encodes l=16 importance values, where a '0' encoded value indicates the highest PDU set importance and a '15' encoded value indicates the lowest PDU set importance.

[0177] In one embodiment, the RTP extended header includes at least a second extended header element, and the second extended header element is located after the extended header element that encapsulates the PDU set information as the first extended header element.

[0178] In one embodiment, the PDU set includes: a coded video frame, a coded video frame partition as a coded video slice, or a coded video tile, a coded video temporal layer, a coded video spatial layer, or a combination thereof.

[0179] In one embodiment, PDU set grouping and PDU set information determination are based on encoding ADU information, encoding configuration, encoding information metadata, application determined configuration, or a combination thereof.

[0180] The controller 1606 may manage input and output signals of the UE 1600. The controller 1606 may also manage peripheral devices that are not integrated into the UE 1600. In some implementations, the controller 1606 may utilize a processor such as Or other operating systems. In some implementations, the controller 1606 can be implemented as part of the processor 1602.

[0181] In some implementations, the UE 1600 may include at least one transceiver 1608. In some other implementations, the UE 1600 may have more than one transceiver 1608. The transceiver 1608 may represent a wireless transceiver. The transceiver 1608 may include one or more receiver chains 1610, one or more transmitter chains 1612, or a combination thereof.

[0182] The receiver chain 1610 may be configured to receive a signal (e.g., control information, data, packet) over a wireless medium. For example, the receiver chain 1610 may include one or more antennas for receiving signals over the air or wireless medium. The receiver chain 1610 may include at least one amplifier (e.g., a low noise amplifier (LNA)) configured to amplify the received signal. The receiver chain 1610 may include at least one demodulator configured to demodulate the received signal and obtain the transmitted data by inverting the modulation technique applied during the transmission of the signal. The receiver chain 1610 may include at least one decoder for decoding and processing the demodulated signal to receive the transmitted data.

[0183] The transmitter chain 1612 may be configured to generate and transmit a signal (e.g., control information, data, packet). The transmitter chain 1612 may include at least one modulator for modulating data onto a carrier signal, preparing the signal for transmission over a wireless medium. The at least one modulator may be configured to support one or more techniques, such as amplitude modulation (AM), frequency modulation (FM), or a digital modulation scheme, such as phase shift keying (PSK) or quadrature amplitude modulation (QAM). The transmitter chain 1612 may also include at least one power amplifier configured to amplify the modulated signal to an appropriate power level suitable for transmission over a wireless medium. The transmitter chain 1612 may also include one or more antennas for transmitting the amplified signal into the air or wireless medium.

[0184] Fig.17 An example of a processor 1700 according to aspects of the present disclosure is shown. The processor 1700 may be an example of a processor configured to perform various operations according to the examples as described herein. The processor 1700 may include a controller 1702 configured to perform various operations according to the examples as described herein. The processor 1700 may optionally include at least one memory 1704, which may be, for example, an L1 / L2 / L3 cache. In addition, or alternatively, the processor 1700 may optionally include one or more arithmetic logic units (ALUs) 1706. One or more of these components may be in electronic communication or otherwise coupled (e.g., operatively, communicatively, functionally, electronically, electrically) via one or more interfaces (e.g., buses).

[0185] Processor 1700 may be a processor chipset and include a protocol stack (e.g., a software stack) executed by the processor chipset to perform various operations (e.g., receive, obtain, retrieve, send, output, forward, store, determine, identify, access, write, read) according to examples as described herein. The processor chipset may include one or more cores, one or more caches (e.g., memory local to or included in the processor chipset (e.g., processor 1700)), or other memory (e.g., random access memory (RAM), read-only memory (ROM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), static RAM (SRAM), ferroelectric RAM (FeRAM), magnetic RAM (MRAM), resistive RAM (RRAM), flash memory, phase change memory (PCM), etc.).

[0186] The controller 1702 may be configured to manage and coordinate various operations (e.g., signaling, receiving, obtaining, retrieving, sending, outputting, forwarding, storing, determining, identifying, accessing, writing, reading) of the processor 1700 so that the processor 1700 supports various operations according to the examples described herein. For example, the controller 1702 may operate as a control unit of the processor 1700, generating control signals that manage the operation of various components of the processor 1700. These control signals include enabling or disabling functional units, selecting data paths, initiating memory accesses, and coordinating the timing of operations.

[0187] The controller 1702 may be configured to extract (e.g., obtain, retrieve, receive) instructions from the memory 1704 and determine the (multiple) subsequent instructions to be executed so that the processor 1700 supports various operations according to the examples described herein. The controller 1702 may be configured to track the memory addresses of the instructions associated with the memory 1704. The controller 1702 may be configured to decode the instructions to determine the operations to be performed and the operands involved. For example, the controller 1702 may be configured to interpret the instructions and determine the control signals to be output to other components of the processor 1700 so that the processor 1700 supports various operations according to the examples described herein. In addition, or alternatively, the controller 1702 may be configured to manage the flow of data within the processor 1700. The controller 1702 may be configured to control the transmission of data between registers, arithmetic logic units (ALUs), and other functional units of the processor 1700.

[0188] The memory 1704 may include one or more caches (e.g., memory local to or included in the processor 1700) or other memory such as RAM, ROM, DRAM, SDRAM, SRAM, MRAM, flash memory, etc. In some implementations, the memory 1704 may reside within or on the processor chipset (e.g., local to the processor 1700). In some other implementations, the memory 1704 may reside external to the processor chipset (e.g., remote from the processor 1700).

[0189] The memory 1704 may store computer-readable, computer-executable code including instructions that, when executed by the processor 1700, cause the processor 1700 to perform various functions described herein. The code may be stored in a non-transitory computer-readable medium, such as a system memory or another type of memory. The controller 1702 and / or the processor 1700 may be configured to execute computer-readable instructions stored in the memory 1704 so that the processor 1700 performs various functions. For example, the processor 1700 and / or the controller 1702 may be coupled to or coupled to the memory 1704, and the processor 1700, the controller 1702, and the memory 1704 may be configured to perform various functions described herein. In some examples, the processor 1700 may include multiple processors, and the memory 1704 may include multiple memories. One or more of the multiple processors may be coupled to one or more of the multiple memories, which may be configured individually or collectively to perform various functions described herein.

[0190] One or more ALUs 1706 may be configured to support various operations according to the examples as described herein. In some implementations, one or more ALUs 1706 may reside in or on a processor chipset (e.g., processor 1700). In some other implementations, one or more ALUs 1706 may reside outside a processor chipset (e.g., processor 1700). One or more ALUs 1706 may perform one or more calculations on data, such as addition, subtraction, multiplication, and division. For example, one or more ALUs 1706 may receive input operands and an opcode that determines the operation to be performed. One or more ALUs 1706 may be configured with various logic and arithmetic circuits, including adders, subtractors, shifters, and logic gates, to process and manipulate data according to the operation. Additionally, or alternatively, the one or more ALUs 1706 may support logical operations such as AND, OR, exclusive OR (XOR), not OR (NOR), and not AND (NAND), enabling the one or more ALUs 1706 to process conditional operations, comparisons, and bitwise operations.

[0191] The processor 1700 may support wireless communication according to the examples disclosed herein. The processor 1700 may be configured or operable to support components for encoding the media of the application into a plurality of ADUs as logically independent information units, grouping each ADU into one or more PDU sets by an application RTP sender routine, determining PDU set information for each PDU set in the PDU set, encapsulating the PDU set information for each PDU in the PDU set in an extended header element, the extended header element being packaged within an RTP extended header of the RTP PDU, the RTP extended header corresponding to the RTP PDU payload of each PDU in the one or more PDUs in the PDU set, and sending the RTP PDU including the PDU set information.

[0192] In one embodiment, the PDU set information optimizes media transmission in view of QoS requirements of the application. In one embodiment, optimizing media transmission based on the PDU set information in view of QoS requirements includes: mapping PDU sets to data radio bearers, group scheduling of radio resources for air transmission of PDUs in the PDU set, dynamic adjustment of radio modulation and coding scheme strategy for air transmission of PDU sets, dynamic decision to delete one or more PDUs in one or more PDU sets from air transmission queues, determining that the deleted PDUs are unnecessary for the application based on at least one of signaled PDU set discard flags and their dependencies on other PDUs or PDU sets that failed to be successfully transmitted, or a combination thereof.

[0193] In one embodiment, the extended header element encapsulating the PDU set information has a fixed size and includes a plurality of information fields, a plurality of reserved fields, or a combination thereof.

[0194] In one embodiment, the set of PDU set attributes of the PDU set information includes a PDU set sequence number / identifier field, a PDU sequence number field, a PDU set start marker field, a PDU set end marker field, a PDU set size field indicating the PDU set size in bytes, a PDU set size field indicating the PDU set size in the number of included PDUs, a PDU set importance field, a PDU set discard flag bit field, a PDU set coding layer identifier field, a PDU set error recovery information field, a PDU set reference field containing a reference list to one or more PDU set sequence numbers / identifiers, or a combination thereof.

[0195] In one embodiment, the PDU set importance field consists of log2|I| bits of information indicating a PDU set importance value in a set of I importance values ​​comprising at least two elements, where at least one value corresponds to "high" importance and at least one value corresponds to "low" importance.

[0196] In one embodiment, the PDU set importance field linearly encodes l=16 importance values, where a '0' encoded value indicates the highest PDU set importance and a '15' encoded value indicates the lowest PDU set importance.

[0197] In one embodiment, the RTP extended header includes at least a second extended header element, and the second extended header element is located after the extended header element that encapsulates the PDU set information as the first extended header element.

[0198] In one embodiment, the PDU set includes: a coded video frame, a coded video frame partition as a coded video slice, or a coded video tile, a coded video temporal layer, a coded video spatial layer, or a combination thereof.

[0199] In one embodiment, PDU set grouping and PDU set information determination are based on encoding ADU information, encoding configuration, encoding information metadata, application determined configuration, or a combination thereof.

[0200] Fig.18 An example of a NE 1800 according to various aspects of the present disclosure is shown. The NE 1800 may include a processor 1802, a memory 1804, a controller 1806, and a transceiver 1808. The processor 1802, the memory 1804, the controller 1806, or the transceiver 1808, or various combinations thereof, or various components thereof, may be examples of components for performing various aspects of the present disclosure as described herein. These components may be coupled via one or more interfaces (e.g., operatively, communicatively, functionally, electronically, electrically).

[0201] The processor 1802, the memory 1804, the controller 1806, or the transceiver 1808, or various combinations or components thereof may be implemented in hardware (e.g., circuits). The hardware may include a processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or other programmable logic device, or any combination thereof, which is configured to or otherwise supports components for performing the functions described in the present disclosure.

[0202] The processor 1802 may include an intelligent hardware device (e.g., a general-purpose processor, a DSP, a CPU, an ASIC, an FPGA, or any combination thereof). In some implementations, the processor 1802 may be configured to operate the memory 1804. In some other implementations, the memory 1804 may be integrated into the processor 1802. The processor 1802 may be configured to execute computer-readable instructions stored in the memory 1804 so that the NE 1800 performs various functions of the present disclosure.

[0203] Memory 1804 may include volatile or non-volatile memory. Memory 1804 may store computer-readable, computer-executable code including instructions that, when executed by processor 1802, cause NE 1800 to perform various functions described herein. Code may be stored in a non-transitory computer-readable medium, such as memory 1804 or another type of memory. Computer-readable media include both non-transitory computer storage media and communication media, including any medium that facilitates transferring a computer program from one place to another. Non-transitory storage media may be any available media that can be accessed by a general or special purpose computer.

[0204] In some implementations, the processor 1802 and the memory 1804 coupled to the processor 1802 may be configured to cause the NE 1800 to perform one or more functions described herein (e.g., the processor 1802 executes instructions stored in the memory 1804). For example, the processor 1802 may support wireless communication at the NE 1800 according to the examples disclosed herein. The NE 1800 may be configured to support components for performing the following operations: encoding the media of the application into a plurality of ADUs as logically independent information units, grouping each ADU into one or more PDU sets by an application RTP sender routine, determining PDU set information for each PDU set in the PDU set, encapsulating the PDU set information of each PDU in the PDU set in an extended header element, the extended header element being packaged within an RTP extended header of the RTP PDU, the RTP extended header corresponding to the RTP PDU payload of each PDU in the one or more PDUs in the PDU set, and sending the RTP PDU including the PDU set information.

[0205] In one embodiment, the PDU set information optimizes media transmission in view of QoS requirements of the application. In one embodiment, optimizing media transmission based on the PDU set information in view of QoS requirements includes: mapping PDU sets to data radio bearers, group scheduling of radio resources for air transmission of PDUs in the PDU set, dynamic adjustment of radio modulation and coding scheme strategy for air transmission of PDU sets, dynamic decision to delete one or more PDUs in one or more PDU sets from air transmission queues, determining that the deleted PDUs are unnecessary for the application based on at least one of signaled PDU set discard flags and their dependencies on other PDUs or PDU sets that failed to be successfully transmitted, or a combination thereof.

[0206] In one embodiment, the extended header element encapsulating the PDU set information has a fixed size and includes a plurality of information fields, a plurality of reserved fields, or a combination thereof.

[0207] In one embodiment, the set of PDU set attributes of the PDU set information includes a PDU set sequence number / identifier field, a PDU sequence number field, a PDU set start marker field, a PDU set end marker field, a PDU set size field indicating the PDU set size in bytes, a PDU set size field indicating the PDU set size in the number of included PDUs, a PDU set importance field, a PDU set discard flag bit field, a PDU set coding layer identifier field, a PDU set error recovery information field, a PDU set reference field containing a reference list to one or more PDU set sequence numbers / identifiers, or a combination thereof.

[0208] In one embodiment, the PDU set importance field consists of log2|I| bits of information indicating a PDU set importance value in a set of I importance values ​​comprising at least two elements, where at least one value corresponds to "high" importance and at least one value corresponds to "low" importance.

[0209] In one embodiment, the PDU set importance field linearly encodes l=16 importance values, where a '0' encoded value indicates the highest PDU set importance and a '15' encoded value indicates the lowest PDU set importance.

[0210] In one embodiment, the RTP extended header includes at least a second extended header element, and the second extended header element is located after the extended header element that encapsulates the PDU set information as the first extended header element.

[0211] In one embodiment, the PDU set includes: a coded video frame, a coded video frame partition as a coded video slice, or a coded video tile, a coded video temporal layer, a coded video spatial layer, or a combination thereof.

[0212] In one embodiment, PDU set grouping and PDU set information determination are based on encoding ADU information, encoding configuration, encoding information metadata, application determined configuration, or a combination thereof.

[0213] Controller 1806 can manage input and output signals of NE 1800. Controller 1806 can also manage peripheral devices that are not integrated into NE 1800. In some implementations, controller 1806 can utilize such as Or other operating systems. In some implementations, the controller 1806 can be implemented as part of the processor 1802.

[0214] In some implementations, the NE 1800 may include at least one transceiver 1808. In some other implementations, the NE 1800 may have more than one transceiver 1808. The transceiver 1808 may represent a wireless transceiver. The transceiver 1808 may include one or more receiver chains 1810, one or more transmitter chains 1812, or a combination thereof.

[0215] The receiver chain 1810 may be configured to receive a signal (e.g., control information, data, packet) over a wireless medium. For example, the receiver chain 1810 may include one or more antennas for receiving signals over the air or wireless medium. The receiver chain 1810 may include at least one amplifier (e.g., a low noise amplifier (LNA)) configured to amplify the received signal. The receiver chain 1810 may include at least one demodulator configured to demodulate the received signal and obtain the transmitted data by inverting the modulation technique applied during the transmission of the signal. The receiver chain 1810 may include at least one decoder for decoding and processing the demodulated signal to receive the transmitted data.

[0216] The transmitter chain 1812 may be configured to generate and transmit signals (e.g., control information, data, packets). The transmitter chain 1812 may include at least one modulator for modulating data onto a carrier signal, preparing the signal for transmission over a wireless medium. The at least one modulator may be configured to support one or more techniques, such as amplitude modulation (AM), frequency modulation (FM), or a digital modulation scheme, such as phase shift keying (PSK) or quadrature amplitude modulation (QAM). The transmitter chain 1812 may also include at least one power amplifier configured to amplify the modulated signal to an appropriate power level suitable for transmission over a wireless medium. The transmitter chain 1812 may also include one or more antennas for transmitting the amplified signal into the air or in a wireless medium.

[0217] Fig.19 A flow chart showing a method according to various aspects of the present disclosure is shown. The operation of the method can be performed as shown in FIG. Fig.16 The UE described in reference Fig.17 The processor and / or base station described, or as referenced Fig.18 In some implementations, the UE, processor and / or NE may execute instruction sets to control the functional elements of the UE to perform the described functions.

[0218] At 1905, the method may include encoding the media of the application into a plurality of ADUs as logically independent information units. The operations of 1905 may be performed according to examples as described herein. In some implementations, aspects of the operations of 1905 may be performed as described in reference to Fig.16 The UE described in reference Fig.17The processor and / or base station described, or as referenced Fig.18 The described application server executes.

[0219] At 1910, the method may include grouping, by an application RTP sender routine, each ADU into one or more PDU sets, wherein the PDU set includes one or more PDUs encapsulating ADU information, the ADU information being packaged into an RTP PDU payload. The operations of 1910 may be performed according to examples as described herein. In some implementations, aspects of the operations of 1910 may be performed as described in reference to Fig.16 The UE described in reference Fig.17 The processor and / or base station described, or as referenced Fig.18 The described application server executes.

[0220] At 1915, the method may include determining, for each PDU set in the PDU set, PDU set information including: an identifier of the PDU set and the PDUs within the PDU set, and a set of PDU set attributes for the PDU set. The operations of 1915 may be performed according to examples as described herein. In some implementations, aspects of the operations of 1915 may be performed as described in reference to Fig.16 The UE described in reference Fig.17 The processor and / or base station described, or as referenced Fig.18 The described application server executes.

[0221] At 1920, the method may include encapsulating PDU set information for each PDU in the PDU set in an extended header element, the extended header element being packaged within an RTP extended header of the RTP PDU, the RTP extended header corresponding to the RTP PDU payload of each PDU in the one or more PDUs in the PDU set. The operations of 1920 may be performed according to examples as described herein. In some implementations, aspects of the operations of 1920 may be performed as described in reference to Fig.16 The UE described in reference Fig.17 The processor and / or base station described, or as referenced Fig.18 The described application server executes.

[0222] At 1925, the method may include sending an RTP PDU containing PDU set information. The operations of 1925 may be performed according to examples as described herein. In some implementations, aspects of the operations of 1925 may be performed as described in reference to Fig.16 The UE described in reference Fig.17 The processor and / or base station described, or as referenced Fig.18 The described application server executes.

[0223] It should be noted that the methods described herein describe possible implementations, and that the operations and steps may be rearranged or otherwise modified, and other implementations are possible.

[0224] Fig. 20 A flow chart showing a method according to various aspects of the present disclosure is shown. The operation of the method can be performed as shown in FIG. Fig.16 The UE described in reference Fig.17 The processor, and / or base station described, or as referenced Fig.18 The described network functions (eg, UPF) on the NE are implemented. In some implementations, the UE, processor and / or NE may execute instruction sets to control the functional elements of the UE to perform the described functions.

[0225] At 2005, the method may include receiving an RTP PDU set, the RTP PDU including an extended header having an extended header element, the extended header element including PDU set information and a PDU payload. The operations of 2005 may be performed according to examples as described herein. In some implementations, aspects of the operations of 2005 may be performed as described in reference to Fig.16 The UE described in reference Fig.17 The processor, and / or base station described, or as referenced Fig.18 The network functions (e.g., UPF) on the described NE are executed.

[0226] At 2010, the method may include extracting PDU set information from an extended header element of each RTP PDU in the PDU set, the PDU set information including: an identifier of the PDU set and the PDUs within the PDU set, and a set of PDU set attributes for the PDU set. The operations of 2010 may be performed according to examples as described herein. In some implementations, aspects of the operations of 2010 may be performed as described in reference to Fig.16 The UE described in reference Fig.17 The processor, and / or base station described, or as referenced Fig.18 The network functions (e.g., UPF) on the described NE are executed.

[0227] At 2015, the method may include packaging each RTP PDU in the RTP PDU of the PDU set and corresponding PDU set information within a GTP-U PDU, the GTP-U PDU including: each RTP PDU in the RTP PDU in the PDU set encapsulated as a GTP-U payload, and each PDU set information in the corresponding PDU set information encapsulated as a GTP-U header field. The operations of 2015 may be performed according to the examples described herein. In some implementations, aspects of the operations of 2015 may be performed as described in reference to Fig.16 The UE described in reference Fig.17 The processor, and / or base station described, or as referenced Fig.18 The network functions (e.g., UPF) on the described NE are executed.

[0228] At 2020, the method may include sending a GTP-U encapsulated PDU set and corresponding PDU set information. The operations of 2020 may be performed according to the examples described herein. In some implementations, aspects of the operations of 2020 may be performed as described in reference to Fig.16 The UE described in reference Fig.17 The processor, and / or base station described, or as referenced Fig.18 The network functions (e.g., UPF) on the described NE are executed.

[0229] It should be noted that the methods described herein describe possible implementations, and that the operations and steps may be rearranged or otherwise modified, and other implementations are possible.

[0230] Embodiments may be practiced in other specific forms. The described embodiments should be considered in all respects as merely illustrative and not restrictive. Therefore, the scope of the present invention is indicated by the appended claims rather than by the foregoing description. All changes that fall within the equivalent meaning and scope of the claims should be included within their scope.

Claims

1. A network device (NE) for wireless communication, comprising: at least one memory; at least one processor, the at least one processor being coupled to the at least one memory and configured to cause the NE to: Encode the application's media into multiple application data units (ADUs), which are logically independent information units; grouping, by an application real-time transport protocol (RTP) sender routine, each ADU into one or more protocol data unit (PDU) sets, wherein a PDU set includes one or more PDUs encapsulating ADU information, the ADU information being packaged into an RTP PDU payload; For each of the PDU sets in the PDU set, determining PDU set information, the PDU set information comprising: identifiers of the PDU set and the PDUs in the PDU set, and a set of PDU set attributes of the PDU set; encapsulating the PDU set information of each of the PDUs in the PDU set in an extended header element, the extended header element being packaged within an RTP extended header of an RTP PDU, the RTP extended header corresponding to the RTP PDU payload of each of the one or more PDUs in the PDU set; and The RTP PDU including the PDU set information is sent.

2. The NE of claim 1, wherein the PDU set information optimizes media transmission in view of a Quality of Service (QoS) requirement of the application.

3. The NE of claim 2, wherein optimizing the media transmission based on the PDU set information in view of the QoS requirement comprises: mapping the PDU sets to data radio bearers, group scheduling of radio resources for over-the-air transmission of the PDUs in the PDU sets, dynamic adjustment of a radio modulation and coding scheme strategy for over-the-air transmission of the PDU sets, dynamic decision to delete one or more PDUs in one or more PDU sets from an over-the-air transmission queue, determining that the deleted PDUs are unnecessary for the application based on at least one of signaled PDU set discard flags and their dependencies on other PDUs or PDU sets that failed to be successfully transmitted, or a combination thereof.

4. The NE according to claim 1, wherein the extended header element encapsulating the PDU set information has a fixed size and includes a plurality of information fields, a plurality of reserved fields, or a combination thereof.

5. The NE according to claim 1, wherein the set of PDU set attributes of the PDU set information comprises: PDU Set Sequence Number / Identifier field; PDU sequence number field; PDU set start marker field; PDU set end marker field; a PDU Set Size field indicating the size of the PDU Set in bytes; a PDU Set Size field indicating the size of the PDU Set in units of the number of included PDUs; PDU set importance field; PDU set discard flag bit field; PDU set coding layer identifier field; PDU set error recovery information field; A PDU Set Reference field containing a list of references to one or more PDU Set sequence numbers / identifiers; or a combination thereof.

6. The NE according to claim 5, wherein the PDU set importance field consists of log2|I| bits of information indicating the PDU set importance value in a set of I importance values ​​containing at least two elements, wherein at least one value corresponds to "high" importance and at least one value corresponds to "low" importance.

7. The NE of claim 6, wherein the PDU set importance field linearly encodes l=16 importance values, wherein a '0' encoded value indicates the highest PDU set importance and a '15' encoded value indicates the lowest PDU set importance.

8. The NE according to claim 1, wherein the RTP extended header includes at least a second extended header element, and the second extended header element is located after the extended header element that encapsulates the PDU set information as the first extended header element.

9. The NE according to claim 1, wherein the PDU set comprises: A coded video frame, a coded video frame partition as a coded video slice, or a coded video tile, a coded video temporal layer, a coded video spatial layer, or a combination thereof.

10. The NE of claim 1, wherein the PDU set grouping and the PDU set information determination are based on encoding ADU information, encoding configuration, encoding information metadata, application determined configuration or a combination thereof.

11. A processor for wireless communication, comprising: at least one controller coupled to the at least one memory and configured to cause the processor to: Encode the application's media into multiple application data units (ADUs), which are logically independent information units; grouping, by an application real-time transport protocol (RTP) sender routine, each ADU into one or more protocol data unit (PDU) sets, wherein a PDU set includes one or more PDUs encapsulating ADU information, the ADU information being packaged into an RTP PDU payload; For each of the PDU sets in the PDU set, determining PDU set information, the PDU set information comprising: identifiers of the PDU set and the PDUs in the PDU set, and a set of PDU set attributes of the PDU set; encapsulating the PDU set information of each of the PDUs in the PDU set in an extended header element, the extended header element being packaged within an RTP extended header of an RTP PDU, the RTP extended header corresponding to the RTP PDU payload of each of the one or more PDUs in the PDU set; and The RTP PDU including the PDU set information is sent.

12. The processor of claim 11, wherein the PDU set information optimizes media transmission in view of quality of service (QoS) requirements of the application.

13. The processor of claim 12, wherein optimizing the media transmission based on the PDU set information in view of the QoS requirement comprises: mapping the PDU sets to data radio bearers, group scheduling of radio resources for over-the-air transmission of the PDUs in the PDU sets, dynamic adjustment of a radio modulation and coding scheme strategy for over-the-air transmission of the PDU sets, dynamic decision to delete one or more PDUs in one or more PDU sets from an over-the-air transmission queue, determining that the deleted PDUs are unnecessary for the application based on at least one of signaled PDU set discard flags and their dependencies on other PDUs or PDU sets that failed to be successfully transmitted, or a combination thereof.

14. The processor of claim 11, wherein the extended header element encapsulating the PDU set information has a fixed size and includes a plurality of information fields, a plurality of reserved fields, or a combination thereof.

15. The processor of claim 11, wherein the set of PDU set attributes of the PDU set information comprises: PDU Set Sequence Number / Identifier field; PDU sequence number field; PDU set start marker field; PDU set end marker field; a PDU Set Size field indicating the size of the PDU Set in bytes; a PDU Set Size field indicating the size of the PDU Set in units of the number of included PDUs; PDU set importance field; PDU set discard flag bit field; PDU set coding layer identifier field; PDU set error recovery information field; A PDU Set Reference field containing a list of references to one or more PDU Set sequence numbers / identifiers; or a combination thereof.

16. The processor of claim 15 , wherein the PDU set importance field consists of log2|I| bits of information indicating the PDU set importance value in a set of I importance values ​​comprising at least two elements, wherein at least one value corresponds to “high” importance and at least one value corresponds to “low” importance.

17. The processor of claim 16, wherein the PDU set importance field linearly encodes l=16 importance values, wherein a '0' encoded value indicates the highest PDU set importance and a '15' encoded value indicates the lowest PDU set importance.

18. The processor of claim 11, wherein the RTP extended header comprises at least a second extended header element, the second extended header element being located after an extended header element encapsulating the PDU set information as a first extended header element.

19. A method performed by a NE, the method comprising: Encode the application's media into multiple application data units (ADUs), which are logically independent information units; grouping, by an application real-time transport protocol (RTP) sender routine, each ADU into one or more protocol data unit (PDU) sets, wherein a PDU set includes one or more PDUs encapsulating ADU information, the ADU information being packaged into an RTP PDU payload; For each of the PDU sets in the PDU set, determining PDU set information, the PDU set information comprising: identifiers of the PDU set and the PDUs in the PDU set, and a set of PDU set attributes of the PDU set; encapsulating the PDU set information of each of the PDUs in the PDU set in an extended header element, the extended header element being packaged within an RTP extended header of an RTP PDU, the RTP extended header corresponding to the RTP PDU payload of each of the one or more PDUs in the PDU set; and The RTP PDU including the PDU set information is sent.

20. A network device (NE) for wireless communication, comprising: at least one memory; at least one processor, the at least one processor being coupled to the at least one memory and configured to cause the NE to: receiving a set of real-time transport protocol (RTP) protocol data units (PDUs), the RTP PDU comprising an extended header having an extended header element, the extended header element comprising PDU set information and a PDU payload; Extracting the PDU set information from the extended header element of each RTP PDU in the PDU set, the PDU set information comprising: identifiers of the PDU set and the PDUs within the PDU set, and a set of PDU set attributes of the PDU set; Packing each of the RTP PDUs in the PDU set and corresponding PDU set information in a General Packet Radio Service (GPRS) User Plane Tunneling Protocol (GTP-U) PDU, the GTP-U PDU comprising: each of the RTP PDUs in the PDU set encapsulated as a GTP-U payload, and each of the PDU set information in the corresponding PDU set information encapsulated as a GTP-U header field; and Send the GTP-U encapsulated PDU set and corresponding PDU set information.

Images