Signaling of the use of PDU sets and burst termination markings for communicating WEBRTC media data.

Abstract

An exemplary device for communicating media data includes a memory configured to store media data and a processing system, which includes one or more processors implemented within a circuit mechanism, the processing system being configured to receive a Session Description Protocol (SDP) message containing configuration information representing at least one of a protocol data unit (PDU) set marking or burst end (EoB) marking relating to a communication session, transmit information representing a PDU set marking or EoB marking relating to a communication session to a real-time communication (RTC) application function, and process the media data of the communication session, the media data including at least one of a PDU set having a PDU set marking or an EoB having an EoB marking according to the configuration information.

Description

Technical Field

[0001]

[0001] This PCT application claims the benefit of U.S. Provisional Application No. 63 / 495,201, filed on April 10, 2023, the entire content of each application being incorporated herein by reference, and has a U.S. utility priority of U.S. Patent Application No. 18 / 629,617, filed on April 8, 2024.

[0002]

[0002] This disclosure relates to the storage and transfer of encoded video data.

Background Art

[0003]

[0003] Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, digital cameras, digital recording devices, digital media players, video gaming devices, video game consoles, cellular phones or satellite radiotelephones, video teleconferencing devices, etc. Digital video devices implement video compression techniques such as those defined by MPEG-2, MPEG-4, ITU-T H.263 or ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.265 (also known as High Efficiency Video Coding (HEVC)), and those described in extensions of such standards.

[0004]

[0004] After the video data has been encoded, it can be packetized for transmission or storage. The video data can be assembled into a video file conforming to any of the various standards, such as the International Organization for Standardization (ISO) based media file format and its extensions, such as AVC. [Overview of the project]

[0005]

[0005] In general, this disclosure describes techniques relating to communicating (e.g., transmitting, receiving, or forwarding) Web Real-time Communication (WebRTC) data. WebRTC data may include extended reality (XR) media data, which may include any or all of the following: text data, audio data, video data, mixed reality (MR) data, augmented reality (AR) data, and / or virtual reality (VR) data. WebRTC data can be segmented and encapsulated into protocol data units (PDUs), which can be communicated in bursts of activity over radio signals. Similarly, PDUs can be organized into PDU sets, which may include a set of PDUs that will be consumed collectively by a receiver. For example, a PDU set may include corresponding PDUs that include audio data, video data, and XR data. Furthermore, PDU sets and ends of bursts (EoBs) can be marked to help identify XR traffic and optimize its delivery. According to the technology of this disclosure, various devices involved in communicating XR data and WebRTC data can indicate whether or not PDU set marking and / or EoB marking is enabled for a particular WebRTC session. In this way, network devices can detect XR data and WebRTC data and, accordingly, apply quality of service (QoS) policies to such data.

[0006]

[0006] In one embodiment, a method for communicating media data includes receiving a session description protocol (SDP) message which includes configuration information which represents at least one of a protocol data unit (PDU) set marking or an end-of-burst (EoB) marking relating to a communication session; transmitting information which represents a PDU set marking or an EoB marking relating to a communication session to a Real-time Communication (RTC) application function; and processing media data of a communication session which includes processing which, according to the configuration information, the media data includes at least one of a PDU set having a PDU set marking or an EoB having an EoB marking.

[0007]

[0007] In another embodiment, a device for communicating media data includes a memory configured to store media data and a processing system including one or more processors implemented within a circuit mechanism, the processing system being configured to receive a session description protocol (SDP) message which includes configuration information which represents at least one of protocol data unit (PDU) set markings or burst end (EoB) markings relating to a communication session, and to transmit information which represents PDU set markings or EoB markings relating to a communication session to a real-time communication (RTC) application function and to process media data of a communication session, the media data which, according to the configuration information, includes at least one of PDU sets with PDU set markings or EoBs with EoB markings.

[0008]

[0008] In another embodiment, a computer-readable storage medium stores instructions, which, when executed, cause a processor to receive a session description protocol (SDP) message containing configuration information representing at least one of protocol data unit (PDU) set markings or burst end (EoB) markings relating to a communication session, to transmit information representing the PDU set markings or EoB markings relating to the communication session to a real-time communication (RTC) application function, to process media data of the communication session, the media data containing at least one of PDU sets having PDU set markings or EoBs having EoB markings according to the configuration information.

[0009]

[0009] In another embodiment, a device for communicating media data includes means for receiving a session description protocol (SDP) message, which includes configuration information representing at least one of protocol data unit (PDU) set markings or burst end (EoB) markings relating to a communication session; means for transmitting information representing PDU set markings or EoB markings relating to a communication session to a real-time communication (RTC) application function; and means for processing media data of a communication session, wherein the media data includes at least one of PDU sets having PDU set markings or EoBs having EoB markings according to the configuration information.

[0010]

[0010] Details of one or more embodiments are described in the accompanying drawings and the following description. Other features, purposes, and advantages will become apparent from this description and drawings, as well as from the claims. [Brief explanation of the drawing]

[0011] [Figure 1]

[0011] This block diagram shows an architecture relating to a system that can be configured to perform immersive real-time communication for Web Real-Time Communication (WebRTC) according to the technology of the present disclosure. [Figure 2]

[0012] This is a block diagram illustrating the elements of an example video file. [Figure 3]

[0013] This flowchart illustrates an exemplary method for using protocol data unit (PDU) sets and burst end-of-burst (EoB) markings using the technology of the present disclosure. [Figure 4]

[0014] This is a flowchart illustrating one embodiment of a method for communicating XR media data using the technology disclosed herein. [Modes for carrying out the invention]

[0012]

[0015] In general, this disclosure describes techniques related to communicating media data, such as Extended Reality (XR) media data. XR media data may include any or all of the following: text data, voice data, audio data, still image data, video data, mixed reality (MR) data, augmented reality (AR) data, and / or virtual reality (VR) data. XR traffic marking is a mechanism that helps a network identify XR traffic and optimize its delivery. The concept of protocol data unit (PDU) sets has been introduced specifically for this purpose, but can also be used for other types of traffic. A PDU set is a set of PDUs that are consumed collectively by the receiver and therefore should be processed collectively by the network. End of Burst (EoB) provides another tool for optimizing the delivery of XR traffic, by enabling the appropriate use of Connected mode Discontinuous Reception (CDRX) to conserve power on the receiver side.

[0013]

[0016] PDU set marking can be performed on real-time transport protocol (RTP) or secure RTP (SRTP) traffic, through the use of RTP header extensions, which are appended to the RTP packet header of each PDU in the RTP stream where PDU set marking is activated. A user plane function (UPF) can inspect downlink traffic to extract information about PDU set marking and pass that PDU set to a base station such as a gNode-B (gNB). This disclosure describes signaling-related techniques for configurations of PDU set marking using a policy control function (PCF), which may constitute a UPF.

[0014]

[0017] Generally, user equipment (UE) and an application server (AS) or other remote UE can initially negotiate the use of PDU sets and EoB markings during the offer / answer exchange, for example, through a Session Initiation Protocol (SIP) re-invite, during session establishment or renewal. According to RFC8285, negotiation of the RTP header extensions used is performed by including the "extmap" attribute. The uniform resource name (URN) for PDU set markings can be set to "urn:3gpp:pdus-marking:rel-18".

[0015]

[0018] Regarding RTP streams, the following options are supported and apply to RTP packets in that RTP stream throughout the entire lifespan of the RTP session: A 1-byte or 2-byte header extension format, appropriately identified through the extension attribute "short" or "long". If not present, the header extension format shall be inferred from the preamble byte 0xBEDE or 0x100 (+appbits). In either case, the application server shall not change the format during the RTP session. The PDU set size in bytes, identified through the presence of the string flag "pdu-set-size". If it is not present, the receiver assumes that the PDU set size field does not exist. This results in a shorter header extension for that RTP session. • The end of a burst is marked by the presence of the string flag "end-of-burst". If it is not present, the receiver should ignore the EoB bit.

[0016]

[0019] The augmented Backus-Naur form (ABNF) syntax for the extmap attribute, according to RFC 8285, is as follows: extmap-attr=“a=extmap:”1 * 5DIGIT[“ / ”direction]SP“urn:3gpp:pdus-marking:rel-18”SP extensionattributes extensionattributes= * 3(format / “pdu-set-size” / “end-of-burst”) format="short" / "long"

[0017]

[0020] Figure 1 is a block diagram showing an architecture 100 relating to a system that can be configured to perform immersive real-time communication for Web Real-Time Communication (WebRTC) using the technology of this disclosure. In particular, architecture 100 can be used with respect to 5G media streaming (5GMS) using WebRTC. That is, architecture 100 can be used to perform WebRTC real-time communication over a 5G network connection.

[0018]

[0021] Architecture 100 can be used to provide WebRTC in a variety of scenarios. For example, Architecture 100 can be used in conjunction with a 5G network to provide "over-the-top" (OTT) WebRTC. Another example is that a mobile network operator (MNO) can use Architecture 100 to provide reliable WebRTC functionality and / or facility WebRTC services. Yet another example is that Architecture 100 can provide interoperable WebRTC services. Architecture 100 can also be used in a variety of other scenarios. Architecture 100 offers flexibility through a set of functions and interfaces that can be combined in various ways based on the needs of a particular scenario.

[0019]

[0022] In the embodiment shown in Figure 1, the architecture 100 includes a 5G RTC application provider 102, a 5G RTC application function 104, and a user device (UE) 150. Generally, the 5G RTC application provider 102 interacts with the functions of the 5G RTC application function 104 and supplies 5G RTC-enabled applications, such as a web application 152, to the user device 150.

[0020]

[0023] User equipment 150 may also be referred to as a "UE" or "client device". User equipment 150 can be, for example, a laptop or desktop computer, digital camera, digital recording device, digital media player, video gaming device, video game console, cellular phone or satellite radio phone, video teleconference device, and the like. In this embodiment, user equipment 150 includes a web application 152, a native WebRTC application 154, and a media session handler (MSH) 158. An interface 156 couples the native WebRTC application 154 and the MSH 158. Interface 156 may be referred to as the "RTC-6" interface. The UE 150 and the 5G RTC application provider 102 are coupled by an interface 174, which may be referred to as the "RTC-8" interface.

[0021]

[0024] MSH158 is a function within UE150 that provides WebRTC applications, such as web application 152, with access to 5G RTC support functions, such as 5G RTC application functions 104. These functions may be provided upon request through interface 156 (RTC-6 interface) or transparently without direct involvement of web application 154. MSH158 can indirectly assist interactive connectivity establishment (ICE) negotiation by, for example, providing a list of candidate Session Traversal Utilities for Network Address Translation (STUN) server candidates and / or Traversal Using Relay around NAT (TURN) server candidates that provide 5G RTC functionality. MSH158 can also collect quality of experience (QoE) metric reports and submit consumption reports. MSH158 can also provide media configuration recommendations to web application 152 through interface 156 (RTC-6).

[0022]

[0025] Interface 170 (which may be referred to as the "RTC-1" interface) enables the 5G RTC application provider 102 to provision support for the provided RTC session as the 5G RTC application function 104. This provisioning can cover functionality including, but not limited to, service quality (QoS) for WebRTC sessions, charging provisioning for WebRTC sessions, collection of consumption data and QoE metric data related to WebRTC sessions, provision of ICE functionality such as STUN servers and TURN servers, and / or provision of a WebRTC signaling server that potentially has interoperability with other signaling servers.

[0023]

[0026] In this embodiment, the 5G RTC application function 104 includes a 5G RTC support application function (AF) 110, a 5G RTC configuration (config) AF 112, a 5G RTC provisioning AF 114, a 5G RTC data channel AF 116, a 5G RTC signaling server AF 118, a 5G RTC interoperability (interop) AF 120, a 5G RTC STUN AF 122, and a 5G RTC TURN AF 124. In this embodiment, the 5G RTC application function 104 is also interoperable with a policy and charging function (PCF) 160, a network exposure function (NEF) 162, and a session management function (SMF) 164.

[0024]

[0027] Interface 170, sometimes referred to as the "provisioning interface," is not necessarily relevant to all collaborative scenarios, and some of the 5G support functionality can be provided by the application provider without provisioning.

[0025]

[0028] Interface 172 (sometimes referred to as the "RTC-5" interface) is an interface between MSH158 and 5G RTC application function 104. Interface 172 can be used to transmit configuration information from 5G RTC application function 104 to MSH158 and to request support regarding a WebRTC session being started / in progress. The configuration information may include static information such as recommendations regarding media configuration, configuration of STUN and TURN server locations, configuration for consumption and QoE reporting, or discovery information regarding WebRTC signaling and data channel servers and their capabilities.

[0026]

[0029] MSH158 can provide support functions to the 5G RTC application function 104 or the web application 152, such as notifying them about WebRTC sessions and their status, requesting QoS allocation for WebRTC sessions being started or modified, receiving notifications about changes in QoS allocation for ongoing WebRTC sessions, or, for example, receiving, updating, or exchanging information about WebRTC sessions with the 5G RTC STUN / TURN / signaling server in order to identify WebRTC sessions and associate those sessions with QoS templates.

[0027]

[0030] In some embodiments, the 5G functionality that provides application functions to a WebRTC application (including the 5G RTC data channel AF116, 5G RTC signaling server AF118, 5G RTC interop AF120, 5G RTC STUN AF122, and 5G RTC TURN AF124) can be provided by an application server (5G RTC AS) instead of the AF. In this case, the 5G RTC AS can use a dedicated RTC-3 interface to request configuration and network support for an ongoing WebRTC session from the 5G RTC AF.

[0028]

[0031] The 5G RTC application provider 102, the 5G RTC application function 104, and the functionality belonging to UE150 can be implemented in hardware, software, firmware, or any combination thereof. When implemented in software or firmware, it can provide memory for storing instructions that can be executed by one or more processors implemented within the circuit mechanism. The processors may include one or more of the following: microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic circuit mechanisms, or any combination thereof.

[0029]

[0032] According to the technology of this disclosure, one or more of the 5G RTC application functions 104 (e.g., 5G RTC signaling server AF118) can be configured to use the Npcf_PolicyAuthorization procedure (according to TS29.514) or the N33 Nnef_AFSessionWithQoS procedure (according to TS29.122) to request the PCF160 or NEF162 to allocate quality of service (QoS) for a communication session between the UE150 and a remote UE or application server (not shown in Figure 1). These methods can be extended to add support for signaling PDU sets and EoB markings to the PCF160 or NEF162 in accordance with the technology of this disclosure. For example, the MediaSubComponent data type according to TS29.514 can be extended as follows, where " <added> Additional text< / added> " refers to additional text for the MediaSubComponent data type in TS29.514:

[0030] [Table 1]

[0031]

[0033] In other words, in this embodiment, the row "pduSetMarking" is added, which includes configuration information relating to PDU sets and EoB marking according to the technology of the present disclosure.

[0032]

[0034] The PDUSetMarking data type can be defined as follows:

[0033] [Table 2]

[0034]

[0035] Furthermore, or alternatively, the MSH158 can be configured to pass information about the session, including the session's media components and the PDU set and EoB marking configuration related to the session. The dynamic policy information configured by the MSH158 regarding the session may include the following:

[0035] [Table 3]

[0036]

[0036] The MediaComponentQoS object may contain the following information:

[0037] [Table 4]

[0038]

[0037] The QoS policy template can be extended to include names for each subcomponent of a session. These names can then be used to associate the actual media streams with the QoS subcomponent policies. Any or all of the various RTC AFs, as shown in Figure 1, can then use this mapping to associate network assistance requests for each media stream with the corresponding subcomponents of their QoS policies. The RTC AF can also use this information to verify that the requested / desired QoS for each component is consistent with the QoS policy provisioned by the 5G RTC application provider 102, which may be referred to as an application service provider (ASP).

[0039]

[0038] Therefore, according to the technology of the present disclosure, the 5G RTC signaling server AF118 can receive configuration data relating to an XR communication session (executed via WebRTC) from the 5G RTC application provider 102. In general, real-time media communication sessions such as XR communication sessions require low latency data transmission between endpoints (e.g., UEs) to ensure that participants in the XR communication session can experience events in the XR communication session (e.g., the movement and interaction of other participants with the virtual environment, the participant's own interaction with the virtual environment, etc.) in near real-time. Therefore, XR communication sessions may require a high level of quality of service (QoS).

[0040]

[0039] The configuration data may include either or both of the protocol data unit (PDU) set markings and / or burst end (EoB) markings relating to the XR communication session. Such PDU set markings or EoB markings can be used to associate traffic with the requested QoS. The 5G RTC signaling server AF118 may transmit data indicating the PDU set markings and / or EoB markings relating to the XR communication session to one or more other RTC AF104. One or more of the RTC AF104 may also interact with the PCF160 to negotiate the requested QoS relating to the XR communication session. In this way, network devices (e.g., base stations such as gNBs) between participants involved in an XR communication session can examine PDU set markings and / or EoB markings for data transmitted as part of the XR communication session, determine that such markings are associated with a negotiated QoS level, and therefore prioritize the transmission of data for the XR communication session to satisfy that QoS level. Such PDU set markings and / or EoB markings can be placed within the RTP header extension of the RTP packet containing the data for the XR communication session.

[0041]

[0040] Figure 2 is a block diagram showing the elements of an exemplary video file 250. Video files according to the ISO-based media file format and its extensions store data within a set of objects referred to as "boxes". In the embodiment of Figure 2, the video file 250 includes a file type (FTYP) box 252, a movie (MOOV) box 254, a segment index (sidx) box 262, a movie fragment (MOOF) box 264, and a movie fragment random access (MFRA) box 266. Figure 2 represents one embodiment of a video file, but it should be understood that other media files may contain other types of media data (e.g., audio data, timed text data, etc.) that are structured similarly to the data in the video file 250, according to the ISO-based media file format and its extensions.

[0042]

[0041] The File Type (FTYP) box 252 generally describes the file type for the video file 250. The File Type box 252 may contain data that identifies a specification describing the best use for the video file 250. The File Type box 252 may also be placed before the MOOV box 254, the Movie Fragment box 264, and / or the MFRA box 266.

[0043]

[0042] In the embodiment shown in Figure 2, the MOOV box 254 includes a movie header (MVHD) box 256, a track (TRAK) box 258, and one or more movie extends (MVEX) boxes 260. Generally, the MVHD box 256 can describe the general characteristics of the video file 250. For example, the MVHD box 256 may include data describing when the video file 250 was first created, data describing when the video file 250 was last modified, data describing the timescale of the video file 250, data describing the duration of playback of the video file 250, or other data that describes the video file 250 in general.

[0044]

[0043] The TRAK box 258 may contain data relating to the track of the video file 250. The TRAK box 258 may contain a track header (TKHD) box that describes the characteristics of the track corresponding to the TRAK box 258. In some embodiments, the TRAK box 258 may contain a coded video picture, while in other embodiments, the coded video picture of that track may be contained within a movie fragment 264 that can be referenced by the data in the TRAK box 258 and / or the sidx box 262.

[0045]

[0044] In some embodiments, the video file 250 may contain two or more tracks. Therefore, the MOOV box 254 may contain a number of TRAK boxes equal to the number of tracks in the video file 250. The TRAK boxes 258 can describe the characteristics of the corresponding tracks in the video file 250. For example, the TRAK box 258 can describe temporal and / or spatial information about the corresponding track. If the encapsulation unit 30 (Figure 1) includes a parameter set track in a video file such as the video file 250, a TRAK box similar to the TRAK box 258 in the MOOV box 254 can describe the characteristics of the parameter set track. The encapsulation unit 30 can signal in the TRAK box describing the parameter set track that a sequence-level SEI message is present in that parameter set track.

[0046]

[0045] The MVEX box 260 can describe the characteristics of the corresponding movie fragment 264 to signal that, for example, the video file 250 contains movie data in addition to the movie fragment 264, if present. In the context of streaming video data, coded video pictures can be contained within the movie fragment 264 rather than within the MOOV box 254. Thus, all coded video samples can be contained within the movie fragment 264 rather than within the MOOV box 254.

[0047]

[0046] A MOOV box 254 may contain a number of MVEX boxes 260 equal to the number of movie fragments 264 in the video file 250. Each MVEX box 260 can describe one corresponding characteristic of the movie fragment 264. For example, each MVEX box may contain a movie extends header box (MEHD) that describes the duration of one corresponding movie fragment 264.

[0048]

[0047] As described above, the encapsulation unit 30 can store a sequence dataset within a video sample that does not contain the actual coded video data. A video sample can generally correspond to an access unit, which is a representation of a coded picture in a particular time instance. In the context of AVC, a coded picture includes one or more VCL NAL units containing information for constructing all the pixels of the access unit, and other related non-VCL NAL units, such as SEI messages. Thus, the encapsulation unit 30 can include a sequence dataset, which may include sequence-level SEI messages, within one of the movie fragments 264. The encapsulation unit 30 can further signal the presence of the sequence dataset and / or sequence-level SEI messages as being present within one of the movie fragments 264 in one of the MVEX boxes 260 corresponding to that movie fragment 264.

[0049]

[0048] The SIDX box 262 is an optional element of the video file 250. That is, a video file conforming to the 3GPP file format, or any other such file format, does not necessarily include a SIDX box 262. According to an embodiment of the 3GPP file format, the SIDX box can be used to identify a subsegment of a segment (for example, a segment contained within the video file 250). The 3GPP file format defines a subsegment as "a self-contained set of one or more consecutive movie fragment boxes having corresponding media data boxes(s) or more, where the media data box containing the data referenced by the movie fragment box must follow that movie fragment box and precede the next movie fragment box containing information about the same track." The 3GPP file format also states that a SIDX box "contains a sequence of references to subsegments of the (sub)segment documented by that box. The referenced subsegments are contiguous within presentation time. Similarly, the bytes referenced by a segment index box are always contiguous within the segment. The referenced size gives a count of the number of bytes in that reference material."

[0050]

[0049] The SIDX box 262 generally provides information representing one or more subsegments of a segment contained within the video file 250. For example, such information may include the playback time at which the subsegment begins and / or ends, a byte offset relative to the subsegment, whether the subsegment includes a stream access point (SAP) (e.g., starting from an SAP), the type of SAP (e.g., whether the SAP is an instantaneous decoder refresh (IDR) picture, a clean random access (CRA) picture, a broken link access (BLA) picture, etc.), and the position of the SAP within the subsegment (in terms of playback time and / or byte offset).

[0051]

[0050] A movie fragment 264 may contain one or more coded video pictures. In some embodiments, a movie fragment 264 may contain one or more groups of pictures (GOPs), each GOP may contain several coded video pictures, e.g., frames or pictures. Furthermore, as described above, in some embodiments, a movie fragment 264 may contain a sequence dataset. Each movie fragment 264 may contain a movie fragment header box (MFHD, not shown in Figure 2). The MFHD box can describe the characteristics of the corresponding movie fragment, such as the sequence number for that movie fragment. The movie fragments 264 may be included in the video file 250 in order of sequence number.

[0052]

[0051] The MFRA box 266 can describe random access points within movie fragments 264 of the video file 250. This can help perform trick modes, such as performing seeks to specific temporal positions (i.e., playback time) within segments encapsulated by the video file 250. In some embodiments, the MFRA box 266 is generally optional and does not need to be included in the video file. Similarly, a client device such as the client device 40 does not necessarily need to refer to the MFRA box 266 in order to accurately decode and display the video data of the video file 250. The MFRA box 266 may include a number of track fragment random access (TFRA) boxes (not shown) equal to the number of tracks in the video file 250, or, in some embodiments, a number equal to the number of media tracks (e.g., non-hint tracks) in the video file 250.

[0053]

[0052] In some embodiments, the movie fragment 264 may include one or more stream access points (SAPs), such as IDR pictures. Similarly, the MFRA box 266 may provide indication of the location of those SAPs within the video file 250. Thus, a temporal subsequence of the video file 250 can be formed from the SAPs of the video file 250. This temporal subsequence may also include other pictures, such as P frames and / or B frames, that depend on the SAPs. Frames and / or slices of a temporal subsequence may be placed within a segment so that frames / slice of the temporal subsequence can be properly decoded, depending on other frames / slice of that subsequence. For example, in a hierarchical arrangement of data, data used for predictions about other data may also be included within that temporal subsequence.

[0054]

[0053] Figure 3 is a flowchart illustrating an exemplary method for using PDU sets and EoB marking according to the technology of the present disclosure. UE1 in Figure 3 may correspond to UE150 in Figure 1. An intermediate server, such as a reliable WebRTC signaling server or a proxy-call session control function (P-CSCF), may be configured to inspect session description protocol (SDP) offers / answers and extract information related to PDU sets and burst end marking. The WebRTC signaling server or P-CSCF in Figure 3 may correspond to the 5G RTC signaling server AF118 in Figure 1. The RTC AF in Figure 3 may correspond to one or more of the other RTC AFs 104 in Figure 1. The policy and billing function (PCF) in Figure 3 may correspond to PCF160 in Figure 1. The application server in Figure 3 may correspond to the 5G RTC application provider 102 in Figure 1.

[0055]

[0054] First, in the embodiment shown in Figure 3, UE1 sends an SDP offer for establishing an XR session to the WebRTC signaling server or P-CSCF, and the signaling server forwards the request to the application server (300).

[0056]

[0055] The application server then responds with an SDP answer (302). That is, the application server sends an SDP answer to the signaling server. The application server includes instructions for PDU set marking and / or burst end marking in the SDP answer, as described above. For example, the application server may signal a MediaSubComponent data element as described above, which may contain data indicating whether PDU set marking and / or eOb marking is active for that XR communication session.

[0057]

[0056] The signaling server examines the SDP answer and extracts information related to PDU set and burst end marking (304). For example, the signaling server may, for example, use the PDU set marking data element described above to extract data indicating whether the PDU set size and / or EoB marking is active for the corresponding XR communication session.

[0058]

[0057] The signaling server can then notify the RTC AF(s)(s) of a new IMS / WebRTC session, including information about the configuration of the PDU set and EoB marking (306).

[0059]

[0058] The RTC AF then requests the PCF to allocate QoS to the PDU set marking and / or EoB marking (308). For example, the RTC AF may use the Npcf_PolicyAuthorization procedure of TS29.514 to request the allocation of QoS to the media session of the IMS / WebRTC session.

[0060]

[0059] The RTC AF can then confirm with the signaling server the allocation of QoS for its media stream (310). The signaling server can receive this confirmation.

[0061]

[0060] Upon receiving confirmation, the signaling server can forward an SDP answer to UE1 (312). UE1 can then begin participating in the XR communication session. For example, UE1 can send and receive media data for the XR communication session. The RTC AF can add PDU set markings and / or EoB markings to appropriate packets of the XR communication session transmitted by UE1, and / or read the PDU set markings and / or EoB markings of packets of the XR communication session that will be received by UE1, and apply the corresponding QoS, such as that authorized by the PCF, to such packets.

[0062]

[0061] If the SDP message is encrypted, a dedicated WebRTC signaling protocol message can be defined to transmit the configuration from the application server to the WebRTC signaling server.

[0063]

[0062] Figure 4 is a flowchart showing one embodiment of a method for communicating XR media data according to the technology of this disclosure. The method in Figure 4 can be performed by a signaling server such as a WebRTC signaling server or P-CSCF.

[0064]

[0063] First, the signaling server can receive an invitation from the UE for a new WebRTC session (320). The UE can send an invitation to another UE to join a WebRTC session, and the signaling server can intercept that invitation. The signaling server can then forward that invitation to the application server (322).

[0065]

[0064] The application server can process the invitation and respond with an SDP answer that includes data representing at least one of the PDU set markings and / or EoB markings for the WebRTC session. Thus, the signaling server can receive an SDP answer that includes the PDU set markings and / or EoB marking data (324). The signaling server can then extract the PDU set and / or EoB marking configuration data (326). The signaling server can send the PDU set and / or EoB marking data to an RTC application function (AF) (328) to request a Quality of Service (QoS) allocation associated with the PDU set markings and / or EoB markings. Thus, the signaling server can receive a QoS allocation configuration from the RTC AF (330). Accordingly, the signaling server can forward the SDP answer to the UE (332).

[0066]

[0065] Thus, the method shown in Figure 4 represents one embodiment of a method for communicating media data, which includes receiving a session description protocol (SDP) message containing configuration information representing at least one of protocol data unit (PDU) set markings or burst end (EoB) markings relating to a communication session; transmitting information representing PDU set markings or EoB markings relating to a communication session to a real-time communication (RTC) application function; and processing media data of a communication session, wherein the media data includes at least one of PDU sets having PDU set markings or EoBs having EoB markings according to the configuration information.

[0067]

[0066] Various embodiments of the technology of this disclosure are summarized in the following clauses:

[0067] Clause 1: A method for communicating media data, comprising: communicating configuration information representing at least one of protocol data unit (PDU) set markings or burst end (EoB) markings; and processing at least one of PDU sets having PDU set markings according to the configuration information or EoBs having EoB markings according to the configuration information.

[0068]

[0068] Clause 2: The method of Clause 1, wherein communicating configuration information includes communicating a MediaSubComponent data type containing configuration information.

[0069]

[0069] Clause 3: Either method of Clauses 1 and 2, wherein the configuration information includes at least one of the following: a version value, a local identifier value, a format value, a value indicating whether the PDU set size is active or not, or a value indicating whether EoB marking is active or not.

[0070]

[0070] Clause 4: The method of Clause 1, wherein communicating configuration information includes communicating a list of quality of service (QoS) allocation specifications for web real-time communication (WebRTC) sessions, each of which includes a name value, a flow description, and a QoS allocation value.

[0071]

[0071] Clause 5: Communication of configuration information by any of the methods of Clauses 1 to 4, including communication of configuration information by the application server (AS).

[0072]

[0072] Clause 6: The method of Clause 5, further comprising receiving an offer for a new Web Real-Time Communication (WebRTC) session and communicating configuration information, which includes sending a response to the offer, including the configuration information, to a WebRTC signaling server or proxy call session control function (P-CSCF).

[0073]

[0073] Clause 7: Any method of Clauses 1 to 3, wherein the communication of configuration information includes communicating the configuration information by a Web Real-Time Communication (WebRTC) signaling server or a Proxy Call Session Control Function (P-CSCF).

[0074]

[0074] Clause 8: The method of Clause 7, wherein communicating configuration information includes receiving an answer to a request relating to a new WebRTC session, the answer comprising configuration information.

[0075]

[0075] Clause 9: Further comprising transmitting information indicating a new WebRTC session to a Real-Time Communication (RTC) application function, wherein the information indicates that the new WebRTC will use at least one of a PDU set having a PDU set marking or an EoB having an EoB marking, in any way of Clauses 7 and 8.

[0076]

[0076] Clause 10: Any method of Clauses 7 to 9, further comprising receiving confirmation of Quality of Service (QoS) allocation.

[0077]

[0077] Clause 11: Any method of Clauses 8 to 10, further comprising forwarding the answer to the user equipment (UE).

[0078]

[0078] Clause 12: The method of Clause 1, wherein communicating configuration information includes communicating a MediaSubComponent data type containing configuration information.

[0079]

[0079] Clause 13: The method of Clause 1, wherein the configuration information includes at least one of the following: a version value, a local identifier value, a format value, a value indicating whether the PDU set size is active or not, or a value indicating whether EoB marking is active or not.

[0080]

[0080] Clause 14: The method of Clause 1, wherein communicating configuration information includes communicating a list of quality of service (QoS) allocation specifications for a web real-time communication (WebRTC) session, each of which includes a name value, a flow description, and a QoS allocation value.

[0081]

[0081] Clause 15: The method of Clause 1, wherein the communication of configuration information is performed by an application server (AS).

[0082]

[0082] Clause 16: The method of Clause 15, further comprising receiving an offer for a new Web Real-Time Communication (WebRTC) session and communicating configuration information, which includes sending a response to the offer, including the configuration information, to a WebRTC signaling server or proxy call session control function (P-CSCF).

[0083]

[0083] Clause 17: The method of Clause 1, wherein the communication of configuration information includes communicating configuration information by a Web Real-Time Communication (WebRTC) signaling server or a Proxy Call Session Control Function (P-CSCF).

[0084]

[0084] Clause 18: The method of Clause 17, wherein communicating configuration information includes receiving an answer to a request relating to a new WebRTC session, the answer comprising configuration information.

[0085]

[0085] Clause 19: The method of Clause 7, further comprising transmitting information indicating a new WebRTC session to a Real-Time Communication (RTC) application function, wherein the information indicates that the new WebRTC will use at least one of a PDU set having a PDU set marking or an EoB having an EoB marking.

[0086]

[0086] Clause 20: The method of Clause 7, further comprising receiving confirmation of a Quality of Service (QoS) allocation.

[0087]

[0087] Clause 21: The method of Clause 20, further comprising transferring an answer to a user device (UE).

[0088]

[0088] Clause 22: A device for extracting media data, comprising one or more means for performing any of the methods of Clauses 1 to 21.

[0089]

[0089] Clause 23: The device of Clause 22, wherein one or more means include a memory and a processor implemented within a circuit mechanism.

[0090]

[0090] Clause 24: The device according to Clause 22, wherein the device includes at least one of an integrated circuit, a microprocessor, or a wireless communication device.

[0091]

[0091] Clause 25: A computer-readable storage medium in which instructions are stored, wherein when the instructions are executed, the instructions cause a processor to execute any of the methods of Clauses 1 to 21.

[0092]

[0092] Clause 26: A device for retrieving media data, comprising means for communicating configuration information representing at least one of protocol data unit (PDU) set markings or burst end (EoB) markings, and means for processing at least one of a PDU set having PDU set markings according to the configuration information or an EoB having EoB markings according to the configuration information.

[0093]

[0093] Clause 27: A method for communicating media data, comprising: receiving a session description protocol (SDP) message which includes configuration information relating to a communication session which represents at least one of a protocol data unit (PDU) set marking or an end-of-burst (EoB) marking; transmitting information relating to a communication session which represents a PDU set marking or an EoB marking to a real-time communication (RTC) application function; and processing media data of a communication session which the media data includes at least one of a PDU set having a PDU set marking or an EoB having an EoB marking according to the configuration information.

[0094]

[0094] Clause 28: The method of Clause 27, wherein the SDP message includes a MediaSubComponent data type containing configuration information.

[0095]

[0095] Clause 29: The method of Clause 27, wherein the configuration information includes at least one of the following: a version value, a local identifier value, a format value, a value indicating whether the PDU set size is active or not, or a value indicating whether EoB marking is active or not.

[0096]

[0096] Clause 30: The method of Clause 27, wherein the communication session includes a Web Real-Time Communication (WebRTC) session, the configuration information includes a list of Quality of Service (QoS) allocation specifications for the WebRTC session, and each QoS allocation specification includes a name value, a flow description, and a QoS allocation value.

[0097]

[0097] Clause 31: The method of Clause 27, wherein receiving an SDP message includes receiving an SDP message by a Web Real-Time Communication (WebRTC) signaling server or a Proxy Call Session Control Function (P-CSCF).

[0098]

[0098] Clause 32: The method of Clause 31, wherein an SDP message includes an SDP answer to an SDP request relating to a new WebRTC session.

[0099]

[0099] Clause 33: The method of Clause 32, further comprising transferring an SDP answer to a user device (UE).

[0100]

[0100] Clause 34: The method of Clause 32, which includes transmitting information representing a PDU set marking or an EoB marking to an RTC application function, wherein the information indicates that the new WebRTC will use at least one of a PDU set having a PDU set marking or an EoB having an EoB marking.

[0101]

[0101] Clause 35: The method of Clause 31, further comprising receiving confirmation of Quality of Service (QoS) allocation.

[0102]

[0102] Clause 36: A device for communicating media data, comprising a memory configured to store media data and a processing system comprising one or more processors implemented within a circuit mechanism, wherein the processing system is configured to receive a session description protocol (SDP) message, which includes configuration information representing at least one of protocol data unit (PDU) set markings or burst end (EoB) markings relating to a communication session, transmit information representing PDU set markings or EoB markings relating to a communication session to a real-time communication (RTC) application function, and process media data of a communication session, wherein the media data includes at least one of PDU sets having PDU set markings or EoBs having EoB markings according to the configuration information.

[0103]

[0103] Clause 37: A device under Clause 27 whose SDP message includes a MediaSubComponent data type containing configuration information.

[0104]

[0104] Clause 38: A device according to Clause 36, wherein the configuration information includes at least one of the following: a version value, a local identifier value, a format value, a value indicating whether the PDU set size is active or not, or a value indicating whether EoB marking is active or not.

[0105]

[0105] Clause 39: A device of Clause 36, wherein the communication session includes a Web Real-Time Communication (WebRTC) session, and the configuration information includes a list of Quality of Service (QoS) allocation specifications for WebRTC sessions, each of which includes a name value, a flow description, and a QoS allocation value.

[0106]

[0106] Clause 40: A device under Clause 36, which receives SDP messages, including receiving SDP messages by a Web Real-Time Communication (WebRTC) signaling server or a Proxy Call Session Control Function (P-CSCF).

[0107]

[0107] Clause 41: A device under Clause 40 whose SDP message includes an SDP answer to an SDP request relating to a new WebRTC session.

[0108]

[0108] Clause 42: The device of Clause 41, wherein the processing system is further configured to transfer SDP answers to a user device (UE).

[0109]

[0109] Clause 43: A device under Clause 41, wherein the processing system is configured to transmit information indicating a new WebRTC session to an RTC application function in order to transmit information representing a PDU set marking or an EoB marking, the information indicating that the new WebRTC will use at least one of a PDU set having a PDU set marking or an EoB having an EoB marking.

[0110]

[0110] Clause 44: The device of Clause 40, further configured to receive Quality of Service (QoS) allocation confirmations.

[0111]

[0111] Clause 45: A device for communicating media data, comprising: means for receiving a session description protocol (SDP) message, which includes configuration information representing at least one of protocol data unit (PDU) set markings or burst end (EoB) markings relating to a communication session; means for transmitting information representing PDU set markings or EoB markings relating to a communication session to a real-time communication (RTC) application function; and means for processing media data of a communication session, wherein the media data includes at least one of PDU sets having PDU set markings or EoBs having EoB markings according to the configuration information.

[0112]

[0112] Clause 46: The device of Clause 45, further comprising means for running a Web Real-Time Communication (WebRTC) signaling server or means for running a Proxy Call Session Control Function (P-CSCF).

[0113]

[0113] In one or more embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted via computer-readable media as one or more instructions or codes and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media corresponding to tangible media such as data storage media, or communication media including any media that facilitates the transfer of computer programs from one place to another according to a communication protocol, for example. Thus, computer-readable media may generally correspond to (1) non-temporary tangible computer-readable storage media, or (2) communication media such as signals or carrier waves. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementation of the technology described herein. Computer program products may include computer-readable media.

[0114]

[0114] For example, but not limited to, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other media that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then those coaxial cables, fiber optic cables, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals, or other temporary media, but instead refer to non-temporary tangible storage media. As used herein, "disk" and "disc" include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where a "disk" typically reproduces data magnetically, while a "disc" reproduces data optically using a laser. Any combination of the above should also be included within the scope of computer-readable media.

[0115]

[0115] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuit mechanisms. Thus, the term “processor” as used herein may refer to any of the above-described structures or any other structure suitable for implementing the technologies described herein. Furthermore, in some embodiments, the functionality described herein can be provided in dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a composite codec. It is also possible to fully implement these technologies in one or more circuits or logic elements.

[0116]

[0116] The technology of the disclosed herein can be implemented in a wide variety of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chipsets). Various components, modules, or units are described herein to highlight the functional aspects of devices configured to perform the disclosed technology, but these do not necessarily require implementation by different hardware units. Rather, as described above, various units can be combined within a codec hardware unit, or they can be provided by a collection of interoperable hardware units, including one or more processors as described above, in conjunction with suitable software and / or firmware.

[0117]

[0117] Various embodiments have been described. These embodiments and other embodiments are within the scope of the following claims.

Claims

1. A method for communicating media data, Receiving a Session Description Protocol (SDP) message that includes configuration information representing at least one of a protocol data unit (PDU) set marking or burst end (EoB) marking relating to a communication session, The media application function transmits information representing the PDU set marking or the EoB marking related to the aforementioned communication session. A method for processing media data of the communication session, wherein the media data includes at least one of the PDU set marking or the EoB marking according to the configuration information.

2. The method according to claim 1, wherein the SDP message includes a Real-Time Transport Protocol (RTP) session message containing the configuration information.

3. The method according to claim 1, wherein the configuration information includes at least one of a version value, a local identifier value, a format value, a value indicating whether the PDU set size is active or not, or a value indicating whether EoB marking is active or not.

4. The method according to claim 1, wherein the communication session includes a web real-time communication (WebRTC) session, the configuration information includes a list of quality of service (QoS) allocation specifications for the WebRTC session, and each of the QoS allocation specifications includes a name value, a flow description, and a QoS allocation value.

5. The method according to claim 1, wherein receiving the SDP message includes receiving the SDP message by a web real-time communication (WebRTC) signaling server or a proxy call session control function (P-CSCF).

6. The method according to claim 5, wherein the SDP message includes an SDP answer to an SDP request relating to a new WebRTC session.

7. The method according to claim 6, further comprising transferring the SDP answer to a user device (UE).

8. The method according to claim 6, wherein transmitting the information representing the PDU set marking or the EoB marking includes extracting information indicating the new WebRTC session from the SDP message and transmitting the information to the media application function so that the media application function transfers the information to a policy control function (PCF), wherein the information indicates that the new WebRTC session will use at least one of the PDU set marking or the EoB marking.

9. The method according to claim 5, further comprising receiving confirmation of a Quality of Service (QoS) allocation.

10. A device for communicating media data, Memory configured to store media data, A processing system comprising one or more processors implemented within a circuit mechanism, wherein the processing system A session description protocol (SDP) message is received, which includes configuration information representing at least one of a protocol data unit (PDU) set marking or burst end (EoB) marking relating to a communication session. Information representing the PDU set marking or the EoB marking related to the aforementioned communication session is transmitted to the media application function. A device configured to process media data of the aforementioned communication session, wherein the media data includes at least one of the PDU set marking or the EoB marking, according to the configuration information.

11. The device according to claim 10, wherein the SDP message includes a Real-Time Transport Protocol (RTP) session message containing the configuration information.

12. The device according to claim 10, wherein the configuration information includes at least one of a version value, a local identifier value, a format value, a value indicating whether the PDU set size is active or not, or a value indicating whether EoB marking is active or not.

13. The device according to claim 10, wherein the communication session includes a web real-time communication (WebRTC) session, the configuration information includes a list of quality of service (QoS) allocation specifications for the WebRTC session, and each of the QoS allocation specifications includes a name value, a flow description, and a QoS allocation value.

14. The device according to claim 10, wherein receiving the SDP message includes receiving the SDP message by a web real-time communication (WebRTC) signaling server or a proxy call session control function (P-CSCF).

15. The device according to claim 14, wherein the SDP message includes an SDP answer to an SDP request relating to a new WebRTC session.

16. The device according to claim 15, wherein the processing system is further configured to transfer the SDP answer to a user device (UE).

17. The device according to claim 15, wherein, in order to transmit the information representing the PDU set marking or the EoB marking, the processing system is configured to extract information indicating the new WebRTC session from the SDP message and transmit the information to the media application function so that the media application function transfers the information to the policy control function (PCF), the information indicating that the new WebRTC session will use at least one of the PDU set marking or the EoB marking.

18. The device according to claim 14, wherein the processing system is further configured to receive confirmation of quality of service (QoS) allocation.

19. A device for communicating media data, Means for receiving a Session Description Protocol (SDP) message, which includes configuration information representing at least one of a protocol data unit (PDU) set marking or a burst end (EoB) marking relating to a communication session; Means for transmitting information representing the PDU set marking or the EoB marking related to the communication session to the media application function, A device comprising means for processing media data of the communication session, wherein the media data includes at least one of the PDU set marking or the EoB marking according to the configuration information.

20. The device according to claim 19, further comprising means for executing a web real-time communication (WebRTC) signaling server, or means for executing a proxy call session control function (P-CSCF).

Images