Methods and devices for media data communication

Description

1 / 41 “METHODS AND DEVICES FOR MEDIA DATA COMMUNICATION

[0001] This application is a US utility priority PCT of US patent application No. 18 / 629,617, filed April 8, 2024, which claims the benefit of US provisional application No. 63 / 495,201, filed April 10, 2023, the contents of each of which are incorporated herein by reference in their entirety. TECHNICAL FIELD

[0002] This disclosure relates to the storage and transport of encoded video data. BACKGROUND

[0003] Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, digital cameras, digital recording devices, digital media players, video game consoles, cellular or satellite radiotelephones, video conferencing devices, and the like. Digital video devices implement video compression techniques, such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263 or ITU-T H.264 / MPEG-4 Part 10, advanced video coding (AVC), ITU-T H.265 (also called high-efficiency video coding (HEVC)), and extensions of such standards, to more efficiently transmit and receive digital video information.

[0004] After the video data has been Petition 870250089258, dated 10 / 01 / 2025, page 8 / 104 2 / 41 encoded, video data can be packaged for transmission or storage. Video data can be assembled into a video file conforming to any of several standards, such as the International Organization for Standardization (ISO) base media file format and extensions thereof, such as AVC. SUMMARY

[0005] In general, this disclosure describes techniques relating to the communication (e.g., sending, receiving, or forwarding) of web real-time communication (WebRTC) data. WebRTC data may include extended reality (XR) media data, which may include any or all of the following: text data, audio data, video data, mixed reality (MR) data, augmented reality (AR) data, and / or virtual reality (VR) data. WebRTC data may be partitioned and encapsulated into protocol data units (PDUs), which may be communicated in bursts of activity over radio signals. Similarly, PDUs may be organized into PDU sets, which may include a set of PDUs to be consumed together by a receiver. For example, a PDU set may include respective PDUs containing audio, video, and XR data.Furthermore, PDU sets and burst ends (EoBs) can be tagged to help identify XR traffic and optimize its delivery. According to the techniques of this disclosure, various devices involved in XR data communication and data... Petition 870250089258, dated 10 / 01 / 2025, p. 9 / 104 3 / 41 of WebRTC can indicate whether one or both of the PDU set marking and / or EoB marking are enabled for a particular WebRTC session. In this way, network devices can detect XR data and WebRTC data and apply quality of service (QoS) policies to such data appropriately.

[0006] In one example, a media data communication method includes: receiving a session description protocol (SDP) message including configuration information representing at least one of the protocol data unit (PDU) set marking or the end-of-burst (EoB) marking for a communication session; sending information representing the PDU set marking or the EoB marking for the communication session to a real-time communication (RTC) application function; and processing media data from the communication session, the media data including at least one of a PDU set having the PDU set marking or an EoB having the EoB marking according to the configuration information.

[0007] In another example, a device for media data communication includes: a memory configured to store media data; and a processing system comprising one or more processors implemented in a circuit array, the processing system being configured to: receive a Session Description Protocol (SDP) message including configuration information representing at least one of the Protocol Data Unit (PDU) set marking or the End-of-Burst Marking (EoB) for a session of Petition 870250089258, dated 10 / 01 / 2025, page 10 / 104 4 / 41 communication; send information representing the PDU set marking or the EoB marking for the communication session to a real-time communication (RTC) application function; and process media data from the communication session, the media data including at least one PDU set having the PDU set marking or an EoB having the EoB marking according to the configuration information.

[0008] In another example, a computer-readable storage medium stored instructions on it that, when executed, cause a processor to: receive a Session Description Protocol (SDP) message including configuration information representing at least one of the Protocol Data Unit (PDU) set marking or the End of Burst (EoB) marking for a communication session; send information representing the PDU set marking or the EoB marking for the communication session to a Real-Time Communication (RTC) application function; and process media data from the communication session, the media data including at least one of a PDU set having the PDU set marking or an EoB having the EoB marking according to the configuration information.

[0009] In another example, a device for media data communication includes: means for receiving a Session Description Protocol (SDP) message including configuration information representing at least one of the Protocol Data Unit (PDU) set marking or the End-of-Burst (EoB) marking for a communication session; means for sending information Petition 870250089258, dated 10 / 01 / 2025, page 11 / 104 5 / 41 representing the PDU set marking or the EoB marking for the communication session for a real-time communication (RTC) application function; and means to process media data from the communication session, the media data including at least one of a PDU set having the PDU set marking or an EoB having the EoB marking according to the configuration information.

[0010] Details of one or more examples are presented in the attached drawings and in the description below. Other attributes, objectives, and advantages will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram illustrating an architecture for a system that can be configured to perform immersive real-time communication for real-time web communication (WebRTC) according to techniques in this disclosure.

[0012] Figure 2 is a block diagram illustrating elements of a sample video file.

[0013] Figure 3 is a flow diagram illustrating an example method for using Protocol Data Unit (PDU) and End of Burst (EoB) Set Marking in accordance with the techniques of this disclosure.

[0014] Figure 4 is a flowchart illustrating an example of an XR media data communication method according to the techniques of this disclosure. DETAILED DESCRIPTION

[0015] In general, this disclosure describes techniques related to media data communication, such as extended reality (XR) media data. XR media data Petition 870250089258, dated 10 / 01 / 2025, page 12 / 104 6 / 41 can include any or all of the following: text data, voice data, audio data, still image data, video data, mixed reality (MR) data, augmented reality (AR) data, and / or virtual reality (VR) data. XR traffic tagging is a mechanism that helps the network identify XR traffic and optimize its delivery. The concept of Protocol Data Unit (PDU) sets was introduced specifically for this purpose, but can also be used for other types of traffic. PDU sets are PDUs that are consumed together by the receiver and, as such, must be treated together by the network. Burst End (EoB) provides another tool to optimize XR traffic delivery by enabling the proper use of connected mode discontinuous reception (CDRX) to save power on the receiver side.

[0016] PDU set marking can be performed for real-time transport protocol (RTP) or secure RTP (SRTP) traffic through the use of an RTP header extension that is appended to an RTP set packet header of each PDU in an RTP stream that has PDU set marking enabled. A user plane function (UPF) can inspect uplink traffic and extract information about the PDU set marking and pass the PDU set to a base station, such as a gNode-B (gNB). This disclosure describes techniques related to configuration signaling for PDU set marking with a policy control function (PCF), which in turn can configure Petition 870250089258, dated 10 / 01 / 2025, p. 13 / 104 7 / 41 at UPF.

[0017] In general, a user equipment (UE) and an application server (AS) or other remote UE can initially negotiate the use of PDU sets and EoB marking during an offer / response exchange during session establishment or during an update, for example, through a session initiation protocol (SIP) re-invitation. According to RFC8285, the negotiation of used RTP header extensions is performed by including an extmap attribute. The uniform resource name (URN) for PDU set marking can be defined as urn:3gpp: pdus-marking:rel-18.

[0018] The following options are supported for an RTP stream and apply to RTP packets within the RTP stream, throughout the entire RTP session lifecycle: • One-byte or two-byte header extension format identified using the corresponding short or long extension attribute. If not present, the header extension format should be deduced from the preamble bytes 0xBEDE or 0x100 (+appbits). In either case, the application server must not change the format during the RTP session. • The size of the PDU set in bytes is identified by the presence of the pdu-set-size string flag. If it is not present, the receiver should assume that the PDU set size field is not present. Petition 870250089258, dated 10 / 01 / 2025, p. 14 / 104 8 / 41 This results in a shorter header extension for that RTP session. • End-of-burst marking identified by the presence of the end-of-burst string flag. When not present, the receiver should ignore the EoB bits.

[0019] The augmented Backus-Naur form (ABNF) syntax for the extmap attribute, according to RFC8285, is: extmap-attr=a=extmap:1*5DIGIT[ / direction] Figure 1 is a block diagram illustrating a 100 architecture for a system that can be configured to perform immersive real-time communication for web real-time communication (WebRTC) according to the techniques of this disclosure. In particular, the 100 architecture can be used for 5G media streaming (5GMS) using WebRTC. That is, the 100 architecture can be used to perform real-time communication in WebRTC over a 5G network connection.

[0021] Architecture 100 can be used to provide WebRTC in a variety of scenarios. As one example, architecture 100 can be used in conjunction with a 5G network to provide WebRTC over the top (OTT). As another example, a mobile network operator (MNO) can provide reliable WebRTC functions and / or services. Petition 870250089258, dated 10 / 01 / 2025, page 15 / 104 9 / 41 WebRTC in installations using the 100 architecture. As another example, the 100 architecture can provide interoperable WebRTC services. The 100 architecture can also be used for several other scenarios as well. The 100 architecture provides flexibility through a set of functions and interfaces that can be combined in different ways based on the needs of a particular scenario.

[0022] In the example in Figure 1, architecture 100 includes 5G RTC application provider 102, 5G RTC application functions 104, and user equipment (UE) 150. In general, 5G RTC application provider 102 interacts with functions of 5G RTC application functions 104 and provides a 5G RTC-aware application, such as web application 152, to user equipment 150.

[0023] User equipment 150 may also be called UE or a client device. User equipment 150 may be, for example, a laptop or desktop computer, a digital camera, a digital recording device, a digital media player, a video game console, a cellular or satellite radiotelephone, a video conferencing device, or similar devices. In this example, user equipment 150 includes web application 152, native WebRTC application 154, and media session handler (MSH) 158. Interface 156 couples native WebRTC application 154 and MSH 158. Interface 156 may be referred to as an RTC-6 interface. UE 150 and 5G RTC application provider 102 are coupled by interface 174, which may be referred to as an RTC-8 interface. Petition 870250089258, dated 10 / 01 / 2025, page 16 / 104 10 / 41

[0024] MSH 158 is a function in UE 150 that provides WebRTC applications, such as web application 152, and access to 5G RTC support functions, such as 5G RTC application functions 104. These functions can be offered upon request through interface 156 (the RTC-6 interface) or transparently without direct involvement of web application 154. MSH 158 can, for example, indirectly assist in interactive connectivity establishment (ICE) negotiation by providing a list of candidate session traversal utilities for network address translation (STUN) and / or traversal using relay around NAT (TURN) servers that offer 5G RTC functionality. MSH 158 can also collect Quality of Experience (QoE) metric reports and present consumption reports.MSH 158 can also offer media configuration recommendations to the web application 152 via interface 156 (RTC-6).

[0025] Interface 170 (which may be referred to as an RTC-1 interface) allows the 5G RTC 102 application provider to provide support for RTC sessions offered as 5G RTC 104 application functions. Provisioning may cover functionalities including Quality of Service (QoS) for WebRTC sessions, billing provisioning for WebRTC sessions, collection of consumption data and QoE metrics related to WebRTC sessions, offering ICE functionality such as STUN and TURN servers, and / or offering WebRTC signaling servers, potentially with interoperability with other servers. Petition 870250089258, dated 10 / 01 / 2025, page 17 / 104 11 / 41 of signaling.

[0026] In this example, the 5G RTC 104 application functions include 5G RTC 110 support application function (AF), 5G RTC 112 configuration AF, 5G RTC 114 provisioning AF, 5G RTC 116 data channel AF, 5G RTC 118 signaling server AF, 5G RTC 120 interoperability AF, 5G RTC 122 STUN AF, and 5G RTC 124 TURN AF. In this example, the 5G RTC 104 application functions are also interoperable with the policy and billing function (PCF) 160, the network exposure function (NEF) 162, and the session management function (SMF) 164.

[0027] Interface 170, which can be called a provisioning interface, is not necessarily relevant for all collaboration scenarios, and some of the 5G support functionalities can be offered without application provider provisioning.

[0028] Interface 172 (which may be referred to as an RTC-5 interface) is an interface between MSH 158 and the 5G RTC 104 application functions. Interface 172 can be used to transport configuration information from the 5G RTC 104 application functions to MSH 158 and to request support for a WebRTC session initiating / in progress. Configuration information may include static information such as recommendations for media settings, STUN and TURN server location settings, configuration on consumption and QoE reports, or discovery information for WebRTC signaling and data channel servers and their capabilities. Petition 870250089258, dated 10 / 01 / 2025, page 18 / 104 12 / 41

[0029] MSH 158 can provide supporting functionality, such as informing 5G RTC application functions 104 or web application 152 about a WebRTC session and its status, requesting QoS allocation for a starting or modified WebRTC session, receiving notification about changes in QoS allocation for an ongoing WebRTC session, or receiving, updating, or exchanging information about the WebRTC session with the 5G RTC STUN / TURN / signaling server, for example, to identify a WebRTC session and associate it with a QoS template.

[0030] In some instances, 5G functionality that provides application functions for the WebRTC application (including 5G RTC 116 data channel AF, 5G RTC 118 signaling server AF, 5G RTC 120 interoperability AF, 5G RTC 122 STUN AF, and 5G RTC 124 TURN AF) may instead be provided by application servers (5G RTC AS) rather than AFs. The 5G RTC AS could then use a dedicated RTC-3 interface to request network configuration and support for ongoing WebRTC sessions from the 5G RTC AF.

[0031] The functionality assigned to the 5G RTC 102 application provider, the 5G RTC 104 application functions, and the UE 150 can be implemented in hardware, software, firmware, or any combination thereof. When implemented in software or firmware, memory can be provided to store instructions that can be executed by one or more processors implemented in a circuit array. The processors may include one or more microprocessors, digital signal processors (DSPs). Petition 870250089258, dated 10 / 01 / 2025, page 19 / 104 13 / 41 - digital signal processors), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic circuits or combinations thereof.

[0032] According to the techniques of this disclosure, one or more of the 5G RTC 104 application functions (e.g., 5G RTC AF 118 signaling server) can be configured to use an Npcf_PolicyAuthorization procedure (according to TS 29.514) or an Nnef_AFSessionWithQoS N33 procedure (according to TS 29.122) to request Quality of Service (QoS) allocation from PCF 160 or NEF 162 for a communication session between UE 150 and a remote UE or application server (not shown in Figure 1). These methods can be extended to add support for PDU set marking and EoB signaling for PCF 160 or NEF 162, according to the techniques of this disclosure. For example, a MediaSubComponent data type, according to TS 29.514, can be extended as follows, where <adicionado>text added< / adicionado> This means added text in relation to the MediaSubComponent data type of TS 29.514: Table 1 Petition 870250089258, dated 10 / 01 / 2025, page 20 / 104 14 / 41 Attribute Name Data Type P Cardinality Description Applicability afSigProtocol AfSigProtocol l O 0..1 Indicates the protocol used for signaling between the UE and the NF service consumer. It can only be included if the flowUsage attribute is set to the value AF_SIGNALLING. ProvAFsignal Flow ethfDescs array(EthFlowDescription) O 1..2 Contains the flow description for uplink and / or downlink Ethernet flows. fNum integer M Identifies the ordinal number of the service data flow. fDescs array(FlowDescription) O 1..2 Contains the flow description for uplink and / or downlink IP flows. Petition 870250089258, dated 10 / 01 / 2025, page 21 / 104 15 / 41 Attribute Name Data Type P Cardinality Description Applicability fStatus FlowStatus O 0..1 Indicates whether the status of service data flows is enabled or disabled. flowUsage FlowUsage O 0..1 Flow usage of flows (e.g., RTCP signaling, AF). marBwUl BitRate O 0..1 Maximum bandwidth requested for the uplink. marBwDl BitRate ..1 Maximum bandwidth requested for the downlink. tosTrCl TosTrafficCl ass O 0..1 Type of service or traffic class. <adicionada> pdu SetMarking PDUSetMarking 0..1 Configuration information for PDU set marking and burst end marking.< / adicionada>

[0033] In other words, in this example, the row Petition 870250089258, dated 10 / 01 / 2025, page 22 / 104 16 / 41 pduSetMarking has been added and includes configuration information for PDU and EoB set marking, in accordance with the techniques of this disclosure.

[0034] The PDUSetMarking data type can be defined as follows: Table 2 Attribute Name Data Type Cardinality Description Applicability Version Integer O 0..1 localIdentifier Integer M 0..1 format Boolean O 0..1 pduSetSizeActive Boolean O 0..1 eobMarkingActive Boolean O 0..1

[0035] Alternatively, or as an additional option, MSH 158 can be configured to pass session information, including session media components and the PDU and EoB tagging configuration for the session. The dynamic policy information configured by MSH 158 for the session can contain the following information: Table 3 Attribute Name Data Type Cardinality Description PolicyTemplate Id ResourceId O 0..1 Identifier of the provisioned policy template Petition 870250089258, dated 10 / 01 / 2025, page 23 / 104 17 / 41 Attribute Name Data Type Cardinality Description mediaComponentQoS Arrangement (MediaComponentQoS) M 0..n A list of QoS allocation specifications for the media sessions of the WebRTC session.

[0036] The MediaComponentQoS object can contain the following information: Table 4 Attribute Name Data Type Cardinality Description String Name M 1 Unique identifier of this media component in the session flowDescription FlowDescription M 1 The flow description, provided as a 5-Tuple, for the media component. qosAllocation M5 QoSSpecification M 1 The QoS description as described in 3GPP TS 26.512, which applies to the flow with the given flow description.

[0037] A QoS policy template can be extended to include a name for each subcomponent of the session. This name can then be used to associate the actual media stream with the QoS subcomponent policy. Any or all of the various RTC AFs, as shown in Figure 1, can then use this mapping to associate a network assistance request for each media stream. Petition 870250089258, dated 10 / 01 / 2025, page 24 / 104 18 / 41 to the corresponding subcomponent of the QoS policy. RTC AFs can also use this information to verify whether the requested / desired QoS for each component is aligned with the QoS policy provisioned by the 5G RTC 102 application provider, which may also be called an application service provider (ASP).

[0038] Thus, according to the techniques of this disclosure, the RTC 5G AF 118 signaling server can receive configuration data for an XR communication session (conducted via WebRTC) from the RTC 5G 102 application provider. It is generally desirable that real-time media communication sessions, such as XR communication sessions, transmit data between endpoints (e.g., UEs) with low latency, to ensure that participants in the XR communication session can experience events in the XR communication session (e.g., other participant movements and interactions with a virtual environment, the participant's own interactions with the virtual environment, or similar) in very near real-time. Thus, the XR communication session can request a high level of Quality of Service (QoS).

[0039] The configuration data may include one or both of the Protocol Data Unit (PDU) set marking and / or the End-of-Burst (EoB) marking for the XR communication session. Such PDU set marking or EoB marking may be used to associate traffic with a requested QoS. The 5G RTC AF 118 signaling server may send, to one or more other RTC AF 104s, indicative data of the PDU set marking and / or the EoB marking for the XR communication session. One or more of the Petition 870250089258, dated 10 / 01 / 2025, p. 25 / 104 19 / 41 RTC AFs 104 can also interact with PCF 160 to negotiate the requested QoS for the XR communication session. In this way, network devices (e.g., base stations such as gNBs) among the participants involved in the XR communication session can examine the PDU set markings and / or EoB markings for data transmitted as part of the XR communication session and determine that such markings are associated with the negotiated QoS level and, therefore, prioritize the transmission of the XR communication session data to satisfy the QoS level. Such PDU set markings and / or EoB markings can be placed in RTP header extensions of RTP packets including the XR communication session data.

[0040] Figure 2 is a block diagram illustrating elements of an example 250 video file. Video files, according to the ISO base media file format and their extensions, store data in a series of objects, called boxes. In the example in Figure 2, video file 250 includes file type box (FTYP) 252, movie box (MOOV) 254, segment index boxes (sidx) 262, movie fragment box (MOOF) 264, and movie fragment random access box (MFRA) 266. Although Figure 2 represents an example of a video file, it should be understood that other media files may include other types of media data (e.g., audio data, timed text data, or similar) that are structured similarly to the data in video file 250, according to the ISO base media file format and its extensions.

[0041] File type box (FTYP) 252 Petition 870250089258, dated 10 / 01 / 2025, page 26 / 104 20 / 41 generally describes a file type for video file 250. File type box 252 may include data identifying a specification describing a best use for video file 250. File type box 252 may alternatively be placed before MOOV box 254, film fragment boxes 264, and / or MFRA box 266.

[0042] The MOOV box 254, in the example in Figure 2, includes the movie header (MVHD) box 256, the track (TRAK) box 258, and one or more movie extends (MVEX) boxes 260. In general, the MVHD box 256 can describe general characteristics of the video file 250. For example, the MVHD box 256 can include data describing when the video file 250 was originally created, when the video file 250 was last modified, a timescale for the video file 250, a playback duration for the video file 250, or other data that generally describes the video file 250.

[0043] The TRAK box 258 may include data for a video file track 250. The TRAK box 258 may include a track header box (TKHD) that describes characteristics of the track corresponding to the TRAK box 258. In some examples, the TRAK box 258 may include video-encoded images, while in other examples, the video-encoded images of the track may be included in the movie fragments 264, which may be referenced by TRAK box 258 data and / or sidx boxes 262.

[0044] In some examples, the video file 250 Petition 870250089258, dated 10 / 01 / 2025, page 27 / 104 21 / 41 can include more than one track. Consequently, MOOV box 254 can include multiple TRAK boxes equal to the number of tracks in video file 250. TRAK box 258 can describe characteristics of a corresponding track from video file 250. For example, TRAK box 258 can describe temporal and / or spatial information for the corresponding track. A TRAK box similar to TRAK box 258 of MOOV box 254 can describe characteristics of a parameter set track when encapsulation unit 30 (Figure 1) includes a parameter set track in a video file, such as video file 250. Encapsulation unit 30 can signal the presence of sequence-level SEI messages in the parameter set track in the TRAK box describing the parameter set track.

[0045] MVEX boxes 260 can describe characteristics of the corresponding film fragments 264, for example, to signal that video file 250 includes film fragments 264, in addition to the video data included in MOOV box 254, if any. In the context of streaming video data, encoded video images can be included in film fragments 264, rather than in MOOV box 254. Consequently, all encoded video samples can be included in film fragments 264, rather than in MOOV box 254.

[0046] The MOOV 254 box may contain multiple MVEX 260 boxes equal to the number of film fragments 264 in the video file 250. Each of the MVEX 260 boxes may describe characteristics of a fragment. Petition 870250089258, dated 10 / 01 / 2025, page 28 / 104 22 / 41 corresponding to the 264 film fragments. For example, each MVEX box may include a movie extends header box (MEHD) that describes a time duration for the corresponding fragment of the 264 film fragments.

[0047] As mentioned above, encapsulation unit 30 can store a sequence data set in a video sample that does not include actual encoded video data. A video sample can generally correspond to an access unit, which is a representation of an encoded image at a specific time instance. In the context of AVC, the encoded image includes one or more VCL NAL units, which contain the information to construct all the pixels of the access unit and other associated non-VCL NAL units, such as SEI messages. Consequently, encapsulation unit 30 can include a sequence data set, which may include sequence-level SEI messages, in one of the film fragments 264.The encapsulation unit 30 can additionally signal the presence of a sequence data set and / or sequence-level SEI messages as present in one of the film fragments 264 in one of the MVEX boxes 260 corresponding to one of the film fragments 264.

[0048] SIDX 262 boxes are optional elements of the 250 video file. That is, video files that conform to the 3GPP file format, or other file formats, do not necessarily include SIDX 262 boxes. Following the example of the 3GPP file format, a SIDX box can be used for Petition 870250089258, dated 10 / 01 / 2025, page 29 / 104 23 / 41 identify a subsegment of a segment (e.g., a segment contained in video file 250). The 3GPP file format defines a subsegment as a self-contained set of one or more consecutive movie fragment boxes with corresponding media data box(es), and a media data box containing data referenced by a movie fragment box must follow the movie fragment box and precede the subsequent movie fragment box containing information about the same track. The 3GPP file format also indicates that a SIDX box contains a sequence of references to subsegments of the (sub)segment documented by the box. The referenced subsegments are contiguous at presentation time. Similarly, the bytes called from a segment index box are always contiguous within the segment. The referenced size gives the count of the number of bytes in the referenced material.

[0049] The SIDX boxes 262 generally provide representative information for one or more subsegments of a segment included in the video file 250. For example, such information may include playback times at which the subsegments begin and / or end, byte offsets for the subsegments, whether the subsegments include (e.g., begin with) a stream access point (SAP), a type for the SAP (e.g., whether the SAP is an instantaneous decoder refresh (IDR) image, a clean random access (CRA) image, a broken link access (BLA) image, or similar), a position of the SAP (in Petition 870250089258, dated 10 / 01 / 2025, page 30 / 104 24 / 41 playback time terms and / or byte offset) in the subsegment and similar.

[0050] Film fragments 264 may include one or more encoded video images. In some examples, film fragments 264 may include one or more groups of pictures (GOPs), where each of these may include multiple encoded video images, for example, frames or images. Additionally, as described above, film fragments 264 may include sequence datasets in some examples. Each film fragment 264 may include a Film Fragment Header Box (MFHD, not shown in Figure 2). The MFHD box may describe characteristics of the corresponding film fragment, such as a sequence number for the film fragment. Film fragments 264 may be included in sequence number order in the video file 250.

[0051] The MFRA 266 box can describe random access points in the 264 movie fragments of the 250 video file. This can help perform trick modes, such as searching for particular temporal locations (i.e., playback times) in a segment encapsulated by the 250 video file. The MFRA 266 box is generally optional and does not need to be included in video files in some instances. Similarly, a client device, such as client device 40, does not necessarily need a reference to the MFRA 266 box to correctly decode and display the video data from the 250 video file. The MFRA 266 box can include multiple track fragment random access (TFRA) boxes (not shown) equal to the number of tracks. Petition 870250089258, dated 10 / 01 / 2025, page 31 / 104 25 / 41 of the 250 video file or, in some examples, equal to the number of media tracks (e.g., tracks without cues) in the 250 video file.

[0052] In some examples, film fragments 264 may include one or more stream access points (SAPs), such as IDR images. Similarly, MFRA box 266 may provide indications of the locations of SAPs in video file 250. Consequently, a temporal subsequence of video file 250 may be formed from SAPs in video file 250. The temporal subsequence may also include other images, such as P-frames and / or B-frames that depend on SAPs. The frames and / or slices of the temporal subsequence may be arranged in the segments so that frames / slices of the temporal subsequence that depend on other frames / slices of the subsequence can be properly decoded. For example, in hierarchical data arrangement, data used for predicting other data may also be included in the temporal subsequence.

[0053] Figure 3 is a flow diagram illustrating an example method for using PDU and EoB set marking in accordance with the techniques of this disclosure. UE1 in Figure 3 may correspond to UE150 in Figure 1. An intermediary server, such as a trusted WebRTC signaling server or a proxy-call session control function (P-CSCF), can be configured to inspect a Session Description Protocol (SDP) offer / response and extract information related to PDU and end-of-burst set marking. The WebRTC or P-CSCF signaling server of Petition 870250089258, dated 10 / 01 / 2025, page 32 / 104 26 / 41 Figure 3 may correspond to the 5G RTC AF 118 signaling server from Figure 1. The RTC AF in Figure 3 may correspond to one or more of the other RTC AFs 104 from Figure 1. The policy and charging function (PCF) in Figure 3 may correspond to PCF 160 from Figure 1. The application server in Figure 3 may correspond to the 5G application provider RTC 102 from Figure 1.

[0054] Initially, in the example in Figure 3, UE1 sends an SDP offer to establish an XR session to the WebRTC or P-CSCF signaling server, and the signaling server forwards the request to the application server (300).

[0055] The application server then responds with an SDP response (302). That is, the application server sends an SDP response to the signaling server. The application server includes an indication of the PDU set marking and / or the burst end marking in the SDP response, as discussed above. For example, the application server might signal a MediaSubComponent data element as discussed above, which might include data representing whether the PDU set marking and / or the eOb marking is active for the XR communication session.

[0056] The signaling server inspects the SDP response and extracts information related to PDU set size and burst end marking (304). For example, the signaling server can extract data indicating whether PDU set size and / or EoB marking are active for the corresponding XR communication session, for example, according to the element of Petition 870250089258, dated 10 / 01 / 2025, page 33 / 104 27 / 41 data from the PDU set marking discussed above.

[0057] The signaling server can then inform the RTC AF(s) about the new IMS / WebRTC session, including information about the configuration for PDU and EoB set marking (306).

[0058] The RTC AF then requests QoS allocation from the PCF corresponding to the PDU set marking and / or EoB marking (308). For example, the RTC AF can use the Npcf_PolicyAuthorization procedure from TS 29.514 to request QoS allocation for the IMS / WebRTC session media sessions.

[0059] The RTC AF can then confirm the QoS allocation for the media streams to the signaling server (310). Similarly, the signaling server can receive this confirmation.

[0060] In response to receiving the confirmation, the signaling server can forward the SDP response to UE1 (312). UE1 can then begin participating in the XR communication session. For example, UE1 can send and receive media data from the XR communication session. RTC AFs can add PDU set marking and / or EoB marking to appropriate packets from the XR communication session sent by UE1 and / or read PDU set marking and / or EoB marking from packets from the XR communication session to be received by UE1 and apply the corresponding QoS to such packets as authorized by the PCF.

[0061] In the case where the SDP message is encrypted, a dedicated WebRTC signaling protocol message can be defined to carry the application server configuration to the server. Petition 870250089258, dated 10 / 01 / 2025, page 34 / 104 28 / 41 WebRTC signaling.

[0062] Figure 4 is a flowchart illustrating an example of an XR media data communication method according to the techniques of this disclosure. The method in Figure 4 can be implemented by a signaling server, such as a WebRTC signaling server or a P-CSCF.

[0063] Initially, the signaling server may receive an invitation to a new WebRTC session from a UE (320). The UE may send the invitation to another UE to participate in the WebRTC session and the signaling server may intercept the invitation. The signaling server may then forward the invitation to an application server (322).

[0064] The application server can process the invitation and respond with an SDP response including data representing at least one of the PDU set marking and / or EoB marking for the WebRTC session. Thus, the signaling server can receive the SDP response including the PDU set marking and / or EoB marking data (324). The signaling server can then extract the PDU set marking and / or EoB marking configuration data (326). The signaling server can send the PDU set marking and / or EoB marking data to an RTC application function (AF) (328) to cause the RTC AF to request Quality of Service (QoS) allocation associated with the PDU set marking and / or EoB marking. Thus, the signaling server can receive a QoS allocation configuration from the RTC AF (330). In response, the signaling server can forward the SDP response to UE (332). Petition 870250089258, dated 10 / 01 / 2025, page 35 / 104 29 / 41

[0065] In this way, the method in Figure 4 represents an example of a media data communication method, including: receiving a Session Description Protocol (SDP) message including configuration information representing at least one of the Protocol Data Unit (PDU) set marking or the End of Burst (EoB) marking for a communication session; sending information representing the PDU set marking or the EoB marking for the communication session to a Real-Time Communication (RTC) application function; and processing media data from the communication session, the media data including at least one of a PDU set having the PDU set marking or an EoB having the EoB marking according to the configuration information.

[0066] Several examples of the techniques of this disclosure are summarized in the following clauses:

[0067] Clause 1: A media data communication method, the method comprising: communicating configuration information representing at least one of the protocol data unit (PDU) set marking or the burst end marking (EoB); and processing at least one of a PDU set having a PDU set marking according to the configuration information or an EoB having an EoB marking according to the configuration information.

[0068] Clause 2: The method of clause 1, wherein the communication of configuration information comprises the communication of a MediaSubComponent data type including the configuration information. Petition 870250089258, dated 10 / 01 / 2025, page 36 / 104 30 / 41

[0069] Clause 3: The method of either clause 1 and 2, wherein the configuration information includes at least one of a version value, a local identifier value, a format value, a value indicating whether a PDU set size is active, or a value indicating whether an EoB tag is active.

[0070] Clause 4: The method of clause 1, wherein the communication of configuration information comprises communicating a list of Quality of Service (QoS) allocation specifications for a Web Real-Time Communication (WebRTC) session, each of the QoS allocation specifications including a name value, a flow description, and a QoS allocation value.

[0071] Clause 5: The method of any of clauses 1 to 4, wherein the communication of configuration information comprises the communication, by an application server (AS), of configuration information.

[0072] Clause 6: The method of clause 5, which further comprises receiving an offer for a new web real-time communication (WebRTC) session, wherein the communication of configuration information comprises sending a response to the offer including the configuration information to a WebRTC signaling server or a proxy call session control function (PCSCF).

[0073] Clause 7: The method of any of clauses 1 to 3, wherein the communication of configuration information comprises communication, by a web real-time communication signaling server (WebRTC) or a proxy call session control function (P-CSCF), Petition 870250089258, dated 10 / 01 / 2025, page 37 / 104 31 / 41 of the configuration information.

[0074] Clause 8: The method of clause 7, wherein the communication of configuration information comprises receiving a response to a request for a new WebRTC session, the response including the configuration information.

[0075] Clause 9: The method of either clause 7 and 8, which further comprises sending indicative information of the new WebRTC session, information indicating that the new WebRTC uses at least one of the PDU sets having the PDU set marking or the EoB having the EoB marking, for a real-time communication (RTC) application function.

[0076] Clause 10: The method of any of clauses 7 to 9, which additionally includes receiving confirmation of a quality of service (QoS) allocation.

[0077] Clause 11: The method of any of clauses 8 to 10, which additionally comprises forwarding the response to a user equipment (UE).

[0078] Clause 12: The method of clause 1, wherein the communication of configuration information comprises the communication of a MediaSubComponent data type including the configuration information.

[0079] Clause 13: The method of clause 1, wherein the configuration information includes at least one of a version value, a local identifier value, a format value, a value indicating whether a PDU set size is active, or a value indicating whether an EoB tag is active.

[0080] Clause 14: The method of clause 1, in which Petition 870250089258, dated 10 / 01 / 2025, page 38 / 104 32 / 41 The communication of configuration information comprises communicating a list of Quality of Service (QoS) allocation specifications for a Web Real-Time Communication (WebRTC) session, each QoS allocation specification including a name value, a flow description, and a QoS allocation value.

[0081] Clause 15: The method of clause 1, wherein the communication of configuration information comprises the communication, by an application server (AS), of configuration information.

[0082] Clause 16: The method of clause 15, which further comprises receiving an offer for a new web real-time communication (WebRTC) session, wherein the communication of configuration information comprises sending a response to the offer including the configuration information to a WebRTC signaling server or a proxy call session control function (PCSCF).

[0083] Clause 17: The method of clause 1, wherein the communication of configuration information comprises the communication, by a web real-time communication signaling server (WebRTC) or a proxy call session control function (P-CSCF), of the configuration information.

[0084] Clause 18: The method of clause 17, wherein the communication of configuration information comprises receiving a response to a request for a new WebRTC session, the response including the configuration information.

[0085] Clause 19: The method of clause 7, which Petition 870250089258, dated 10 / 01 / 2025, page 39 / 104 33 / 41 further includes sending indicative information about the new WebRTC session, information indicating that the new WebRTC uses at least one of the PDU sets having the PDU set marking or the EoB having the EoB marking, for a real-time communication (RTC) application function.

[0086] Clause 20: The method of clause 7, which additionally includes receiving confirmation of a quality of service (QoS) allocation.

[0087] Clause 21: The method of clause 20, which additionally comprises forwarding the response to a user device (UE).

[0088] Clause 22: A device for retrieving data from media, wherein the device comprises one or more means for carrying out the method of any of clauses 1 to 21.

[0089] Clause 23: The device of clause 22, in which one or more means comprise a memory and a processor implemented in a circuit assembly.

[0090] Clause 24: The device of clause 22, wherein the apparatus comprises at least one of: an integrated circuit; a microprocessor; or a wireless communication device.

[0091] Clause 25: A computer-readable storage medium having instructions stored on it which, when executed, cause a processor to perform the method of any of clauses 1 to 21.

[0092] Clause 26: A device for retrieving media data, the device comprising: means for communicating configuration information represented by Petition 870250089258, dated 10 / 01 / 2025, page 40 / 104 34 / 41 less one of either a Protocol Data Unit (PDU) set marking or an End-of-Burst (EoB) marking; and means to process at least one of a set of PDUs having a PDU set marking according to the configuration information or an EoB having an EoB marking according to the configuration information.

[0093] Clause 27: A media data communication method, the method comprising: receiving a Session Description Protocol (SDP) message including configuration information representing at least one of the Protocol Data Unit (PDU) set marking or the End of Burst (EoB) marking for a communication session; sending information representing the PDU set marking or the EoB marking for the communication session to a Real-Time Communication (RTC) application function; and processing media data from the communication session, the media data including at least one of a PDU set having the PDU set marking or an EoB having the EoB marking according to the configuration information.

[0094] Clause 28: The method of clause 27, where the SDP message comprises a MediaSubComponent data type including configuration information.

[0095] Clause 29: The method of clause 27, wherein the configuration information includes at least one of a version value, a local identifier value, a format value, a value indicating whether a PDU set size is active, or a value indicating whether an EoB tag is active.

[0096] Clause 30: The method of clause 27, in Petition 870250089258, dated 10 / 01 / 2025, page 41 / 104 35 / 41 whereby the communication session comprises a real-time web communication session (WebRTC) and wherein the configuration information comprises a list of Quality of Service (QoS) allocation specifications for the WebRTC session, each of the QoS allocation specifications including a name value, a flow description and a QoS allocation value.

[0097] Clause 31: The method of clause 27, wherein receiving the SDP message comprises receiving, by a web real-time communication signaling server (WebRTC) or a proxy call session control function (P-CSCF), the SDP message.

[0098] Clause 32: The method of clause 31, where the SDP message comprises an SDP response to an SDP request for a new WebRTC session.

[0099] Clause 33: The method of clause 32, which additionally comprises forwarding the SDP response to a user device (UE).

[0100] Clause 34: The method of clause 32, whereby sending information representing the PDU set markup or the EoB markup comprises sending information indicating the new WebRTC session, information indicating that the new WebRTC uses at least one of the PDU sets having the PDU set markup or the EoB having the EoB markup, for the RTC application function.

[0101] Clause 35: The method of clause 31, which additionally includes receiving confirmation of a quality of service (QoS) allocation.

[0102] Clause 36: A device for Petition 870250089258, dated 10 / 01 / 2025, p. 42 / 104 36 / 41 media data communication, the device comprising: a memory configured to store media data; and a processing system comprising one or more processors implemented in a circuit array, the processing system being configured to: receive a Session Description Protocol (SDP) message including configuration information representing at least one of the Protocol Data Unit (PDU) set marking or the End of Burst (EoB) marking for a communication session; send information representing the PDU set marking or the EoB marking for the communication session to a real-time communication (RTC) application function; and process media data from the communication session, the media data including at least one of a PDU set having the PDU set marking or an EoB having the EoB marking according to the configuration information.

[0103] Clause 37: The device of clause 27, where the SDP message comprises a MediaSubComponent data type including configuration information.

[0104] Clause 38: The device of clause 36, wherein the configuration information includes at least one of a version value, a local identifier value, a format value, a value indicating whether a PDU set size is active, or a value indicating whether an EoB tag is active.

[0105] Clause 39: The device of clause 36, wherein the communication session comprises a real-time web communication session (WebRTC) and wherein the configuration information comprises a list of Petition 870250089258, dated 10 / 01 / 2025, page 43 / 104 37 / 41 WebRTC session quality of service (QoS) allocation specifications, each QoS allocation specification including a name value, a flow description, and a QoS allocation value.

[0106] Clause 40: The device of clause 36, wherein receiving the SDP message includes receiving, by a web real-time communication signaling server (WebRTC) or a proxy call session control function (P-CSCF), the SDP message.

[0107] Clause 41: The device of clause 40, where the SDP message comprises an SDP response to an SDP request for a new WebRTC session.

[0108] Clause 42: The device in clause 41, where the processing system is further configured to forward the SDP response to a user device (UE).

[0109] Clause 43: The device of clause 41, whereby in order to send information representing the PDU set markup or the EoB markup, the processing system is configured to send information indicating the new WebRTC session, the information indicating that the new WebRTC uses at least one of the PDU sets having the PDU set markup or the EoB having the EoB markup, for the RTC application function.

[0110] Clause 44: The device of clause 40, in which the processing system is additionally configured to receive confirmation of a quality of service (QoS) allocation.

[0111] Clause 45: A device for Petition 870250089258, dated 10 / 01 / 2025, page 44 / 104 38 / 41 Media data communication, the device comprising: means for receiving a Session Description Protocol (SDP) message including configuration information representing at least one of the Protocol Data Unit (PDU) set marking or the Burst End Marking (EoB) for a communication session; means for sending information representing the PDU set marking or the EoB marking for the communication session to a Real-Time Communication (RTC) application function; and means for processing media data from the communication session, the media data including at least one of a PDU set having the PDU set marking or an EoB having the EoB marking according to the configuration information.

[0112] Clause 46: The device of clause 45, which additionally comprises means for operating a web real-time communication signaling server (WebRTC) or means for operating a proxy call session control function (P-CSCF).

[0113] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted as one or more instructions or code in a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which correspond to a tangible medium, such as data storage media, or communication media, including any medium that facilitates the transfer of a computer program from Petition 870250089258, dated 10 / 01 / 2025, page 45 / 104 39 / 41 one place to another, for example, according to a communication protocol. In this way, computer-readable media can generally correspond to (1) a tangible, non-transient, computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for the implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0114] By way of example, and not limitation, such computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory or any other media that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer. In addition, any connection is properly termed a computer-readable medium.For example, if the instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair cable, DSL, or wireless technologies such as infrared, radio, and microwaves will be included in the... Petition 870250089258, dated 10 / 01 / 2025, page 46 / 104 40 / 41 Definition of media. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead refer to tangible non-transient storage media. As used in the present invention, disks (disk and disc) include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks generally reproduce data magnetically, while discs reproduce data optically by means of lasers. Combinations of the above items should also be included within the scope of computer-readable media.

[0115] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent set of discrete or integrated logic circuits. Consequently, the term processor, as used in the present invention, may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described in the present invention. Furthermore, in some respects, the functionality described in the present invention may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or may be incorporated into a combined codec. Additionally, the techniques may be fully implemented in one or more logic circuits or elements. Petition 870250089258, dated 10 / 01 / 2025, page 47 / 104 41 / 41

[0116] The techniques in this disclosure can be implemented in a wide variety of devices or appliances, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Several components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require implementation by different hardware units. Instead, as described above, several units can be combined into a codec hardware unit or provided by a collection of interoperable hardware units, including one or more processors, as described above, in conjunction with suitable software and / or firmware.

[0117] Several examples have been described. These and other examples are within the scope of the following claims. Petition 870250089258, dated 10 / 01 / 2025, page 48 / 104

Claims

1 / 5 CLAIMS 1. Media data communication method, the method characterized by comprising: receiving a Session Description Protocol (SDP) message including configuration information representing at least one of the Protocol Data Unit (PDU) set marking or the End of Burst (EoB) marking for a communication session; sending information representing the PDU set marking or the EoB marking for the communication session to a media application function; and processing media data from the communication session, the media data including at least one of the PDU set marking or the EoB marking according to the configuration information.

2. A method according to claim 1, characterized in that the SDP message comprises a Real-Time Transport Protocol (RTP) session message including configuration information.

3. A method according to claim 1, characterized in that the configuration information includes at least one of a version value, a local identifier value, a format value, a value indicating whether a PDU set size is active, or a value indicating whether an EoB tag is active.

4. Method, according to claim 1, characterized in that the communication session comprises a real-time web communication session (WebRTC), and in that the configuration information comprises a list of Quality of Service (QoS) allocation specifications from Petition 870250089258, dated 10 / 01 / 2025, page 91 / 104 2 / 5 WebRTC session, each of the QoS allocation specifications including a name value, a flow description and a QoS allocation value.

5. Method, according to claim 1, characterized by receiving the SDP message, understood as receiving, by a web real-time communication signaling server (WebRTC) or a proxy call session control function (P-CSCF), the SDP message.

6. A method according to claim 5, characterized in that the SDP message comprises an SDP response to an SDP request for a new WebRTC session.

7. Method according to claim 6, characterized by further comprising forwarding the SDP response to a user device (UE).

8. A method according to claim 6, characterized by sending information representing the PDU set markup or EoB markup, comprising extracting information indicative of the new WebRTC session from the SDP message and sending the information to the media application function to have the media application function forward the information to a policy control function (PCF), the information indicating that the new WebRTC session uses at least one of the PDU set markup or EoB markup.

9. A method according to claim 5, characterized by further comprising receiving confirmation of a quality of service (QoS) allocation.

10. Device for media data communication, the device characterized by comprising: Petition 870250089258, 10 / 01 / 2025, p. 92 / 104 3 / 5 a memory configured to store media data; and a processing system comprising one or more processors implemented in a circuit array, the processing system being configured to: receive a Session Description Protocol (SDP) message including configuration information representing at least one of the Protocol Data Unit (PDU) set marking or the End of Burst (EoB) marking for a communication session; send information representing the PDU set marking or the EoB marking for the communication session to a media application function; and process media data from the communication session, the media data including at least one of the PDU set marking or the EoB marking according to the configuration information.

11. Device according to claim 10, characterized in that the SDP message comprises a Real-Time Transport Protocol (RTP) session message including configuration information.

12. Device according to claim 10, characterized in that the configuration information includes at least one of a version value, a local identifier value, a format value, a value indicating whether a PDU set size is active, or a value indicating whether an EoB tag is active.

13. Device according to claim 10, characterized in that the communication session comprises a real-time web communication session (WebRTC), and in which the configuration information comprises a list of Quality of Service (QoS) allocation specifications for the WebRTC session, each of the QoS allocation specifications including a name value, a flow description, and a QoS allocation value.

14. Device, according to claim 10, characterized in that the SDP message reception comprises receiving, by a web real-time communication signaling server (WebRTC) or a proxy call session control function (P-CSCF), the SDP message.

15. Device according to claim 14, characterized in that the SDP message comprises an SDP response to an SDP request for a new WebRTC session.

16. Device according to claim 15, characterized in that the processing system is additionally configured to forward the SDP response to a user device (UE).

17. Device according to claim 15, characterized in that, to send information representing the PDU set markup or EoB markup, the processing system is configured to extract information indicative of the new WebRTC session from the SDP message and send the information to the media application function to have the media application function forward the information to a policy control function (PCF), the information indicating that the new WebRTC session uses at least one of the PDU set markup or EoB markup. Petition 870250089258, October 1, 2025, pp. 94 / 104 5 / 5 18. Device according to claim 14, characterized in that the processing system is additionally configured to receive confirmation of a quality of service (QoS) allocation.

19. Media data communication device, the device characterized by comprising: means for receiving a Session Description Protocol (SDP) message including configuration information representing at least one of the Protocol Data Unit (PDU) set marking or the End of Burst (EoB) marking for a communication session; means for sending information representing the PDU set marking or the EoB marking for the communication session to a media application function; and means for processing media data from the communication session, the media data including at least one of the PDU set marking or the EoB marking according to the configuration information.

20. Device according to claim 19, characterized by further comprising means for running a real-time web communication signaling server (WebRTC) or means for performing a proxy call session control function (P-CSCF). Petition 870250089258, dated 10 / 01 / 2025, pp. 95 / 104

Images