Tethered devices for WebRTC in cellular systems

Abstract

A tethered user equipment (UE) device, such as a mobile phone, may be tethered to a tethered device, such as augmented reality (AR) glasses. The tethered device may execute a Web Real-Time Communication Protocol (WebRTC) endpoint application, and the tethered UE device may execute a WebRTC endpoint support function. A tethered UE device may access a WebRTC signaling function of a WebRTC operator network on behalf of a tethered device to transmit and receive AR media data for a WebRTC communication session. The tethered device may present received AR media data (e.g., audio, video, and image data), and may collect and send user movement data (e.g., view orientation, entity movement data, controller interaction data, etc.) to the tethered UE device to cause the tethered UE device to send the user movement data to other devices participating in the WebRTC communication session.

Description

[0001] This application claims priority to U.S. Patent Application No. 18 / 938,067, filed November 5, 2024, and U.S. Provisional Application No. 63 / 596,530, filed November 6, 2023, the entire contents of each of which are incorporated herein by reference. U.S. Patent Application No. 18 / 938,067 claims the benefit of U.S. Provisional Application No. 63 / 596,530. Technical Field

[0002] This disclosure relates to the transmission of media data. Background Technology

[0003] Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcasting systems, wireless broadcasting systems, personal digital assistants (PDAs), laptops or desktop computers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite broadcast phones, video conferencing equipment, and more. Digital video devices implement video compression technologies (such as those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263 or ITU-T H.264 / MPEG-4, Part 10, Advanced Video Decoding (AVC), ITU-T H.265 (also known as High Efficiency Video Decoding (HEVC)), and extensions to these standards) to transmit and receive digital video information more efficiently.

[0004] Video compression techniques perform spatial and / or temporal prediction to reduce or remove inherent redundancy in video sequences. For block-based video decoding, video frames or slices can be divided into macroblocks. Each macroblock can be further subdivided. Macroblocks in intra-frame decoding (I) frames or slices can be encoded using spatial prediction of adjacent macroblocks. Macroblocks in inter-frame decoding (P or B) frames or slices can be encoded using spatial prediction of adjacent macroblocks in the same frame or slice, or temporal prediction of other reference frames.

[0005] After video data is encoded, it can be packaged for transmission or storage. Video data can be assembled into video files that conform to any of various standards, such as the International Organization for Standardization (ISO) Basic Media File Format and its extensions, such as AVC. Summary of the Invention

[0006] Generally speaking, this disclosure describes techniques for exchanging media data. Specifically, this disclosure describes techniques for tethering a user equipment (UE) device to a tethered UE device, wherein the UE device and the tethered UE device can jointly participate in a communication session, such as a WebRTC communication session.

[0007] In one example, a method for exchanging media data includes: a tethered user equipment (UE) performing a WebRTC endpoint support function to access WebRTC signaling functions of a WebRTC operator network and receiving media data of a WebRTC communication session from the WebRTC operator network, wherein the tethered UE is tethered to a tethered device performing a WebRTC endpoint application; and the tethered UE sending the received media data of the WebRTC communication session to the tethered device to cause the tethered device to present the media data.

[0008] In another example, a tethered user equipment (UE) device for participating in a Web Real-Time Communication Protocol (WebRTC) communication session includes: a memory configured to store media data; a communication interface communicatively coupled to a tethered device performing a WebRTC endpoint application; and a processing system implemented in circuitry and configured to: perform WebRTC endpoint support functions to access WebRTC signaling functions of a WebRTC operator network and receive media data of a WebRTC communication session from the WebRTC operator network; and transmit the received media data of the WebRTC communication session to the tethered device to cause the tethered device to present the media data.

[0009] In another example, a method for exchanging media data includes: a tethered device executing a WebRTC endpoint application to participate in a WebRTC communication session, wherein the tethered device is tethered to a tethered UE device performing WebRTC endpoint support functions; the tethered device receiving media data of the WebRTC communication session from the tethered UE device; and the tethered device presenting the media data of the WebRTC communication session.

[0010] In another example, a tethered device for exchanging media data includes: a memory configured to store media data; at least one display; a communication interface communicatively coupled to a tethered user equipment (UE) device performing Web Real-Time Communication Protocol (WebRTC) endpoint support functions; and a processing system implemented in circuitry and configured to: execute a WebRTC endpoint application to participate in a WebRTC communication session; receive media data of the WebRTC communication session from the tethered UE device; and present the media data of the WebRTC communication session via the at least one display.

[0011] In another example, a system for exchanging media data in a Web Real-Time Communication Protocol (WebRTC) communication session includes: a tethered device for participating in the WebRTC communication session; and a tethered user equipment (UE) device for participating in the WebRTC communication session, the tethered UE device including: a memory configured to store media data; a communication interface communicatively coupled to the tethered device; and a processing system implemented in circuitry and configured to: perform WebRTC endpoint support functions to access WebRTC signaling functions of a WebRTC operator network and from the WebRTC operator... The network receives media data of the WebRTC communication session; and transmits the received media data of the WebRTC communication session to the tethered device to cause the tethered device to present the media data, wherein the tethered device includes: a memory configured to store media data; at least one display; a communication interface communicatively coupled to the tethered user UE; and a processing system implemented in circuitry and configured to: execute a WebRTC endpoint application to participate in the WebRTC communication session; receive media data of the WebRTC communication session from the tethered UE device; and present the media data of the WebRTC communication session via the at least one display.

[0012] Details of one or more examples are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from these descriptions and drawings, and from the claims. Attached Figure Description

[0013] Figure 1 This is a block diagram illustrating an example system for implementing a technology for streaming media data over a network.

[0014] Figure 2 This is a block diagram illustrating the elements of the example video file.

[0015] Figure 3 This is a block diagram illustrating an example RTP control protocol (RCP) architecture that can support the Web Real-time Communication Protocol (WebRTC) in a 3GPP-based system.

[0016] Figure 4 This is a block diagram illustrating an example enhanced immersive real-time communication (ieRTCW) architecture for WebRTC, including a carrier network that includes a WebRTC domain for communicating with a UE including a WebRTC endpoint.

[0017] Figure 5This is a conceptual diagram illustrating a set of example components that may be involved in a WebRTC session where a UE is tethered to another device.

[0018] Figures 6 to 8 This is a block diagram illustrating an example architecture in which the technologies of this disclosure can be implemented.

[0019] Figure 9 This is a flowchart illustrating an example WebRTC setup procedure for an iRTCW architecture according to the technology of this disclosure.

[0020] Figures 10 to 12 This is a block diagram illustrating an additional example architecture in which the technologies of this disclosure can be implemented.

[0021] Figure 13 This is a flowchart illustrating an example method that can be performed by a tethered user equipment (UE) device (such as a mobile phone) and a tethered device (such as augmented reality (AR) glasses). Detailed Implementation

[0022] This disclosure describes a technique by which a tethered user equipment (UE) device (such as a cellular phone) and a tethered device (such as augmented reality (AR) glasses) can operate together to exchange and use AR data during an AR communication session. For example, the tethered UE device can be configured to interact with a radio access network (RAN) component, such as a WebRTC operator network. The tethered device can send and receive WebRTC data as part of the AR communication session and send media data of the AR communication session to the AR glasses for display. In this way, the AR glasses do not need to include all client actions configured to perform the WebRTC communication session, but still participate in the communication session.

[0023] Figure 1 This is a block diagram illustrating an example system 10 for implementing techniques for streaming media data over a network. In this example, system 10 includes a content preparation device 20, a server device 60, and a client device 40. Client device 40 and server device 60 are communicatively coupled via a network 74, which may include the Internet. In some examples, content preparation device 20 and server device 60 may also be coupled via network 74 or another network, or may be directly communicatively coupled. In some examples, content preparation device 20 and server device 60 may include the same device.

[0024] exist Figure 1In the example, content preparation device 20 includes an audio source 22 and a video source 24. Audio source 22 may include, for example, a microphone that generates electrical signals representing captured audio data to be encoded by audio encoder 26. Alternatively, audio source 22 may include: a storage medium storing previously recorded audio data; an audio data generator, such as a computerized synthesizer; or any other audio data source. Video source 24 may include: a video camera that generates video data to be encoded by video encoder 28; a storage medium encoding previously recorded video data; a video data generation unit, such as a computer graphics source; or any other video data source. Content preparation device 20 is not necessarily communicatively coupled to server device 60 in all examples, but multimedia content may be stored on a separate medium that is read by server device 60.

[0025] The raw audio and video data may include analog or digital data. Analog data may be digitized before being encoded by audio encoder 26 and / or video encoder 28. While a speaker is speaking, audio source 22 may obtain audio data from that speaker, and video source 24 may simultaneously obtain video data of that speaker. In other examples, audio source 22 may include a computer-readable storage medium containing stored audio data, and video source 24 may include a computer-readable storage medium containing stored video data. Thus, the techniques described in this disclosure can be applied to live, streaming, real-time audio and video data, or to archived, pre-recorded audio and video data.

[0026] An audio frame corresponding to a video frame is typically an audio frame containing audio data, which is simultaneously captured (or generated) by audio source 22 and video data captured (or generated) by video source 24 and contained within the video frame. For example, when a speaker typically generates audio data by speaking, audio source 22 captures the audio data, and video source 24 simultaneously (i.e., while audio source 22 is capturing audio data) captures the speaker's video data. Therefore, an audio frame can temporally correspond to one or more specific video frames. Thus, an audio frame corresponding to a video frame typically corresponds to the following situation: in which audio data and video data are captured simultaneously, and in this situation, the audio frame and video frame respectively include the simultaneously captured audio data and video data.

[0027] In some examples, audio encoder 26 may encode a timestamp into each encoded audio frame, where the timestamp indicates the time when the audio data for the encoded audio frame was recorded, and similarly, video encoder 28 may encode a timestamp into each encoded video frame, where the timestamp indicates the time when the video data for the encoded video frame was recorded. In such examples, the audio frame corresponding to the video frame may include: an audio frame including a timestamp, and a video frame including the same timestamp. Content preparation device 20 may include an internal clock, which audio encoder 26 and / or video encoder 28 may use to generate timestamps, or audio source 22 and video source 24 may use the internal clock to associate audio and video data with timestamps, respectively.

[0028] In some examples, audio source 22 may send data to audio encoder 26 corresponding to the time when the audio data was recorded, and video source 24 may send data to video encoder 28 corresponding to the time when the video data was recorded. In some examples, audio encoder 26 may encode sequence identifiers into the encoded audio data to indicate the relative time order of the encoded audio data, rather than indicating the absolute time when the audio data was recorded, and similarly, video encoder 28 may use sequence identifiers to indicate the relative time order of the encoded video data. Similarly, in some examples, sequence identifiers may be mapped or otherwise associated with timestamps.

[0029] Audio encoder 26 typically produces encoded audio data streams, while video encoder 28 produces encoded video data streams. Each individual data stream (whether audio or video) can be referred to as an elementary stream. An elementary stream is a single, digitally decoded (possibly compressed) component of a media presentation. For example, the decoded video or audio portion of a media presentation can be an elementary stream. Elementary streams can be converted into Packed Elementary Streams (PES) before being encapsulated into a video file. Within the same media presentation, stream IDs can be used to distinguish PES packets belonging to one elementary stream from those belonging to another. The basic unit of data in an elementary stream is the packed elementary stream (PES) packet. Therefore, decoded video data typically corresponds to an elementary video stream. Similarly, audio data corresponds to one or more corresponding elementary streams.

[0030] exist Figure 1In one example, the encapsulation unit 30 of the content preparation device 20 receives a base stream of video data including decoded data from the video encoder 28 and a base stream of audio data including decoded data from the audio encoder 26. In some examples, both the video encoder 28 and the audio encoder 26 may include packers for forming PES packets based on the encoded data. In other examples, both the video encoder 28 and the audio encoder 26 may interface with corresponding packers for forming PES packets based on the encoded data. In other examples, the encapsulation unit 30 may include packers for forming PES packets based on the encoded audio and video data.

[0031] Video encoder 28 is capable of encoding video data of multimedia content in various ways to produce different representations of the multimedia content at various bit rates and utilizing various characteristics such as pixel resolution, frame rate, compliance with various decoding standards, compliance with various profiles and / or profile levels used for various decoding standards, representations with one or more views (e.g., for two-dimensional or three-dimensional playback), or other such characteristics. As used in this disclosure, the representation may include one of the following: audio data, video data, text data (e.g., for closed captions), or other such data. The representation may include a primary stream, such as an audio primary stream or a video primary stream. Each PES packet may include a stream_id, which identifies the primary stream to which the PES packet belongs. Encapsulation unit 30 is responsible for assembling the primary streams into streamable media data.

[0032] Encapsulation unit 30 receives PES packets from audio encoder 26 and video encoder 28 for media presentation of the basic stream, and forms corresponding Network Abstraction Layer (NAL) units based on the PES packets. Decoded video segments can be organized into NAL units that provide a “network-friendly” video representation for addressing applications such as video telephony, storage, broadcasting, or streaming. NAL units can be classified as Video Decoding Layer (VCL) NAL units and non-VCL NAL units. VCL units may contain the core compression engine and may include block, macroblock, and / or slice-level data. Other NAL units may be non-VCL NAL units. In some examples, a picture of the decoded data in a time instance (typically presented as a picture of the main decoded data) may be included in an access unit, which may include one or more NAL units.

[0033] Non-VCL NAL units can include parameter set NAL units and SEI NAL units, etc. Parameter sets can contain sequence-level header information (in the Sequence Parameter Set (SPS)) and picture-level header information that changes very little (in the Picture Parameter Set (PPS)). Using parameter sets (e.g., PPS and SPS), it is not necessary to repeat information that changes very little for each sequence or picture; therefore, decoding efficiency can be improved. Furthermore, the use of parameter sets allows for out-of-band transmission of important header information, thus avoiding redundant transmissions required for error recovery. In an example of out-of-band transmission, parameter set NAL units can be transmitted on a different channel than other NAL units (such as SEI NAL units).

[0034] Supplemental Enhancement Information (SEI) can contain information that is not essential for decoding image samples from VCL NAL units but can assist in processes related to decoding, display, error recovery, and other purposes. SEI messages can be included in non-VCL NAL units. SEI messages are a specification part of some standards and are therefore not always mandatory for specific implementations of standards-compliant decoders. SEI messages can be sequence-level or image-level. Some sequence-level information can be included in SEI messages, such as the scalability information SEI message in the SVC example and the view scalability information SEI message in MVC. These example SEI messages can send information about, for example, the extraction and characteristics of operation points.

[0035] Server device 60 includes a Real-Time Transport Protocol (RTP) sending unit 70 and a network interface 72. In some examples, server device 60 may include multiple network interfaces. Furthermore, any or all of the features of server device 60 may be implemented on other devices in the content delivery network, such as routers, bridges, proxy devices, switches, or other devices. In some examples, intermediate devices in the content delivery network may cache data for multimedia content 64 and include components that substantially conform to the components of server device 60. Generally, network interface 72 is configured to send and receive data via network 74.

[0036] RTP sending unit 70 is configured to deliver media data to client device 40 via network 74 according to RTP, which is standardized in Internet Engineering Task Force (IETF) Request for Comments (RFC) 3550. RTP sending unit 70 may also implement RTP-related protocols such as RTP Control Protocol (RTCP), Real-time Streaming Protocol (RTSP), Session Initiation Protocol (SIP), and / or Session Description Protocol (SDP). RTP sending unit 70 can send media data via network interface 72, which implements Uniform Datagram Protocol (UDP) and / or Internet Protocol (IP). Therefore, in some examples, server device 60 may use network 74 to send media data via RTP and RTSP over UDP.

[0037] RTP sending unit 70 may receive RTSP description requests from, for example, client device 40. The RTSP description request may include data indicating what types of data the client device 40 supports. RTP sending unit 70 may respond to client device 40 with data indicating a media stream (such as media content 64), which may be sent to client device 40 together with a corresponding network location identifier (such as a Uniform Resource Locator (URL) or Uniform Resource Name (URN)).

[0038] Then, RTP sending unit 70 can receive an RTSP establishment request from client device 40. The RTSP establishment request typically indicates how the media stream will be transmitted. The RTSP establishment request may include a network location identifier and transmission specifier for the requested media data (e.g., media content 64), such as the local port used to receive RTP data and control data (e.g., RTCP data) on client device 40. RTP sending unit 70 can respond to the RTSP establishment request with acknowledgments and data representing the port of server device 60, through which the RTP data and control data will be transmitted. RTP sending unit 70 can then receive an RTSP playback request to allow the media stream to be "played," i.e., transmitted to client device 40 via network 74. RTP sending unit 70 can also receive an RTSP teardown request to terminate the streaming session; in response to this RTSP teardown request, RTP sending unit 70 can stop sending media data for the corresponding session to client device 40.

[0039] Similarly, RTP receiving unit 52 can initiate a media stream by initially sending an RTSP description request to server device 60. The RTSP description request can indicate the data types supported by client device 40. RTP receiving unit 52 can then receive from server device 60 a reply specifying an available media stream (such as media content 64) that can be sent to client device 40 along with a corresponding network location identifier (such as a Uniform Resource Locator (URL) or Uniform Resource Name (URN)).

[0040] Then, RTP receiving unit 52 can generate an RTSP establishment request and send it to server device 60. As noted above, the RTSP establishment request may include a network location identifier for the requested media data (e.g., media content 64) and a transport specification, such as the local port used to receive RTP data and control data (e.g., RTCP data) on client device 40. In response, RTP receiving unit 52 can receive an acknowledgment from server device 60, including the port of server device 60 used to send the media data and control data.

[0041] After a media streaming session is established between server device 60 and client device 40, the RTP sending unit 70 of server device 60 can send media data (e.g., packets of media data) to client device 40 according to the media streaming session. Server device 60 and client device 40 can exchange control data (e.g., RTCP data) indicating, for example, the receive statistics of client device 40, so that server device 60 can perform congestion control or otherwise diagnose and resolve transmission failures.

[0042] Network interface 54 can receive selected media presentation and provide it to RTP receiving unit 52, which in turn can provide the media data to decapsulation unit 50. Decapsulation unit 50 can decapsulate the elements of a video file into a PES stream, unpack the PES stream to retrieve encoded data, and send the encoded data to audio decoder 46 or video decoder 48, depending on whether the encoded data is part of an audio stream or a video stream (e.g., as indicated by the PES packet header of the stream). Audio decoder 46 decodes the encoded audio data and sends the decoded audio data to audio output 42, while video decoder 48 decodes the encoded video data and sends the decoded video data (which may include multiple views of the stream) to video output 44.

[0043] The video encoder 28, video decoder 48, audio encoder 26, audio decoder 46, encapsulation unit 30, RTP receiver unit 52, and decapsulation unit 50 can all be implemented as any of a variety of suitable processing circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic circuits, software, hardware, firmware, or any combination thereof. Each of the video encoder 28 and video decoder 48 can be included in one or more encoders or decoders, and either the video encoder or the video decoder can be integrated as part of a combined video encoder / decoder (CODEC). Similarly, each of the audio encoder 26 and audio decoder 46 can be included in one or more encoders or decoders, and either the audio encoder or the audio decoder can be integrated as part of a combined CODEC. The apparatus including the video encoder 28, video decoder 48, audio encoder 26, audio decoder 46, encapsulation unit 30, RTP receiver unit 52, and / or decapsulation unit 50 can include integrated circuits, microprocessors, and / or wireless communication devices, such as cellular phones.

[0044] Client device 40, server device 60, and / or content preparation device 20 may be configured to operate according to the techniques of this disclosure. For illustrative purposes, these techniques are described with respect to client device 40 and server device 60. However, it should be understood that content preparation device 20 may also be configured to perform these techniques as an alternative to (or other than) server device 60.

[0045] Encapsulation unit 30 can form NAL units, which include a header identifying the program to which the NAL unit belongs and a payload, such as audio data, video data, or data describing the transport or program stream corresponding to the NAL unit. For example, in H.264 / AVC, a NAL unit includes a 1-byte header and a payload of varying size. NAL units whose payloads include video data can include video data at various granularities. For example, a NAL unit can include video data blocks, multiple blocks, video data slices, or entire frames of video data. Encapsulation unit 30 can receive encoded video data in PES packet format with elementary streams from video encoder 28. Encapsulation unit 30 can associate each elementary stream with its corresponding program.

[0046] The encapsulation unit 30 can also assemble access units based on multiple NAL units. Generally, an access unit may include one or more NAL units representing a video data frame, and the corresponding audio data (when the audio data is available). Access units typically include all NAL units for a single output time instance, such as all audio and video data for a single time instance. For example, if each view has a frame rate of 20 frames per second (fps), each time instance may correspond to a time interval of 0.05 seconds. During this time interval, specific frames for all views with the same access unit (same time instance) can be rendered simultaneously. In one example, an access unit may include a decoded image within a time instance, which can be rendered as the primary decoded image.

[0047] Therefore, an access unit can include all audio and video frames of a common time instance, such as all views corresponding to time X. This disclosure also refers to the encoded picture of a particular view as a "view component." That is, a view component can include pictures (or frames) encoded for a particular view at a particular time. Therefore, an access unit can be defined as including all view components of a common time instance. The decoding order of the access units need not be the same as the output or display order.

[0048] After the encapsulation unit 30 has assembled the NAL units and / or access units into a video file based on the received data, the encapsulation unit 30 passes the video file to the output interface 32 for output. In some examples, the encapsulation unit 30 may store the video file locally or send the video file to a remote server via the output interface 32, instead of sending the video file directly to the client device 40. The output interface 32 may include, for example, a transmitter, a transceiver, a device for writing data to a computer-readable medium (such as, for example, an optical drive, a magnetic media drive (e.g., a floppy disk drive)), a universal serial bus (USB) port, a network interface, or other output interface. The output interface 32 outputs the video file to a computer-readable medium, such as, for example, a transmitting signal, a magnetic medium, an optical medium, a memory, a flash drive, or other computer-readable media.

[0049] Network interface 54 can receive NAL units or access units via network 74 and provide NAL units or access units to decapsulation unit 50 via RTP receiving unit 52. Decapsulation unit 50 can decapsulate the elements of the video file into a PES stream, unpack the PES stream to retrieve encoded data, and send the encoded data to audio decoder 46 or video decoder 48, depending on whether the encoded data is part of an audio stream or a video stream (e.g., as indicated by the PES packet header of the stream). Audio decoder 46 decodes the encoded audio data and sends the decoded audio data to audio output 42, while video decoder 48 decodes the encoded video data and sends the decoded video data (which may include multiple views of the stream) to video output 44.

[0050] Figure 2 This is a block diagram illustrating the elements of example video file 150. As described above, video files, according to the ISO Basic Media File Format and its extensions, store data in a series of objects called "boxes". Figure 2 In the example, video file 150 includes a file type (FTYP) box 152, a movie (MOOV) box 154, a segment index (sidx) box 162, a movie clip (MOOF) box 164, and a movie clip random access (MFRA) box 166. Although Figure 2 The example shown is a video file, but it should be understood that other media files may include other types of media data (e.g., audio data, timed text data, etc.) constructed similarly to those in video file 150, depending on the ISO basic media file format and its extensions.

[0051] The File Type (FTYP) box 152 generally describes the file type of the video file 150. The File Type box 152 may include data identifying the specifications for the best use of the video file 150. The File Type box 152 may optionally be placed before the MOOV box 154, the Movie Clip box 164, and / or the MFRA box 166.

[0052] exist Figure 2 In the example, the MOOV box 154 includes a Movie Header (MVHD) box 156, a Track (TRAK) box 158, and one or more Movie Extension (MVEX) boxes 160. Typically, the MVHD box 156 describes general characteristics of the video file 150. For example, the MVHD box 156 may include data describing when the video file 150 was initially created, when the video file 150 was last modified, the timestamp of the video file 150, the playback duration of the video file 150, or other data generally describing the video file 150.

[0053] TRAK box 158 may include track data of video file 150. TRAK box 158 may include a Track Header (TKHD) box that describes the characteristics of the track corresponding to TRAK box 158. In some examples, TRAK box 158 may include decoded video images, while in other examples, the decoded video images of the track may be included in movie clip 164, which may be referenced by data from TRAK box 158 and / or sidx box 162.

[0054] In some examples, video file 150 may include more than one track. Therefore, MOOV box 154 may include TRAK boxes in a number equal to the number of tracks in video file 150. TRAK boxes 158 may describe the characteristics of the corresponding tracks in video file 150. For example, TRAK boxes 158 may describe the temporal and / or spatial information of the corresponding tracks. When encapsulation unit 30 ( Figure 1 When a parameter set track is included in a video file (such as video file 150), a TRAK box similar to the TRAK box 158 of the MOOV box 154 can describe the characteristics of the parameter set track. The encapsulation unit 30 can signal the presence of a sequence level SEI message in the parameter set track within the TRAK box describing the parameter set track.

[0055] The MVEX box 160 can describe the characteristics of the corresponding movie clip 164, for example, by signaling to the video file 150 that, in addition to the video data (if any) included in the MOOV box 154, the movie clip 164 is also included. In the context of streaming video data, decoded video images may be included in the movie clip 164 instead of the MOOV box 154. Therefore, all decoded video samples may be included in the movie clip 164 instead of the MOOV box 154.

[0056] MOOV boxes 154 may include an equal number of MVEX boxes 160 as the number of movie segments 164 in the video file 150. Each MVEX box in the MVEX boxes 160 may describe the characteristics of a corresponding movie segment in the movie segment 164. For example, each MVEX box may include a Movie Extended Header Box (MEHD) box, which describes the time duration of a corresponding movie segment in the movie segment 164.

[0057] As noted above, encapsulation unit 30 may store a set of sequence data in a video sample that does not include the actual decoded video data. The video sample may typically correspond to an access unit, which is a representation of a decoded picture at a specific time instance. In the context of AVC, the decoded picture may include one or more VCL NAL units (containing information about all pixels used to construct the access unit) and other associated non-VCL NAL units (such as SEI messages). Therefore, encapsulation unit 30 may include a set of sequence data in a movie clip of movie clip 164, which may include a sequence-level SEI message. Encapsulation unit 30 may also signal the presence of the set of sequence data and / or the sequence-level SEI message as being present in a movie clip of movie clip 164 within an MVEX box of MVEX box 160 corresponding to a movie clip of movie clip 164.

[0058] SIDX box 162 is an optional element of video file 150. That is, video files conforming to 3GPP file formats or other such file formats do not necessarily need to include SIDX box 162. According to the example of the 3GPP file format, SIDX boxes can be used to identify sub-segments of a segment (e.g., a segment contained within video file 150). The 3GPP file format defines a sub-segment as "a self-contained set of one or more consecutive movie clip boxes having a corresponding media data box and a media data box containing data referenced by a movie clip box that must follow that movie clip box and precede the next movie clip box containing information about the same track." The 3GPP file format also instructs that the SIDX box "contains a series of references to sub-segments of the (sub)segment recorded by that box. The referenced sub-segments are consecutive in presentation time. Similarly, bytes referenced by the segment index box are always consecutive within the segment. The size of the reference gives a count of the number of bytes in the referenced material."

[0059] SIDX box 162 typically provides information representing one or more sub-segments of a segment included in video file 150. For example, such information may include the playback time of the start and / or end of the sub-segment, the byte offset of the sub-segment, whether the sub-segment includes (e.g., begins at) a Stream Access Point (SAP), the type of SAP (e.g., whether the SAP is an Instant Decoder Refresh (IDR) picture, a Clean Random Access (CRA) picture, a Broken Link Access (BLA) picture, etc.), the location of the SAP within the sub-segment (in terms of playback time and / or byte offset), etc.

[0060] Movie clip 164 may include one or more decoded video pictures. In some examples, movie clip 164 may include one or more groups of pictures (GOPs), where each GOP may include several decoded video pictures, such as frames or images. Additionally, as described above, in some examples, movie clip 164 may include a sequence of data. Each movie clip in movie clip 164 may include a movie clip header box (MFHD). Figure 2 (Not shown in the image). The MFHD box can describe the characteristics of the corresponding movie clip, such as the movie clip's sequence number. Movie clips 164 can be included in video file 150 in the order of their sequence numbers.

[0061] MFRA box 166 can describe random access points within movie segments 164 of video file 150. This can help perform special effects modes, such as searching for specific time positions (i.e., playback times) within segments encapsulated by video file 150. In some examples, MFRA box 166 is typically optional and does not need to be included in the video file. Similarly, client devices (such as client device 40) do not necessarily need to refer to MFRA box 166 to correctly decode and display the video data of video file 150. MFRA box 166 can include a number equal to the number of tracks in video file 150, or in some examples, a number equal to the number of media tracks (e.g., non-cue tracks) in video file 150 (TFRA) boxes (not shown).

[0062] In some examples, movie clip 164 may include one or more streaming access points (SAPs), such as IDR pictures. Similarly, MFRA box 166 can provide an indication of the location within the SAP video file 150. Therefore, a temporal subsequence of video file 150 can be formed from the SAP of video file 150. The temporal subsequence may also include other pictures, such as SAP-dependent P-frames and / or B-frames. Frames and / or slices of the temporal subsequence can be arranged within segments such that frames / slices of the temporal subsequence that depend on other frames / slices of the subsequence can be correctly decoded. For example, in a hierarchical arrangement of data, data used for prediction of other data may also be included in the temporal subsequence.

[0063] Figure 3 This is a block diagram illustrating an example RTP control protocol (RCP) architecture that supports the Web Real-time Communication Protocol (WebRTC) in a 3GPP-based system. This example RCP architecture corresponds to 3GPP TS 26.506. In this example, the RCP architecture includes a User Equipment (UE) device 200, an RTC Application Function (AF) 240, an RTC Application Server (AS) 250, and an RTC Application Provider (AP) 270.

[0064] RTC AF 240 includes Network Support Functions (NS-AF) 242, Configuration Functions 244, and Provisioning Functions 246. RTC AS 250 includes Interactive Connectivity Establishment (ICE) Functions 252, Media Functions 254, Transport Gateway Functions 256, WebRTC Signaling Functions 258, Application Support Web Page Functions 260, and Interactive Working Functions 262. RTC AF 240 is also coupled to Policy and Charging Functions 230, Network Exposure Functions 232, and Session Management Functions 234. RTC AF 240 is communicatively coupled to RTC AS 250 via RTC-3 interface 224. RTC AF 240 is communicatively coupled to RTC Application Provider 270 via RTC-1 interface 226.

[0065] UE 200 includes an RTC endpoint that includes a WebRTC API 210 (which includes an RTC Media Session Handler (MSH) 212 and a WebRTC framework 214). UE 200 also includes a native WebRTC application 202 and a web application 204. The native WebRTC application 202 is communicatively coupled to the RTC MSH 212 via an RTC-6 interface 216. The native WebRTC application 202 is communicatively coupled to the WebRTC framework 214 via an RTC-7 interface 218. The RTC MSH 212 is communicatively coupled to the RTC AF via an RTC-5 interface 220. The WebRTC framework 214 is communicatively coupled to the RTC AS 250 via an RTC-4 interface 222. The native WebRTC application 202 is communicatively coupled to the RTC application provider 270 via an RTC-8 interface 228.

[0066] This disclosure recognizes that conventional techniques do not define the RTC function partitioning and signaling required for proper functioning when another device (such as XR / AR / VR / MR glasses) is tethered to the UE.

[0067] Figure 4 This is a block diagram illustrating an example enhanced immersive real-time communication (ieRTCW) architecture for WebRTC, including a carrier network 320, which includes a WebRTC domain communicating with a UE device 300 including a WebRTC endpoint 302. The WebRTC endpoint 302 may include both native WebRTC applications and web applications, such as... Figure 3 As shown. Figure 4 The WebRTC endpoint 302 can correspond to Figure 3 The RTC endpoint.

[0068] In this example, Rh-u represents the reference point between the Conference Session Function (CSF) 310 and the UE 300. The CSF 310 can provide functions such as conference session management and application usage assistance, such as downloading applications to the UE 300.

[0069] In this example, Rs-u represents the reference point between the WebRTC Signalling Function (WSF) 312 and the UE. The WSF 312 can handle offer / acknowledge exchanges and can access Session Description Protocol (SDP) messages in both directions. The WSF 312 is also communicatively coupled to the WebRTC NNI Signalling Gateway Function (WNSGF) 316 via the Rs-i interface.

[0070] In this example, Rm-u represents the reference point between the WebRTC Media Center Function (WMCF) 314 and the UE 300. The WMCF 314 provides functions including: delivering content to the WebRTC endpoint 302, mixing, acting as a multipoint control unit (MCU), selective transfer unit (SFU), relay, etc. The WMCF 314 is communicatively coupled to the WebRTC NNI Media Gateway Function (WNMGF) 318 via the Rm-i interface.

[0071] Figure 5 This is a conceptual diagram illustrating a set of example components that may be involved in a WebRTC session where a UE is tethered to another device. Figure 3 Examples include mobile phone 352, augmented reality (AR) glasses 350, gNodeB 354, user plane function (UPF) 356, and application server 358. In some examples, the UE device according to the technology of this disclosure may be a cellular phone, such as mobile phone 352; and the tethered device may be AR glasses, such as AR glasses 350, as... Figure 5 As shown in the examples. This disclosure describes various examples of devices that instruct the execution of WebRTC media functions and control functions, as well as the signaling used to make the resulting architecture work.

[0072] Generally speaking, mobile phone 352 can be configured to communicate with radio access networks, such as... Figure 5 The 5G core network (5GC) is shown. The AR glasses 350 can be configured to act as a WebRTC endpoint and participate in communication with another UE device (…). Figure 5 The WebRTC communication session (not shown) can be performed by mobile phone 352, but RAN-specific communication functions can be executed by mobile phone 352. Therefore, for example, mobile phone 352 can be configured to perform functions that generally belong to the Media Session Handler (MSH).

[0073] Figures 6 to 8 and Figures 10 to 12This is a block diagram illustrating an example architecture in which the technologies of this disclosure can be implemented. Generally speaking, WebRTC functionality includes a WebRTC application (which controls the startup of the application, the codecs of media data, bitrate control, congestion control, and the like) and WebRTC support functionality (which provides access to RTC support functionality, including interactive connectivity establishment (ICE) negotiation to establish a connection, quality of experience (QoE) metric collection, media configuration recommendations for the application, and the like).

[0074] WebRTC applications and WebRTC support features can be separated in several ways. In one example, both the WebRTC application and support features reside on the tethered device (e.g., AR glasses), such as... Figure 6 and Figure 7 As shown in the example. In another example, the WebRTC application resides on the tethered device (e.g., AR glasses), while the supporting functionality resides on the tethered device (e.g., a phone or UE), for example, as... Figure 8 and Figure 10 As shown, in this case, the tethered device can act as a relay device. In yet another example, both the WebRTC application and supporting functions reside on the tethered device, and the tethered device is used solely as a display, for example, as... Figure 11 and Figure 12 As shown.

[0075] WebRTC applications can be mapped to native WebRTC applications or web applications within the iRTCW architecture, or to WebRTC endpoint applications within the eiRTCW architecture. WebRTC support features can be mapped to RTC endpoints within the iRTCW architecture, and to WebRTC endpoint support features within the eiRTCW architecture.

[0076] Specifically, Figure 6 Examples include tethered device 400, relay UE device 436, RTC application function (AF) 440, RTC application server (AS) 450, RTC application provider 470, PCF 430, NEF 432, and SMF 434. Tethered device 400 includes native WebRTC application 402, web application 404, and WebRTC API 410 including RTC MSH 412 and WebRTC framework 414. Native WebRTC application 402 is coupled to RTC MSH 412 via RTC-6 interface 416. Native WebRTC application 402 is coupled to WebRTC framework 414 via RTC-7 interface 418. Native WebRTC application 402 is coupled to RTC application provider 470 via RTC-8 interface 428.

[0077] RTC AF 440 includes NS-AF 442, configuration function 444, and provisioning function 446. RTC MSH 412 is coupled to RTC AF 440 via RTC-5 interface 420 through relay UE 436. RTC AS 450 includes ICE function 452, media function 454, transport gateway function 456, WebRTC signaling function 458, application support webpage function 460, and interactive functionality 462. WebRTC framework 414 is coupled to RTC AS 450 via RTC-4 interface 422 through relay UE 436. RTC AF 440 is coupled to RTC AS 450 via RTC-3 interface 424. RTC AF 440 is coupled to RTC application provider 470 via RTC-1 interface 426.

[0078] Figure 7 An example is depicted in which the tethered device 400 includes a WebRTC endpoint 502 and a display 506. The tethered device 500 participates in a WebRTC communication session via a relay UE 504, which communicates with the Conference Support Function (CSF) 510, the WebRTC Signaling Function (WSF) 512, and the WebRTC Media Center Function (WMCF) 514. The WSF 512 communicates with the WebRTC NNI Signaling Gateway Function (WNSGF) 516, and the WMCF 514 communicates with the WebRTC NNI Media Gateway Function (WNMGF) 518. The tethered device 500 may correspond to... Figure 6 The tethered device 400, and the relay UE 504 can correspond to Figure 6 The relay UE 436.

[0079] exist Figure 6 and Figure 7 In these examples, the WebRTC endpoints reside on either tethered device 400 or tethered device 500, respectively. In these examples, relay UE 436 acts as a repeater. In this way, Figure 6 400 tethered devices and Figure 7 The tethered device 500 represents an example of a tethered device that performs a Web Real-Time Communication Protocol (WebRTC) endpoint application, wherein the tethered device is tethered to a tethered UE device that performs WebRTC endpoint support functions, and wherein the tethered device receives media data of a WebRTC session from the tethered UE device.

[0080] same, Figure 6Relay UE 436 and Relay UE 504 represent examples of tethered UE devices that perform Web Real-Time Communication Protocol (WebRTC) endpoint support functions, wherein the tethered UE device is tethered to a tethered device that performs a WebRTC endpoint application, and wherein the tethered UE device sends media data of a WebRTC session to the tethered UE device.

[0081] Figure 8 Examples include tethered device 520, UE device 556, RTC Application Function (AF) 560, RTC Application Server (AS) 570, RTC Application Provider 590, PCF 550, NEF 552, and SMF 554. Tethered device 520 includes a native WebRTC application 522 and a web application 524. UE device 556 includes an RTC endpoint 530, which includes an RTC MSH 532 and a WebRTC framework 534. Native WebRTC application 522 is coupled to RTC MSH 532 via RTC-6' interface 536. Native WebRTC application 522 is coupled to WebRTC framework 534 via RTC-7' interface 538. Native WebRTC application 522 is coupled to RTC application provider 590 via RTC-8 interface 548. Web application 524 is coupled to WebRTC framework 534 via WebRTC API.

[0082] RTC AF 560 includes NS-AF 562, configuration function 564, and provisioning function 566. RTC MSH 532 is coupled to RTC AF 560 via RTC-5 interface 420. RTC AS 570 includes ICE function 572, media function 574, transport gateway function 576, WebRTC signaling function 578, application support webpage function 580, and interactive functionality 582. WebRTC framework 534 is coupled to RTCAS 450 via RTC-4 interface 542. RTC AF 560 is coupled to RTC AS 570 via RTC-3 interface 544. RTCAF 560 is coupled to RTC application provider 590 via RTC-1 interface 546.

[0083] Figure 9This is a flowchart illustrating an example WebRTC setup procedure for an iRTCW architecture according to the technology of this disclosure. A provisioning session may initially be established between the application provider (AP) and the mobile network operator (MNO). This provides information about the STUN / TURN server used in the ICE procedure. The tethered device and the tethered device can then establish a tether link (600). The tether link can be a 3GPP link (e.g., a side link on PC5) or a non-3GPP link (e.g., Wi-Fi). The setup may involve authentication.

[0084] The tethered device can then launch a WebRTC application (e.g., a browser application or a non-browser WebRTC application) (602).

[0085] The tethered device and the tethered device can then establish a communication channel for RTC-6' between the WebRTC application and the RTC MSH (604). The difference between RTC-6' and RTC-6 is that RTC-6' is between two different devices, while RTC-6 is within the same device, such as a device with an RTC-6' interface 536. Figure 8 As shown. This setting may involve selecting the protocol to be used (e.g., TCP, UDP, etc.) and the port number (e.g., the port number for the WebRTC application and the port number for the RTC MSH). Additionally or alternatively, the control channel RTC-7' and / or the WebRTC API can be configured similarly.

[0086] The RTC MSH can then extract configuration parameters (606) from the trusted ICE function. For example, the RTC MSH can receive a list of trusted STUN / TURN servers that the UE can use to establish an RTC session.

[0087] WebRTC applications can then retrieve a list of trusted ICE servers from the RTC MSH via RTC-6 (608).

[0088] The UE can discover and test ICE candidates to find suitable candidates for connection (610).

[0089] The WebRTC application on the tethered device and the remote RTC endpoint can then establish a WebRTC communication session (612), for example, via SDP.

[0090] Figure 10Examples include a tethered device 650 (which includes a WebRTC endpoint application 652 and a display 658) and a UE device 654 (which includes a WebRTC endpoint support function 656). The tethered device 650 participates in a WebRTC communication session via the UE device 654, which communicates with the Conference Support Function (CSF) 660, the WebRTC Signaling Function (WSF) 662, and the WebRTC Media Center Function (WMCF) 664. The WSF 662 communicates with the WebRTC NNI Signaling Gateway Function (WNSGF) 666, and the WMCF 664 communicates with the WebRTC NNI Media Gateway Function (WNMGF) 668. The tethered device 650 may correspond to... Figure 8 The tethered device 520, and the UE device 654 can correspond to Figure 8 UE device 556.

[0091] exist Figure 10 In the example, RT-u represents the interface between WebRTC endpoint application 652 and WebRTC endpoint support feature 656. RT-u can be mapped to... Figure 8 The iRTCW architecture shown includes the RTC-6' interface 536, the WebRTC API 2 interface, and the RTC-7' interface 538.

[0092] When WebRTC endpoint application 652 is launched, it can communicate with WebRTC endpoint support function 656 on UE 654 via the Rt-u interface. This communication can trigger further communication between UE 654 and WebRTC support function in the cellular system via the Rh-u, Rs-u, and Rm-u interfaces.

[0093] The Rt-u interface is not in the same device as UE 654, and therefore may require the configuration of a communication channel, which can be defined by a protocol (e.g., TCP, UDP, SCTP, etc.) and a pair of port numbers (one for the tethered device and the other for the tethered device).

[0094] In this way, Figure 8 UE 556 and Figure 10UE 654 represents an example of a tethered UE device for participating in a Web Real-Time Communication Protocol (WebRTC) communication session, the tethered UE device comprising: a memory configured to store media data; a communication interface communicatively coupled to a tethered device performing a WebRTC endpoint application; and a processing system implemented in circuitry and configured to: perform WebRTC endpoint support functions to access WebRTC signaling functions of a WebRTC operator network and receive media data of a WebRTC communication session from the WebRTC operator network; and transmit the received media data of the WebRTC communication session to the tethered device to cause the tethered device to present the media data.

[0095] same, Figure 8 The tethered equipment 520 and Figure 10 The tethered device 650 represents an example of a tethered device for exchanging media data, the tethered device comprising: a memory configured to store media data; at least one display; a communication interface communicatively coupled to a tethered user equipment (UE) device performing Web Real-Time Communication Protocol (WebRTC) endpoint support functions; and a processing system implemented in circuitry and configured to: execute a WebRTC endpoint application to participate in a WebRTC communication session; receive media data of the WebRTC communication session from the tethered UE device; and present the media data of the WebRTC communication session via the at least one display.

[0096] Figure 11 Examples include UE device 700, AR device 736, RTC Application Function (AF) 740, RTC Application Server (AS) 750, RTC Application Provider 770, PCF 730, NEF 732, and SMF 734. UE device 700 includes a native WebRTC application 702, a web application 704, and a WebRTC API 710 including RTC MSH 712 and WebRTC Framework 714. Native WebRTC application 702 is coupled to RTC MSH 712 via RTC-6 interface 716. Native WebRTC application 702 is coupled to WebRTC Framework 714 via RTC-7 interface 718. Native WebRTC application 702 is coupled to RTC Application Provider 770 via RTC-8 interface 728.

[0097] RTC AF 740 includes NS-AF 742, configuration function 744, and provisioning function 746. RTC MSH 712 is coupled to RTC AF 740 via RTC-5 interface 720. RTC AS 750 includes ICE function 752, media function 754, transport gateway function 756, WebRTC signaling function 758, application support webpage function 760, and interactive functionality 762. WebRTC framework 714 is coupled to RTCAS 750 via RTC-4 interface 722. RTC AF 740 is coupled to RTC AS 750 via RTC-3 interface 724. RTCAF 740 is coupled to RTC application provider 770 via RTC-1 interface 726.

[0098] In this example, UE device 700 provides media data for WebRTC communication sessions (e.g., AR communication sessions that include audio, video, images, and / or AR data such as MR, XR, and / or VR data) to AR device 736. UE device 700 otherwise performs most of the WebRTC communication session functions with RTC AF 740, RTC AS 750, and RTC application provider 770.

[0099] Figure 12 Examples include a tethered device 800 (which includes a display 802) and a UE device 804 (which includes a WebRTC endpoint 806). The WebRTC endpoint 806 communicates with a Conference Support Function (CSF) 810, a WebRTC Signaling Function (WSF) 812, and a WebRTC Media Center Function (WMCF) 814. The WSF 812 communicates with a WebRTC NNI Signaling Gateway Function (WNSGF) 816, and the WMCF 814 communicates with a WebRTC NNI Media Gateway Function (WNMGF) 818. The tethered device 800 may correspond to... Figure 11 AR device 736, while UE device 804 can correspond to Figure 11 UE device 700.

[0100] In this way, Figure 11 UE equipment 700 and Figure 12 The UE device 804 represents an example of a tethered UE device that performs Web Real-Time Communication Protocol (WebRTC) endpoint support and WebRTC endpoint application to receive media data from a WebRTC communication session and send the media data to the tethered device for output to the user.

[0101] same, Figure 11 AR device 736 and Figure 12The tethered device 800 represents an example of a tethered device that receives media data from a WebRTC (Web Real-Time Communication Protocol) communication session from a tethered UE device that performs WebRTC endpoint support functions and WebRTC endpoint applications.

[0102] Figure 13 This is a flowchart illustrating an example method that can be performed by a tethered user equipment (UE) device (such as a mobile phone) and a tethered device (such as augmented reality (AR) glasses). The tethered UE device typically performs WebRTC endpoint support functions, and the tethered device typically performs WebRTC endpoint applications. For example, the tethered UE device may correspond to... Figure 8 UE 556 or Figure 10 UE 654, and the tethered device can correspond to Figure 8 520 or tethered equipment Figure 10 The tethered equipment is 650.

[0103] Initially, the tethered UE and the tethered device can establish a tether link and communication channel between each other (850, 852). The tethered UE can then retrieve Interactive Connectivity Establishment (ICE) configuration information from one or more ICE servers (854). The tethered UE can then provide the ICE configuration information to the tethered device.

[0104] The tethered device can then use the ICE configuration information to perform ICE negotiation (856). For example, the tethered device can retrieve a list of pre-configured ICE functions and discover ICE candidates. The tethered device can then exchange Session Description Protocol (SDP) offer or acknowledgment and the list of ICE candidates with the remote UE to establish a WebRTC communication session with the remote UE (858).

[0105] The tethered UE device can then receive media data of the WebRTC communication session (860) and send the media data to the tethered device (862). The tethered device can then display the media data, for example, via a display, speaker, etc. (864).

[0106] In this way, Figure 13 The method represents an example of a method for exchanging media data, the method comprising: a tethered user equipment (UE) device performing a Web Real-Time Communication Protocol (WebRTC) endpoint support function to access WebRTC signaling functions of a WebRTC operator network and receiving media data of a WebRTC communication session from the WebRTC operator network, wherein the tethered UE device is tethered to a tethered device performing a WebRTC endpoint application; and the tethered UE device transmitting the received media data of the WebRTC communication session to the tethered device to cause the tethered device to present the media data.

[0107] Figure 13 The method also represents an example of a method for exchanging media data, the method comprising: a tethered device executing a Web Real-Time Communication Protocol (WebRTC) endpoint application to participate in a WebRTC communication session, wherein the tethered device is tethered to a tethered UE device performing WebRTC endpoint support functions; the tethered device receiving media data of the WebRTC communication session from the tethered UE device; and the tethered device presenting the media data of the WebRTC communication session.

[0108] The following clauses illustrate various examples of the technology disclosed herein:

[0109] Clause 1: A method for exchanging media data, the method comprising: a user equipment (UE) device performing a Web Real-Time Communication Protocol (WebRTC) endpoint support function, wherein the UE device is tethered to a tethered UE device performing a WebRTC endpoint application; and the UE device transmitting media data of a WebRTC session to the tethered UE device.

[0110] Clause 2: A method for exchanging media data, the method comprising: a tethered user equipment (UE) device executing a Web Real-Time Communication Protocol (WebRTC) endpoint application, wherein the tethered UE device is tethered to a tethered UE device performing WebRTC endpoint support functions; and the tethered UE device receiving media data of a WebRTC session from the tethered UE device.

[0111] Clause 3: A method for exchanging media data, the method comprising: a user equipment (UE) device performing a Web Real-Time Communication Protocol (WebRTC) endpoint support function and a WebRTC endpoint application, wherein the UE device is tethered to a tethered UE device; and the UE device transmitting media data of a WebRTC session to the tethered UE device.

[0112] Clause 4: A method for exchanging media data, the method comprising: receiving media data of a WebRTC session from a tethered UE device performing Web Real-Time Communication Protocol (WebRTC) endpoint support functionality and a WebRTC endpoint application.

[0113] Clause 5: A method for exchanging media data, the method comprising: a user equipment (UE) device sending media data of a WebRTC session to a tethered UE device performing Web Real-Time Communication Protocol (WebRTC) endpoint support functionality and a WebRTC endpoint application.

[0114] Clause 6: A method for exchanging media data, the method comprising: performing a Web Real-Time Communication Protocol (WebRTC) endpoint support function and a WebRTC endpoint application by a tethered user equipment (UE) device, wherein the UE device is tethered to the tethered UE device; and receiving media data of a WebRTC session from the tethered UE device.

[0115] Clause 7: The method according to any one of Clauses 1 to 6, wherein the UE device includes a cellular phone.

[0116] Clause 8: The method according to any one of Clauses 1 to 7, wherein the tethered UE device includes augmented reality (AR) glasses.

[0117] Clause 9: The method according to any one of Clauses 1 to 8, wherein the method further comprises: establishing a tethered link between the UE device and the tethered UE device; launching the WebRTC endpoint application; establishing an RTC-6' communication channel between the UE device and the tethered UE device; retrieving configuration information from one or more Interactive Connectivity Establishment (ICE) servers; retrieving a list of pre-configured ICE functions; discovering ICE candidates from the list of pre-configured ICE functions; and using the ICE candidates to establish a communication session with a remote UE.

[0118] Clause 10: The method described in Clause 9, wherein establishing the communication session includes using a Session Description Protocol (SDP) to establish the communication session.

[0119] Clause 11: The method according to any one of Clauses 1 to 10, wherein the UE device and the tethered UE device are configured to communicate via an Rt-u interface.

[0120] Clause 12: An apparatus for exchanging media data, the apparatus comprising one or more components for performing a method according to any one of Clauses 1 to 11.

[0121] Clause 13: The device according to Clause 12, wherein the one or more components include a processing system, the processing system including one or more processors implemented in a circuit.

[0122] Clause 14: The apparatus according to Clause 12, wherein the apparatus includes at least one of: an integrated circuit; a microprocessor; and a wireless communication device.

[0123] Clause 15: A computer-readable storage medium having instructions stored thereon, which, when executed, cause a processing system to perform the method according to any one of Clauses 1 to 11.

[0124] Clause 16: A user equipment (UE) device for exchanging media data, the UE device comprising: components for performing Web Real-Time Communication Protocol (WebRTC) endpoint support functions, wherein the UE device is tethered to a tethered UE device performing a WebRTC endpoint application; and components for transmitting media data of a WebRTC session to the tethered UE device.

[0125] Clause 17: A tethered user equipment (UE) device for exchanging media data, the tethered UE device comprising: components for performing a Web Real-Time Communication Protocol (WebRTC) endpoint application, wherein the tethered UE device is tethered to a tethered UE device performing WebRTC endpoint support functions; and components for receiving media data of a WebRTC session from the tethered UE device.

[0126] Clause 18: A method for exchanging media data, the method comprising: a tethered user equipment (UE) device performing a Web Real-Time Communication Protocol (WebRTC) endpoint support function to access WebRTC signaling functions of a WebRTC operator network and receiving media data of a WebRTC communication session from the WebRTC operator network, wherein the tethered UE device is tethered to a tethered device performing a WebRTC endpoint application; and the tethered UE device transmitting the received media data of the WebRTC communication session to the tethered device to cause the tethered device to present the media data.

[0127] Clause 19: The method of Clause 18, wherein the tethered UE device includes a cellular phone and the tethered device includes augmented reality (AR) glasses.

[0128] Clause 20: The method according to Clause 18 further includes: establishing a tether link with the tethered device by the tethered UE device; establishing a communication channel with the tethered device by the tethered UE device; retrieving configuration information from one or more Interactive Connectivity Establishment (ICE) servers by the tethered UE device; retrieving a list of pre-configured ICE functions by the tethered UE device; discovering ICE candidates from the list of pre-configured ICE functions by the tethered UE device; and using the ICE candidates to establish the WebRTC communication session with a remote UE by the tethered UE device.

[0129] Clause 21: The method according to Clause 20, wherein establishing the WebRTC communication session includes using the Session Description Protocol (SDP) to establish the WebRTC communication session.

[0130] Clause 22: The method according to Clause 18 further includes establishing a communication channel between the tethered UE device and the tethered device via the Rt-u interface.

[0131] Clause 23: The method according to Clause 18 further includes establishing a Wi-Fi communication connection between the tethered UE device and the tethered device.

[0132] Clause 24: The method according to Clause 18, wherein performing the WebRTC endpoint support function includes performing the WebRTC endpoint support function to collect quality of experience (QoE) metrics and provide media configuration recommendations for the WebRTC endpoint application of the tethered device.

[0133] Clause 25: A tethered user equipment (UE) device for participating in a Web Real-Time Communication Protocol (WebRTC) communication session, the tethered UE device comprising: a memory configured to store media data; a communication interface communicatively coupled to a tethered device performing a WebRTC endpoint application; and a processing system implemented in circuitry and configured to: perform WebRTC endpoint support functions to access WebRTC signaling functions of a WebRTC operator network and receive media data of a WebRTC communication session from the WebRTC operator network; and transmit the received media data of the WebRTC communication session to the tethered device to cause the tethered device to present the media data.

[0134] Clause 26: The tethered UE device as described in Clause 25, wherein the tethered UE device includes a cellular phone and the tethered device includes augmented reality (AR) glasses.

[0135] Clause 27: The tethered UE device as described in Clause 25, wherein the processing system is further configured to: establish a tether link with the tethered device; establish a communication channel with the tethered device; retrieve configuration information from one or more Interactive Connectivity Establishment (ICE) servers; retrieve a list of pre-configured ICE functions; discover ICE candidates from the list of pre-configured ICE functions; and use the ICE candidates to establish the WebRTC communication session with the remote UE.

[0136] Clause 28: Tethering a UE device as described in Clause 27, wherein the processing system is configured to use the Session Description Protocol (SDP) to establish the WebRTC communication session.

[0137] Clause 29: The tethered UE device as described in Clause 25, wherein the processing system is further configured to establish a communication channel with the tethered device via an Rt-u interface.

[0138] Clause 30: The tethered UE device as described in Clause 25, wherein the processing system is further configured to establish a Wi-Fi communication connection with the tethered device.

[0139] Clause 31: A tethered UE device as described in Clause 25, wherein the WebRTC endpoint support function is configured to collect quality of experience (QoE) metrics and provide media configuration recommendations for the WebRTC endpoint application of the tethered device.

[0140] Clause 32: A method for exchanging media data, the method comprising: a tethered device executing a WebRTC endpoint application to participate in a WebRTC communication session, wherein the tethered device is tethered to a tethered UE device performing WebRTC endpoint support functions; the tethered device receiving media data of the WebRTC communication session from the tethered UE device; and the tethered device presenting the media data of the WebRTC communication session.

[0141] Clause 33: The method according to Clause 32, wherein the tethered UE device includes a cellular phone and the tethered device includes augmented reality (AR) glasses.

[0142] Clause 34: The method according to Clause 32 further includes: establishing a tether link between the tethered device and the tethered UE device; establishing a communication channel between the tethered device and the tethered UE device; and executing the WebRTC endpoint application by the tethered device to participate in the WebRTC communication session.

[0143] Clause 35: The method according to Clause 34, wherein performing the WebRTC endpoint application to participate in the WebRTC communication session includes performing the WebRTC endpoint application to collect user mobility data and sending the user mobility data to the tethered UE device so that the tethered UE device sends the user mobility data as part of the WebRTC communication session.

[0144] Clause 36: The method according to Clause 32 further includes establishing a communication session between the tethered device and the tethered UE device via the Rt-u interface.

[0145] Clause 37: The method according to Clause 32 further includes establishing a Wi-Fi communication connection between the tethered device and the tethered UE device.

[0146] Clause 38: The method according to Clause 32, wherein performing the WebRTC endpoint application includes performing the WebRTC endpoint application to encode and decode media data of the WebRTC communication session, perform bit rate control, and perform network congestion control.

[0147] Clause 39: A tethered device for exchanging media data, the tethered device comprising: a memory configured to store media data; at least one display; a communication interface communicatively coupled to a tethered user equipment (UE) device performing Web Real-Time Communication Protocol (WebRTC) endpoint support functionality; and a processing system implemented in circuitry and configured to: execute a WebRTC endpoint application to participate in a WebRTC communication session; receive media data of the WebRTC communication session from the tethered UE device; and present the media data of the WebRTC communication session via the at least one display.

[0148] Clause 40: The device as described in Clause 39, wherein the tethered UE device includes a cellular phone and the tethered device includes augmented reality (AR) glasses.

[0149] Clause 41: The device according to Clause 39, wherein the processing system is further configured to: establish a tether link with the tethered UE device; establish a communication channel with the tethered UE device; and execute the WebRTC endpoint application to participate in the WebRTC communication session.

[0150] Clause 42: The device according to Clause 41 further includes one or more motion detection sensors, wherein, in order to execute the WebRTC endpoint application to participate in the WebRTC communication session, the processing system is configured to collect user motion data via the motion detection sensors and send the user motion data to the tethered UE device so that the tethered UE device sends the user motion data as part of the WebRTC communication session.

[0151] Clause 43: The device as described in Clause 39, wherein the processing system is further configured to establish a communication session with the tethered UE device via the Rt-u interface.

[0152] Clause 44: The device according to Clause 39, wherein the processing system is further configured to establish a Wi-Fi communication connection with the tethered UE device.

[0153] Clause 45: The device according to Clause 39, wherein the WebRTC endpoint application is configured to encode and decode media data of the WebRTC communication session, perform bit rate control, and perform network congestion control.

[0154] Clause 46: A system for exchanging media data in a Web Real-Time Communication Protocol (WebRTC) communication session, the system comprising: a tethered device for participating in the WebRTC communication session; and a tethered user equipment (UE) device for participating in the WebRTC communication session, the tethered UE device including: a memory configured to store media data; a communication interface communicatively coupled to the tethered device; and a processing system implemented in circuitry and configured to: perform WebRTC endpoint support functions to access WebRTC signaling functions of a WebRTC operator network and from the WebRTC operator network. The system receives media data from the WebRTC communication session and transmits the received media data from the WebRTC communication session to the tethered device to present the media data on the tethered device, wherein the tethered device includes: a memory configured to store media data; at least one display; a communication interface communicatively coupled to the tethered user UE; and a processing system implemented in circuitry and configured to: execute a WebRTC endpoint application to participate in the WebRTC communication session; receive media data from the tethered UE device for the WebRTC communication session; and present the media data of the WebRTC communication session via the at least one display.

[0155] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on a computer-readable medium or transmitted via a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium (which corresponds to a tangible medium such as a data storage medium) or a communication medium, including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to extract instructions, code, and / or data structures for implementing the techniques described in this disclosure. Computer program products may include computer-readable media.

[0156] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium capable of storing desired program code in the form of instructions or data structures and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies (such as infrared, radio, and microwave), then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies (such as infrared, radio, and microwave) are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead refer to non-transient tangible storage media. As used herein, disks and optical discs include: compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. The combinations described above should also be included within the scope of computer-readable media.

[0157] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, as used herein, the term "processor" can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Furthermore, these techniques can be fully implemented in one or more circuit or logic elements.

[0158] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or IC sets (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but implementation by different hardware units is not necessarily required. Rather, as described above, various units can be combined in a codec hardware unit, or various units can be provided by a collection of interoperable hardware units (including one or more processors as described above) combined with appropriate software and / or firmware.

[0159] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. A method for exchanging media data, the method comprising: The tethered user equipment (UE) device performs Web Real-Time Communication Protocol (WebRTC) endpoint support functions to access WebRTC signaling functions of the WebRTC operator network and receives media data of WebRTC communication sessions from the WebRTC operator network, wherein the tethered UE device is tethered to the tethered device performing the WebRTC endpoint application; as well as The tethered UE device sends the media data received in the WebRTC communication session to the tethered device, so that the tethered device can display the media data.

2. The method of claim 1, wherein the tethered UE device comprises a cellular phone and the tethered device comprises augmented reality (AR) glasses.

3. The method according to claim 1, further comprising: The tethered UE device establishes a tethering link with the tethered device; The tethered UE device establishes a communication channel with the tethered device; The tethered UE device retrieves configuration information from one or more Interactive Connectivity Establishment (ICE) servers; as well as The tethered UE device sends the configuration information to the tethered device.

4. The method of claim 3, wherein establishing the WebRTC communication session includes using a Session Description Protocol (SDP) to establish the WebRTC communication session.

5. The method according to claim 1, further comprising establishing a communication channel between the tethered UE device and the tethered device via the Rt-u interface.

6. The method according to claim 1, further comprising establishing a Wi-Fi communication connection between the tethered UE device and the tethered device.

7. The method of claim 1, wherein performing the WebRTC endpoint support function includes performing the WebRTC endpoint support function to collect quality of experience (QoE) metrics and provide media configuration recommendations for the WebRTC endpoint application on the tethered device.

8. A tethered user equipment (UE) device for participating in a Web Real-Time Communication Protocol (WebRTC) communication session, the tethered UE device comprising: A memory configured to store media data; A communication interface, which is communicatively coupled to a tethered device that performs a WebRTC endpoint application; and A processing system, implemented in a circuit and configured to: Perform WebRTC endpoint support functions to access the WebRTC signaling functions of the WebRTC operator network and receive media data of the WebRTC communication session from the WebRTC operator network; as well as The received media data from the WebRTC communication session is sent to the tethered device so that the tethered device can display the media data.

9. The tethered UE device of claim 8, wherein the tethered UE device comprises a cellular phone and the tethered device comprises augmented reality (AR) glasses.

10. The tethered UE device according to claim 8, wherein the processing system is further configured to: Establish a tethering link with the tethered device; Establish a communication channel with the tethered device; Retrieve configuration information from one or more Interactive Connectivity Establishment (ICE) servers; and The configuration information is sent to the tethered device.

11. The tethered UE device of claim 10, wherein the processing system is configured to use the Session Description Protocol (SDP) to establish the WebRTC communication session.

12. The tethered UE device of claim 8, wherein the processing system is further configured to establish a communication channel with the tethered device via an Rt-u interface.

13. The tethered UE device of claim 8, wherein the processing system is further configured to establish a Wi-Fi communication connection with the tethered device.

14. The tethered UE device of claim 8, wherein the WebRTC endpoint support function is configured to collect quality of experience (QoE) metrics and provide media configuration recommendations for the WebRTC endpoint application of the tethered device.

15. A method for exchanging media data, the method comprising: The tethered device executes a Web Real-Time Communication Protocol (WebRTC) endpoint application to participate in a WebRTC communication session, wherein the tethered device is tethered to a tethered UE device that performs WebRTC endpoint support functions; The tethered device receives media data of the WebRTC communication session from the tethered UE device; as well as The media data of the WebRTC communication session is presented by the tethered device.

16. The method of claim 15, wherein the tethered UE device comprises a cellular phone and the tethered device comprises augmented reality (AR) glasses.

17. The method according to claim 15, further comprising: A tethering link is established between the tethered device and the tethered UE device; A communication channel is established between the tethered device and the tethered UE device; The tethered device receives a list of pre-configured ICE functions; The tethered device discovers ICE candidates from the list of pre-configured ICE functions; The tethered device uses the ICE candidate to establish the WebRTC communication session with the remote UE; as well as The tethered device executes the WebRTC endpoint application to participate in the WebRTC communication session.

18. The method of claim 17, wherein executing the WebRTC endpoint application to participate in the WebRTC communication session includes executing the WebRTC endpoint application to collect user mobility data and sending the user mobility data to the tethered UE device so that the tethered UE device sends the user mobility data as part of the WebRTC communication session.

19. The method of claim 15, further comprising establishing a communication session with the tethered UE device via an Rt-u interface.

20. The method of claim 15, further comprising establishing a Wi-Fi communication connection between the tethered device and the tethered UE device.

21. The method of claim 15, wherein executing the WebRTC endpoint application includes executing the WebRTC endpoint application to encode and decode media data of the WebRTC communication session, perform bit rate control, and perform network congestion control.

22. A tethered device for exchanging media data, the tethered device comprising: A memory configured to store media data; At least one display; A communication interface, which is communicatively coupled to a tethered user equipment (UE) device that performs Web Real-Time Communication Protocol (WebRTC) endpoint support functions; as well as A processing system, implemented in a circuit and configured to: Execute a WebRTC endpoint application to participate in a WebRTC communication session; Receive media data of the WebRTC communication session from the tethered UE device; as well as The media data of the WebRTC communication session is presented via the at least one display.

23. The device of claim 22, wherein the tethered UE device comprises a cellular phone and the tethered device comprises augmented reality (AR) glasses.

24. The apparatus of claim 22, wherein the processing system is further configured to: Establish a tethering link with the tethered UE device; Establish a communication channel with the tethered UE device; Retrieve a list of pre-configured ICE features; Discover ICE candidates from the list of pre-configured ICE functions; Use the ICE candidate to establish the WebRTC communication session with the remote UE; as well as Execute the WebRTC endpoint application to participate in the WebRTC communication session.

25. The device of claim 24, further comprising one or more motion detection sensors, wherein, in order to execute the WebRTC endpoint application to participate in the WebRTC communication session, the processing system is configured to collect user motion data via the motion detection sensors and transmit the user motion data to the tethered UE device so that the tethered UE device transmits the user motion data as part of the WebRTC communication session.

26. The device of claim 22, wherein the processing system is further configured to establish a communication session with the tethered UE device via an Rt-u interface.

27. The device of claim 22, wherein the processing system is further configured to establish a Wi-Fi communication connection with the tethered UE device.

28. The device of claim 22, wherein the WebRTC endpoint application is configured to encode and decode media data of the WebRTC communication session, perform bit rate control, and perform network congestion control.

29. A system for exchanging media data in a Web Real-Time Communication Protocol (WebRTC) communication session, the system comprising: A tethered device, the tethered device being used to participate in the WebRTC communication session; as well as A tethered user equipment (UE) device, the tethered user equipment (UE) device being used to participate in the WebRTC communication session, the tethered UE device comprising: A memory configured to store media data; A communication interface, which is communicatively coupled to the tethered device; and A processing system, implemented in a circuit and configured to: Perform WebRTC endpoint support functions to access the WebRTC signaling functions of a WebRTC operator network and receive media data from the WebRTC operator network for the WebRTC communication session; and The received media data from the WebRTC communication session is sent to the tethered device so that the tethered device can display the media data. The tethered device includes: A memory configured to store media data; At least one display; A communication interface, which is communicatively coupled to the tethered user UE; and A processing system, implemented in a circuit and configured to: Execute a WebRTC endpoint application to participate in the WebRTC communication session; Receive media data of the WebRTC communication session from the tethered UE device; and The media data of the WebRTC communication session is presented via the at least one display.

Images