User interaction data transmission using real-time transport protocol header extension

Abstract

Aspects of the present disclosure relate to user interaction data transmission using real-time transport protocol header extension. One or more physical device interactions by a user are sampled to obtain interaction data inputs, such as for users playing cloud games or using augmented reality on their user devices. These interactive data inputs are frequently generated, such as about 250 to 1000 times per second. Time-sensitive one or more interaction short messages (e.g., less than 100 bytes) are generated from user interaction data and included in one or more transport header extensions of a real-time transport protocol data unit. The real-time transport protocol data unit is transmitted to a remote device, such as a network entity.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. patent application serial number 63 / 420,885, filed on October 31, 2022, entitled “USER INTERACTION DATATRANSPORTATION USING REAL-TIME TRANSPORT PROTOCOL HEADER EXTENSION,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to wireless communications, and more particularly to transmitting user interaction data using a Real-time Transport Protocol (RTP) header. Background Art

[0004] A wireless communication system may include one or more network communication devices (such as base stations), which may also be referred to as eNodeB (eNB), next generation NodeB (gNB) or other suitable terms. Each network communication device (such as a base station) may support wireless communication of one or more user communication devices, which may also be referred to as user equipment (UE) or other suitable terms. A wireless communication system may support wireless communication with one or more user communication devices by utilizing resources of the wireless communication system (e.g., time resources (e.g., symbols, time slots, subframes, frames, etc.) or frequency resources (e.g., subcarriers, carriers). In addition, a wireless communication system may support wireless communication across various radio access technologies, including third generation (3G) radio access technology, fourth generation (4G) radio access technology, fifth generation (5G) radio access technology, and other suitable radio access technologies beyond 5G (e.g., sixth generation (6G)).

[0005] Extended reality (XR) systems can use wireless communication systems to transmit data between various devices. XR refers to any one or more of various types of reality, such as virtual reality (VR), augmented reality (AR), and mixed reality (MR). The data sent includes physical interaction data, which is data that describes the user's physical interactions in the physical world. Examples of physical interaction data include posture tracking data, gesture tracking data, eye tracking data, etc. Summary of the invention

[0006] The present disclosure relates to methods, apparatus and systems for supporting the transmission of user interactive data using real-time transport protocol header extensions. One or more physical device interactions performed by a user are sampled to obtain interactive data inputs, such as for users playing cloud games or using XR on their UEs. These interactive data inputs are generated frequently, such as approximately 250 to 1000 times per second. One or more time-sensitive interactive short messages (e.g., less than 100 bytes) are generated from the user interaction data and included in one or more transport header extensions of an RTP data unit. The RTP data unit is sent to a remote device such as a network entity. By sending the user interaction data in the header extension of the RTP data unit, the user interaction data can be quickly sent to the remote device along the time-critical path of the RTP, thereby allowing the user interaction data to be sent to the remote device in a time-sensitive manner.

[0007] Some implementations of the methods and apparatus described herein may also include: sampling one or more physical device interactions performed by a user to obtain interaction data corresponding to action input; generating one or more gesture information elements from the interaction data; including the one or more gesture information elements in one or more transport header extensions of a real-time transport protocol data unit corresponding to a payload of at least one media component; and sending the real-time transport protocol data unit to a remote device.

[0008] In some implementations of the methods and devices described herein, the methods and devices also include causing the device to encrypt one or both of a transport layer security (TLS) process and a datagram transport layer security (DTLS) process to encrypt one or more transport header extensions. Additionally or alternatively, the device includes a UE. Additionally or alternatively, the method and device also include generating a message authentication signature based at least in part on one or more transport header extensions; and attaching the message authentication signature to a real-time transport protocol data unit. Additionally or alternatively, the method and device also include generating one or more posture information elements based on an interactive data format syntax, the interactive data format syntax including one or more of the following: a syntax protocol identifier, an interactive short message type, an interactive short message timestamp, an interactive short message raw data length, or an interactive short message raw data payload. Additionally or alternatively, the payload of the real-time transport protocol data unit does not contain data. Additionally or alternatively, the frequency of generating the posture information element is based on a processing mode indicating one of a periodic interactive short message and an event-based interactive short message. Additionally or alternatively, the method and apparatus further include: when the processing mode indicates an event-based interactive short message, using an event threshold to determine generating one or more gesture information elements.

[0009] Some implementations of the methods and apparatus described herein may also include receiving from a UE a real-time transport protocol data unit corresponding to a payload of at least one media component, and one or more posture information elements included in one or more transport header extensions of the real-time transport protocol data unit; processing the one or more posture information elements to obtain graphics rendering interaction and graphics rendering output; and sending the graphics rendering output and graphics rendering interaction to the UE.

[0010] In some implementations of the methods and devices described herein, the methods and devices further include: determining graphics rendering interaction data based at least in part on one or more of the following: a syntax protocol identifier, an interaction short message type, an interaction short message timestamp, an interaction short message raw data length, or an interaction short message raw data payload included in each of one or more interaction short messages including a gesture information element, and generating graphics rendering output based on the graphics rendering interaction data. Additionally or alternatively, determining the graphics rendering interaction data further includes at least one of the following: selection of one of the interaction short messages, or interpolation of one or more interaction short messages. Additionally or alternatively, the methods and devices further include receiving interaction data based at least in part on out-of-band session description signaling, the out-of-band session description signaling determining a session configuration for receiving a real-time transport protocol data unit containing one or more interaction data, and transmitting graphics rendering interactions and graphics rendering outputs. Additionally or alternatively, the methods and devices further include decrypting one or more transport header extensions using one or both of a TLS process and a DTLS process. Additionally or alternatively, the method and apparatus further comprises authenticating one or more transport header extensions based at least in part on a message authentication signature appended to the real-time transport protocol data unit.Additionally or alternatively, the payload of the real-time transport protocol data unit does not contain data. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 An example of a wireless communication system supporting user interactive data transmission using a real-time transport protocol header extension in accordance with aspects of the present disclosure is illustrated.

[0012] Figure 2 An example of an RTP and Real-time Transport Control Protocol (RTCP) stack supporting user interactive data transmission using a Real-time Transport Protocol header extension in accordance with aspects of the present disclosure is illustrated.

[0013] Figure 3 An example of a Web Real-Time Communications (WebRTC) stack supporting user interactive data transmission using Real-time Transport Protocol header extensions in accordance with aspects of the present disclosure is illustrated.

[0014] Figure 4Illustrated are RTP and Secure Real-time Transport Protocol (SRTP) packet formats and header information supporting user interactive data transmission using a Real-time Transport Protocol header extension in accordance with aspects of the present disclosure.

[0015] Figure 5 An RTP or SRTP header extension format and syntax supporting user interactive data transmission using a real-time transport protocol header extension according to aspects of the present disclosure is illustrated.

[0016] Figure 6 An example of an RTP or SRTP header extension payload format for a generic interactivity message supporting user interactivity data transmission using a real-time transport protocol header extension is illustrated in accordance with aspects of the present disclosure.

[0017] Figure 7 An example of an RTP or SRTP header extension format for a generic interactivity message supporting user interactive data transmission using a real-time transport protocol header extension is illustrated in accordance with aspects of the present disclosure.

[0018] Figure 8 An example of a single-byte RTP or SRTP header extension format with an interactive message data element supporting user interactive data transmission using a real-time transport protocol header extension in accordance with aspects of the present disclosure is illustrated.

[0019] Fig. 9 An example of a two-byte RTP or SRTP header extension format with interactive message data elements supporting user interactive data transmission using a real-time transport protocol header extension is illustrated in accordance with aspects of the present disclosure.

[0020] Fig.10 Illustrated is an example 1000 of a Session Description Protocol (SDP) answer supporting user interactive data transmission using a Real-time Transport Protocol header extension in accordance with aspects of the present disclosure.

[0021] Fig.11 Illustrated is an example 1100 of an SDP answer supporting user interactive data transmission using a real-time transport protocol header extension in accordance with aspects of the present disclosure.

[0022] Fig.12 and Fig.13 An example of a block diagram of a device supporting user interactive data transmission using a real-time transport protocol header extension is illustrated in accordance with aspects of the present disclosure.

[0023] Figures 14 to 18 A flow chart of a method for supporting user interactive data transmission using a real-time transport protocol header extension according to aspects of the present disclosure is illustrated. DETAILED DESCRIPTION

[0024] Interactive communication means that various information streams carry potentially time-sensitive inputs from one device to transmit them to a remote device over a network. Interactive applications that rely on this communication mode are becoming increasingly popular in large-scale online games, cloud games, and XR. This interactive real-time information stream contains input to a game engine, a graphics engine, or a rendering engine, which processes the information and returns a reaction to the exciting input under some latency constraints. The format and syntax of this interactive information stream are mainly application-dependent and contradictory to established media formats (e.g., audio or video codecs), and no mainstream encoding has been established. In addition, this interaction develops rapidly and requires rapid adaptation to new versions at a higher speed than the typical traditional media codec development cycle. Therefore, it is necessary to transmit such information streams over the network in a real-time, flexible, and encoding / grammar-independent manner to provide application developers with the modern tools needed for rapidly emerging and disruptive interactive applications.

[0025] Using the techniques discussed in this article, one or more physical device interactions performed by a user are sampled to obtain interaction data inputs, such as for users playing cloud games or using XR on their UE. These physical device interactions of the user are recorded or sampled by various sensors and may include posture information, tracking of gaze and eye movements, tracking of hands and identification of gestures, tracking of facial expressions and identification of emotions, or spatial tracking, AR anchoring or viewport description. Interaction data input refers to data describing physical device interactions. These interaction data inputs are generated frequently, such as approximately 250 to 1000 times per second.

[0026] One or more time-sensitive interaction short messages (e.g., less than 100 bytes) are generated from the user interaction data and included in one or more transport header extensions of the RTP data unit. These interaction short messages refer to messages that include data describing the interaction of physical devices. The RTP data unit is sent to a remote device such as a network entity. The network entity processes the received interaction short message, which may include transmitting the data in the received interaction short message to one or more other network entities or cloud-based services, thereby generating an interaction response that the network entity sends to the UE. The interaction response refers to an action to be taken (e.g., feedback to be taken by the UE, such as a change in the content displayed by the UE to the user).

[0027] By sending user interaction data in a header extension of an RTP data unit, the user interaction data can be quickly sent to a remote device along the time-critical path of RTP, thereby allowing the user interaction data to be sent to the remote device in a time-sensitive manner. The techniques discussed herein allow interactive data input to be transmitted in a header extension in a time-critical path (e.g., of a WebRTC stack), thereby allowing interactive data input to be transmitted to a network device more reliably and faster than using a non-time-critical path (e.g., of a WebRTC data channel stack). In addition, the header extension transmission mechanism discussed herein is equally applicable to both the RTP protocol and its secure version (SRTP protocol), although in some implementations and examples, they are mentioned in the context of RTP or SRTP.

[0028] Furthermore, typically in order to send data in the time-critical path of the WebRTC stack, a profile describing the appearance of the payload needs to be defined (e.g., so that a decoder at the receiver can understand the payload), reviewed by the Internet Engineering Task Force (IETF), and incorporated into a proposed standard. By including the interactive data input in a header extension to RTP, rather than in the RTP payload, the time-consuming process of obtaining profile approval and incorporating the profile into a standard can be avoided.

[0029] Aspects of the present disclosure are described in the context of a wireless communication system. Aspects of the present disclosure are further illustrated and described with reference to device diagrams and flow charts.

[0030] Figure 1 An example of a wireless communication system 100 that supports user interactive data transmission using a real-time transport protocol header extension according to various aspects of the present disclosure is illustrated. The wireless communication system 100 may include one or more network entities 102, one or more UEs 104, a core network 106, and a packet data network 108. The wireless communication system 100 may support various radio access technologies. In some implementations, the wireless communication system 100 may be a 4G network, such as an LTE network or an advanced LTE (LTE-A) network. In some other implementations, the wireless communication system 100 may be a 5G network, such as an NR network. In other implementations, the wireless communication system 100 may be a combination of a 4G network and a 5G network, or other suitable radio access technologies, including the Institute of Electrical and Electronics Engineers (IEEE) 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20. The wireless communication system 100 may support radio access technologies other than 5G. In addition, the wireless communication system 100 may support technologies such as time division multiple access (TDMA), frequency division multiple access (FDMA), or code division multiple access (CDMA).

[0031] One or more network entities 102 may be dispersed throughout a geographic area to form a wireless communication system 100. One or more of the network entities 102 described herein may be or include or may be referred to as a network node, a base station, a network element, a radio access network (RAN), a base transceiver station, an access point, a NodeB, an eNodeB (eNB), a next generation NodeB (gNB), or other suitable terms. The network entity 102 and the UE 104 may communicate via a communication link 110, which may be a wireless or wired connection. For example, the network entity 102 and the UE 104 may perform wireless communication (e.g., receive signaling, send signaling) over a Uu interface.

[0032] The network entity 102 may provide a geographic coverage area 112 for which the network entity 102 supports services (e.g., voice, video, packet data, messaging, broadcast, etc.) for one or more UEs 104 within the geographic coverage area 112. For example, the network entity 102 and the UE 104 may support wireless communications of signals associated with services (e.g., voice, video, packet data, messaging, broadcast, etc.) according to one or more radio access technologies. In some implementations, the network entity 102 may be mobile, for example, a satellite associated with a non-terrestrial network. In some implementations, different geographic coverage areas 112 associated with the same or different radio access technologies may overlap, but different geographic coverage areas 112 may be associated with different network entities 102. The information and signals described herein may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referred to throughout the description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0033] One or more UEs 104 may be dispersed throughout the geographic area of ​​the wireless communication system 100. UE 104 may include or may be referred to as a mobile device, a wireless device, a remote device, a remote unit, a handheld device, a subscriber device, or some other suitable term. In some implementations, UE 104 may be referred to as a unit, a station, a terminal, or a client, etc. Additionally or alternatively, UE 104 may be referred to as an Internet of Things (IoT) device, an Internet of Everything (IoE) device, or a machine type communication (MTC) device, etc. In some implementations, UE 104 may be stationary in the wireless communication system 100. In some other implementations, UE 104 may be mobile in the wireless communication system 100.

[0034] One or more UEs 104 may be devices of different forms or with different capabilities. Figure 1 Some examples of UE 104 are illustrated in FIG. Figure 1 As shown, UE 104 can communicate with various types of devices, such as network entity 102, other UE 104, or network equipment (e.g., core network 106, packet data network 108, relay device, integrated access and backhaul (IAB) node, or another network device). Additionally or alternatively, UE 104 can support communication with other network entities 102 or UE 104 that can act as relays in wireless communication system 100.

[0035] The UE 104 may also be capable of supporting wireless communications directly with other UEs 104 via a communication link 114. For example, the UE 104 may support wireless communications directly with another UE 104 via a device-to-device (D2D) communication link. In some implementations, such as vehicle-to-vehicle (V2V) deployments, vehicle-to-everything (V2X) deployments, or cellular V2X deployments, the communication link 114 may be referred to as a sidelink. For example, the UE 104 may support wireless communications directly with another UE 104 via a PC5 interface.

[0036] The network entity 102 may support communication with the core network 106 or with another network entity 102 or both. For example, the network entity 102 may interface with the core network 106 via one or more backhaul links 116 (e.g., via S1, N2, N6, or another network interface). The network entities 102 may communicate with each other via the backhaul links 116 (e.g., via X2, Xn, or another network interface). In some implementations, the network entities 102 may communicate directly with each other (e.g., between the network entities 102). In some other implementations, the network entities 102 may communicate with each other or indirectly (e.g., via the core network 106). In some implementations, one or more network entities 102 may include subcomponents, such as an access network entity, which may be an example of an access node controller (ANC). The ANC may communicate with one or more UEs 104 via one or more other access network transport entities (which may be referred to as radio heads, smart radio heads, or transport reception points (TRPs)).

[0037] In some implementations, the network entity 102 may be configured in a decomposed architecture that may be configured to utilize a protocol stack that is physically or logically distributed between two or more network entities 102, such as an integrated access backhaul (IAB) network, an open RAN (O-RAN) (e.g., a network configuration sponsored by the O-RAN Alliance), or a virtualized RAN (vRAN) (e.g., a cloud RAN (C-RAN)). For example, the network entity 102 may include one or more of a central unit (CU), a distributed unit (DU), a radio unit (RU), a RAN intelligent controller (RIC) (e.g., a near real-time RIC (near RT RIC), a non-real-time RIC (non-RT RIC)), a service management and orchestration (SMO) system, or any combination thereof.

[0038] The RU may also be referred to as a radio head, a smart radio head, a remote radio head (RRH), a remote radio unit (RRU), or a transmission reception point (TRP). In a decomposed RAN architecture, one or more components of the network entity 102 may be co-located, or one or more components of the network entity 102 may be located at distributed locations (e.g., separate physical locations). In some implementations, one or more network entities 102 of the decomposed RAN architecture may be implemented as virtual units (e.g., virtual CU (VCU), virtual DU (VDU), virtual RU (VRU)).

[0039] The functional split between CU, DU and RU can be flexible, and different functions can be supported according to the functions performed at the CU, DU or RU (e.g., network layer functions, protocol layer functions, baseband functions, radio frequency functions and any combination thereof). For example, a functional split of a protocol stack can be adopted between the CU and the DU so that the CU can support one or more layers of the protocol stack and the DU can support one or more different layers of the protocol stack. In some implementations, the CU can host higher-level protocol layer (e.g., Layer 3 (L3), Layer 2 (L2)) functions and signaling (e.g., Radio Resource Control (RRC), Service Data Adaptation Protocol (SDAP), Packet Data Convergence Protocol (PDCP)). The CU can be connected to one or more DUs or RUs, and one or more DUs or RUs can host lower protocol layers, such as Layer 1 (L1) (e.g., physical (PHY) layer) or L2 (e.g., radio link control (RLC) layer, media access control (MAC) layer) functions and signaling, and each can be at least partially controlled by the CU.

[0040] Additionally or alternatively, a functional split of the protocol stack may be employed between the DU and the RU such that the DU may support one or more layers of the protocol stack and the RU may support one or more different layers of the protocol stack. The DU may support one or more different cells (e.g., via one or more RUs). In some implementations, the functional split between the CU and the DU or between the DU and the RU may be within the protocol layer (e.g., some functions of the protocol layer may be performed by one of the CU, the DU, or the RU, while other functions of the protocol layer may be performed by different ones of the CU, the DU, or the RU).

[0041] The CU may be further functionally split into CU control plane (CU-CP) and CU user plane (CU-UP) functions. The CU may be connected to one or more DUs via a medium-range communication link (e.g., F1, F1-c, F1-u), and the DU may be connected to one or more RUs via a fronthaul communication link (e.g., an open fronthaul (FH) interface). In some implementations, the medium-range communication link or the fronthaul communication link may be implemented based on interfaces (e.g., channels) between layers of a protocol stack supported by a corresponding network entity 102 communicating via such a communication link.

[0042] The core network 106 may support user authentication, access authorization, tracking, connection, and other access, routing, or mobility functions. The core network 106 may be an evolved packet core (EPC) or a 5G core (5GC), which may include control plane entities that manage access and mobility (e.g., mobility management entity (MME), access and mobility management function (AMF)), and user plane entities that route packets or interconnections to external networks (e.g., serving gateway (S-GW), packet data network (PDN) gateway (P-GW), or user plane function (UPF)). In some implementations, the control plane entities may manage non-access stratum (NAS) functions, such as mobility, authentication, and bearer management (e.g., data bearers, signal bearers, etc.) of one or more UEs 104 served by one or more network entities 102 associated with the core network 106.

[0043] The core network 106 may communicate with the packet data network 108 via one or more backhaul links 116 (e.g., via S1, N2, N6, or another network interface). The packet data network 108 may include an application server 118. In some implementations, one or more UEs 104 may communicate with the application server 118. The UE 104 may establish a session (e.g., a protocol data unit (PDU) session, etc.) with the core network 106 via the network entity 102. The core network 106 may use the established session (e.g., the established PDU session) to route traffic (e.g., control information, data, etc.) between the UE 104 and the application server 118. The PDU session may be an example of a logical connection between the UE 104 and the core network 106 (e.g., one or more network functions of the core network 106).

[0044] In the wireless communication system 100, the network entity 102 and the UE 104 may use the resources of the wireless communication system 100 (e.g., time resources (e.g., symbols, time slots, subframes, frames, etc.) or frequency resources (e.g., subcarriers, carriers) to perform various operations (e.g., wireless communication). In some implementations, the network entity 102 and the UE 104 may support different resource structures. For example, the network entity 102 and the UE 104 may support different frame structures. In some implementations, such as in 4G, the network entity 102 and the UE 104 may support a single frame structure. In some other implementations, such as in 5G and other suitable radio access technologies, the network entity 102 and the UE 104 may support various frame structures (i.e., multiple frame structures). The network entity 102 and the UE 104 may support various frame structures based on one or more digital technologies.

[0045] One or more digital technologies may be supported in the wireless communication system 100, and the digital technologies may include subcarrier spacing and cyclic prefixes. A first digital technology (e.g., μ = 0) may be associated with a first subcarrier spacing (e.g., 15kHz) and a normal cyclic prefix. A first digital technology (e.g., μ = 0) associated with a first subcarrier spacing (e.g., 15kHz) may utilize one slot per subframe. A second digital technology (e.g., μ = 1) may be associated with a second subcarrier spacing (e.g., 30kHz) and a normal cyclic prefix. A third digital technology (e.g., μ = 2) may be associated with a third subcarrier spacing (e.g., 60kHz) and a normal cyclic prefix or an extended cyclic prefix. A fourth digital technology (e.g., μ = 3) may be associated with a fourth subcarrier spacing (e.g., 120kHz) and a normal cyclic prefix. A fifth digital technology (e.g., μ = 4) may be associated with a fifth subcarrier spacing (e.g., 240kHz) and a normal cyclic prefix.

[0046] The time intervals of resources (e.g., communication resources) can be organized according to frames (also referred to as radio frames). Each frame can have a duration, for example, a duration of 10 milliseconds (ms). In some implementations, each frame can include multiple subframes. For example, each frame can include 10 subframes, and each subframe can have a duration, for example, a duration of 1 ms. In some implementations, each frame can have the same duration. In some implementations, each subframe of a frame can have the same duration.

[0047] Additionally or alternatively, the time intervals of resources (e.g., communication resources) can be organized according to time slots. For example, a subframe may include a certain number (e.g., quantity) of time slots. Each time slot may include a certain number (e.g., quantity) of symbols (e.g., orthogonal frequency division multiplexing (OFDM) symbols). In some implementations, the number (e.g., quantity) of time slots of a subframe may depend on the digital technology. For a normal cyclic prefix, a time slot may include 14 symbols. For an extended cyclic prefix (e.g., applicable to a 60kHz subcarrier spacing), a time slot may include 12 symbols. For a normal cyclic prefix and an extended cyclic prefix, the relationship between the number of symbols per time slot, the number of time slots per subframe, and the number of time slots per frame may depend on the digital technology. It should be understood that reference to a first digital technology (e.g., μ=0) associated with a first subcarrier spacing (e.g., 15kHz) may be used interchangeably between subframes and time slots.

[0048] In the wireless communication system 100, the electromagnetic (EM) spectrum can be divided into various categories, frequency bands, frequency channels, etc. based on frequency or wavelength. For example, the wireless communication system 100 can support one or more operating frequency bands, such as frequency range names FR1 (410 MHz-7.125 GHz), FR2 (24.25 GHz-52.6 GHz), FR3 (7.125 GHz-24.25 GHz), FR4 (52.6 GHz-114.25 GHz), FR4a or FR4-1 (52.6 GHz-71 GHz), and FR5 (114.25 GHz-300 GHz). In some implementations, the network entity 102 and the UE 104 can perform wireless communications on the one or more operating frequency bands. In some implementations, FR1 can be used by the network entity 102 and the UE 104 and other devices or apparatuses for cellular communication services (e.g., control information, data). In some implementations, FR2 can be used by the network entity 102 and the UE 104 and other equipment or devices for short-range, high data rate capabilities.

[0049] FR1 may be associated with one or more digital technologies (e.g., at least three digital technologies). For example, FR1 may be associated with a first digital technology (e.g., μ=0) including a subcarrier spacing of 15 kHz; a second digital technology (e.g., μ=1) including a subcarrier spacing of 30 kHz; a third digital technology (e.g., μ=2) including a subcarrier spacing of 60 kHz. FR2 may be associated with one or more digital technologies (e.g., at least 2 digital technologies). For example, FR2 may be associated with a third digital technology (e.g., μ=2) including a subcarrier spacing of 60 kHz; a fourth digital technology (e.g., μ=3) including a subcarrier spacing of 120 kHz.

[0050] The technology discussed herein supports the transmission of interactive short messages from one device or equipment to another device or equipment. For example, UE 104 samples one or more interactions of a user (e.g., interactions with UE 104) to obtain interactive data input. Message generation 120 at UE 104 generates one or more interactive short messages from the interactive data input, which are included in one or more transport header extensions (also referred to as header extensions) of a real-time transport protocol (RTP) data unit. These (multiple) RTP data units are sent to a network entity as (multiple) RTP data units 122. In one or more implementations, at least a portion of each RTP data unit 122 is encrypted, such as using one or both of a transport layer security (TLS) process and a datagram transport layer security (DTLS) process. Then, the network entity 102 is able to decrypt the encrypted portion of the RTP data unit 122. Additionally or alternatively, a message authentication signature generated at least in part based on one or more transport header extensions is attached to each RTP data unit 122. Then, the network entity 102 is able to verify that one or more transport header extensions have not been modified.

[0051] The response generation 124 at the network entity 102 (aggregating the network element distributed processing to the application server 118) processes the one or more interactive short messages in the received (multiple) RTP data units 122, and based on the processing, generates a response to the interaction identified in the one or more interactive short messages. The network entity 102 sends the response as an interactive response 126 to the UE 104.

[0052] As discussed in more detail below, the interactive data input is a small size (e.g., up to 100 bytes) real-time interactive data specific to an interactive application (e.g., XR or cloud gaming (CG)), and the interactive short message is transmitted through an RTP header extension element independently or together with other header extension elements. The header extension element that transmits the interactive data is optionally authenticated, encrypted, or both authenticated and encrypted.

[0053] The term extended reality (XR) is used as an umbrella term for different types of reality, including virtual reality (VR), augmented reality (AR), and mixed reality (MR). VR is a rendered version of a delivered visual and audio scene. The rendering in this case is designed to mimic the visual and audio sensory stimulation of the real world as naturally as possible as the viewer or user moves within a range defined by the application. Virtual reality typically (but not necessarily) requires the user to wear a head-mounted display (HMD) that completely replaces the user's field of view with simulated visual components, and wear headphones to provide the user with accompanying audio. Some form of head and motion tracking of the user in VR is typically used to allow the simulated visual and audio components to be updated, ensuring that objects and sound sources are consistent with the user's movements from the user's perspective. In some implementations, additional components for interacting with the virtual reality simulation may be provided.

[0054] AR is when the user is provided with additional information or artificially generated items, or content overlaid onto their current environment. Such additional information or content will typically be visual and / or auditory, and their observation of their current environment may be direct, without intermediate sensing, processing, and rendering; or indirect, with their perception of their environment transmitted via sensors and may be augmented or processed.

[0055] MR is an advanced form of AR in which some virtual elements are inserted into the physical scene with the goal of providing the illusion that these elements are part of the real scene.

[0056] XR refers to all real and virtual environments and human-computer interactions generated by computer technology and wearable devices. It includes representative forms such as AR, MR and VR, as well as areas inserted between them. The degree of virtuality ranges from partial sensory input to fully immersive VR. The key aspect of XR is to expand human experience, especially related to presence (represented by VR) and cognitive acquisition (represented by AR).

[0057] At the heart of a successful immersive XR experience is the interaction and spatial computation associated with XR application activities. The same applies to other mainstream interaction-driven applications, such as CG. Therefore, the interaction data and related spatial computation that determine the response of the XR rendering engine or CG game engine to the user's physical input help create an immersive cyber-physical illusion between the physical world and the virtual world.

[0058] In one or more implementations, data associated with user interaction events has one or more of the following characteristics: 1) each message typically has a low data footprint of up to about 100 bytes (thus, these messages are also referred to as short messages or interaction short messages); 2) a high sampling rate in the range of 250 Hertz (Hz)-1000 Hz; 3) the data can trigger responses with low latency requirements (for example, up to 1 second end-to-end from interaction to response as perceived by the user); 4) can be synchronized with other media streams (for example, video or audio media streams); 5) can be synchronized with other interaction data; 6) reliability is optional and determined by individual application requirements; 7) data encoding typically follows proprietary / non-standardized or rapidly evolving formats that depend on applications and interactions; 8) carries privacy-sensitive interaction events and data, such as user viewport descriptions, user posture / orientation tracking data (up to 6 degrees of freedom (DoF)), user gesture tracking data, user body tracking data, customer facial expression tracking data, employee eye tracking data, user motion tracking data, application and AR anchor data and descriptions, etc.

[0059] In some cases, much of this data is transmitted over the network via the WebRTC data channel or similar solutions. However, the WebRTC data channel is not suitable for time-sensitive transport methods that rely on the Stream Control Transmission Protocol (SCTP), and therefore, a real-time transport solution is desirable.

[0060] The services of the above immersive and interactive real-time applications use transport architectures and protocols suitable for real-time. As part of the latter, one of them is the Real-time Transport Protocol (RTP), its corresponding security configuration SRTP and its network-oriented stack Web Real-Time Communication (WebRTC).

[0061] RTP is a media codec-independent network protocol with application layer frames for real-time delivery of multimedia (such as audio, video, etc.) data over Internet Protocol (IP) networks. It is used in conjunction with a sister protocol for control (i.e., RTCP) to provide end-to-end functions such as jitter compensation, packet loss and out-of-order delivery detection, synchronization, and source stream multiplexing.

[0062] Figure 2 An example of an RTP and RTCP stack 200 supporting user interactive data transmission using a real-time transport protocol header extension in accordance with aspects of the present disclosure is illustrated. The stack 200 is an RTP and RTCP protocol stack over an IP network.

[0063] SRTP is a secure version of RTP that provides encryption (primarily by means of payload confidentiality), message authentication and integrity protection (by means of PDU, e.g. header and payload, signatures), and replay attack protection. Similar to RTP, the SRTP sister protocol is the Secure Real-Time Control Protocol (SRTCP). This provides the same functionality as its RTCP counterpart. Thus, in the normal SRTP version, the RTP header information remains accessible but not modifiable, while the payload is encrypted. These security provisions are described below. Figure 4 The bottom part is shown. In addition, the key exchange and additional security parameters required for using SRTP are based on the Datagram Transport Layer Security (DTLS) key exchange process. For these reasons, SRTP is used as the transport protocol for media in the WebRTC stack to ensure secure real-time communication (RTC) multimedia communication through a web browser interface.

[0064] Figure 3 An example of a WebRTC stack 300 for supporting user interactive data transmission using a real-time transport protocol header extension according to various aspects of the present disclosure is illustrated. Stack 300 is a WebRTC (SRTP) protocol stack on an IP network. As shown, the IP layer carries signaling from the data plane and the control plane. The data plane stack includes functions for user datagram protocol (UDP), interactive connection establishment (ICE), datagram transport layer security (DTLS), SRTP, SRTCP, media codecs, quality control, and SCTP. ICE can use the session traversal utility of the NAT (STUN) protocol and traverse using a relay around the NAT (TURN) to address real-time media content delivery across heterogeneous networks, NAT rules, and firewalls. The SCTP data plane is primarily dedicated to application data channels and can be non-time critical, while SRTP-based stacks (including control elements (i.e., SRTCP), encoding (i.e., media codecs), and quality of service (QoS) (i.e., quality control)) are dedicated to time-critical transmission.

[0065] Figure 4 The RTP and SRTP packet formats and header information 400 supporting user interactive data transmission using the real-time transport protocol header extension according to aspects of the present disclosure are illustrated. The RTP and SRTP header information is shared as follows: Figure 4 Same format as shown.

[0066] The individual fixed header information and complete header information (including header extension) of the RTP / SRTP packet are briefly summarized as follows. The fixed header information includes V, P, X, CC, M, PT, sequence number, timestamp, synchronization source (SSRC) identifier and contribution source identifier (CSRC). V is 2 bits, which indicates the protocol version used. P is a 1-bit field, which indicates that there are one or more zero-filled octets at the end of the payload, where the filling can be used for a fixed-size encryption block or for carrying multiple RTP / SRTP packets through a lower layer protocol. X is 1 bit, which indicates that the standard fixed RTP / SRTP header will be followed by an RTP header extension that is usually associated with a specific data / profile, which will carry more information about the data (e.g., a frame of an RTP header extension marking video data, or a general RTP header extension such as an RTP / SRTP extension protocol). CC is 4 bits, which indicates the number of contributing media sources (CSRC) following the fixed header.

[0067] M is 1 bit, which is used to mark the information frame boundary in the packet stream, and its behavior is precisely specified by the RTP profile (e.g., H.264, H.265, H.266, AV1, etc.). PT is 7 bits, which indicates the payload type, which is dynamic in the case of video profiles and is negotiated with the help of SDP (e.g., 96 for H.264, 97 for H.265, 98 for AV1, etc.). The sequence number is 16 bits, which indicates that the sequence number of each RTP data packet sent through the session is incremented by 1. The timestamp is 32 bits, which indicates the timestamp of the payload type clock reflecting the sampling moment of the first octet of the RTP data packet (video stream associated with a video frame), while the first timestamp of the first RTP packet is randomly selected. The SSRC identifier is 32 bits, which indicates a random identifier for the source of the RTP packet stream that forms part of the same timing and sequence number space, so that the receiver can group the packets into groups for playback based on the synchronization source. The CSRC identifier is a list of up to 16 CSRC items, each 32 bits, identifying the contributing sources of the payload contained in this packet, given the SSRC identifiers of the contributing sources.

[0068] The complete header information includes the RTP header extension. If the X bit is marked, the RTP header extension is a variable length field; if present, the header extension is attached to the RTP fixed header information after the CSRC list; and the RTP header extension is 32-bit aligned and consists of the following fields: 1) a 16-bit extension identifier, which is defined by the profile and is usually negotiated and determined via the SDP signaling mechanism; 2) a 16-bit length field, which describes the extended header length in multiples of 32 bits, excluding the first 32 bits corresponding to the 16-bit extension identifier and the 16-bit length field itself; and 3) a 32-bit aligned header extension raw data field formatted according to a certain RTP header extension identifier specified format.

[0069] Figure 5 The RTP or SRTP header extension format and syntax 500 for supporting user interactive data transmission using a real-time transport protocol header extension according to aspects of the present disclosure is illustrated. The RTP header extension format and syntax are similar to those of SRTP. In addition, in both RTP and SRTP, only one RTP extension header can be appended to the fixed header information. However, for both RTP and SRTP extensions, there is an underlying protocol to allow multiple RTP header extensions of predetermined types to be appended to the fixed header information of the protocol.

[0070] In one or more implementations, an RTP header extension generated at a source may be ignored by a destination endpoint that does not know how to interpret and process the RTP header extension sent by the source endpoint.

[0071] The techniques discussed in this article use SRTP header extensions as a mechanism to transmit real-time interactive data related to emerging interactive and immersive applications in the context of XR, CG or similar application categories. Specifically, the technical solutions discussed in this article use the SRTP protocol with a generic header extension framework, and in addition, the security provisions for header extension encryption in the SRTP protocol provide additional encryption for header extensions including interactive data. Therefore, the proposed solution creates a customizable lightweight transport subprotocol on the SRTP extension header, which is agnostic and extensible to transmit any data format and syntax according to some predetermined extension header profile during the SDP establishment of the SRTP connection. It is worth noting that the same mechanism can also be used in the WebRTC context, where SRTP is the underlying transport protocol for media and time-sensitive data.

[0072] In one or more implementations, user physical interactions in the physical world are converted to the virtual domain by XR, CG devices. To this end, many embedded sensors record physical device interactions, that is, physical interactions of users with XR, CG or similar devices, which sample physical stimuli and further process measurements at a high rate. In some implementations, this physical interaction can represent one or more of the following: sampling of posture information, tracking of gaze and eye movements, tracking of hands and recognition of gestures, tracking of facial expressions and recognition of emotions, spatial tracking, AR anchoring or viewport description, etc. All of the latter are represented in current systems by various data encodings averaging up to 100 bytes.

[0073] In one or more implementations, the data is regularly generated by a sensor device coupled to a processor at a rate of up to 250 Hz (e.g., 250 times per second). Additionally or alternatively, the output frequency can be up to 1000 Hz (e.g., 1000 times per second). Additionally or alternatively, the sensor device and processor coupled together generate events for interactive data, such as pointer detection on a virtual object, a nod or change in field of view, focusing of elements within a viewport, etc. In this case, the event is determined by the processor applying a threshold to the sampled sensor data to determine whether the event trigger condition is met. When the trigger condition is reached, the processor creates an interactive data event. In this case, generating the interactive short messages discussed herein is also referred to as event-based.

[0074] Since the syntax of the interactive data may vary from implementation to implementation and may also vary from application to application, a common syntax for interactive message data is supported. In one or more implementations, the data format of the interactive message includes, at least in part, a subsequent continuation element, a syntax identifier (ID) field, an interactive message type field, an interactive message timing attribute field, an interactive message raw data length field, and an interactive message raw data payload field. The syntax identifier field may be a fixed length (e.g., 8 bits) and signals the format of the raw interactive data payload included in the RTP extension header.

[0075] The interaction message type field may be of fixed length and the type of the message is determined from a dictionary of possible types corresponding to the grammar identifier. For example, in one or more implementations, a grammar identifier may have L possible message types - pose tracking, head tracking, hand tracking, gesture detection, etc., and therefore, the length of this field is at least log2(L) bits.

[0076] The Interaction Message Timing Attributes field may be of fixed length (e.g., 16 bits) and determine a fine-tuning timing element used for fine synchronization of the Interaction Message content relative to the RTP timestamp associated with the media payload type of the RTP extension header carrying the Interaction data. For example, in one or more implementations, this field may encode a signed 16-bit duration in milliseconds relative to the RTP timestamp for synchronization, while in one or more implementations, a value of 0 for such a timing attribute field will indicate that fine synchronization is not used and the RTP timestamp will be used for any further synchronization purposes.

[0077] The interactive message raw data length field may be of a fixed length (eg, 8 bits) and signals the number of bytes of interactive message raw data that will follow.

[0078] The interactive message raw data payload field may be of variable length and contain interactive information encoded according to the format determined by the syntax identifier field indication. In one or more implementations, the length of this field may be determined by the indication provided by the preceding interactive message raw data length field.

[0079] Figure 6 An example of an RTP or SRTP header extension payload format of a generic interactivity message 600 supporting user interactivity data transmission using a real-time transport protocol header extension in accordance with aspects of the present disclosure is illustrated. Figure 6 The interaction message 600 illustrates basic elements of a general interaction message as data extended by an RTP header.

[0080] In one or more implementations, the above and Figure 6 The basic elements of the interactive messages shown are byte aligned. Therefore, in one or more implementations, Figure 6 As shown, padding may be appended at the end of the interactive message raw data payload to ensure a specific bit alignment (eg, 32-bit alignment corresponding to the RTP header extension).

[0081] Figure 7 An example of an RTP or SRTP header extension format of a generic interaction message 700 supporting user interaction data transmission using a real-time transport protocol header extension in accordance with aspects of the present disclosure is illustrated. The interaction message 700 illustrates a complete RTP header extension representation, including an identifier and length determined by the RTP header extension profile.

[0082] In one or more implementations, Figure 7The RTP header extension format of may be sufficient. Additionally or alternatively, other RTP header extensions are sent by the source device simultaneously with the interactive message data. In this case, a generic RTP header extension format (e.g., as described in "Generic Mechanism for RTP Header Extensions" in IETF Request for Comments (RFC) 8285, October 2017) is used to allow additional extension elements to be signaled within the RTP header extension along with the interactive message data.

[0083] In one or more implementations, for very short interactive data messages up to 16 bytes in length, a single-byte extension format of the general RTP header extension mechanism may be used. This allows other header extension elements to be multiplexed with interactive data on the same RTP / SRTP protocol data unit (PDU).

[0084] Figure 8 An example 800 of a single-byte RTP or SRTP header extension format of an interactive message data element supporting user interactive data transmission using a real-time transport protocol header extension according to various aspects of the present disclosure is illustrated. Example 800 implements the single-byte extended format and the basic interactive message payload described above as one of the RTP header extension elements. In example 800, the interactive message data of 15 bytes in size is transmitted through a single-byte general RTP header extension mechanism (identified by profile ID 0xBEDE) with a total length of 12 words. The interactive message data is the last RTP header extension element after the other two extension elements. The RTP header extension is additionally padded with 3 bytes to ensure that the RTP header extension is 32-bit aligned.

[0085] In one or more implementations, for longer short interactive data messages up to 256 bytes in length, a two-byte extension format of the general RTP header extension mechanism may be used. This also allows the header extension element to be multiplexed with interactive data on the same RTP / SRTP PDU.

[0086] Fig. 9An example 900 of a double-byte RTP or SRTP header extension format of an interactive message data element supporting user interactive data transmission using a real-time transport protocol header extension according to various aspects of the present disclosure is illustrated. Example 900 implements the double-byte extended format and the basic interactive message payload described above as one of the RTP header extension elements. In example 800, interactive message data of 82 bytes in size is transmitted by a double-byte general RTP header extension mechanism (identified by the bit (default is 0x0) determined by the profile ID 0x100+4 application (app)) with a total length of 46 words. The interactive message data is the last RTP header extension element after the other three extension elements, wherein the first extension element is empty. The RTP header extension is additionally filled with 2 bytes to ensure that the RTP header extension is 32-bit aligned.

[0087] In one or more implementations, the signaling of the local ID and extension mapping corresponding to the RTP header extension element of the interactive message data follows the SDP signaling. In this case, during the SDP offer / answer process, the sender and the receiver negotiate and determine the RTP extension mapping, local extension element ID, uniform resource identifier (URI), direction (e.g., 'sendrecv', 'sendonly', 'recvonly', 'inactive'), and namespace (e.g., RTP session scope or RTP media scope) of the interactive message data extension element transmitted by the RTP PDU on the session.

[0088] Fig.10 An example 1000 of an SDP answer supporting user interactive data transmission using a real-time transport protocol header extension in accordance with various aspects of the present disclosure is illustrated. The SDP answer in example 1000 is, for example, for the case of a two-byte generic RTP header extension mechanism that maps a local ID 42 to a URI corresponding to an OpenXR interactive message data model at a session level. The SDP answer indicates that it will only send such interactive message data for both video and audio media associated with the RTP session. In addition, the URI is obtained from a central registry hosting the OpenXR data models and syntaxes, which are associated with and mapped to the local ID 42. The SDP answer in example 1000 also contains another extension element, namely the 3GPP Coordinated Video Direction (CVO), which has a precision of 6 bits and is valid only for the video media of the RTP session.

[0089] Thus, in one or more implementations, the URI of the SDP answer / offer process serves as a unique syntactic identifier and, therefore, does not require Figure 6The information contained in the fixed-length syntax identifier field shown. In this case, the syntax identifier field therefore does not need to be part of the RTP header extension payload format of the interactive message, and the URI is used to determine the syntax of the interactive message data.

[0090] In one or more implementations, the URI is a URN defined by the IETF, which is used to uniquely identify the model and syntax of the interaction message data, while in other implementations, the URI is registered with a central repository (e.g., 'https: / / example.org / 122021 / interaction-ext.html') that collects multiple models and grammars that uniquely identify various interaction message data formats (e.g., 'https: / / example.org / 122021 / interaction-ext.html##openxr', 'https: / / example.org / 122021 / interaction-ext.html#thinkreality', etc.), from which the application can select a preferred interaction message data format.

[0091] In one or more implementations, since the rate of interactive message data (e.g., 250 Hz or 1000 Hz) may be different from the rate of the associated media (e.g., video @ 60 fps and / or audio @ 48 kHz), an empty or 'dummy' RTP PDU may be used to transport the interactive message data. In such an implementation, the RTP PDU contains no payload (e.g., no data) or equivalently contains a zero-length payload. In one or more implementations, prefix padding may be used to ensure that the RTP PDU is 32-bit aligned.

[0092] Back to Figure 1 ,Since the interaction data contains privacy-sensitive inputs that record and track user physical actions (e.g., head movements, gestures, touch interactions, eye movements, gaze, and direction elements), it is meaningful to ensure the security of such data transmitted over heterogeneous networks and terminals in one or more implementations.,In such implementations, the integrity and confidentiality of the data are ensured, and therefore, the interaction message data uses both,authentication and encryption.

[0093] To this end, in one or more implementations, the SRTP protocol is used rather than an unsecured version of RTP to authenticate RTP header extensions that transport interactive message data. Additionally or alternatively, where any of the previously discussed single-byte or two-byte generic RTP header extension mechanisms are used to transport interactive message data as an extension element along with other potential extension elements, SRTP also provides authenticity for interactive message data through a message authentication signature that it appends to a typical RTP PDU to form an SRTP PDU.

[0094] Additionally or alternatively, to further provide confidentiality for interactive message data, an encrypted SRTP header extension confidentiality mechanism is used. This utilizes the same security process that SRTP applies to encrypt the SRTP payload to encrypt selected header extension elements of the SRTP header extension in conjunction with the single-byte or double-byte mechanism of RTP described above. Thus, during the SDP offer / answer process, the encryption of the extension element corresponding to the local ID mapped to the interactive message data can be selected and determined. This in turn provides that the extension element is first encrypted and secondly used to generate the SRTP message signature, thereby providing integrity and confidentiality for the interactive message data.

[0095] In one or more implementations, the encryption-based SRTP header extension confidentiality mechanism may be used with any of the single-byte or two-byte generic RTP header extension mechanisms described previously to provide an extensible and secure multi-extension element framework for transporting interactive message data along with other potential SRTP header extension elements.

[0096] Fig.11 An example 1100 of an SDP answer for supporting user interactive data transmission using a real-time transport protocol header extension in accordance with various aspects of the present disclosure is illustrated. The SDP answer in example 1100 is, for example, an SDP answer for a secure RTP header extension element for interactive message data. This illustrates an example of an SDP offer / answer process for an SRTP transport that selects encryption of an extension element containing interactive message data.

[0097] In example 1100, only the interactive message data is encrypted as Local ID=1 and sent by the responder (e.g., UE device as a head mounted display, input controller, etc.) along with any session video or audio related SRTP PDUs. On the other hand, the CVO extension is not encrypted but authenticated via the SRTP message authentication signature prefixed to the PDU.

[0098] In one or more implementations, key generation and negotiation for SRTP header extension encryption may rely on the SRTP key exchange process, which includes DTLS key exchange and security parameter determination for the encryption scheme used in SRTP. Additionally or alternatively, the SDP security description of the SRTP key (e.g.,

[0099] a=crypto:1AESCM128HMACSHA132\

[0100] inline:NzB4d1BINUAvLEw6UzF3WSJ+PSdFcGdUJShpX1Zj]2~201:32'

[0101] Security parameters that may be used for key exchange and selection of an encryption scheme for protecting one or more SRTP header extension elements.

[0102] In one or more implementations, the above-described SRTP-based process for encrypting and authenticating interactive message data as an SRTP header extension element may be embedded in a WebRTC protocol stack serving interactive applications (such as XR or CG applications) via WebRTC. In such an implementation, SDP signaling, key exchange, and security parameter establishment will be performed based on DTLS key exchange as part of the WebRTC session establishment process. Such one or more implementations benefit the privacy of interactive message data through encryption and authentication, thereby protecting selected elements of the RTP header extension from eavesdropping or tracking by means of a man-in-the-middle or network attack on the WebRTC protocol stack attack surface (e.g., an untrusted or malicious TURN relay server).

[0103] In one or more implementations, since the rate of interactive message data (e.g., 250 Hz or 1000 Hz) may be different from the rate of the associated media (e.g., video @ 60 fps and / or audio @ 48 kHz), empty or 'dummy' SRTP PDUs may be used to securely transport the interactive message data. In this case, the SRTP PDU contains no payload, or equivalently contains a zero-length payload. In one or more implementations, prefix padding may be used to ensure that the SRTP PDUs are aligned (e.g., 32-bit aligned).

[0104] Therefore, the technology discussed in this article provides for transmitting small real-time interactive data specific to interactive applications (e.g., XR / CG) through RTP header extension elements, independently or together with other header extension elements, and authenticating and encrypting header extension elements used to transmit interactive data.

[0105] Fig.12An example of a block diagram 1200 of a device 1202 supporting user interaction data transmission using a real-time transport protocol header extension according to aspects of the present disclosure is illustrated. The device 1202 may be an example of a UE 104 as described herein. The device 1202 may support wireless communications with one or more network entities 102, UEs 104, or any combination thereof. The device 1202 may include components for bidirectional communications, including components for sending and receiving communications (such as a processor 1204, a memory 1206, a transceiver 1208, and an I / O controller 1210). These components may be in electronic communication or otherwise coupled (e.g., operatively, communicatively, functionally, electronically, electrically) via one or more interfaces (e.g., a bus).

[0106] The processor 1204, the memory 1206, the transceiver 1208, or various combinations thereof, or various components thereof may be examples of means for performing various aspects of the present disclosure described herein. For example, the processor 1204, the memory 1206, the transceiver 1208, or various combinations thereof, or components thereof may support methods for performing one or more of the operations described herein.

[0107] In some implementations, the processor 1204, memory 1206, transceiver 1208, or various combinations or components thereof may be implemented in hardware (e.g., in a communications management circuit system). The hardware may include a processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, which is configured to or otherwise supports means for performing the functions described in the present disclosure. In some implementations, the processor 1204 and the memory 1206 coupled to the processor 1204 may be configured to perform one or more functions described herein (e.g., by executing instructions stored in the memory 1206 by the processor 1204).

[0108] For example, according to examples disclosed herein, the processor 1204 can support wireless communication at the device 1202. The processor 1204 can be configured to or otherwise support: sampling one or more physical device interactions by a user to obtain interaction data corresponding to a motion input; generating one or more gesture information elements from the interaction data; including the one or more gesture information elements in one or more transport header extensions of a real-time transport protocol data unit corresponding to a payload of at least one media component; and sending the real-time transport protocol data unit to a remote device.

[0109] Additionally or alternatively, processor 1204 may be configured to or otherwise support: wherein the processor is further configured to cause the device to encrypt one or both of a TLS process and a DTLS process; to generate a message authentication signature based at least in part on the one or more transport header extensions; and to attach the message authentication signature to a real-time transport protocol data unit; wherein the device includes a UE; to generate one or more posture information elements based on an interactive data format syntax, the interactive data format syntax including one or more of the following: a syntax protocol identifier, an interactive short message type, an interactive short message timestamp, an interactive short message original data length, or an interactive short message original data payload; wherein the payload of the real-time transport protocol data unit does not contain data; wherein the frequency of generating posture information elements is based on a processing mode indicating one of periodic interactive short messages and event-based interactive short messages; when the processing mode indicates an event-based interactive short message, an event threshold is used to determine the generation of one or more posture information elements.

[0110] For example, according to examples disclosed herein, the processor 1204 can support wireless communication at the device 1202. The processor 1204 can be configured to or otherwise support a component for: sampling one or more physical device interactions by a user to obtain interaction data corresponding to a motion input; generating one or more gesture information elements from the interaction data; including the one or more gesture information elements in one or more transport header extensions of a real-time transport protocol data unit corresponding to a payload of at least one media component; and sending the real-time transport protocol data unit to a remote device.

[0111] Additionally or alternatively, the processor 1204 may be configured to or otherwise support: encrypting one or more transport header extensions using one or both of a TLS process and a DTLS process; generating a message authentication signature based at least in part on the one or more transport header extensions; and attaching the message authentication signature to a real-time transport protocol data unit; generating one or more posture information elements based on an interactive data format syntax, the interactive data format syntax including one or more of the following: a syntax protocol identifier, an interactive short message type, an interactive short message timestamp, an interactive short message raw data length, or an interactive short message raw data payload; wherein the payload of the real-time transport protocol data unit does not contain data; wherein the frequency of generating the posture information element is based on a processing mode indicating one of a periodic interactive short message and an event-based interactive short message; when the processing mode indicates an event-based interactive short message, using an event threshold to determine the generation of the one or more posture information elements. The processor 1204 may include an intelligent hardware device (e.g., a general-purpose processor, a DSP, a CPU, a microcontroller, an ASIC, an FPGA, a programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, or any combination thereof). In some implementations, the processor 1204 may be configured to operate the memory array using a memory controller. In some other implementations, the memory controller may be integrated into the processor 1204. The processor 1204 may be configured to execute computer-readable instructions stored in a memory (e.g., memory 1206) to cause the device 1202 to perform various functions of the present disclosure.

[0112] According to examples disclosed herein, a processor 1204 of a device 1202, such as a UE 104, may support wireless communication. The processor 1204 includes at least one controller coupled to at least one memory and configured to cause the processor to: sample one or more physical device interactions by a user to obtain interaction data corresponding to an action input; generate one or more gesture information elements from the interaction data input; include the one or more gesture information elements in one or more transport header extensions of a real-time transport protocol data unit corresponding to a payload of at least one media component; and send the real-time transport protocol data unit to a remote device.

[0113] The memory 1206 may include random access memory (RAM) and read-only memory (ROM). The memory 1206 may store computer-readable computer executable code, which includes instructions that, when executed by the processor 1204, cause the device 1202 to perform various functions described herein. The code may be stored in a non-transitory computer-readable medium such as a system memory or another type of memory. In some implementations, the code may not be directly executed by the processor 1204, but may cause a computer (e.g., when compiled and executed) to perform the functions described herein. In some implementations, the memory 1206 may include a basic I / O system (BIOS), which may control basic hardware or software operations, such as interaction with peripheral components or devices.

[0114] I / O controller 1210 can manage input and output signals of device 1202. I / O controller 1210 can also manage peripheral devices that are not integrated into device 1202. In some implementations, I / O controller 1210 can represent a physical connection or port to an external peripheral device. In some implementations, I / O controller 1210 can utilize an operating system, such as

[0115] Or other known operating systems. In some implementations, I / O controller 1210 can be implemented as part of a processor such as processor 1204. In some implementations, a user can interact with device 1202 via I / O controller 1210 or via hardware components controlled by I / O controller 1210.

[0116] In some implementations, the device 1202 may include a single antenna 1212. However, in some other implementations, the device 1202 may have more than one antenna 1212 (i.e., multiple antennas), including multiple antenna panels or antenna arrays, which are capable of simultaneously sending or receiving multiple wireless transmissions. The transceiver 1208 can communicate bidirectionally via one or more antennas 1212, wired or wireless links as described herein. For example, the transceiver 1208 can represent a wireless transceiver and can communicate bidirectionally with another wireless transceiver. The transceiver 1208 may also include a modem for modulating packets, providing the modulated packets to one or more antennas 1212 for transmission, and demodulating packets received from one or more antennas 1212.

[0117] Fig.13An example of a block diagram 1300 of a device 1302 supporting user interaction data transmission using a real-time transport protocol header extension in accordance with aspects of the present disclosure is illustrated. The device 1302 may be an example of a network entity 102 as described herein. The device 1302 may support wireless communications with one or more network entities 102, UEs 104, or any combination thereof. The device 1302 may include components for bidirectional communications, including components for sending and receiving communications (such as a processor 1304, a memory 1306, a transceiver 1308, and an I / O controller 1310). These components may be in electronic communication or otherwise coupled (e.g., operatively, communicatively, functionally, electronically, electrically) via one or more interfaces (e.g., a bus).

[0118] The processor 1304, the memory 1306, the transceiver 1308, or various combinations thereof, or various components thereof may be examples of means for performing various aspects of the present disclosure described herein. For example, the processor 1304, the memory 1306, the transceiver 1308, or various combinations thereof, or components thereof may support methods for performing one or more of the operations described herein.

[0119] In some implementations, the processor 1304, memory 1306, transceiver 1308, or various combinations or components thereof may be implemented in hardware (e.g., in a communications management circuit system). The hardware may include a processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, configured to or otherwise support means for performing the functions described in the present disclosure. In some implementations, the processor 1304 and the memory 1306 coupled to the processor 1304 may be configured to perform one or more functions described herein (e.g., by executing instructions stored in the memory 1306 by the processor 1304).

[0120] For example, according to the examples disclosed herein, the processor 1304 can support wireless communication at the device 1302. The processor 1304 can be configured to or otherwise support: receiving a real-time transport protocol data unit corresponding to a payload of at least one media component from a UE, and one or more posture information elements included in one or more transport header extensions of the real-time transport protocol data unit; processing the one or more posture information elements to obtain a graphics rendering interaction and a graphics rendering output; and sending the graphics rendering output and the graphics rendering interaction to the UE.

[0121] Additionally or alternatively, the processor 1304 may be configured to or otherwise support: determining graphics rendering interaction data based at least in part on one or more of the following: a syntax protocol identifier, an interaction short message type, an interaction short message timestamp, an interaction short message raw data length, or an interaction short message raw data payload included in each of one or more interaction short messages including a gesture information element, and generating graphics rendering output based on the graphics rendering interaction data; wherein determining the graphics rendering interaction data further comprises at least one of the following: selection of one of the interaction short messages, or interpolation of one or more interaction short messages; receiving interaction data based at least in part on out-of-band session description signaling, the out-of-band session description signaling determining a session configuration for receiving a real-time transport protocol data unit containing one or more interaction data, and transmitting graphics rendering interaction and graphics rendering output; decrypting one or more transport header extensions using one or both of a TLS process and a DTLS process; authenticating one or more transport header extensions based at least in part on a message authentication signature attached to the real-time transport protocol data unit; wherein the payload of the real-time transport protocol data unit does not contain data.

[0122] For example, according to the examples disclosed herein, the processor 1304 may support wireless communication at the device 1302. The processor 1304 may be configured to or otherwise support a component for: receiving a real-time transport protocol data unit corresponding to a payload of at least one media component from a UE, and one or more posture information elements included in one or more transport header extensions of the real-time transport protocol data unit; processing the one or more posture information elements to obtain a graphics rendering interaction and a graphics rendering output; and sending the graphics rendering output and the graphics rendering interaction to the UE.

[0123] Additionally or alternatively, the processor 1304 may be configured to or otherwise support: determining graphics rendering interaction data based at least in part on one or more of the following: a syntax protocol identifier, an interaction short message type, an interaction short message timestamp, an interaction short message raw data length, or an interaction short message raw data payload included in each of one or more interaction short messages including a gesture information element; and generating graphics rendering output based on the graphics rendering interaction data; wherein determining the graphics rendering interaction data includes at least one of the following: selection of one of the interaction short messages, or interpolation of one or more interaction short messages; wherein the interaction data is received at least in part based on out-of-band session description signaling, the out-of-band session description signaling determines a session configuration, the session configuration being used for receiving a real-time transport protocol data unit containing one or more interaction data, and transmitting graphics rendering interactions and graphics rendering outputs; decrypting one or more transport header extensions using one or both of a TLS process and a DTLS process; authenticating one or more transport header extensions based at least in part on a message authentication signature attached to the real-time transport protocol data unit; wherein the payload of the real-time transport protocol data unit does not contain data.

[0124] According to examples disclosed herein, a processor 1304 of a device 1302 (such as a network entity 102) may support wireless communication. The processor 1304 includes at least one controller, the at least one controller is coupled to at least one memory and is configured to cause the processor to: receive a real-time transport protocol data unit corresponding to a payload of at least one media component from a UE, and one or more posture information elements included in one or more transport header extensions of the real-time transport protocol data unit; process the one or more posture information elements to obtain a graphics rendering interaction and a graphics rendering output; and send the graphics rendering output and the graphics rendering interaction to the UE.

[0125] The processor 1304 may include an intelligent hardware device (e.g., a general purpose processor, a DSP, a CPU, a microcontroller, an ASIC, an FPGA, a programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, or any combination thereof). In some implementations, the processor 1304 may be configured to operate a memory array using a memory controller. In some other implementations, the memory controller may be integrated into the processor 1304. The processor 1304 may be configured to execute computer-readable instructions stored in a memory (e.g., memory 1306) to cause the device 1302 to perform various functions of the present disclosure.

[0126] The memory 1306 may include random access memory (RAM) and read-only memory (ROM). The memory 1306 may store computer-readable computer executable code, which includes instructions that, when executed by the processor 1304, cause the device 1302 to perform various functions described herein. The code may be stored in a non-transitory computer-readable medium such as a system memory or another type of memory. In some implementations, the code may not be directly executed by the processor 1304, but may cause a computer (e.g., when compiled and executed) to perform the functions described herein. In some implementations, the memory 1306 may include a basic I / O system (BIOS), which may control basic hardware or software operations, such as interaction with peripheral components or devices.

[0127] I / O controller 1310 can manage input and output signals of device 1302. I / O controller 1310 can also manage peripheral devices that are not integrated into device 1302. In some implementations, I / O controller 1310 can represent a physical connection or port to an external peripheral device. In some implementations, I / O controller 1310 can utilize an operating system, such as

[0128] Or other known operating systems. In some implementations, I / O controller 1310 can be implemented as part of a processor such as processor 1304. In some implementations, a user can interact with device 1302 via I / O controller 1310 or via hardware components controlled by I / O controller 1310.

[0129] In some implementations, the device 1302 may include a single antenna 1312. However, in some other implementations, the device 1302 may have more than one antenna 1312 (i.e., multiple antennas), including multiple antenna panels or antenna arrays, which are capable of simultaneously sending or receiving multiple wireless transmissions. The transceiver 1308 can communicate bidirectionally via one or more antennas 1312, wired or wireless links as described herein. For example, the transceiver 1308 can represent a wireless transceiver and can communicate bidirectionally with another wireless transceiver. The transceiver 1308 may also include a modem for modulating packets, providing the modulated packets to one or more antennas 1312 for transmission, and demodulating packets received from one or more antennas 1312.

[0130] Fig.14 A flow chart of a method 1400 for supporting user interactive data transmission using a real-time transport protocol header extension according to aspects of the present disclosure is illustrated. The operations of the method 1400 may be implemented by the apparatus described herein or components thereof. For example, the operations of the method 1400 may be implemented by reference to Figures 1 to 13The UE 104 described herein performs. In some implementations, the device may execute a set of instructions to control the functional elements of the device to perform the described functions. Additionally or alternatively, the device may use dedicated hardware to perform various aspects of the described functions.

[0131] At 1405, the method may include sampling one or more physical device interactions performed by the user to obtain interaction data corresponding to the action input. The operations of 1405 may be performed according to the examples described herein. In some implementations, aspects of the operations of 1405 may be described by reference to Figure 1 The device is used to perform the above.

[0132] At 1410, the method may include generating one or more gesture information elements from the interaction data input. The operations of 1410 may be performed according to the examples described herein. In some implementations, aspects of the operations of 1410 may be described by reference to Figure 1 The device is used to perform the above.

[0133] At 1415, the method may include including one or more posture information elements in one or more transport header extensions of a real-time transport protocol data unit corresponding to a payload of at least one media component. The operations of 1415 may be performed according to examples described herein. In some implementations, aspects of the operations of 1415 may be described by reference to Figure 1 The device is used to perform the above.

[0134] At 1420, the method may include sending a real-time transport protocol data unit to the remote device. The operations of 1420 may be performed according to the examples described herein. In some implementations, aspects of the operations of 1420 may be described by reference to Figure 1 The device is used to perform the above.

[0135] Fig.15 1 is a flow chart of a method 1500 for supporting user interactive data transmission using a real-time transport protocol header extension according to aspects of the present disclosure. The operations of the method 1500 may be implemented by the apparatus described herein or components thereof. For example, the operations of the method 1500 may be implemented by reference to Figures 1 to 13 The UE 104 described herein performs. In some implementations, the device may execute a set of instructions to control the functional elements of the device to perform the described functions. Additionally or alternatively, the device may use dedicated hardware to perform various aspects of the described functions.

[0136] At 1505, the method may include encrypting one or more transport header extensions using one or both of a TLS process and a DTLS process. The operations of 1505 may be performed according to the examples described herein. In some implementations, aspects of the operations of 1505 may be described by reference to Figure 1 The device is used to perform the above.

[0137] Fig.16 1 shows a flow chart of a method 1600 for supporting user interactive data transmission using a real-time transport protocol header extension according to aspects of the present disclosure. The operations of the method 1600 may be implemented by the devices described herein or components thereof. For example, the operations of the method 1600 may be implemented by reference to Figures 1 to 13 The UE 104 described herein performs. In some implementations, the device may execute a set of instructions to control the functional elements of the device to perform the described functions. Additionally or alternatively, the device may use dedicated hardware to perform various aspects of the described functions.

[0138] At 1605, the method may include generating a message authentication signature based at least in part on one or more transport header extensions. The operations of 1605 may be performed according to examples described herein. In some implementations, aspects of the operations of 1605 may be described with reference to Figure 1 The device is used to perform the above.

[0139] At 1610, the method may include appending a message authentication signature to a real-time transport protocol data unit. The operations of 1610 may be performed according to examples described herein. In some implementations, aspects of the operations of 1610 may be described by reference to Figure 1 The device is used to perform the above.

[0140] Fig.17 1700 is a flowchart of a method 1700 for supporting user interactive data transmission using a real-time transport protocol header extension according to aspects of the present disclosure. The operations of the method 1700 may be implemented by the devices described herein or components thereof. For example, the operations of the method 1700 may be implemented by reference to Figures 1 to 13 The network entity 102 performs. In some implementations, the device may execute a set of instructions to control the functional elements of the device to perform the described functions. Additionally or alternatively, the device may use dedicated hardware to perform various aspects of the described functions.

[0141] At 1705, the method may include receiving from the UE a real-time transport protocol data unit corresponding to a payload of at least one media component, and one or more posture information elements included in one or more transport header extensions of the real-time transport protocol data unit. The operations of 1705 may be performed according to the examples described herein. In some implementations, aspects of the operations of 1705 may be described with reference to Figure 1 The device is used to perform the above.

[0142] At 1710, the method may include processing one or more gesture information elements to obtain a graphics rendering interaction and a graphics rendering output. The operations of 1710 may be performed according to the examples described herein. In some implementations, aspects of the operations of 1710 may be described by reference to Figure 1 The device is used to perform the above.

[0143] At 1715, the method may include sending graphics rendering output and graphics rendering interactions to the UE. The operations of 1715 may be performed according to the examples described herein. In some implementations, aspects of the operations of 1715 may be described by reference to Figure 1 The device is used to perform the above.

[0144] Fig.18 A flow chart of a method 1800 for supporting user interactive data transmission using a real-time transport protocol header extension according to aspects of the present disclosure is shown. The operations of the method 1800 may be implemented by the devices described herein or components thereof. For example, the operations of the method 1800 may be implemented by reference to Figures 1 to 13 The network entity 102 performs. In some implementations, the device may execute a set of instructions to control the functional elements of the device to perform the described functions. Additionally or alternatively, the device may use dedicated hardware to perform various aspects of the described functions.

[0145] At 1805, the method may include obtaining a syntax protocol identifier from one or more interactive short messages. The operations of 1805 may be performed according to the examples described herein. In some implementations, aspects of the operations of 1805 may be described by reference to Figure 1 The device is used to perform the above.

[0146] At 1810, the method may include processing one or more interactive short messages based at least in part on the grammatical protocol identifier to obtain an interactive response. The operations of 1810 may be performed according to the examples described herein. In some implementations, aspects of the operations of 1810 may be described by reference to Figure 1 The device is used to perform the above.

[0147] It should be noted that the methods described herein describe possible implementations, and that the operations and steps may be rearranged or otherwise modified, and that other implementations are possible. Furthermore, aspects from two or more methods may be combined.

[0148] The various illustrative blocks and components described in conjunction with the disclosure herein may be implemented or performed with a general purpose processor, DSP, ASIC, CPU, FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in an alternative, the processor may be any processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in combination with a DSP core, or any other such configuration).

[0149] The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored on or sent via a computer-readable medium as one or more instructions or codes. Other examples and implementations are within the scope of this disclosure and the appended claims. For example, due to the nature of software, the functions described herein may be implemented using software executed by a processor, hardware, firmware, hard wiring, or any combination of these. Features that implement the functions may also be physically located in various locations, including being distributed so that portions of the functions are implemented in different physical locations.

[0150] Computer readable media include both non-transient computer storage media and communication media, and communication media include any medium that facilitates the transfer of computer programs from one place to another. Non-transient storage media can be any available medium that can be accessed by a general or special-purpose computer. As an example and not limitation, non-transient computer readable media can include RAM, ROM, electrically erasable programmable ROM (EEPROM), flash memory, compact disk (CD) ROM or other optical disk storage, disk storage or other magnetic storage device, or can be used to carry or store desired program code components in the form of instructions or data structures and can be accessed by a general or special-purpose computer, or any other non-transient medium of a general or special-purpose processor.

[0151] Any connection may be appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology (such as infrared, radio, and microwave) is used to send software from a website, server, or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology (such as infrared, radio, and microwave) is included in the definition of computer-readable medium. Disks and optical disks as used herein include CDs, laser disks, optical disks, digital versatile disks (DVDs), floppy disks, and Blu-ray disks, where disks typically reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above are also included within the scope of computer-readable media.

[0152] As used herein, including in the claims, "or" used in a list of items (e.g., a list of items beginning with a phrase such as "at least one of..." or "one or more of..." or "one or both of...") indicates an inclusive list, so that, for example, a list of at least one of A, B, or C means A or B or C or AB or AC or BC or ABC (i.e., A and B and C). Similarly, a list of at least one of A, B, or C refers to A or B or C or AB or AC or BC or ABC (i.e., A and B and C). In addition, as used herein, the phrase "based on" should not be interpreted as a reference to a set of closed conditions. For example, without departing from the scope of the present disclosure, an example step described as "based on condition A" may be based on both condition A and condition B. In other words, as used herein, the phrase "based on" should be interpreted in the same manner as the phrase "based at least in part on". In addition, as used herein, including in the claims, a "set" may include one or more elements.

[0153] When referring to a network entity, the terms "send," "receive," or "transmit" may refer to any part of a network entity of a RAN (e.g., base station, CU, DU, RU) that communicates with another device (e.g., directly or via one or more other network entities).

[0154] The descriptions described herein in conjunction with the accompanying drawings describe example configurations and do not represent all examples that may be implemented or within the scope of the claims. The term "example" as used herein means "used as an example, instance, or illustration," rather than "preferred" or "better than other examples." The detailed description includes specific details to facilitate understanding of the described techniques. However, these techniques may be practiced without these specific details. In some cases, known structures and devices are shown in block diagram form to avoid confusing the concepts of the described examples.

[0155] The description herein is provided to enable one of ordinary skill in the art to make or use the present disclosure. Various modifications to the present disclosure will be apparent to one of ordinary skill in the art, and the general principles defined herein may be applied to other variations without departing from the scope of the present disclosure. Therefore, the present disclosure is not limited to the examples and designs described herein, but conforms to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A device for wireless communication, comprising: at least one memory; as well as at least one processor coupled to the at least one memory and configured to cause the apparatus to: sampling one or more physical device interactions performed by the user to obtain interaction data corresponding to the action input; generating one or more gesture information elements from the interaction data; including the one or more posture information elements in one or more transport header extensions of a real-time transport protocol data unit corresponding to a payload of at least one media component; The real-time transport protocol data unit is sent to a remote device.

2. The apparatus of claim 1, wherein the at least one processor is further configured to cause the apparatus to: encrypt the one or more transport header extensions using one or both of a Transport Layer Security (TLS) process and a Datagram Transport Layer Security (DTLS) process.

3. The apparatus of claim 1, wherein the apparatus comprises a user equipment (UE).

4. The device according to claim 1, wherein the at least one processor is further configured to enable the device to: generate the one or more posture information elements based on an interactive data format syntax, wherein the interactive data format syntax includes one or more of the following: a syntax protocol identifier, an interactive short message type, an interactive short message timestamp, an interactive short message original data length, or an interactive short message original data payload.

5. The apparatus of claim 1, wherein the payload of the real-time transport protocol data unit does not contain data. 6 . The apparatus of claim 1 , wherein a frequency of generating the gesture information element is based on a processing mode indicating one of a periodic interactive short message and an event-based interactive short message.

7. The apparatus according to claim 6, wherein the at least one processor is further configured to enable the apparatus to: when the processing mode indicates the event-based interactive short message, use an event threshold to determine generation of the one or more gesture information elements.

8. An apparatus for wireless communication, comprising: at least one memory; as well as at least one processor coupled to the at least one memory and configured to cause the apparatus to: receiving from a user equipment (UE) a real-time transport protocol data unit corresponding to a payload of at least one media component and one or more posture information elements included in one or more transport header extensions of the real-time transport protocol data unit; Processing the one or more gesture information elements to obtain graphics rendering interaction and graphics rendering output; The graphics rendering output and the graphics rendering interaction are sent to the UE.

9. The apparatus of claim 8, wherein the at least one processor is further configured to cause the apparatus to: The graphics rendering interaction data is determined based at least in part on one or more of: a syntax protocol identifier, an interaction short message type, an interaction short message timestamp, an interaction short message raw data length, or an interaction short message raw data payload included in each of the one or more interaction short messages including the gesture information element, and The graphics rendering output is generated based on the graphics rendering interaction data.

10. The apparatus according to claim 9, wherein determining the graphics rendering interaction data further comprises at least one of the following: selection of one of the interaction short messages, or interpolation of the one or more interaction short messages.

11. An apparatus according to claim 9, wherein the at least one processor is further configured to enable the apparatus to: receive the interaction data at least partially based on out-of-band session description signaling, the out-of-band session description signaling determining a session configuration, the session configuration being used for receiving the real-time transport protocol data unit containing the one or more interaction data, and transmitting the graphics rendering interaction and the graphics rendering output.

12. The apparatus of claim 8, wherein the at least one processor is further configured to cause the apparatus to: decrypt the one or more transport header extensions using one or both of a Transport Layer Security (TLS) process and a Datagram Transport Layer Security (DTLS) process.

13. A method comprising: sampling one or more physical device interactions performed by the user to obtain interaction data corresponding to the action input; generating one or more gesture information elements from the interaction data; including the one or more posture information elements in one or more transport header extensions of a real-time transport protocol data unit corresponding to a payload of at least one media component; as well as The real-time transport protocol data unit is sent to a remote device.

14. A processor for wireless communication, comprising: at least one controller coupled to at least one memory and configured to cause the processor to: sampling one or more physical device interactions performed by the user to obtain interaction data corresponding to the action input; generating one or more gesture information elements from the interaction data input; including the one or more posture information elements in one or more transport header extensions of a real-time transport protocol data unit corresponding to a payload of at least one media component; The real-time transport protocol data unit is sent to a remote device.

15. The processor of claim 14, wherein the at least one controller is further configured to cause the processor to: encrypt the one or more transport header extensions using one or both of a Transport Layer Security (TLS) process and a Datagram Transport Layer Security (DTLS) process.

16. The processor of claim 14, wherein the at least one controller is further configured to cause the processor to: generating a message authentication signature based at least in part on the one or more transport header extensions; and The message authentication signature is appended to the real-time transport protocol data unit.

17. A processor according to claim 14, wherein the at least one controller is further configured to enable the processor to: generate the one or more posture information elements based on an interactive data format syntax, wherein the interactive data format syntax includes one or more of the following: a syntax protocol identifier, an interactive short message type, an interactive short message timestamp, an interactive short message raw data length, or an interactive short message raw data payload.

18. The processor of claim 14, wherein the payload of the real-time transport protocol data unit does not contain data.

19. The processor of claim 14, wherein a frequency of generating the gesture information element is based on a processing mode indicating one of a periodic interactive short message and an event-based interactive short message.

20. The processor according to claim 19, wherein the at least one controller is further configured to enable the processor to: use an event threshold to determine the generation of the one or more gesture information elements when the processing mode indicates the event-based interactive short message.

Images