Communicating pre-rendered media

Abstract

Embodiments of systems and methods for communicating rendered media to a user equipment (UE) may include generating pre-rendered content for processing by the UE based on pause information received from the UE; generating description information based on the pre-rendered content configured to enable the UE to perform a rendering operation using the pre-rendered content; and transmitting the description information and the pre-rendered content to the UE.

Description

[Technical Field]

[0001] (Related Applications)

[0001] This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 383,478, entitled "Communicating Pre-rendered Media," filed November 11, 2022, and U.S. Non-Provisional Patent Application No. 18 / 506,024, entitled "Communicating Pre-rendered Media," filed November 9, 2023, the entire contents of which are incorporated herein by reference for all purposes. [Background technology]

[0002]

[0002] Devices such as augmented reality (AR) glasses may execute applications that provide rich media or multimedia output. However, applications that generate AR output and other similar outputs require a large amount of computation to be performed in a relatively short period of time. Some endpoint devices are unable to perform such computations under such constraints. To accomplish such computations, some endpoint devices may send a portion of their computational workload to another computing device and receive completed computational output from the other computing device. In some contexts, such as AR, virtual reality games, and other similar computationally intensive implementations, such collaborative processing may be referred to as "split rendering." Summary of the Invention

[0003]

[0003] Various aspects include a method and a network computing device configured to perform the method for communicating information needed to enable communication of rendered media to a user equipment (UE). Various aspects may include receiving pause information from the UE, generating pre-rendered content for processing by the UE based on the pause information received from the UE, generating description information based on the pre-rendered content configured to enable the UE to perform a rendering operation using the pre-rendered content, transmitting the description information to the UE, and transmitting the pre-rendered content to the UE.

[0004] In some aspects, the description information may be configured to indicate buffer information for one or more buffers to which the network computing device will stream the pre-rendered content. In some aspects, the description information may be configured to indicate view configuration information for the pre-rendered content. In some aspects, the description information may be configured to indicate an array of layer view objects. In some aspects, the description information may be configured to indicate eye visibility information for the pre-rendered content. In some aspects, the description information may be configured to indicate composition layer information for the pre-rendered content. In some aspects, the description information may be configured to indicate composition layer type information for the pre-rendered content. In some aspects, the description information may be configured to indicate audio configuration properties for the pre-rendered content.

[0005] Some aspects may include receiving from the UE an uplink data description that may be configured to indicate information regarding content to be pre-rendered for processing by the UE, and generating the pre-rendered content for processing by the UE based on the pause information received from the UE may include generating the pre-rendered content based on the uplink data description. In some aspects, transmitting the description information to the UE may include transmitting a packet header extension to the UE that includes information that may be configured to enable the UE to process the pre-rendered content. In some aspects, transmitting the description information to the UE may include transmitting a data channel message to the UE that includes information that may be configured to enable the UE to process the pre-rendered content.

[0006]

[0006] Further aspects include a network computing device having a memory and a processing system including one or more processors configured to perform one or more operations of any of the methods summarized above. Further aspects include a network computing device configured with processor-executable instructions for performing the operations of any of the methods summarized above. Further aspects include a non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of the network computing device to perform the operations of any of the methods summarized above. Further aspects include a network computing device having means for performing the functions of any of the methods summarized above. Further aspects include a system-on-chip for use in a network device including a processor configured to perform one or more operations of any of the methods summarized above.

[0007] Further aspects include a method executed by a processor of a UE, the method may include transmitting pose information to a network computing device, receiving description information from the network computing device configured to enable the UE to perform rendering operations using the pre-rendered content, and transmitting rendered frames to an extended reality (XR) runtime for compositing and display. Some aspects may further include transmitting information regarding UE capabilities and configuration to the network computing device and receiving a scene description for the split rendering session from the network computing device. Some aspects may further include determining whether to select a 3D rendering configuration or a 2D rendering configuration based at least in part on the received scene description, receiving the pre-rendered content via a buffer described in the description information extension of the scene description in response to determining to select the 2D rendering configuration, and receiving information for rendering a 3D scene image and for rendering one or more 3D scene images in response to determining to select the 3D rendering configuration.

[0008]

[0008] Further aspects include a UE having a processor configured to perform one or more operations of any of the methods summarized above. Further aspects include a UE configured with processor-executable instructions to perform operations of any of the methods summarized above. Further aspects include a non-transitory processor-readable storage medium storing processor-executable instructions configured to cause a processor of the UE to perform operations of any of the methods summarized above. Further aspects include a UE having means for performing the functions of any of the methods summarized above. Further aspects include a system-on-chip for use in a UE, including a processor configured to perform one or more operations of any of the methods summarized above. [Brief explanation of the drawings]

[0009] [Figure 1A]

[0009] FIG. 1 is a system block diagram illustrating an exemplary communication system suitable for implementing any of the various embodiments. [Figure 1B]

[0010] FIG. 1 is a system block diagram illustrating an exemplary split base station architecture suitable for implementing any of the various embodiments. [Figure 1C]

[0011] FIG. 1 is a system block diagram illustrating an example of a split rendering operation suitable for implementing any of the various embodiments. [Figure 2]

[0012] FIG. 1 is a component block diagram illustrating an exemplary computing and wireless modem system suitable for implementing any of the various embodiments. [Figure 3]

[0013] FIG. 1 is a component block diagram illustrating a software architecture including radio protocol stacks for user and control planes in wireless communications suitable for implementing any of the various embodiments. [Figure 4A]

[0014] FIG. 1 is a conceptual diagram illustrating operations performed by an application and an XR runtime, according to various embodiments. [Figure 4B]

[0015] FIG. 1 is a block diagram illustrating the operation of a rendering loop that may be performed by an XR system, according to various embodiments. [Figure 4C]

[0016] FIG. 1 is a conceptual diagram illustrating an XR device view, according to various embodiments. [Figure 4D]

[0017] FIG. 2 is a conceptual diagram illustrating operations performed by a synthesizer, according to various embodiments. [Figure 4E]

[0018] FIG. 1 is a conceptual diagram illustrating an extension configured to include descriptive information, according to various embodiments. [Figure 5A]

[0019] 1 illustrates aspects of descriptive information, according to various embodiments. [Figure 5B] 1 illustrates aspects of descriptive information, according to various embodiments. [Figure 5C] 1 illustrates aspects of descriptive information, according to various embodiments. [Figure 5D] 1 illustrates aspects of descriptive information, according to various embodiments. [Figure 5E] 1 illustrates aspects of descriptive information, according to various embodiments. [Figure 5F] 1 illustrates aspects of descriptive information, according to various embodiments. [Figure 5G] 1 illustrates aspects of descriptive information, according to various embodiments. [Figure 6A]

[0020] FIG. 10 is a process flow diagram illustrating a method performed by a processor of a network computing device to communicate pre-rendered media to a UE, according to various embodiments. [Figure 6B]

[0021] FIG. 10 is a process flow diagram illustrating operations that may be performed by a processor of a network element as part of a method for communicating pre-rendered media to a UE, according to various embodiments. [Figure 6C]

[0022] FIG. 10 is a process flow diagram illustrating operations that may be performed by a processor of a UE according to various embodiments. [Figure 7]

[0023] FIG. 1 is a component block diagram of a network computing device suitable for use with various embodiments. [Figure 8]

[0024] FIG. 1 is a component block diagram of a UE suitable for use with the various embodiments. [Figure 9]

[0025] FIG. 1 is a component block diagram of a UE suitable for use with the various embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0010]

[0026] Various embodiments will be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. References made to specific examples and implementations are for illustrative purposes only and do not limit the scope of the claims.

[0011]

[0027] Various embodiments may include a computing device configured to perform operations for communicating necessary information to enable communication of the rendered media to user equipment, including generating, based on the generated image, description information configured to enable the UE to present the rendered content, and transmitting the description information and the rendered content to the UE. In various embodiments, the description information may be configured to indicate buffer information for one or more buffers to which the network computing device will stream the rendered content, view configuration information for the rendered content, an array of layer view objects, eye visibility information for the rendered content, compositing layer information for the rendered content, compositing layer type information for the rendered content, and / or audio configuration properties for the rendered content.

[0012]

[0028] The term "network computing device" or "network element" is used herein to refer to any one or all of the computing devices that are part of or communicate with a communications network, such as a server, router, gateway, hub device, switch device, bridge device, repeater device, or another electronic device that includes memory, communication components, and a programmable processor.

[0013]

[0029] The term “user equipment (UE)” is used herein to refer to any one or all of computing devices, wireless devices, cellular telephones, smartphones, portable computing devices, personal or mobile multimedia players, laptop computers, tablet computers, smartbooks, ultrabooks, palmtop computers, smart glasses, XR devices, wireless email receivers, multimedia Internet-enabled cellular telephones, medical devices and appliances, biometric sensors / devices, wearable devices including smart watches, smart clothing, smart wristbands, smart jewelry (e.g., smart rings and smart bracelets), entertainment devices (e.g., wireless game controllers, music and video players, satellite radio, etc.), wireless network-enabled Internet of Things (IoT) devices including smart meters / sensors, industrial manufacturing equipment, large and small machines and appliances for home or business use, wireless communication elements in autonomous and semi-autonomous vehicles, wireless devices fixed to or embedded in various mobile platforms, global positioning system devices, and similar electronic devices that include memory, wireless communication components, and a programmable processor.

[0014]

[0030] As used herein, the terms "network," "communications network," and "system" may interchangeably refer to part or all of a communications network or internetwork. A network may include multiple network elements. A network may include a wireless network and / or support one or more functions or services of a wireless network.

[0015]

[0031] As used herein, the terms “wireless network,” “cellular network,” and “wireless communications network” may interchangeably refer to some or all of a wireless network of a carrier associated with a wireless device and / or a subscription on the wireless device. The techniques described herein may be used for various wireless communications networks, such as code division multiple access (CDMA), time division multiple access (TDMA), FDMA, orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), and other networks. Generally, any number of wireless networks may be deployed in a given geographic area. Each wireless network may support at least one radio access technology, which may operate on one or more frequencies or ranges of frequencies. For example, a CDMA network may implement Universal Terrestrial Radio Access (UTRA) (including the Wideband Code Division Multiple Access (WCDMA) standard), CDMA2000 (including the IS-2000, IS-95, and / or IS-856 standards), etc. In another example, a TDMA network may implement GSM Enhanced Data Rates for GSM Evolution (EDGE). In another example, an OFDMA network may implement Evolved UTRA (E-UTRA) (including the LTE standard), Institute of Electrical and Electronics Engineers (IEEE) 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, Flash-OFDM, etc. Reference may be made to wireless networks using the LTE standard, and thus the terms "Universal Terrestrial Radio Access," "E-UTRAN," and "eNodeB" may also be used interchangeably herein to refer to wireless networks. However, such references are provided by way of example only and do not exclude wireless networks using other communication standards. For example, while various third-generation (3G), fourth-generation (4G), and fifth-generation (5G) systems are described herein, these systems are mentioned by way of example only, and future generation systems (e.g., sixth-generation (6G) or higher systems) may be used instead in various examples.

[0016]

[0032] The term "system on a chip" (SOC) is used herein to refer to a single integrated circuit (IC) chip that includes multiple resources or processors integrated on a single substrate. A single SOC may include circuits for digital, analog, mixed-signal, and radio frequency functions. A single SOC may also include any number of general-purpose or special-purpose processors (such as digital signal processors, modem processors, video processors, etc.), memory blocks (such as ROM, RAM, flash, etc.), and resources (such as timers, voltage regulators, oscillators, etc.). A SOC may also include software for controlling the integrated resources and processors, as well as for controlling peripheral devices.

[0017]

[0033] The term "system in package" (SIP) may be used herein to refer to a single module or package that includes multiple resources, computing units, cores, or processors on two or more IC chips, substrates, or SOCs. For example, a SIP may include a single substrate on which multiple IC chips or semiconductor dies are stacked in a vertical configuration. Similarly, a SIP may include one or more multi-chip modules (MCMs) on which multiple ICs or semiconductor dies are packaged in a singulated substrate. A SIP may also include multiple independent SOCs packaged in close proximity and coupled to each other via high-speed communication circuits, such as on a single motherboard or within a single wireless device. The proximity of the SOCs facilitates high-speed communication and sharing of memory and resources.

[0018]

[0034] An endpoint UE may be configured to run various extended reality (XR) applications. XR may include or refer to a variety of services, including virtual reality (VR), augmented reality (AR), mixed reality (MR), and other similar services. The operations performed by applications that generate XR outputs and other similar outputs are computationally intensive, requiring a large amount of calculations to be performed in a relatively short period of time (e.g., ray and path tracing, global lighting calculations, dynamic scene lighting, etc.). Some UEs are unable to meet the required computational load. In some embodiments, a UE may send a portion of its computational workload to another computing device and receive completed computational output from the other computing device. In some embodiments, a UE may request that another computing device generate or pre-render image information for use by the UE in rendering a scene, display, or video frame. In some contexts, such as XR applications, such collaborative processing may be referred to as "split rendering." In various embodiments, the other computing device may perform various pre-rendering operations and provide the UE with pre-rendered content as well as information (e.g., metadata or other suitable information) configured to enable the UE to use the pre-rendered content when rendering a scene, display, or video frame.

[0019]

[0035] In some split rendering operations, the UE may send information about the UE's view (e.g., the UE's pose and / or field of view) and the UE's compositing layer capabilities, as well as information about the UE's rendering capabilities, to a network computing device. The network computing device may pre-render content (e.g., images, image elements, or visual, audio, and / or haptic elements) according to a rendering format(s) that matches the UE's rendering capabilities, and provide the UE with a scene description document that includes information about the rendering format and a location (i.e., a network location) for accessing the stream to obtain the pre-rendered content. The UE may select an appropriate rendering format that matches the UE's capabilities and perform a rendering operation using the pre-rendered content to render an image, display, or video frame, such as an augmented reality image in the case of an AR / XR application.

[0020]

[0036] To communicate information to and from XR applications, UEs and network computing devices may use an interface protocol such as OpenXR. In some embodiments, the protocol may provide an application programming interface (API) that enables communication between XR applications, XR device hardware, and an XR rendering system (sometimes referred to as an "XR runtime"). While various examples and embodiments are described herein with reference to OpenXR as an example, this is not intended as limitation, and various embodiments may employ different interface protocols and other operations for communicating with XR applications.

[0021]

[0037] In OpenXR, an XR application may send a query message to the XR system. In response, the XR system may create an instance (e.g., XrInstance) and generate a session (e.g., XrSession) for the XR application. The application may then start a rendering loop. The application may wait for a display frame opportunity (e.g., xrWaitFrame) and signal the start of frame rendering (e.g., xrBeginFrame). Once rendering is complete, the swap chain may be handed over to a compositor (e.g., xrEndFrame) or another appropriate function of the XR runtime configured to fuse (combine) images from multiple sources into a frame. A "swap chain" is multiple memory buffers used by a device to display image frames. Each time an application presents a new frame for display, the first buffer in the swap chain replaces the displayed buffer. This process is called swapping or flipping. Swap chains (e.g., xrSwapchains) may be limited by the capabilities of the XR system (e.g., xrSystem). Swapchains can be customized when created based on the requirements of the XR application.

[0022]

[0038] Information about the UE's view may also be provided to the XR system. For example, a smartphone or tablet running an XR application may provide a single view on a touchscreen display, while AR glasses or VR goggles may provide two views, such as a stereoscopic view, by presenting a view to each of the user's eyes. Information about the UE's view capabilities may be enumerated to the XR system.

[0023]

[0039] The XR runtime may include a compositor that is responsible for, among other things, compositing layers, reprojecting layers, applying lens distortion, and sending the final image to the UE for display. In some embodiments, an XR application may use multiple layers. Various compositors may support different composite layer types, such as stereo, quad (e.g., a 2D plane in 3D space), cubemap, equirectangular, cylindrical, depth, alpha-blended, and / or other vendor compositing layers.

[0024]

[0040] When operating in split rendering mode, a computing device requested by a UE to perform a pre-rendering operation needs to know information about the UE view and the UE compositing layer capabilities and may negotiate the configuration to be used based on such information. Furthermore, because the computing device may stream the generated pre-rendered content (e.g., images, image elements, or visual, audio, and / or haptic elements) to the UE, the computing device also needs information about the stream. Such configuration may be static or dynamic.

[0025]

[0041] Various embodiments include a method and a network computing device configured to execute the method for communicating pre-rendered media content to a UE. Various embodiments enable the network computing device to describe the output of the pre-rendering operation ("pre-rendered content") to the UE. The pre-rendered content may include images, audio information, haptic information, or other information that the UE may process for presentation to a user by performing a rendering operation. In various embodiments, the pre-rendered content output may be streamed to the UE by the network computing device (acting as a pre-rendering server device) via one or more streamed buffers, such as one or more visual data buffers, one or more audio data buffers, one or more haptic data buffers, etc. The network computing device may describe the pre-rendered content in a scene description document ("description information") that the network computing device sends to the UE. The network computing device may dynamically update the description information, such as during the lifetime of a split rendering session. Additionally, the UE may provide a description of the information (data) sent from the UE to the network computing device as input for use by the network computing device when it is to perform the pre-rendering operation. The UE may transmit such information (data) as one or more uplink streamed buffers.

[0026]

[0042] In various embodiments, the network computing device may generate pre-rendered content for presentation by the UE based on the pause information received from the UE, generate description information based on the generated images configured to enable the UE to perform a rendering operation using the pre-rendered content, and transmit the description information and the pre-rendered content to the UE. In some embodiments, the network computing device may transmit the pre-rendered content via one or more streamed buffers. In some embodiments, the network computing device may configure a Graphics Language Transmission Format (glTF) extension to include information describing buffers that carry the streamed, pre-rendered content. In some embodiments, the network computing device may configure a Moving Picture Experts Group (MPEG) media extension (e.g., MPEG_media extension) to include information describing a stream source (e.g., network location information for the data stream(s)).

[0027]

[0043] In some embodiments, the network computing device may configure the description information with an extension (e.g., which may be called "3GPP_node_prerendered") that describes a pre-rendered content node type (e.g., a new OpenXR node type). In some embodiments, the pre-rendered content node type may indicate the presence of pre-rendered content. In some embodiments, the extension may include visual, audio, and / or haptic information components or information elements. In some embodiments, each information component or information element may describe a set of buffers and associated buffer configurations, such as a raw format (pre-rendered buffer data after decoding, e.g., a red-green-blue-alpha (RGBA) texture image). In some embodiments, the extension may include information describing an uplink buffer for conveying information from the UE to the network computing device, which may include time-dependent metadata such as UE pose information and information about user input. In this way, the network computing device may send information to the UE describing a downlink stream over which the network computing device may send description information and pre-rendered content to the UE, and an uplink stream over which the UE may send information (e.g., UE configuration information, UE capability information, UE pose information, UE field of view information, UE sensor input, etc.) and image information (e.g., scene description information, etc.) to the network computing device.

[0028]

[0044] In various embodiments, the network computing device may configure the description information to include various information usable by the UE to perform rendering operations using the pre-rendered content. In some embodiments, the description information may be configured to indicate buffer information for one or more buffers to which the network computing device will stream the pre-rendered content. The buffers may include one or more streaming buffers, such as a visual data buffer, an audio data buffer, and / or a haptic data buffer. In some embodiments, the description information may be configured to indicate view configuration information for the pre-rendered content. In some embodiments, the description information may be configured to indicate an array of layer view objects. In some embodiments, the description information may be configured to indicate eye visibility information for the pre-rendered content. In some embodiments, the description information may be configured to indicate composition layer information and / or composition layer type information for the pre-rendered content. In some embodiments, the description information may be configured to indicate audio configuration properties for the pre-rendered content.

[0029]

[0045] In some embodiments, the network computing device may receive from the UE an uplink data description configured to indicate information regarding content to be pre-rendered for processing by the UE, and may generate the pre-rendered content based on the uplink data description. In some embodiments, the network computing device may send to the UE a packet header extension including information configured to enable the UE to process the pre-rendered content. In some embodiments, the network computing device may send to the UE a data channel message including information configured to enable the UE to process the pre-rendered content.

[0030]

[0046] Various embodiments improve the operation of network computing devices and UEs by enabling the network computing devices and UEs to describe outputs and / or inputs for split rendering operations. Various embodiments improve the operation of network computing devices and UEs by increasing the efficiency with which the UEs and network computing devices communicate information regarding and perform split rendering operations.

[0031]

[0047] 1A is a system block diagram illustrating an exemplary communication system 100 suitable for implementing any of the various embodiments. The communication system 100 may be a 5G New Radio (NR) network or any other suitable network, such as a Long Term Evolution (LTE) network. While FIG. 1 illustrates a 5G network, later generation networks may include the same or similar elements. Accordingly, references to a 5G network or 5G network elements in the following description are for purposes of illustration and not intended to be limiting.

[0032]

[0048] The communications system 100 may include a heterogeneous network architecture including a core network 140 and various wireless devices (shown in FIG. 1 as user equipment (UE) 120a-120e). The communications system 100 may include an edge network 142 that provides network computing resources in proximity to the wireless devices. The communications system 100 may also include several base stations (shown as BS 110a, BS 110b, BS 110c, and BS 110d) and other network entities. A base station is an entity that communicates with the wireless devices and may also be called a Node B (Node B), an LTE evolved Node B (eNodeB or eNB), an access point (AP), a radio head, a transmit / receive point (TRP), a new radio base station (NR BS), a 5G Node B (NB), a next generation Node B (gNodeB or gNB), etc. Each base station may provide communication coverage for a particular geographic area. In 3GPP, the term "cell" can refer to a coverage area of ​​a base station, a base station subsystem serving this coverage area, or a combination thereof, depending on the context in which the term is used. Core network 140 can be any type of core network, such as an LTE core network (e.g., an EPC network), a 5G core network, etc.

[0033]

[0049] The base stations 110a-110d may provide communication coverage for a macro cell, a pico cell, a femto cell, another type of cell, or a combination thereof. A macro cell may cover a relatively large geographic area (e.g., a few kilometers in radius) and may allow unrestricted access by wireless devices with service subscriptions. A pico cell may cover a relatively small geographic area and may allow unrestricted access by wireless devices with service subscriptions. A femto cell may cover a relatively small geographic area (e.g., a home) and may allow restricted access by wireless devices that have an association with the femto cell (e.g., wireless devices in a Closed Subscriber Group (CSG)). A base station for a macro cell may be referred to as a Macro BS. A base station for a pico cell may be referred to as a Pico BS. A base station for a femto cell may be referred to as a Femto BS or Home BS. 1, base station 110a may be a macro BS for macro cell 102a, base station 110b may be a pico BS for pico cell 102b, and base station 110c may be a femto BS for femto cell 102c. Base stations 110a-110d may support one or multiple (e.g., three) cells. The terms “eNB,” “base station,” “NR BS,” “gNB,” “TRP,” “AP,” “node B,” “5G NB,” and “cell” may be used interchangeably herein.

[0034]

[0050] In some examples, the cells may not be fixed, and the geographic area of ​​the cells may move according to the location of the mobile base station. In some examples, the base stations 110a-110d may be interconnected to each other and to one or more other base stations or network nodes (not shown) in the communication system 100 through various types of backhaul interfaces, such as direct physical connections, virtual networks, or combinations thereof, using any suitable transport network.

[0035]

[0051] The base stations 110a-110d may communicate with the core network 140 over wired or wireless communication links 126. The wireless devices 120a-120e may communicate with the base stations 110a-110d over wireless communication links 122.

[0036]

[0052] The wired communication link 126 may use various wired networks (such as Ethernet, TV cable, telephony, optical fiber, and other forms of physical network connections) that may use one or more wired communication protocols, such as Ethernet, Point-to-Point Protocol, High-Level Data Link Control (HDLC), Advanced Data Communication Control Protocol (ADCCP), and Transmission Control Protocol / Internet Protocol (TCP / IP).

[0037]

[0053] Communications system 100 may also include relay stations (such as relay BS 110d). A relay station is an entity that can receive data transmissions from an upstream station (e.g., a base station or a wireless device) and send the data transmissions to a downstream station (e.g., a wireless device or a base station). A relay station may also be a wireless device that can relay transmissions for other wireless devices. In the example shown in FIG. 1, relay station 110d may communicate with base station 110a and wireless device 120d to facilitate communication between base station 110a and wireless device 120d. A relay station may also be referred to as a relay base station, a relay base station, a repeater, etc.

[0038]

[0054] Communications system 100 may be a heterogeneous network including different types of base stations, e.g., macro base stations, pico base stations, femto base stations, relay base stations, etc. These different types of base stations may have different transmit power levels, different coverage areas, and may have different impacts on interference in communications system 100. For example, macro base stations may have high transmit power levels (e.g., 5-40 watts), while pico base stations, femto base stations, and relay base stations may have lower transmit power levels (e.g., 0.1-2 watts).

[0039]

[0055] Network controller 130 may couple to a set of base stations and provide coordination and control for these base stations. Network controller 130 may communicate with the base stations via a backhaul. The base stations may also communicate with each other directly or indirectly, e.g., via wireless or wireline backhaul.

[0040]

[0056] The wireless devices 120a, 120b, 120c may be dispersed throughout the communication system 100, and each wireless device may be fixed or mobile. A wireless device may also be called an access terminal, a terminal, a mobile station, a subscriber unit, a station, a user equipment (UE), etc.

[0041]

[0057] The macro base station 110a may communicate with the communication network 140 over a wired or wireless communication link 126. The wireless devices 120a, 120b, 120c may communicate with the base stations 110a-110d over a wireless communication link 122.

[0042]

[0058] The wireless communication links 122 and 124 may include multiple carrier signals, frequencies, or frequency bands, each of which may include multiple logical channels. The wireless communication links 122 and 124 may use one or more radio access technologies (RATs). Examples of RATs that may be used in the wireless communication links include 3GPP LTE, 3G, 4G, 5G (such as NR), GSM, code division multiple access (CDMA), wideband code division multiple access (WCDMA), Worldwide Interoperability for Microwave Access (WiMAX), time division multiple access (TDMA), and other mobile telephony communication technology cellular RATs. Further examples of RATs that may be used in one or more of the various wireless communication links in communication system 100 include medium-range protocols such as Wi-Fi, LTE-U, LTE-Direct, LAA, and MuLTEfire, as well as relatively short-range RATs such as ZigBee, Bluetooth, and Bluetooth Low Energy (LE).

[0043]

[0059] Some wireless networks (e.g., LTE) use orthogonal frequency division multiplexing (OFDM) on the downlink and single-carrier frequency division multiplexing (SC-FDM) on the uplink. OFDM and SC-FDM partition the system bandwidth into multiple (K) orthogonal subcarriers, which are also commonly referred to as tones, bins, etc. Each subcarrier may be modulated with data. Generally, modulation symbols are sent in the frequency domain with OFDM and in the time domain with SC-FDM. The spacing between adjacent subcarriers may be fixed, and the total number of subcarriers (K) may depend on the system bandwidth. For example, the subcarrier spacing may be 15 kHz, and the minimum resource allocation (called a "resource block") may be 12 subcarriers (or 180 kHz). Thus, the nominal fast file transfer (FFT) size may be equal to 128, 256, 512, 1024, or 2048 for system bandwidths of 1.25, 2.5, 5, 10, or 20 megahertz (MHz), respectively. The system bandwidth may also be partitioned into subbands. For example, a subband may cover 1.08 MHz (i.e., 6 resource blocks), and there may be 1, 2, 4, 8, or 16 subbands for system bandwidths of 1.25, 2.5, 5, 10, or 20 MHz, respectively.

[0044]

[0060] Although the description of some implementations may use terminology and examples related to LTE technology, some implementations may be applicable to other wireless communication systems, such as New Radio (NR) or 5G networks. NR may utilize OFDM with cyclic prefix (CP) on the uplink (UL) and downlink (DL) and may include support for half-duplex operation using time division duplex (TDD). A single component carrier bandwidth of 100 MHz may be supported. An NR resource block may span 12 subcarriers with a subcarrier bandwidth of 75 kHz over a duration of 0.1 milliseconds (ms). Each radio frame may consist of 50 subframes with a length of 10 ms. Thus, each subframe may be 0.2 ms long. Each subframe may indicate a link direction (i.e., DL or UL) for data transmission, and the link direction for each subframe may be dynamically switched. Each subframe may contain DL / UL data as well as DL / UL control data. Beamforming may be supported, and the beam direction may be dynamically configured. Multiple-input multiple-output (MIMO) transmission with precoding may also be supported. MIMO configurations in the DL may support up to eight transmit antennas with multi-layer DL transmission of up to eight streams and up to two streams per wireless device. Multi-layer transmission with up to two streams per wireless device may be supported. Aggregation of multiple cells may be supported with up to eight serving cells. Alternatively, NR may support an air interface other than an OFDM-based air interface.

[0045]

[0061] Some wireless devices may be considered machine-type communication (MTC) or evolved or extended machine-type communication (eMTC) wireless devices. MTC and eMTC wireless devices include, for example, robots, drones, remote devices, sensors, meters, monitors, location tags, etc. that may communicate with a base station, another device (e.g., a remote device), or some other entity. A wireless computing platform may provide, for example, connectivity for or to a network (e.g., a wide area network such as the Internet or a cellular network) via a wired or wireless communication link. Some wireless devices may be considered Internet of Things (IoT) devices or may be implemented as NB-IoT (narrowband Internet of Things) devices. Wireless devices 120a-120e may be included within a housing that houses components of the wireless device 120a-120e, such as a processor component, a memory component, similar components, or a combination thereof.

[0046]

[0062] In general, any number of communication systems and any number of wireless networks may be deployed in a given geographic area. Each communication system and wireless network may support a particular radio access technology (RAT) and may operate on one or more frequencies. A RAT may also be referred to as a radio technology, air interface, etc. A frequency may also be referred to as a carrier, frequency channel, etc. Each frequency may support a single RAT in a given geographic area to avoid interference between communication systems of different RATs. In some cases, 4G / LTE and / or 5G / NR RAT networks may be deployed. For example, a 5G non-standalone (NSA) network may utilize both a 4G / LTE RAT on the 4G / LTE RAN side of the 5G NSA network and a 5G / NR RAT on the 5G / NR RAN side of the 5G NSA network. Both the 4G / LTE RAN and the 5G / NR RAN may connect to each other and to a 4G / LTE core network (e.g., an evolved packet core (EPC) network) within the 5G NSA network. Other example network configurations may include a 5G standalone (SA) network in which a 5G / NR RAN connects to a 5G core network.

[0047]

[0063] In some implementations, two or more wireless devices 120a-120e (e.g., shown as wireless device 120a and wireless device 120e) may communicate directly (e.g., without using base station 110a-110d as an intermediary for communicating with each other) using one or more sidelink channels 124. For example, the wireless devices 120a-120e may communicate using peer-to-peer (P2P) communication, device-to-device (D2D) communication, a vehicle-to-everything (V2X) protocol (which may include a vehicle-to-vehicle (V2V) protocol, a vehicle-to-infrastructure (V2I) protocol, or similar protocols), a mesh network, or similar network, or a combination thereof. In this case, the wireless devices 120a-120e may perform scheduling operations, resource selection operations, and other operations described elsewhere herein as being performed by base station 110a-110d.

[0048]

[0064] 1B is a system block diagram illustrating an exemplary separated base station 160 architecture suitable for implementing any of the various embodiments. Referring to FIGS. 1A and 1B , the separated base station 160 architecture may include one or more central units (CUs) 162 that can communicate directly with the core network 180 via a backhaul link or indirectly with the core network 180 through one or more separated base station units (e.g., a near-real-time (near-RT) RAN intelligent controller (RIC) 164 via an E2 link, or a non-real-time (non-RT) RIC 168 associated with a service management and orchestration (SMO) framework 166, or both). The CUs 162 can communicate with one or more distributed units (DUs) 170 via respective midhaul links, such as an F1 interface. The DUs 170 can communicate with one or more radio units (RUs) 172 via respective fronthaul links. The RUs 172 can communicate with respective UEs 120 via one or more radio frequency (RF) access links. In some implementations, a UE 120 may be served by multiple RUs 172 simultaneously.

[0049]

[0065] Each of the units (i.e., CU 162, DU 170, RU 172), as well as quasi-RT RIC 164, non-RT RIC 168, and SMO framework 166, may include or be coupled to one or more interfaces configured to receive or transmit signals, data, or information (collectively, signals) over a wired or wireless transmission medium. Each of the units, or an associated processor or controller that provides instructions to the unit's communication interface, may be configured to communicate with one or more of the other units over a transmission medium. For example, a unit may include a wired interface configured to receive signals from or transmit signals to one or more of the other units over a wired transmission medium. Furthermore, the units may include a wireless interface, which may include a receiver, transmitter, or transceiver (such as a radio frequency (RF) transceiver) configured to receive, transmit, or transmit signals over a wireless transmission medium to one or more of the other units.

[0050]

[0066] In some aspects, the CU 162 can host one or more higher-layer control functions. Such control functions may include Radio Resource Control (RRC), Packet Data Convergence Protocol (PDCP), Service Data Adaptation Protocol (SDAP), etc. Each control function may be implemented with an interface configured to communicate signals with other control functions hosted by the CU 162. The CU 162 may be configured to handle user plane functions (i.e., Central Unit-User Plane (CU-UP)), control plane functions (i.e., Central Unit-Control Plane (CU-CP)), or a combination thereof. In some implementations, the CU 162 may be logically divided into one or more CU-UP units and one or more CU-CP units. The CU-UP units, when implemented in an O-RAN configuration, may communicate bidirectionally with the CU-CP units via an interface, such as an E1 interface. The CU 162 may be implemented to communicate with the DU 170, as needed, for network control and signaling.

[0051]

[0067] The DU 170 may correspond to a logical unit including one or more base station functions for controlling the operation of one or more RUs 172. In some aspects, the DU 170 may host one or more of a radio link control (RLC) layer, a medium access control (MAC) layer, and one or more upper physical (PHY) layers (such as modules for forward error correction (FEC) encoding and decoding, scrambling, modulation and demodulation, etc.) at least in part according to a functional division such as that defined by the 3rd Generation Partnership Project (3GPP). In some aspects, the DU 170 may further host one or more lower PHY layers. Each layer (or module) may be implemented with an interface configured to communicate signals with other layers (and modules) hosted by the DU 170 or with control functions hosted by the CU 162.

[0052]

[0068] The lower layer functions may be implemented by one or more RUs 172. In some deployments, the RUs 172 controlled by the DU 170 may correspond to logical nodes hosting RF processing functions, lower PHY layer functions (such as performing fast Fourier transforms (FFTs), inverse FFTs (iFFTs), digital beamforming, physical random access channel (PRACH) extraction and filtering, etc.), or both, based at least in part on a functional division, such as a lower layer functional division. In such an architecture, the RU(s) 172 may be implemented to handle over-the-air (OTA) communications with one or more UEs 120. In some implementations, real-time and non-real-time aspects of control and user plane communications with the RU(s) 172 may be controlled by the corresponding DU 170. In some scenarios, this configuration may enable the DU(s) 170 and CU 162 to be implemented in a cloud-based radio access network (RAN) architecture, such as a virtual RAN (vRAN) architecture.

[0053]

[0069] The SMO framework 166 may be configured to support RAN deployment and provisioning of non-virtualized and virtualized network elements. For non-virtualized network elements, the SMO framework 166 may be configured to support deployment of dedicated physical resources related to RAN coverage requirements, which may be managed via an operation and maintenance interface (e.g., an O1 interface). For virtualized network elements, the SMO framework 166 may be configured to interact with a cloud computing platform (e.g., an open cloud (O-cloud) 176) via a cloud computing platform interface (e.g., an O2 interface) to perform network element lifecycle management (e.g., instantiate virtualized network elements). Such virtualized network elements may include, but are not limited to, a CU 162, a DU 170, an RU 172, and a quasi-RT RIC 164. In some implementations, the SMO framework 166 may communicate with hardware aspects of a 4G RAN, such as an open eNB (O-eNB) 174, via the O1 interface. Additionally, in some implementations, the SMO framework 166 may communicate directly with one or more RUs 172 via the O1 interface. The SMO framework 166 may also include a non-RT RIC 168 configured to support the functionality of the SMO framework 166 .

[0054]

[0070] The non-RT RIC 168 may be configured to include logic functions that enable non-real-time control and optimization of RAN elements and resources, artificial intelligence / machine learning (AI / ML) workflows including model training and updates, or policy-based guidance of applications / features in the quasi-RT RIC 164. The non-RT RIC 168 may be coupled to the quasi-RT RIC 125 or may communicate with the quasi-RT RIC 164 (e.g., via an A1 interface). The quasi-RT RIC 164 may be configured to include logic functions that enable near-real-time control and optimization of RAN elements and resources through data collection and action via one or more CUs 162, one or more DUs 170, or both, and an interface connecting the O-eNB to the quasi-RT RIC 164 (e.g., via an E2 interface).

[0055]

[0071] In some implementations, the non-RT RIC 168 may receive parameters or external enrichment information from an external server to generate the AI / ML models deployed to the quasi-RT RIC 164. Such information may be utilized by the quasi-RT RIC 164 or may be received at the SMO framework 166 or the non-RT RIC 168 from non-network data sources or from network functions. In some examples, the non-RT RIC 168 or the quasi-RT RIC 164 may be configured to adjust RAN behavior or performance. For example, the non-RT RIC 168 may employ AI / ML models to monitor long-term trends and patterns in performance and implement corrective actions through the SMO framework 166 (e.g., reconfiguration via O1) or through the creation of RAN management policies (e.g., A1 policies).

[0056]

[0072] FIG. 1C is a system block diagram of an example system 182 configured to perform split rendering operations suitable for implementing any of the various embodiments. With reference to FIGS. 1A-1C, the system 182 may include a network computing device 184 (an “XR server”) and a UE 186 (an “XR device”). In various embodiments, the network computing device 184 may perform operations to pre-render content (e.g., image data for a 3D scene) into a simpler format that can be transmitted to and processed by the UE 186. In some embodiments, the UE 186 may receive the pre-rendered content and perform operations to render the content. The rendering operations performed by the UE 186 may include final rendering of the image data based on a local compensation process, a local pose compensation operation, and other suitable processing operations.

[0057]

[0073] In various embodiments, the UE 186 may transmit tracking and sensor information 188, such as the UE 186's orientation (e.g., pose rotation), the UE 186's field of view information, three-dimensional coordinates of the image pose, and other suitable information, to the network computing device 184. Using the tracking and sensor information 188, the network computing device 184 may perform operations to pre-render content. In some embodiments, the network computing device 184 may perform operation 190a to generate XR media and operation 190b to perform pre-rendering operations of the generated media based on the UE 186's field of view and other display information. The network computing device 184 may perform operation 190c to encode 2D or 3D media and / or operation 190d to generate XR rendering metadata. The network computing device 184 may perform operation 190e to prepare the encoded media and / or XR rendering metadata for transmission to the UE 186.

[0058]

[0074] The network computing device 184 may transmit the encoded 2D or 3D media and XR metadata 192 to the UE 186. The UE 186 may perform operations to render the pre-rendered content. In some embodiments, the UE 186 may perform operation 194a to receive the encoded 2D or 3D media and XR metadata 192. The UE 186 may perform operation 194b to decode the 2D or 3D media and / or operation 194c to receive, parse, and / or process the XR rendering metadata. The UE 186 may perform operation 194d to render the 2D or 3D media using the XR rendering metadata (this operation may include an asynchronous time warping (ATW) operation). In some embodiments, the UE 186 may also perform local correction operations as part of the content rendering operation. The UE 186 may perform operation 194e to display the rendered content using an appropriate display device. The UE 186 may also perform operations 194f for movement and orientation tracking of the UE 186 and / or receiving input from one or more sensors of the XR device 186. The UE 186 may transmit the movement and orientation tracking information and / or sensor input information to the network computing device 184 as tracking and sensor information 188.

[0059]

[0075] 2 is a component block diagram illustrating an exemplary processing system 200 suitable for implementing any of the various embodiments. The various embodiments may be implemented on processing system 200 including several single-core and multi-core processors implemented in a computing system, which may be integrated into a system-on-chip (SOC) or a system-in-package (SIP).

[0060]

[0076] 1A-2 , the illustrated exemplary processing system 200 (which may be a SIP in some embodiments) includes two SOC processing systems 202, 204 coupled to a clock 206, a voltage regulator 208, and a wireless transceiver 266 configured to transmit and receive wireless communications via an antenna (not shown) to and from a wireless device (e.g., 120a-120e) or base station (e.g., 110a-110d). In some implementations, the first SOC processing system 202 may operate as a central processing unit (CPU) of the wireless device, implementing instructions of a software application program by performing arithmetic, logic, control, and input / output (I / O) operations specified by the instructions. In some implementations, the second processing system SOC 204 may operate as a dedicated processing unit. For example, the second SOC processing system 204 may operate as a dedicated 5G processing unit responsible for managing high-volume, high-speed (e.g., 5 Gbps), or ultra-high-frequency, short-wavelength (e.g., 28 GHz mm-wave spectrum) communications.

[0061]

[0077] The first SOC processing system 202 may include a digital signal processor (DSP) 210, a modem processor 212, a graphics processor 214, an application processor 216, one or more coprocessors 218 (such as a vector coprocessor) connected to one or more of the processors, memory 220, custom circuitry 222, system components and resources 224, an interconnect / bus module 226, one or more temperature sensors 230, a thermal management unit 232, and a thermal power envelope (TPE) component 234. The second SOC processing system 204 may include a 5G modem processor 252, a power management unit 254, an interconnect / bus module 264, multiple mm-wave transceivers 256, memory 258, and various additional processors 260, such as an application processor, a packet processor, etc.

[0062]

[0078] In processing systems 200, 202, and 204, each processor 210, 212, 214, 216, 218, 252, and 260 may include one or more cores, and each processor / core may perform operations independent of the other processors / cores. For example, first SOC processing system 202 may include a processor running a first type of operating system (e.g., FreeBSD, Linux, OS X, etc.) and a processor running a second type of operating system (e.g., Microsoft Windows 10). Additionally, any or all of processors 210, 212, 214, 216, 218, 252, and 260 may be included as part of a processor cluster architecture (e.g., a synchronous processor cluster architecture, an asynchronous or heterogeneous processor cluster architecture, etc.).

[0063]

[0079] First SOC processing system 202 and second SOC processing system 204 may include various system components, resources, and custom circuitry for managing sensor data, analog-to-digital conversion, wireless data transmission, and performing other specialized operations, such as decoding data packets and processing encoded audio and video signals for rendering in a web browser. For example, system components and resources 224 of first SOC processing system 202 may include power amplifiers, voltage regulators, oscillators, phase-locked loops, peripheral bridges, data controllers, memory controllers, system controllers, access ports, timers, and other similar components used to support processors and software clients running on wireless devices. System components and resources 224 and / or custom circuitry 222 may also include circuitry for interfacing with peripheral devices, such as cameras, electronic displays, wireless communication devices, external memory chips, etc.

[0064]

[0080] The first SOC processing system 202 and the second SOC processing system 204 may communicate via an interconnect / bus module 250. The various processors 210, 212, 214, 216, 218 within each processing system may be interconnected to one or more memory elements 220, system components and resources 224, and custom circuitry 222, as well as a thermal management unit 232, via an interconnect / bus module 226. Similarly, the processor 252 may be interconnected to a power management unit 254, a mm-wave transceiver 256, memory 258, and various additional processors 260 via an interconnect / bus module 264. The interconnect / bus modules 226, 250, 264 may include arrays of reconfigurable logic gates and / or implement bus architectures (e.g., CoreConnect, AMBA, etc.). Communication may occur via advanced interconnects such as high-performance networks-on-chip (NoCs).

[0065]

[0081] First SOC processing system 202 and / or second SOC processing system 204 may further include input / output modules (not shown) for communicating with resources external to the SOC, such as clock 206 and voltage regulator 208. Resources external to the SOC (clock 206, voltage regulator 208, etc.) may be shared by two or more of the internal SOC processors / cores.

[0066]

[0082] In addition to the exemplary SIP 200 described above, some implementations may be implemented in a wide variety of computing systems, which may include a single processor, multiple processors, multi-core processors, or any combination thereof.

[0067]

[0083] 3 is a component block diagram illustrating a software architecture 300 including radio protocol stacks for user and control planes in wireless communications suitable for implementing any of the various embodiments. Referring to FIGS. 1A-3, a wireless device 320 may implement software architecture 300 to facilitate communications between the wireless device 320 (e.g., wireless devices 120a-120e, 200) and a base station 350 (e.g., base stations 110a-110d) of a communications system (e.g., 100). In various embodiments, layers in software architecture 300 may form logical connections with corresponding layers in the software of base station 350. Software architecture 300 may be distributed among one or more processors (e.g., processors 212, 214, 216, 218, 252, 260) of a processing system. Although illustrated with respect to one radio protocol stack in a multi-SIM (Subscriber Identity Module) wireless device, software architecture 300 may include multiple protocol stacks, each associated with a different SIM (such as two protocol stacks associated with two SIMs in a dual-SIM wireless communication device). Although described below with respect to an LTE communication layer, software architecture 300 may support any of a variety of standards and protocols for wireless communication and / or may include additional protocol stacks supporting any of a variety of standards and protocols for wireless communication.

[0068]

[0084] The software architecture 300 may include a non-access stratum (NAS) 302 and an access stratum (AS) 304. The NAS 302 may include functions and protocols to support packet filtering, security management, mobility control, session management, traffic and signaling between a wireless device's SIM (e.g., SIM 204) and its core network 140. The AS 304 may include functions and protocols to support communication between a SIM(s) (e.g., SIM(s) 204) and supported access network entities (e.g., base stations). Specifically, the AS 304 may include at least three layers (Layer 1, Layer 2, and Layer 3), each of which may include various sublayers.

[0069]

[0085] In the user and control plane, Layer 1 (L1) of the AS 304 may be a physical layer (PHY) 306 that may oversee functions that enable transmission and / or reception over the air interface via a wireless transceiver (e.g., 266). Examples of such physical layer 306 functions may include cyclic redundancy check (CRC) attachment, coding blocks, scrambling and descrambling, modulation and demodulation, signal measurement, MIMO, etc. The physical layer may include various logical channels, including a physical downlink control channel (PDCCH) and a physical downlink shared channel (PDSCH).

[0070]

[0086] In the user and control plane, Layer 2 (L2) of the AS 304 may carry the link between the wireless device 320 and the base station 350 on the physical layer 306. In some implementations, Layer 2 may include a Medium Access Control (MAC) sublayer 308, a Radio Link Control (RLC) sublayer 310, a Packet Data Convergence Protocol (PDCP) sublayer 312, and a Service Data Adaptation Protocol (SDAP) 317 sublayer, each of which forms a logical connection that terminates at the base station 350.

[0071]

[0087] In the control plane, Layer 3 (L3) of the AS 304 may include a radio resource control (RRC) sublayer 3. Although not shown, the software architecture 300 may include additional Layer 3 sublayers, as well as various upper layers above Layer 3. In some implementations, the RRC sublayer 313 may provide functions including broadcasting system information, paging, and establishing and releasing RRC signaling connections between the wireless device 320 and the base station 350.

[0072]

[0088] In various embodiments, the SDAP sublayer 317 may provide mapping between Quality of Service (QoS) flows and Data Radio Bearers (DRBs). In various implementations, the PDCP sublayer 312 may provide uplink functions including multiplexing between different radio bearers and logical channels, sequence numbering, handover data processing, integrity protection, ciphering, and header compression. In the downlink, the PDCP sublayer 312 may provide functions including in-order delivery of data packets, duplicate data packet detection, integrity verification, decryption, and header recovery.

[0073]

[0089] In the uplink, the RLC sublayer 310 may provide segmentation and concatenation of upper layer data packets, retransmission of lost data packets, and automatic repeat request (ARQ). In the downlink, the RLC sublayer 310 functions may include reordering of data packets to compensate for out-of-order reception, reassembly of upper layer data packets, and ARQ.

[0074]

[0090] In the uplink, the MAC sublayer 308 may provide functions including multiplexing between logical and transport channels, random access procedures, logical channel priorities, and Hybrid ARQ (HARQ) operations. In the downlink, MAC layer functions may include channel mapping within a cell, demultiplexing, discontinuous reception (DRX), and HARQ operations.

[0075]

[0091] While the software architecture 300 may provide functionality for transmitting data over a physical medium, the software architecture 300 may further include at least one host layer 314 for providing data transfer services to various applications in the wireless device 320. In some implementations, the application-specific functionality provided by the at least one host layer 314 may provide an interface between the software architecture and the general-purpose processor 206.

[0076]

[0092] In other implementations, software architecture 300 may include one or more upper logical layers (e.g., transport, session, presentation, application) that provide host layer functionality. For example, in some implementations, software architecture 300 may include a network layer (e.g., an Internet Protocol (IP) layer) in which a logical connection terminates at a packet data network (PDN) gateway (PGW). In some implementations, software architecture 300 may include an application layer in which a logical connection terminates at another device (e.g., an end-user device, a server). In some implementations, software architecture 300 may further include a hardware interface 316 between physical layer 306 and communications hardware (e.g., one or more radio frequency (RF) transceivers) in AS 304.

[0077]

[0093] FIG. 4A is a conceptual diagram illustrating operations 400a performed by an application and an XR runtime, according to various embodiments. Referring to FIGS. 1A-4A, an application 402 may use an extensible API (e.g., the OpenXR API) to communicate with an XR runtime 404. The application 402 may begin by sending a query to the XR runtime 404 to create an instance (e.g., xrInstance 406). If the XR runtime is available, a session 408 is created. The XR runtime receives information for rendering from the application and performs the operations of the rendering loop, including xrWaitFrame 410a (waiting for a display frame opportunity), xrBeginFrame 410b (signaling the start of frame rendering), performing rendering operations 410c ("graphics work done"), and xrEndFrame 410d (rendering ends and the swap chain is handed over to the compositor).

[0078]

[0094] 4B is a block diagram illustrating rendering loop operations 400b that may be performed by an XR system, according to various embodiments. Referring to FIGS. 1A-4B, an application running in a UE may create an XR session, and for each visual stream, the UE may create a swap chain image. The application may receive pre-rendered frames from each stream and pass the pre-rendered frames to the XR runtime for rendering. A networked computing device (acting as a split rendering server) may match the format and resolution of the swap chain images when pre-rendering content (e.g., 3D content).

[0079]

[0095] In some embodiments, the XR system may execute an xrCreateSwapchain operation 412 to create a swap chain handle (e.g., an XrSwapchain handle). The xrCreateSwapchain operation 412 may include parameters such as a session identifier (e.g., session parameters) for the session to create the image for processing, a pointer to a data structure (e.g., XrSwapchainCreateInfo) that contains parameters used to create the image (e.g., createInfo parameters), and the created swap chain (e.g., XrSwapchain) is returned. The XR system may execute an xrCreateSwapchainImage operation 414 to create a graphics backend-optimized swap chain image. The XR system may then execute a rendering loop operation, including an xrAcquireSwapchainImage operation 416a to acquire the image for processing, an xrWaitSwapchainImage operation 416b to wait for processing of the image, a graphics work operation 416c to perform processing of the image, and an xrReleaseSwapchainImage operation 416d to release the rendered image. Upon completion of the rendering loop operations, the XR system may perform an xrDestroySwapchain operation 418 to release the swap chain image and associated resources. When a swap chain is created based on the needs of an application, it can be customized by specifying various parameters, such as an XR structure type, a graphics API-specific texture format identifier, the number of sub-data element samples in the image (e.g., sampleCount), the image width, the image height, the number of faces indicating the number of image faces (e.g., 6 for cubemaps), the number of array layers in the image (e.g., arraySize), and the number of levels of detail available for down-sampling of the image (e.g., mipCount).

[0080]

[0096] FIG. 4C is a conceptual diagram illustrating an XR device view 400c according to various embodiments. Referring to FIGS. 1A-4D, the XR system requires configuration information regarding the UE's views to perform rendering operations. For example, a smartphone or tablet (e.g., smartphone 420a) running an XR application may provide a single view on a touchscreen display. In another example, AR glasses or VR goggles (e.g., AR goggles 420b) may provide two views, such as a stereoscopic view, by presenting a view to each of the user's eyes. Information regarding the UE's view capabilities may be enumerated for the XR system in descriptive information (e.g., xrEnumerateViewConfigurations), which may list supported view configuration types and associated parameters.

[0081]

[0097] FIG. 4D is a conceptual diagram illustrating operations 400d performed by a compositor, according to various embodiments. Referring to FIGS. 1A-4D, an XR system may include a compositor 426 that may perform operations including compositing layers, reprojecting layers, applying lens distortion, and / or transmitting a final image for display. For example, the compositor 426 may receive a left-eye image 422a and a right-eye image 422b as input and provide a combined image 424 as output that includes a combination of the left-eye image on the right-eye image. In some embodiments, an application may use multiple layers. Supported compositing layer types may include stereo, quad (e.g., a two-dimensional plane in three-dimensional space), cubemap, equirectangular, cylindrical, depth, alpha-blended, and / or other vendor compositing layers.

[0082]

[0098] 4E is a conceptual diagram illustrating an extension 400e configured to include description information, according to various embodiments. Referring to FIGS. 1A-4E, in some embodiments, a network computing device may configure extension 400e (which may be referred to, for example, as "3GPP_node_prerendered") with description information that describes the rendered content node type 434 of a node 432 in a scene 430. In various embodiments, scene 430 may include a description of a 3D environment. Scene 430 may be formatted as a hierarchical graph, and each graph node may be described by a node 432.

[0083]

[0099] In some embodiments, the rendered content node type may indicate the presence of pre-rendered content. In some embodiments, the extension may include visual 436, audio 440, and / or haptic 442 information components. In some embodiments, visual information component 436 may include information about a first view (“view_1”) 438a, layer projection information 438b, and layer depth information 438c. In some embodiments, each component may describe a set of buffers 450, 452, 454, 456 and associated buffer configurations. In some embodiments, each buffer 450, 452, 454, 456 may be associated with particular information or a particular information component. For example, buffer 450 may be associated with layer projection information 438b, buffer 452 may be associated with layer depth information 438c, and so on. In some embodiments, extension 400e may include information describing an uplink buffer 444 for conveying information from the UE to a network computing device, which may include time-dependent metadata such as UE pause information and information about user input.

[0084]

[0100] 5A-5G illustrate aspects of descriptive information 500a-500f, according to various embodiments. With reference to FIGS. 1A-5G, descriptive information 500a-500f is discussed using the OpenXR protocol as an example, although any suitable configuration of information may be used in various embodiments.

[0085]

[0101] 5A , description information 500a may be configured to describe pre-rendered content 502, e.g., a “glTF extension to describe pre-rendered content.” Description information 500a may be configured to include parameters or configuration information related to visual information 504a (“visual”), audio information 506a (“audio”), and haptic information 508a, such as haptic commands (e.g., “haptics”) executed by the UE. Description information 500a may also be configured to include configuration information related to information 510a that the UE may provide to a network computing device in the uplink. Description information 500a may also be configured to include configuration information or parameters related to streamed buffers for each of the above information, such as “visual streamed buffers” 504b, “audio streamed buffers” 506b, “haptics streamed buffers” 508b, and “uplink streamed buffers” 510b. In some embodiments, audio information 506a, haptic information 508a, and / or uplink information 510a may be optional.

[0086]

[0102] 5B, description information 500b may be configured to describe visual pre-rendered content 512. Description information 500b may be configured to include information describing view configuration 514. Description information 500b may also include an enumeration of view type(s). Description information 500b may be configured to include information describing an array of layer view objects 516.

[0087]

[0103] 5C , description information 500c may be configured to describe a representation of a pre-rendered view 520. Description information 500c may be configured to include properties such as eye visibility information 522 (e.g., for the left eye, right eye, both eyes, or no eyes), a description 524 of an array of glTF timed accessors that carry streamed buffers for each compositing layer of the view, and an array 526 of compositing layer types within the array of compositing layers. In various embodiments, the timed accessors are descriptors in glTF for how timed media is formatted and from which source the timed media should be received. Description information 500c may be configured to include information describing the compositing layer types in the array of compositing layers.

[0088]

[0104] 5D, description information 500d may be configured to include information describing audio pre-rendered media 520. Description information 500d may be configured to include object descriptions 530, type information 532 including a description of the type of audio being rendered, and information regarding higher order ambisonics (HOA), such as an enumeration of audio aspects such as mono, stereo, or information related to a three-dimensional sound scene or sound field. Description information 500d may also be configured to include information regarding components 534, such as information regarding an array of timed accessors to audio component buffers.

[0089]

[0105] 5E, description information 500e may be configured to include information describing uplink data 540 that a UE may transmit to a network computing device. Descriptive information 500e may be configured to include a description of timed metadata 542 including various parameters and an enumeration of types of metadata, such as UE pause, information about user input, or other information that the UE may provide to a network computing device in the uplink. Descriptive information 500e may be configured to include information regarding source information, such as a pointer to a timed accessor that describes the uplink timed metadata.

[0090]

[0106] 5F and 5G, description information 500f may be configured to include information describing a data channel message format for frame-related metadata 550. Description information 500f may be configured to include information describing a unique identifier for XR space 552 in which content is being pre-rendered. Description information 500f may be configured to include information describing pose information for image 554. The pose information may include characteristic information such as orientation (e.g., pose rotation), three-dimensional coordinates of the pose of the image, and other suitable information. Description information 500f may be configured to include information describing field of view information 556, including information regarding the field of view of the projected layer (e.g., left, right, top, and bottom angle information). Description information 500f may be configured to include timestamp information 558 for the image.

[0091]

[0107] 6A is a process flow diagram illustrating a method 600a performed by a processing system of a network computing device to communicate pre-rendered media to a UE, according to various embodiments. With reference to FIGS. 1A-6A, the operations of method 600a may be performed by a processing system (e.g., 200, 202, 204) including one or more processors (e.g., 210, 212, 214, 216, 218, 252, 260) and / or hardware elements, any one or combination of which may be configured to perform any of the operations of method 600a. To encompass any of the processor(s), hardware elements, and software elements that may be involved in performing method 600a, the elements that perform the method operations are generally referred to as a “processing system.” Additionally, the means for performing the operations of method 600a include a processing system (e.g., 200, 202, 204) including one or more processors (e.g., processors 210, 212, 214, 216, 218, 252, 260, etc.) of the network computing device (e.g., 700).

[0092]

[0108] In block 601, the processing system may receive pause information received from the UE.

[0093]

[0109] At block 602, the processing system may generate pre-rendered content for processing by the UE based on the pause information received from the UE.

[0094]

[0110] At block 604, the processing system may generate description information based on the pre-rendered content, the description information being configured to enable the UE to perform a rendering operation using the pre-rendered content. In some embodiments, the processing system may configure the description information to include various information such as those described with respect to description information 500a-500g.

[0095]

[0111] In some embodiments, the processing system may configure the description information to indicate buffer information for one or more buffers to which the networked computing device will stream the pre-rendered content. The buffers may include a visual data buffer, an audio data buffer, and / or a haptic data buffer. In some embodiments, the processing system may configure the description information to indicate view configuration information for the pre-rendered content. In some embodiments, the processing system may configure the description information to indicate an array of layer view objects. In some embodiments, the processing system may configure the description information to indicate eye visibility information for the pre-rendered content. In some embodiments, the processing system may configure the description information to indicate compositing layer information for the pre-rendered content. In some embodiments, the processing system may configure the description information to indicate compositing layer type information for the pre-rendered content. In some embodiments, the processing system may configure the description information to indicate audio configuration properties for the pre-rendered content.

[0096]

[0112] At block 606, the processing system may transmit the description information to the UE. In some embodiments, the processing system may transmit a packet header extension to the UE including information configured to enable the UE to present the pre-rendered content. In some embodiments, the processing system may transmit a data channel message to the UE including information configured to enable the UE to present the pre-rendered content.

[0097]

[0113] In block 608, the processing system may transmit the pre-rendered image to the UE.

[0098]

[0114] 6B is a process flow diagram illustrating operations 600b that may be performed by a processing system of a network element as part of a method 600a for communicating pre-rendered media to a UE, according to various embodiments. With reference to FIGS. 1A-6B, the operations of method 600b may be performed by a processing system (e.g., 200, 202, 204) that includes one or more processors (e.g., 210, 212, 214, 216, 218, 252, 260) and / or hardware elements, any one or combination of which may be configured to perform any of the operations of method 600b. To encompass any of the processor(s), hardware elements, and software elements that may be involved in performing method 600b, the elements that perform the method operations are generally referred to as a "processing system." Additionally, the means for performing operation 600b includes a processing system (e.g., 200, 202, 204) including one or more processors (e.g., processors 210, 212, 214, 216, 218, 252, 260, etc.) of the network computing device (e.g., 700).

[0099]

[0115] At block 610, the processing system may receive from the UE an uplink data description configured to indicate information regarding content to be pre-rendered for processing by the UE.

[0100]

[0116] At block 612, the processing system may generate pre-rendered content (for processing by the UE) based on the uplink data description.

[0101]

[0117] The processing system may transmit the description information and the rendered content to the UE at block 606 as described.

[0102]

[0118] 6C is a process flow diagram illustrating operations 600c that may be performed by a processing system of a UE, according to various embodiments. With reference to FIGS. 1A-6C, the operations of method 600c may be performed by a processing system (e.g., 200, 202, 204) that includes one or more processors (e.g., 210, 212, 214, 216, 218, 252, 260) and / or hardware elements, any one or combination of which may be configured to perform any of the operations of method 600c. The elements that perform the method operations are generally referred to as a "processing system" to encompass any of the processor(s), hardware elements, and software elements that may be involved in performing method 600b. Additionally, the means for performing operation 600c may include a processing system (e.g., 200, 202, 204) including one or more processors (e.g., processors 210, 212, 214, 216, 218, 252, 260, etc.) of the UE (e.g., 800, 900).

[0103]

[0119] The processing system may transmit the pose information to the networked computing device at block 616. In some embodiments, the pose information may include information regarding position, orientation, movement, or similar information useful for the networked computing device to render content suitable for display on the UE.

[0104]

[0120] At block 618, the processing system may receive description information from the network computing device configured to enable the UE to perform a rendering operation using pre-rendered content to be provided by the network computing device.

[0105]

[0121] At block 626, the processing system may receive the pre-rendered content from the networked computing device via the buffer described in the description information extension.

[0106]

[0122] In block 630, the processing system may send the rendered frames to the XR runtime for compositing and display (e.g., on a display device of the UE).

[0107]

[0123] In some embodiments, the UE may have the capability to receive 2D or 3D content, may inform the network computing device of such capability, and may then perform operations to render the received content according to the selected rendering configuration. In such embodiments, the UE processing system may also perform the operations at blocks 620-628.

[0108]

[0124] The processing system may send information about the UE capabilities and configuration to the network computing device at block 620. In some embodiments, the UE information may include information about the UE's display capabilities, rendering capabilities, processing capabilities, and / or other suitable capabilities related to split-rendering operations.

[0109]

[0125] At block 622, the processing system may receive a scene description (e.g., description information) for the split rendering session from a networked computing device.

[0110]

[0126] At decision block 624, the processing system may determine whether to select a 3D rendering configuration or a 2D rendering configuration. In some embodiments, the processing system may select a 3D rendering configuration or a 2D rendering configuration based at least in part on the received scene description for the split rendering session (e.g., based at least in part on the description information).

[0111]

[0127] In response to deciding to select a 2D rendering configuration (i.e., decision block 624 = "Pre-render to 2D"), the processing system may receive the pre-rendered content in block 626 via a buffer described in the description information extension of the scene description (e.g., "3GPP_node_prerendered").

[0112]

[0128] In response to determining to select a 3D rendering configuration (i.e., decision block 624="3D"), the processing system may receive information for rendering the 3D scene image from the networked computing device and may render the 3D scene image(s) using the information for rendering the 3D scene image(s).

[0113]

[0129] Following execution of the operations of block 626 or 628, the processing system may send the rendered frames to the XR runtime in block 630 for compositing and display (e.g., on a display device of the UE).

[0114]

[0130] FIG. 7 is a component block diagram of a network computing device suitable for use with various embodiments. Referring to FIGS. 1A-7, the network computing device may perform functions (e.g., 414, 416, 418) in a communications network (e.g., 100, 150) and may include at least the components shown in FIG. 7. The network computing device 700 may include a processing system 701 coupled to volatile memory 702 and mass non-volatile memory, such as a disk drive 708. The network computing device 700 may also include a peripheral memory access device 706, such as a floppy disk drive, compact disk (CD) drive, or digital video disk (DVD) drive, coupled to the processing system 701. The network computing device 700 may also include a network access port 704 (or interface) coupled to the processing system 701 for establishing a data connection with a network, such as the Internet or a local area network coupled to other system computers and servers. The network computing device 700 may include one or more antennas 707 for transmitting and receiving electromagnetic radiation, which may be connected to a wireless communications link. Network computing device 700 may include additional access ports, such as USB, Firewire, Thunderbolt, etc., for coupling to peripherals, external memory, or other devices.

[0115]

[0131] FIG. 8 is a component block diagram of a UE 800 suitable for use in various embodiments. With reference to FIGS. 1A-8, various embodiments may be implemented in various UEs 800 (e.g., wireless devices 120a-120e, 200, 320, 404), an example of which is shown in FIG. 8 in the form of a smartphone. However, it will be understood that the UE 800 may be implemented in various embodiments, such as an XR device, VR goggles, smart glasses, and / or the like. The UE 800 may include a first SOC processing system 202 (e.g., a SOC-CPU) coupled to a second SOC processing system 204 (e.g., a 5G-enabled SOC). The first SOC processing system 202 and the second processing system SOC 204 may be coupled to an internal memory 816, a display 812, and a speaker 814. Additionally, the UE 800 may include an antenna 804 for transmitting and receiving electromagnetic radiation that may be connected to a transceiver 427 coupled to one or more processors in the first SOC processing system 202 and / or the second SOC processing system 204. The UE 800 may include menu selection buttons or rocker switches 820 for receiving user input.

[0116]

[0132] The UE 800 may include a voice encoding / decoding (CODEC) circuit 810 that digitizes voice received from a microphone into data packets suitable for wireless transmission and decodes the received voice data packets to generate analog signals that are provided to a speaker to generate voice. One or more of the processors, wireless transceiver 266, and codec 810 in the first SOC processing system 202 and the second SOC processing system 204 may include digital signal processor (DSP) circuitry (not separately shown).

[0117]

[0133] FIG. 9 is a component block diagram of a UE suitable for use with various embodiments. Referring to FIGS. 1A-9, various embodiments may be implemented on various UEs, one example of which is shown in FIG. 9 in the form of smart glasses 900. Smart glasses 900 are similar to traditional eyeglasses but may operate with enhanced computing capabilities and sensors, such as a built-in camera 935 and head-up display or augmented reality capabilities located on or near lenses 931. Like any eyeglasses, smart glasses 900 may include a frame 902 coupled to temples 904 that fit along the wearer's head and behind the ears. Frame 902 holds lenses 931 in place in front of the wearer's eyes when nose pads 906 on bridge 908 rest on the wearer's nose.

[0118]

[0134] In some embodiments, the smart glasses 900 may include an image rendering device 914 (e.g., an image projector) embedded in one or both temples 904 of the frame 902 and configured to project an image onto the optical lens 931. In some embodiments, the image rendering device 914 may include a light emitting diode (LED) module, a light tunnel, a homogenizing lens, an optical display, a folding mirror, or other components, known projectors, or head-mounted displays. In some embodiments (e.g., embodiments in which the image rendering device 914 is not included or used), the optical lens 931 may be or include a see-through or partially see-through electronic display. In some embodiments, the optical lens 931 includes an image generating element, such as a see-through organic light emitting diode (OLED) display element or a liquid crystal on silicon (LCOS) display element. In some embodiments, the optical lens 931 may include separate left-eye and right-eye display elements. In some embodiments, the optical lens 931 may include or function as a light guide for delivering light from the display element to the wearer's eye.

[0119]

[0135] The smart glasses 900 may include several external sensors that may be configured to acquire information about the wearer's actions and external conditions, which may be useful in sensing images, sounds, muscle movements, and other phenomena that may be useful in detecting that the wearer is interacting with the virtual user interface. In some embodiments, the smart glasses 900 may include a camera 935 configured to image objects in front of the wearer in a still image or video stream. Additionally, the smart glasses 900 may include a lidar sensor 940 or other distance measurement device. In some embodiments, the smart glasses 900 may include a microphone 910 positioned and configured to record audio in the wearer's vicinity. In some embodiments, multiple microphones may be positioned at different locations on the frame 902, such as on the distal ends of the temples 904 near the chin, to record sounds such as those generated when a user taps an object they are selecting with their hand. In some embodiments, the smart glasses 900 may include pressure sensors, such as on the nose pads 906, configured to sense facial movements to calibrate distance measurements. In some embodiments, the smart glasses 900 may include other sensors (e.g., a thermometer, a heart rate monitor, a body temperature sensor, a pulse oximeter, etc.) for collecting information about the environment and / or user state that may be useful in recognizing a user's interaction with the virtual user interface.

[0120]

[0136] The smart glasses 900 may include a processing system 912 including processing and communications SOCs 202, 204, which may include one or more processors (e.g., 212, 214, 216, 218, 260), one or more of which may be configured with processor-executable instructions for performing the operations of various embodiments. The processing and communications SOCs 202, 204 may be coupled to internal sensors 920, internal memory 922, and communications circuitry 924 coupled to one or more antennas 926 for establishing a wireless data link. The processing and communications SOCs 202, 204 may also be coupled to a sensor interface circuit 928 configured to control and receive data from a camera 935, microphone(s) 910, and other sensors disposed on the frame 902.

[0121]

[0137] The internal sensors 920 may include an internal measurement unit (IMU) including an electronic gyroscope, an accelerometer, and a magnetic compass configured to measure the movement and orientation of the wearer's head. The internal sensors 920 may further include a magnetometer, an altimeter, an odometer, and an atmospheric pressure sensor, as well as other sensors useful in determining the orientation and movement of the smart glasses 900. The processing system 912 may further include a power source, such as a rechargeable battery 930 coupled to the SOCs 202, 204, as well as external sensors on the frame 902.

[0122]

[0138] The processing systems of network computing device 700 and UEs 800 and 900 may include any programmable microprocessor, microcomputer, or multiple processor chip(s) that can be configured by software instructions (applications) to perform various functions, including those of some implementations described below. In some wireless devices, multiple processors may be provided, such as one processor in SOC processing system 204 dedicated to wireless communication functions and one processor in SOC 202 dedicated to running other applications. Software applications may be stored in memory 702, 816, 922 before being accessed and loaded into the processors. The processors may include sufficient internal memory to store application software instructions.

[0123]

[0139] The various embodiments shown and described are provided merely as examples to illustrate various features of the claims. However, features shown and described with respect to any given embodiment are not necessarily limited to the associated embodiment and may be used with or combined with other embodiments shown and described. Furthermore, the claims are not limited by any single exemplary embodiment. For example, one or more aspects of descriptive information 500a-500f and any of methods and operations 600a-600c may be substituted for or combined with one or more aspects of descriptive information 500a-500f and any of methods and operations 600a-600c.

[0124]

[0140] Example implementations are described in the following paragraphs. While some of the implementations below are described with reference to example methods, further example implementations may include the example methods discussed in the following paragraphs implemented by a base station including a processor configured with processor-executable instructions to perform the operations of the methods of the example implementations below, the example methods discussed in the following paragraphs implemented by a base station including means for performing the functions of the methods of the example implementations below, and the example methods discussed in the following paragraphs that may be implemented as a non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of a base station to perform the operations of the methods of the example implementations below.

[0125]

[0141] Example 1. A method for communicating rendered media to a user equipment (UE) performed by a processing system of a network computing device, the method including: receiving pause information from the UE; generating pre-rendered content for processing by the UE based on the pause information received from the UE; generating description information based on the pre-rendered content configured to enable the UE to perform a rendering operation using the pre-rendered content; transmitting the description information to the UE; and transmitting the pre-rendered content to the UE.

[0126]

[0142] Example 2. The method of example 1, wherein the description information is configured to indicate buffer information for one or more buffers to which the networked computing device will stream the pre-rendered content.

[0127]

[0143] Example 3. The method of Examples 1 and / or 2, wherein the description information is configured to indicate view configuration information for the pre-rendered content.

[0128]

[0144] Example 4. The method of any of Examples 1-3, wherein the description information is configured to indicate an array of layer view objects.

[0129]

[0145] Example 5. The method of any of Examples 1-4, wherein the description information is configured to indicate eye visibility information for the pre-rendered content.

[0130]

[0146] Example 6. The method of any of Examples 1-5, wherein the description information is configured to indicate compositing layer information for pre-rendered content.

[0131]

[0147] Example 7. The method of any of Examples 1-6, wherein the description information is configured to indicate compositing layer type information for the pre-rendered content.

[0132]

[0148] Example 8. The method of any of Examples 1-7, wherein the description information is configured to indicate audio configuration properties for the pre-rendered content.

[0133]

[0149] Example 9. The method of any of Examples 1-8, further comprising receiving from the UE an uplink data description configured to indicate information regarding content to be pre-rendered for processing by the UE, and wherein generating the pre-rendered content for processing by the UE based on the pause information received from the UE comprises generating the pre-rendered content based on the uplink data description.

[0134]

[0150] Example 10. The method of any of Examples 1-9, wherein sending the description information to the UE includes sending a packet header extension to the UE including information configured to enable the UE to process the pre-rendered content.

[0135]

[0151] Example 11. The method of any of Examples 1-10, wherein sending the description information to the UE includes sending a data channel message to the UE including information configured to enable the UE to process the pre-rendered content.

[0136]

[0152] Example 12. A method executed by a processor of a user equipment (UE), comprising: sending pose information to a network computing device; receiving description information from the network computing device configured to enable the UE to perform rendering operations using pre-rendered content; receiving the pre-rendered content via a buffer described in the description information extension; and sending the rendered frames to an extended reality (XR) runtime for compositing and display.

[0137]

[0153] Example 13. The method of example 12, further comprising: sending information about the UE capabilities and configuration to the network computing device; and receiving a scene description for the split rendering session from the network computing device.

[0138]

[0154] Example 14. The method of Example 13, further comprising: determining whether to select a 3D rendering configuration or a 2D rendering configuration based at least in part on the received scene description; receiving pre-rendered content via a buffer described in the description information extension of the scene description in response to the determination to select the 2D rendering configuration; and receiving information for rendering a 3D scene image and for rendering one or more 3D scene images in response to the determination to select the 3D rendering configuration.

[0139]

[0155] [More examples may be added depending on the modifications to the method claims.]

[0156] As used herein, terms such as “component,” “module,” and “system” are intended to include, but are not limited to, computer-related entities, such as hardware, firmware, a combination of hardware and software, software, or software in execution, configured to perform particular operations or functions. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, or a computer. By way of example, both an application running on a wireless device and the wireless device may be referred to as a component. One or more components may reside within a process or thread of execution, and a component may be local to one processor or core or distributed among two or more processors or cores. In addition, these components may execute from various non-transitory computer-readable media having various instructions or data structures stored thereon. Components may communicate via local or remote processes, function or procedure calls, electronic signals, data packets, memory read / writes, and other known network-, computer-, processor-, or process-related communication methods.

[0140]

[0157] Several different cellular and mobile communication services and standards are available or are contemplated in the future, all of which may implement and benefit from various embodiments, such as, for example, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE) systems, third generation wireless mobile communication technology (3G), fourth generation wireless mobile communication technology (4G), fifth generation wireless mobile communication technology (5G), and later generations of 3GPP technologies, Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), 3GSM, General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA) systems (e.g., cdmaOne, CDMA1020), Enhanced Data Rates for GSM Evolution (EDGE), Advanced Mobile Phone System (AMPS), Digital AMPS (IS-136 / TDMA), Evolution Data Optimized (EV-DO), Digital Enhanced Cordless Telecommunications (DECT), and others. Examples of technologies include Direct Connectivity (DECT), Worldwide Interoperability for Microwave Access (WiMAX), Wireless Local Area Network (WLAN), Wi-Fi Protected Access I & II (WPA, WPA2), and Integrated Digital Enhanced Network (iDEN). Each of these technologies involves, for example, the transmission and reception of voice, data, signaling, and / or content messages. It should be understood that any reference to terminology and / or technical details relating to a particular telecommunications standard or technology is for illustrative purposes only and does not limit the scope of the claims to any particular communications system or technology unless specifically recited in the claim language.

[0141]

[0158] The above method descriptions and process flow diagrams are provided as illustrative examples only and do not require or imply that the operations of the various embodiments must be performed in the order presented. As will be understood by one of ordinary skill in the art, the order of operations in the above-described embodiments may be performed in any order. Terms such as "thereafter," "then," and "next" do not limit the order of operations. These terms are used to guide the reader through the method descriptions. Furthermore, any reference to claim elements in the singular, for example, using the articles "a," "an," or "the," should not be construed as limiting the element to the singular.

[0142]

[0159] The various illustrative logical blocks, modules, components, circuits, and algorithmic operations described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, the various illustrative components, blocks, modules, circuits, and operations have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the claims.

[0143]

[0160] The hardware used to implement the various exemplary logic, logic blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed using general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of receiver smart objects, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Alternatively, some operations or methods may be performed by circuitry specific to a given function.

[0144]

[0161] In one or more embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium. The operations of a method or algorithm disclosed herein may be embodied in a processor-executable software module or processor-executable instructions, which may reside on a non-transitory computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable storage medium may be any storage medium that can be accessed by a computer or processor. By way of example, and not limitation, such non-transitory computer-readable or processor-readable storage medium may include RAM, ROM, EEPROM, FLASH memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage smart objects, or any other medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically while discs reproduce data optically using lasers. Combinations of the above are also included within the scope of non-transitory computer-readable and processor-readable media. Furthermore, the operations of a method or algorithm may reside as one or any combination or set of code and / or instructions on a non-transitory processor-readable storage medium and / or a non-transitory computer-readable storage medium, which may be incorporated into a computer program product.

[0145]

[0162] The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the claims. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein.

Claims

1. 1. A method for communicating rendered media to user equipment (UE), executed by a processor of a network computing device, comprising: receiving pause information from the UE; generating pre-rendered content for processing by the UE based on the pause information received from the UE; generating description information based on the pre-rendered content, the description information being configured to enable the UE to perform a rendering operation using the pre-rendered content; transmitting the description information to the UE; transmitting the pre-rendered content to the UE; A method comprising:

2. 2. The method of claim 1, wherein the description information is configured to indicate buffer information for one or more buffers to which the networked computing device will stream the pre-rendered content.

3. The method of claim 1 , wherein the description information is configured to indicate view configuration information for the pre-rendered content.

4. The method of claim 1 , wherein the descriptive information is configured to describe an array of layer view objects.

5. The method of claim 1 , wherein the description information is configured to indicate eye visibility information for the pre-rendered content.

6. The method of claim 1 , wherein the description information is configured to indicate compositing layer information for the pre-rendered content.

7. The method of claim 1 , wherein the description information is configured to indicate compositing layer type information for the pre-rendered content.

8. The method of claim 1 , wherein the description information is configured to indicate audio configuration properties for the pre-rendered content.

9. receiving from the UE an uplink data description configured to indicate information regarding content to be pre-rendered for processing by the UE; 2. The method of claim 1, wherein generating the pre-rendered content for processing by the UE based on pause information received from the UE comprises generating the pre-rendered content based on the uplink data description.

10. 2. The method of claim 1, wherein sending the description information to the UE comprises sending a packet header extension to the UE including information configured to enable the UE to process the pre-rendered content.

11. 2. The method of claim 1, wherein transmitting the description information to the UE comprises transmitting a data channel message to the UE including information configured to enable the UE to process the pre-rendered content.

12. 1. A network computing device, comprising: Memory and a processing system coupled to the memory and including one or more processors, the one or more processors: receiving pause information from a user equipment (UE); generating pre-rendered content for processing by the UE based on the pause information received from the UE; generating, based on the pre-rendered content, description information configured to enable the UE to perform a rendering operation using the pre-rendered content; sending the description information to the UE; transmitting the pre-rendered content to the UE; a processing system configured to A network computing device comprising:

13. 13. The network computing device of claim 12, wherein the one or more processors are configured such that the description information is configured to indicate buffer information for one or more buffers to which the network computing device will stream the pre-rendered content.

14. 13. The network computing device of claim 12, wherein the one or more processors are configured such that the description information is configured to indicate view configuration information for the pre-rendered content.

15. The network computing device of claim 12 , wherein the one or more processors are configured such that the description information is configured to describe an array of layer view objects.

16. 13. The network computing device of claim 12, wherein the one or more processors are configured such that the description information is configured to indicate eye visibility information for the pre-rendered content.

17. 13. The network computing device of claim 12, wherein the one or more processors are configured such that the description information is configured to indicate compositing layer information for the pre-rendered content.

18. 13. The network computing device of claim 12, wherein the one or more processors are configured such that the description information is configured to indicate compositing layer type information for the pre-rendered content.

19. 13. The network computing device of claim 12, wherein the one or more processors are configured such that the description information is configured to indicate audio configuration properties for the pre-rendered content.

20. the one or more processors: receiving an uplink data description from the UE configured to indicate information regarding content to be pre-rendered for processing by the UE; generating the pre-rendered content for processing by the UE based on the uplink data description; The network computing device of claim 12 further configured to:

21. 13. The network computing device of claim 12, wherein the one or more processors are further configured to transmit the description information and the pre-rendered content to the UE as a packet header extension including information configured to enable the UE to process the pre-rendered content.

22. 13. The network computing device of claim 12, wherein the one or more processors are further configured to include information configured to enable the UE to process the pre-rendered content in the descriptive information data channel message transmitted to the UE.

23. 1. A method executed by a processor of a user equipment (UE), comprising: transmitting the pause information to a network computing device; receiving description information from the network computing device configured to enable the UE to perform a rendering operation using pre-rendered content; receiving pre-rendered content via a buffer described in the description information extension; sending the rendered frames to an extended reality (XR) runtime for compositing and display; A method comprising:

24. sending information about UE capabilities and configuration to the network computing device; receiving a scene description for a split rendering session from the networked computing device; 24. The method of claim 23, further comprising:

25. determining whether to select a 3D rendering configuration or a 2D rendering configuration based at least in part on the received scene description; In response to determining to select the 2D rendering configuration, receiving pre-rendered content via a buffer described in the description information extension of the scene description; receiving information for rendering a 3D scene image and for rendering the one or more 3D scene images in response to determining to select the 3D rendering configuration; 25. The method of claim 24, further comprising:

26. A user equipment (UE), Memory and A transceiver; a processing system coupled to the memory and the transceiver and including one or more processors, the one or more processors comprising: transmitting the pause information to the network computing device; receiving description information from a network computing device configured to enable the UE to perform a rendering operation using pre-rendered content; receiving pre-rendered content via a buffer described in the description information extension; Sending the rendered frames to an Extended Reality (XR) runtime for compositing and display; a processing system configured to A user equipment (UE) comprising:

27. the one or more processors: Sending information about UE capabilities and configuration to the network computing device; receiving a scene description for a split rendering session from the networked computing device; 27. The UE of claim 26, further configured to:

28. the one or more processors: determining whether to select a 3D rendering configuration or a 2D rendering configuration based at least in part on the received scene description; responsive to determining to select the 2D rendering configuration, receiving pre-rendered content via a buffer described in the description information extension of the scene description; receiving information for rendering a 3D scene image and for rendering the one or more 3D scene images in response to determining to select the 3D rendering configuration; 28. The UE of claim 27, further configured to:

Images