On-demand graphical asset delivery for split rendering

Abstract

A method performed by a first rendering entity on a user equipment is provided. The method includes establishing a split rendering session for rendering a scene in which first objects and second objects in the scene are to be rendered by respectively the first rendering entity, and a second rendering entity on a network device. The method includes receiving graphical assets corresponding to the first objects, including receiving one or more graphical assets corresponding to one or more first objects on demand relative to when the one or more first objects are to be rendered. The method includes rendering the first objects using the graphical assets corresponding to the first objects to produce first rendered media, and receiving second rendered media produced by the second rendering entity rendering the second objects. And the first rendered media and the second rendered media are combined into a view of the scene.

Description

ON-DEMAND GRAPHICAL ASSET DELIVERY FOR SPLIT RENDERING TECHNOLOGICAL FIELD

[0001] The present disclosure relates generally to telecommunications and, in particular, to split graphics rendering across a telecommunications system. BACKGROUND

[0002] A telecommunications system can be seen as a facility that enables communication sessions between two or more entities such as user terminals, base stations and / or other nodes by providing carriers between the various entities involved in the communications path. A telecommunications system can be provided, for example, by means of a communication network and one or more compatible communication devices. The communication sessions may comprise, for example, communication of data for carrying communications such as voice, video, electronic mail (email), text message, multimedia and / or content data and so on. Non-limiting examples of services provided comprise two-way or multi-way calls, data communication or multimedia services and access to a data network system, such as the Internet.

[0003] In a wireless telecommunications system at least a part of a communication session between at least two stations occurs over a wireless link. Examples of wireless systems comprise public land mobile networks (PLMN), satellite based communication systems and different wireless local networks, for example wireless local area networks (WLAN). Some wireless systems can be divided into cells, and are therefore often referred to as cellular systems.

[0004] A user can access the telecommunications system by means of an appropriate communication device or terminal. A communication device of a user may be referred to as user equipment (UE) or user device. A communication device is provided with an appropriate signal receiving and transmitting apparatus for enabling communications, for example enabling access to a communication network or communications directly with other users. The communication device may access a carrier provided by a station, for example a base station of a cell, and transmit and / or receive communications on the carrier.

[0005] The telecommunications system and associated devices typically operate in accordance with a given standard or specification which sets out what the various entities associated with the system are permitted to do and how that should be achieved. Communication protocols and / or parameters which shall be used for the connection are also typically defined. One example of a telecommunications system is the Universal Mobile Telecommunications System (UMTS). Other examples of telecommunications systems are Long-Term Evolution (LTE), LTE Advanced and the so-called 5G or New Radio (NR) networks. NR is being standardized by the 3rd Generation Partnership Project (3GPP). BRIEF SUMMARY

[0006] Example implementations of the present disclosure are directed to telecommunications and, in particular, to split graphics rendering across a telecommunications system. The present disclosure includes, without limitation, the following example implementations for illustrative purposes.

[0007] Some example implementations provide an apparatus implemented by a user equipment including a first rendering entity, the apparatus comprising: at least one memory configured to store instructions; and at least one processing circuitry configured to access the at least one memory, and execute the instructions to cause the apparatus to at least: establish a split rendering session for rendering a scene in which first objects in the scene are to be rendered by the first rendering entity, and second objects in the scene are to be rendered by a second rendering entity on a network device; receive graphical assets corresponding to the first objects, including receiving one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; render the first objects using the graphical assets corresponding to the first objects to produce first rendered media; receive, from the second rendering entity, second rendered media produced by the second rendering entity rendering the second objects; and combine the first rendered media and the second rendered media into a view of the scene.

[0008] Some example implementations provide an apparatus implemented by a first rendering entity on an user equipment, the apparatus comprising: means for establishing a split rendering session for rendering a scene in which first objects in the scene are to be rendered by the first rendering entity, and second objects in the scene are to be rendered by a second rendering entity on a network device; means for receiving graphical assets corresponding to the first objects, including receiving one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; means for rendering the first objects using the graphical assets corresponding to the first objects to produce first rendered media; means for receiving, from the second rendering entity, second rendered media produced by the second rendering entity rendering the second objects; and means for combining the first rendered media and the second rendered media into a view of the scene.

[0009] Some example implementations provide a method performed by a first rendering entity on a user equipment, the method comprising: establishing a split rendering session for rendering a scene in which first objects in the scene are to be rendered by the first rendering entity, and second objects in the scene are to be rendered by a second rendering entity on a network device; receiving graphical assets corresponding to the first objects, including receiving one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; rendering the first objects using the graphical assets corresponding to the first objects to produce first rendered media; receiving, from the second rendering entity, second rendered media produced by thesecond rendering entity rendering the second objects; and combining the first rendered media and the second rendered media into a view of the scene.

[0010] Some example implementations provide a computer-readable storage medium that is non- transitory and has instructions stored therein that, in response to execution by at least one processing circuitry, causes a user equipment including a first rendering entity to at least: establish a split rendering session for rendering a scene in which first objects in the scene are to be rendered by the first rendering entity, and second objects in the scene are to be rendered by a second rendering entity on a network device; receive graphical assets corresponding to the first objects, including receiving one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; render the first objects using the graphical assets corresponding to the first objects to produce first rendered media; receive, from the second rendering entity, second rendered media produced by the second rendering entity rendering the second objects; and combine the first rendered media and the second rendered media into a view of the scene.

[0011] Some example implementations provide an apparatus implemented by a network device including a second rendering entity, the apparatus comprising: at least one memory configured to store instructions; and at least one processing circuitry configured to access the at least one memory, and execute the instructions to cause the apparatus to at least: establish a split rendering session for rendering a scene in which first objects in the scene are to be rendered by a first rendering entity on a user equipment, and second objects in the scene are to be rendered by the second rendering entity; send, to the first rendering entity, graphical assets corresponding to the first objects for rendering the first objects using the graphical assets to produce first rendered media, sending the graphical assets including sending one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; render the second objects to produce second rendered media; and send the second rendered media to the first rendering entity for combining of the first rendered media and the second rendered media into a view of the scene.

[0012] Some example implementations provide an apparatus implemented by a second rendering entity on a network device, the apparatus comprising: means for establishing a split rendering session for rendering a scene in which first objects in the scene are to be rendered by a first rendering entity on an user equipment, and second objects in the scene are to be rendered by the second rendering entity; means for sending, to the first rendering entity, graphical assets corresponding to the first objects for rendering the first objects using the graphical assets to produce first rendered media, sending the graphical assets including sending one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; means for rendering the second objects to produce second rendered media; and means for sending the second rendered media tothe first rendering entity for combining of the first rendered media and the second rendered media into a view of the scene.

[0013] Some example implementations provide a method performed by a second rendering entity on a network device, the method comprising: establishing a split rendering session for rendering a scene in which first objects in the scene are to be rendered by a first rendering entity on a user equipment, and second objects in the scene are to be rendered by the second rendering entity; sending, to the first rendering entity, graphical assets corresponding to the first objects for rendering the first objects using the graphical assets to produce first rendered media, sending the graphical assets including sending one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; rendering the second objects to produce second rendered media; and sending the second rendered media to the first rendering entity for combining of the first rendered media and the second rendered media into a view of the scene.

[0014] Some example implementations provide a computer-readable storage medium that is non- transitory and has instructions stored therein that, in response to execution by at least one processing circuitry, causes a network device including a second rendering entity to at least: establish a split rendering session for rendering a scene in which first objects in the scene are to be rendered by a first rendering entity on a user equipment, and second objects in the scene are to be rendered by the second rendering entity; send, to the first rendering entity, graphical assets corresponding to the first objects for rendering the first objects using the graphical assets to produce first rendered media, sending the graphical assets including sending one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; render the second objects to produce second rendered media; and send the second rendered media to the first rendering entity for combining of the first rendered media and the second rendered media into a view of the scene.

[0015] These and other features, aspects, and advantages of the present disclosure will be apparent from a reading of the following detailed description together with the accompanying figures, which are briefly described below. The present disclosure includes any combination of two, three, four or more features or elements set forth in this disclosure, regardless of whether such features or elements are expressly combined or otherwise recited in a specific example implementation described herein. This disclosure is intended to be read holistically such that any separable features or elements of the disclosure, in any of its aspects and example implementations, should be viewed as combinable unless the context of the disclosure clearly dictates otherwise.

[0016] It will therefore be appreciated that this Brief Summary is provided merely for purposes of summarizing some example implementations so as to provide a basic understanding of some aspects of the disclosure. Accordingly, it will be appreciated that the above described example implementations are merely examples and should not be construed to narrow the scope or spirit of the disclosure in any way.Other example implementations, aspects and advantages will become apparent from the following detailed description taken in conjunction with the accompanying figures which illustrate, by way of example, the principles of some described example implementations. BRIEF DESCRIPTION OF THE FIGURE(S)

[0017] Having thus described example implementations of the disclosure in general terms, reference will now be made to the accompanying figures, which are not necessarily drawn to scale, and wherein:

[0018] FIG.1 illustrates a telecommunications system that includes one or more public land mobile networks (PLMNs) coupled to one or more external data networks, according to some example implementations of the present disclosure;

[0019] FIG.2 illustrates a 5G deployment of a PLMN, referred to at times as the 5G system (5GS), according to some example implementations;

[0020] FIG.3 more particularly depicts aspects of the 5GS of FIG.2, according to some example implementations;

[0021] FIG.4 is an example top level scene graph according to the graphics library transmission format (glTF) 2.0;

[0022] FIG.5 illustrates a split rendering (SR) media service enabler (MSE) architecture, according to some example implementations;

[0023] FIG.6 illustrates an IP multimedia subsystem (IMS) based conversational services (IBACS) split rendering architecture, according to some example implementations;

[0024] FIG.7 illustrates a SR architecture according to some example implementations;

[0025] FIG.8 illustrates a scene graph in a glTF 2.0 based format;

[0026] FIG.9 illustrates the scene graph of FIG.8, as the scene graph may be divided for a split of the rendering operations;

[0027] FIG.10 is a signaling chart of adaptive split rendering including on demand asset delivery, according to some example implementations;

[0028] FIG.11 is a signaling chart of adaptive split rendering in an IBACS split rendering architecture, according to some example implementations;

[0029] FIG.12 illustrates logic to determine and schedule graphical asset delivery, according to some example implementations;

[0030] FIGS.13A, 13B and 13C are flowcharts illustrating various steps in a method performed by a first rendering entity on a user equipment (UE), according to various example implementations;

[0031] FIGS.14A, 14B, 14C and 14D are flowcharts illustrating various steps in a method performed by a second rendering entity on a network device, according to various example implementations; and

[0032] FIG.15 illustrates an apparatus according to some example implementations.DETAILED DESCRIPTION

[0033] Some implementations of the present disclosure will now be described more fully hereinafter with reference to the accompanying figures, in which some, but not all implementations of the disclosure are shown. Indeed, various implementations of the disclosure may be embodied in many different forms and should not be construed as limited to the implementations set forth herein; rather, these example implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Like reference numerals refer to like elements throughout.

[0034] Unless specified otherwise or clear from context, references to first, second or the like should not be construed to imply a particular order. A feature described as being above another feature (unless specified otherwise or clear from context) may instead be below, and vice versa; and similarly, features described as being to the left of another feature else may instead be to the right, and vice versa. Also, while reference may be made herein to quantitative measures, values, geometric relationships or the like, unless otherwise stated, any one or more if not all of these may be absolute or approximate to account for acceptable variations that may occur, such as those due to engineering tolerances or the like.

[0035] As used herein, unless specified otherwise or clear from context, the “or” of a set of operands is the “inclusive or” and thereby true if and only if one or more of the operands is true, as opposed to the “exclusive or” which is false when all of the operands are true. Thus, for example, “[A] or [B]” is true if [A] is true, or if [B] is true, or if both [A] and [B] are true. Further, the articles “a” and “an” mean “one or more,” unless specified otherwise or clear from context to be directed to a singular form. Furthermore, it should be understood that unless otherwise specified, the terms “data,” “content,” “digital content,” “information,” and similar terms may be at times used interchangeably. The term “network” may refer to a group of interconnected computers including clients and servers; and within a network, these computers may be interconnected directly or indirectly by various means including via one or more switches, routers, gateways, access points or the like.

[0036] Reference may be made herein to terms specific to a particular system, architecture or the like, but it should be understood that example implementations of the present disclosure may be equally applicable to any of a number of systems, architectures and the like. For example, reference may be made to 3GPP technologies such as Global System for Mobile Communications (GSM), UMTS, LTE, LTE Advanced, 5G NR, 5G Advanced and 6G; however, it should be understood that example implementations of the present disclosure may be equally applicable to non-3GPP technologies such as IEEE 802, Bluetooth and Bluetooth Low Energy.

[0037] Further, as used in this application, the term “circuitry” may refer to one or more or all of the following: (a) hardware-only circuit implementations (such as implementations in only analog and / or digitalcircuitry); (b) combinations of hardware circuits and software, such as (as applicable): (i) a combination of analog and / or digital hardware circuit(s) with software / firmware and (ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions); or (c) hardware circuit(s) and / or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.

[0038] The above definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

[0039] FIG.1 illustrates a telecommunications system 100 according to various example implementations of the present disclosure. The telecommunications system generally includes one or more telecommunications networks. As shown, for example, the system includes one or more public land mobile networks (PLMNs) 102 coupled to one or more other external data networks 104 – notably including a wide area network (WAN) such as the Internet. Each of the PLMNs includes a core network (CN) 106 backbone such as the Evolved Packet Core (EPC) of LTE, the 5G core network (5GC) or the like; and each of the core networks and the Internet are coupled to one or more radio access networks (RANs) 108, air interfaces or the like that implement one or more radio access technologies (RATs). As used herein, a “network device” refers to any suitable device at a network side of a telecommunications network. Examples of suitable network devices are described in greater detail below.

[0040] In addition, the system includes one or more radio units that may be varyingly known as user equipment (UE) 110, terminal device, terminal equipment, mobile station or the like. The UE is generally a device configured to communicate with a network device or a further UE in a telecommunication network. The UE may be a portable computer (e.g., laptop, notebook, tablet computer), mobile phone (e.g., cell phone, smartphone), wearable computer (e.g., smartwatch), or the like. In other examples, the UE may be an Internet of Things (IoT) device, an industrial IoT (IIoT device), a vehicle equipped with a vehicle-to- everything (V2X) communication technology, or the like. In some examples, as referenced by 3GPP, the UE may be a narrowband IoT (NB-IoT) device, an enhanced machine-type communication (eMTC) device, a reduced capability (RedCap) device, an ambient IoT device, or the like.

[0041] In operation, these UEs may be configured to connect to one or more of the RANs 108 according to their particular radio access technologies to thereby access a particular CN 106 ofa PLMN 102, or to access one or more of the external data networks 104 (e.g., the Internet). The external data network may be configured to provide Internet access, operator services, 3rd party services, etc. For example, the International Telecommunication Union (ITU) has classified 5G mobile network services into three categories: enhanced mobile broadband (eMBB), ultra-reliable and low-latency communications (URLLC), and massive machine type communications (mMTC) or massive internet of things (MIoT).

[0042] Examples of radio access technologies include 3GPP radio access technologies such as GSM, UMTS, LTE, LTE Advanced, 5G NR, 5G Advanced, and 6G. Other examples of radio access technologies include IEEE 802 technologies such as IEEE 802.11 (Wi-Fi), IEEE 802.15 (including 802.15.1 (WPAN / Bluetooth), 802.15.4 (Zigbee) and 802.15.6 (WBAN)), Bluetooth, Bluetooth Low Energy (BLE), ultra wideband (UWB), and the like. Generally, a radio access technology may refer to any 2G, 3G, 4G, 5G, 6G or higher generation mobile communication technology and their different versions, as well as to any other wireless radio access technology that may be arranged to interwork with such a mobile communication technology to provide access to the CN 106 of a mobile network operator (MNO).

[0043] In various examples, a RAN 108 may be configured as one or more macrocells, microcells, picocells, femtocells or the like. The RAN may generally include one or more radio access nodes that are configured to interact with UEs 110. In various examples, a radio access node may be referred to as a base station (BS), access point (AP), base transceiver station (BTS), Node B (NB), evolved NB (eNB), macro BS, NB (MNB) or eNB (MeNB), home BS, NB (HNB) or eNB (HeNB), next generation NB (gNB), enhanced gNB (en-gNB), next generation eNB (ng-eNB), or the like. The RAN may include some type of network controlling / governing entity responsible for control of the radio access nodes. The network controlling / governing entity and radio access node may be separate or integrated into a single apparatus. The network controlling / governing entity may include processing circuity configured to carry out various management functions, etc. The processing circuity may be associated with a memory, computer-readable storage medium or database for maintaining information required in the management functions.

[0044] A RAN 108 may be centralized or distributed. In various examples, components of a RAN may be interconnected by Ethernet, Gigabit Ethernet, Asynchronous Transfer Mode (ATM), optical fiber, dark fiber, passive wavelength division multiplexing (WDM), WDM passive optical network (WDM-PON), optical transport network (OTN), time sensitive networking (TSN) and / or any other data link layer network, possibly including radio links. The RAN may be connected to a CN 106 through one or more gateways, network functions or the like.

[0045] As will be appreciated, a PLMN 102 may be deployed in a number of different manners. In a 4G LTE deployment, the EPC is the CN 106, and the evolved UMTS terrestrial radio access network (E- UTRAN) is the RAN 108; and the E-UTRAN includes one or more eNBs (radio access nodes) configured to connect UEs 110 to the E-UTRAN to thereby access the EPC. FIG.2 illustrates a deployment 200, referred to at times as the 5G system (5GS). As shown, the 5GC 202 is the CN, and the next generation (NG) radioaccess network (NG-RAN) 204 is the RAN; and the NG-RAN includes one or more gNBs 206 (radio access nodes) configured to connect UEs 208 to the NG-RAN to thereby access the 5GC. The term ‘gNB’ in 5G may correspond to the eNB in 4G LTE.

[0046] Some 4G LTE and 5G deployments are considered standalone (SA) deployments. Other deployments combine 4G LTE and 5G technologies, and are referred to as non-standalone (NSA) deployments. In some deployments, the E-UTRAN includes one or more ng-eNBs that are configured to communicate with the 5GC, and that may also be configured to communicate with one or more gNBs. Similarly, in another deployment, the NG-RAN may include one or more en-gNBs that are configured to communicate with the EPC, and that may also be configured to communicate with one or more eNBs. In various instances, a single UE 110, a dual-mode or multimode UE, may support multiple (two or more) RANs—thereby being configured to connect to multiple RANs, such as 4G LTE and 5G.

[0047] FIG.3 more particularly depicts aspects of the 5GS deployment 200 for a MNO, according to some example implementations. As shown, the deployment includes the 5GC 202, and NG-RAN 204 with one or more gNBs 206 configured to connect UEs 208 to the NG-RAN to thereby access the 5GC. The 5GC may include a number of network functions (NFs) divided between the control plane and the user plane. In particular, the 5GC may include, for example, an access and mobility management function (AMF) 302, a session management function (SMF) 304, a user plane function (UPF) 306, a policy and charging function (PCF) 308, a network exposure function (NEF) 310, and / or an application function (AF) 312. Other examples of suitable NFs include a network repository function (NRF), a network slice selection function (NSSF), a unified data management (UDM), or the like. Also shown is a server hosting an application, referred to as application server (AS) 314.

[0048] In the control plane, the AMF 302 is configured to provide UE-based authentication, authorization, mobility management, etc. The PCF 308 may be responsible for policy enforcement and management, controlling Quality of Service (QoS), charging, and resource allocation for subscribers. The PCF may facilitate efficient resource utilization, optimizing network performance, and delivering a quality user experience.

[0049] The SMF 304 is configured to provide various functionality including session management (SM), UE Internet Protocol (IP) address allocation and management, selection and control of UPF(s) 306, control part of policy application and QoS, lawful intercept, termination of SM parts of NAS messages, Downlink Data Notification (DNN), roaming functionality, handle local enforcement to apply QoS for Service Level Agreements (SLAs), charging data collection and charging interface, etc. If the UE 208 has multiple sessions, different SMFs may be allocated to each session to manage them individually and possibly provide different functionalities per session.

[0050] The UPF 306 supports various user plane operations and functionalities, such as packet routing and forwarding, traffic handling (e.g., applying QoS policies), an anchor point for intra-RAT / inter-RAT mobility (when applicable), packet inspection and policy rule application, lawful intercept (UP collection), traffic accounting and reporting, etc. The UPF is the point of interconnect between the 5GC and at least one external data network (DN) 316 (i.e., point of ingress or egress for a DN), and routes packets to and from the DN. The DN may be configured to provide Internet access, operator services, 3rd party services, etc.

[0051] The AF 312 may interact with the 5GC 202 to enable the deployment of specific services and applications. The AF communicates with other NFs to request and manage network resources, ensuring that the network adapts to the requirements of different applications and services. The NEF 310 allows authorized third-party applications and services to access specific NFs and services in a controlled manner. The NEF enables the exposure of network capabilities to external entities, fostering innovation and the development of new services.

[0052] In some deployments, such as deployment 200, operations of the gNB 206 or other radio access node may be carried out, at least partly, in a central / centralized unit (CU), such as a server, host or node, operationally coupled to a distributed unit (DU), such as a radio head / node. It is also possible that node operations may be distributed among a plurality of servers, hosts or nodes. It should also be understood that the distribution of work between 5GC 202 (or other CN) operations and gNB (or other radio access node) operations may vary depending on implementation.

[0053] A 5G network architecture may be based on a so-called CU-DU split. One gNB-CU (central node) may control one or more gNB-DUs. The gNB-CU may control a plurality of spatially separated gNB- DUs, acting at least as transmit / receive (Tx / Rx) nodes. In some example implementations, however, the gNB-DUs (also called DU) may include, for example, a radio link control (RLC), medium access control (MAC) layer and a physical (PHY) layer, whereas the gNB-CU (also called a CU) may include the layers above the RLC layer, such as a packet data convergence protocol (PDCP) layer, a radio resource control (RRC), and an IP layer. Other functional splits are also possible. It is considered that a skilled person is familiar with the open systems interconnection (OSI) model and the functionalities within each layer.

[0054] In some example implementations, the server or CU may generate a virtual network through which the server communicates with the radio node. In general, virtual networking may involve a process of combining hardware and software network resources and network functionality into a single, software- based administrative entity, a virtual network. Such virtual network may provide flexible distribution of operations between the server and the radio head / node. In practice, any digital signal processing task may be performed in either the CU or the DU, and the boundary where the responsibility is shifted between the CU and the DU may be selected according to implementation.

[0055] Networks including 5G are now starting to support interactive media services such as eXtended reality (XR), which encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR). These and a number of other media services involve graphics rendering, which can have compute,render and power intensive requirements to provide high quality graphics for a reasonable quality of experience (QoE). Resource constrained devices like smartphones and head-mounted displays may not have the capacity to provide high quality XR experiences due to limitations of compute, render and energy resources.

[0056] Split rendering (SR) can be a potential solution to this issue. In the split rendering paradigm, the rendering workload is divided between two or more rendering entities, for example, a split rendering client (SRC) and a split rendering server (SRS). In split-rendering scenarios, the SRC regularly sends metadata to the SRS which the SRS uses to render frames of the scene. The SRS then sends the rendered media to the SRC which decodes and displays it. The above example, however is the simplest implementation of SR in that the client does minimal rendering.

[0057] Adaptive split rendering (ASR) improves upon simple SR by allowing the rendering operations to be adaptively split between the SRS and the SRC. In practice, this means a split adaptation logic is executed during the SR session and decides whether a particular part of the rendering operations for a given frame (or a sequence of frames) is executed at the SRS or the SRC. In ASR scenarios, both the SRS and SRC may need information about the scene, including graphical assets, available at runtime for execution and rendering. In typical rendering applications, for example, video games and XR applications, such information may be made available to a rendering entity in the form of a scene graph or scene description.

[0058] Scene graphs and scene descriptions represent a virtual 3D scene as a hierarchical graph comprising nodes. The nodes may represent virtual objects in the scene. A node may be associated with rendering data which includes geometry information such as meshes, material information such as reflectance, transparency, color, color maps (uv maps) and rendering logic. A mesh may comprise so called geometric primitives like points, lines, triangles, polygons, while as color information may comprise textures, bitmaps, uv maps and the rendering logic may comprise shaders. These components of an object may be needed by a rendering entity to render the object. This information may be provided to a rendering entity as a scene graph or a scene description, which is a resource that describes the scene as a hierarchical graph in a format that is understandable by the rendering entity.

[0059] One example format for the distribution of 3D scenes and models is the graphics library transmission format (glTF) 2.0. Another example format is the MPEG-I Scene Description (MPEG-I-SD), which provides extensions to scene description formats such as glTF in order to support Moving Picture Experts Group (MPEG) media, and in particular immersive media. In this regard, the MPEG-I-SD provides a standardized means for distribution of information about a real time immersive scene and the content therein.

[0060] The core capability of glTF2.0 allows description of a scene graph in an interoperable and extensible fashion as a JavaScript Object Notation (JSON) file which comprises a hierarchy of nodesrepresenting the scene graph. FIG.4 illustrates an example glTF2.0 top level scene graph 400. As shown, a node may be associated with rendering data which may include objects such as transformations, meshes, accessors (to meshes), textures, other nodes, references to other resources, etc. The extensions defined by MPEG allow glTF2.0 to reference immersive multimedia content which may be timed, dynamic textures, dynamic scene updates, etc. In the JSON file describing a glTF2.0 / MPEG-I-SD based scene, the objects in a scene may be identified by their index values in the scene graph or by their “name” property.

[0061] Once the scene information is available to a rendering entity, the scene may be rendered according to a rendering process. In one example, the rendering process for a given frame may conceptually be divided into three main stages, namely, the application stage, the geometry stage, and the rasterization stage. In the application stage, the scene and objects therein undergo transformations based on application logic. In the geometry stage, the mesh data comprising objects in the scene is subjected to various geometric transforms to obtain a subset of vertices which fall within the view frustum of the rendering (virtual) camera. In the rasterization stage, the pixels in a framebuffer to be displayed are filled based on the vertices from the geometry stage and associated information like color (e.g. form textures, sprites), illumination, depth from camera near plane, etc.

[0062] The above is an example rendering process, and it should be understood that various other rendering processes exist. Examples of other rendering processes include ray-tracing based rendering, point cloud based rendering, image based rendering, and the like.

[0063] In a split rendering session, both the SRS and the SRC may execute the rendering loop for a given frame to be displayed. In that case the SRS and SRC may each need information about the part of the scene they are rendering. When scene descriptions or graphs are used, this translates to the rendering entity (SRS or SRC) needing rendering data of the objects they render to be provided to them. This includes geometry, color and rendering data.

[0064] Split rendering is envisioned to be available in 5G networks. One example of a proposed split rendering architecture is the SR media service enabler (MSE) architecture, which is a general-purpose split rendering MSE based on 5G system. Another example is the IP multimedia subsystem (IMS) based conversational services (IBACS) split rendering architecture, which was developed specifically for conversational AR calls.

[0065] FIG.5 illustrates a SR MSE architecture 500, according to some example implementations. The SR MSE includes a number of 5G media functions, such as a SR client (SRC) 502, a SR server (SRS) 504, a real-time communication (RTC) AF 506 (e.g., AF 312), an application provider 508, an application 510, and a media session handler (MSH) 512. Also shown, in some examples, the SR MSE includes an XR runtime 514.

[0066] The SRC 502 may reside in a UE 208, and the SRS 504 may reside in a 5G edge server such as a RTC AS (e.g., AS 314) in a DN 316. The SRC may be responsible for acquiring UE media capabilitiesand negotiating with the SRS (e.g., RTC AS) to agree on the split-rendering process at the SRS. The SRS may be responsible for negotiating a SR session with the SRC, monitoring resource usage of the 5G edge server in which the SRS resides (e.g., RTC AS), and managing / running the split rendering process. The RTC AF 506 may be responsible for provisioning, quality of service (QoS) allocation, and edge resource discovery. The application provider 508 may offer the split rendering media service for the application 510 running on the UE 208. And the MSH 512 may be responsible for control plane communication with the RTC AF 506.

[0067] In the SR MSE architecture 500, the application provider 508 may provision resources for split- rendering through RTC-1. The application provider may be authorized to use resources and functionalities provided by the 5GS 200. In this regard, the application provider may request the 5GS to allocate appropriate resources and QoS profiles for a split rendering session. The resources, may for example, include means to carry out compute and render operations for a split rendering session.

[0068] The application provider 508 may deliver (e.g., provide) media to a SRS 502 through RTC-2. The communication between RTC AF 510 and SRS may be through RTC-3, which may include the EDGE- 3 interface. User plane signaling (e.g., WebRTC session setup or reconfiguration) and media delivery between SRC 504 and SRS may be though RTC-4. The RTC AF 506 may provide the split-rendering information to the MSH 512 defined by RTC-5. The SRC may discover the application 508 through RTC-6. The SRC may handle the XR runtime 514, and the SRC may discover the client media capabilities from the MSH through the RTC-7 interface. The application and application provider may interact through RTC-8.

[0069] FIG.6 illustrates an IBACS split rendering architecture 600, according to some example implementations. The IBACS split rendering architecture is based on the IMS architecture with enhancements to support data channel services. The architecture includes a SRC 602, a SRS 604, and a data channel (DC) application server 606 that support the split-rendering process. The IBACS split rendering architecture includes an IMS 608, and the IMS includes, for example, an interrogating / serving / proxy call session control function (I / S / P-CSCF) 610, an IMS AS 612, a DC media function (MF) 614 (or a DC media resource function (MRF)) and a data channel signaling function (DCSF) 616.

[0070] The SRC 602 may be provided by a DC multimedia telephony service for IMS (DC-MTSI) client 618 that resides in a UE 208, and the SRS 604 may be provided by the MF 614. The MF is a NF that interacts with the IMS AS 612 via the service-based interface DC2, and provides media capabilities in support of IMS DC and AR. The SRC may be responsible for acquiring the UE media capabilities and interacting with the SRS during the split-rendering process. The SRS may be responsible for interacting with the SRC during split-rendering process, monitoring resource usage, and managing / running the split rendering process. The DC application server may be responsible for service control related to split- rendering, including session media control and media capability negotiation with the UE via the MF.

[0071] In support of data channel services, the DCSF 616 is a signaling control function that provides data channel control logic. The DCSF may receive event reports from the IMS AS 612, and decide whether AR communication service is allowed to be provided during an IMS session. The DCSF may support HTTP web server functionality to download data channel applications (e.g., IMS-aware application 620) via the MF 614 to the UE 208 based on UE subscription. The DCSF may download data channel applications from a DC application repository 622. The DCSF may interact with the DC application server 606 for DC resource control via DC4 / DC3, and for traffic forwarding via MDC3 / MDC2.

[0072] FIG.7 illustrates a SR architecture 700 according to some example implementations, which may be implemented by the SR MSE architecture 500 or the IBACS split rendering architecture 600. As shown, the SR architecture 700 includes a SRC 702 (residing in a UE 208) and a SRS 704 (residing in the 5G network). The SRC and SRS may communicate with each other across the 5G network 706 (5GC 202 and NG-RAN 204), and receive assets and application logic from an application provider 708. In the SR MSE architecture, for example, the SRC 502 is defined as an entity within the UE that handles the SR related operations and communications and presents frames for display, while the SRS 504 is provided by an AS 314 capable of SR sessions. The SRC and SRS are specified to communicate using the 5G RTC architecture. The IBACS split rendering architecture specifies communication over the IMS architecture; and accordingly, the SRC 602 is provided by a DC-MTSI client 618 and the SRS is provided by a MF (or MRF), which have SR functionalities.

[0073] In a split rendering session where both the SRS 704 and the SRC 702 perform rendering operations, scene rendering data corresponding to the portions rendered by the SRS and SRC may need to be available at the respective rendering entity. Typically, when the rendering is done by a single entity, this rendering data is made available to the rendering entity before executing the rendering session. In case of SR, a native solution would be to make all the rendering data, which comprises the whole scene description, needed for the application being rendered to both the SRS and the SRC. This solution, while ensuring availability of rendering data at the SRS and SRC, may have the drawback that all rendering data, regardless of whether it is used at a rendering entity during the rendering session, is transmitted to it via a network in advance, potentially wasting network resources and slowing down the start of a split rendering session.

[0074] A second solution may be to tailor the rendering data made available to a rendering entity (SRS 704, SRC 702) according to some predefined split of the rendering operation. The predefined rendering split may be hard-coded or may have some flexibility. In the latter case, the rendering data made available to a rendering entity may have to accommodate the maximum extent of the split in rendering operations possible at the rendering entity. This approach may also be wasteful as depending on how the rendering split is adjusted during an SR session, assets already made available to a rendering entity may be not be used at the rendering entity for rendering. This may result in unnecessary usage of networkresources and delay in starting a SR rendering session as an SR session may not be started before all the designated assets are available at the corresponding rendering entities.

[0075] To illustrate this issue, consider an example application to be rendered during a split rendering session. For illustration, as shown in FIG.8, let the graphics scene to be rendered be in the form of a scene graph 800 or scene description in a glTF2.0 based format.

[0076] In the native solution for SR, the scene graph 800 of FIG.8 is made available to both the SRS 704 and SRC 702. It should be noted that each node of FIG.8 may further include graphical assets that may be large in storage size. For example, a node may include mesh data, material information etc. as shown, either embedded in the scene graph resource or as uniform resource names (URNs) that refer to the storage location of the actual binary data corresponding to the graphical assets, as illustrated in FIG.4. In the second solution mentioned above, the scene graph may be divided into two copies, each including the assets according to pre-defined logic for a split of the rendering operations. FIG.9 illustrates the scene graph as the scene graph may be divided for a split of the rendering operations.

[0077] The scene graph may be made available to both SRS 704 and SRC 702 as a graph, but the actual assets (binary files pointed to by, for example, accessors, meshes or textures) made available to the SRS may only include the nodes illustrated in solid lines, while the assets corresponding to nodes illustrated in dashed lines may be made available to the SRC. This solution however, restricts adaptability of the split of rendering operations during a split rendering session to the asset split decided at the start of the session.

[0078] In view of the foregoing, example implementations of the present disclosure therefore provide a solution in which graphical assets may be delivered on demand to rendering entities during an SR session. In this regard, split adaptation logic and possibly additional heuristics may be used to determine whether an asset is needed at a rendering entity, and when the asset is needed, deliver the asset before the asset is needed. The solution of example implementations may improve user QoE by enabling faster start-up times for SR applications. The solution may also reduce network resource usage as graphical assets with potentially large transmission sizes are not transported. The solution may reduce network congestion events as large burst of traffic at the start of a SR session are avoided, and may improve temporal network traffic distribution as relative large transfers of graphics assets may be staggered in time.

[0079] Some example implementations provide a first rendering entity on a UE 208 and a second rendering entity on a network device. In various examples, the first rendering entity is a SRC 702, and the second rendering entity is a SRS 704. The network device may be, for example, a 5G edge server, an AS 314, MF 614, MRF or the like. The SRC and SRS may establish a SR session for rendering a scene in which first objects in the scene are to be rendered by the SRC, and second objects in the scene are to be rendered by a SRS on a network device. The SRS may to send, to the SRC, graphical assets corresponding to the first objects.

[0080] The SRC 702 (first rendering entity) may to render the first objects using the graphical assets corresponding to the first objects to produce first rendered media. The SRS 704 (second rendering entity) may render the second objects to produce second rendered media, such as using graphical assets corresponding to the second objects. The SRS may send the second rendered media to the SRC, which may combine the first rendered media and the second rendered media into a view of the scene.

[0081] In some examples, the SRC 702 and SRS 704 may carry out an adaptation of the split rendering session that includes a change in the first objects to be rendered by the SRC, and the second objects to be rendered by the SRS. In some of these examples, the SRC may receive (and the SRS may send) the graphical assets corresponding to the first objects, and the SRC may render the first objects (and the SRS may render the second objects), in accordance with the change in the first objects and the second objects.

[0082] According to example implementations of the present disclosure, one or more of the graphical assets corresponding to one or more of the first objects may be sent by the SRS 702, and received by the SRC 704, on demand relative to when the first object(s) are to be rendered by the SRC. In this regard, in some examples, the SRC or SRS may make a determination that one or more of the first objects in the scene are to be rendered within a certain upcoming time period (in the near future). In some examples, the certain upcoming time period may be the time period to the next frame, such as 8 or 11 milliseconds (depending on the frame rate of the application), the time period of more than one frame ahead (e.g., 80 or 110 milliseconds), or the time period of a few hundred frames ahead (e.g., 5 to 10 seconds). In various examples, wihtout limitation, the certain upcoming time period may be in the range of 5 to 1,000 milliseconds.

[0083] In some further examples, the SRC 702 or SRS 704 may make the determination that the first object(s) in the scene are to be rendered within the certain upcoming time period, and that the graphical asset(s) corresponding to the first object(s) are locally unavailable to the SRC. When the determination is made by the SRC, the SRC may send a request to the SRS for the graphical asset(s) corresponding to the first object(s), and receive the graphical asset(s) from the SRS in response to the request. In the case of the SRS, the SRS may have knowledge about the SRC, the state of the split rendering session and the graphical assets available at the SRC, which may allow the SRS to proactively determine which assets the SRC will need in near future. When the determination is made by the SRS, the SRS may send the graphical asset(s) corresponding to the first object(s) to the SRC, without a request for the graphical asset(s) from the SRC.

[0084] In some examples, the determination may be made that the first object(s) in the scene are to be rendered within the certain upcoming time period, and that at least one of the graphical asset(s) corresponding to the first object(s) is locally available to the SRC 702. When the determination is made by the SRC, the request for the graphical asset(s) may be a request for an updated version of the at least oneof the graphical asset(s). And when the determination is made by the SRS 704, the SRS may send an updated version of the at least one of the graphical asset(s), again, without a request from the SRC.

[0085] In some examples, a quality of the graphical asset(s) that are sent by the SRS 704 to the SRC 702 (and received by the SRC) may depend on a position of the first object(s) relative to a camera object in the scene. In some of these examples, the request by the SRC for the graphical asset(s) may indicate the position of the first object(s) relative to the camera object.

[0086] In some examples, the SRS 704 may determine a quality for the graphical asset(s) based on one or more network capabilities and / or QoS policies. In some of these examples, the graphical asset(s) that are sent by the SRS 704 to the SRC 702 (and received by the SRC) may have the quality that depends on the one or more network capabilities and / or QoS policies.

[0087] In some examples, the graphical asset(s) that are sent by the SRS 704 to the SRC 702 (and received by the SRC) may be sent (and received) in an order based on a priority of the first object(s) to which the graphical asset(s) correspond. In some of these examples, the request by the SRC for the graphical asset(s) may include information that indicates the priority of the first object(s).

[0088] To further illustrate some example implementations of the present disclosure, FIG.10 is a signaling chart 1000 of on demand asset delivery for split rendering, according to some example implementations. As shown at step 1001, a SR session is established between an SRC 702 and an SRS 704. This may include negotiation of rendering output, meta-data formats, state and timing synchronization, rendering split, available rendering splits and initial asset delivery to SRC. Although not separately shown, other entities, such as 5G NFs and servers and application provider servers may be involved during session establishment. In the illustrated example, the SRS may receive the application logic and assets during session establishment.

[0089] A main loop follows session establishment. The main loop may include the processes and operations needed for a SR session and the application being executed in the SR session to run coherently, and the main loop may be executed once or more for each frame. The main loop may conceptually include multiple repeating computational processes and operations, and may include one or more sub-loops such as for physics updates, light map updates, application logic execution, rendering, split adaptation, and the proposed asset management operations. These sub-loops (each of which may itself be referred to a loop) may be executed sequentially or in parallel to each other, synchronously or asynchronously. Some of these sub-loops are non-exhaustively discussed below to illustrate an example of on demand asset delivery.

[0090] Within the main loop, a rendering sub-loop may include execution of graphics rendering operations for a given frame to be displayed. In the rendering sub-loop, the SRC 702 at step 1002a sends rendering and session related metadata to the SRS 704. The metadata may include pose information, user interaction information, session and application state information, gaze information etc. The SRS at step1002b renders (second) objects in the scene or performs rendering operations assigned to the SRS according to the split of the rendering operations in effect.

[0091] The SRC 702 at step 1002c renders (first) objects in the scene or performs rendering operations assigned to the SRC according to the split of the rendering operations in effect. The SRS 704 at step 1002d sends the (second) rendered media to the SRC. And the SRC at step 1002e combines the received rendered media from the SRS and its own (first) rendered media into a view of the scene for the given frame, which may also include processes like pose correction or re-projection. Also note that although steps 1002a, 1002b, 1002c and 1002d are presented in a particular order, the steps may occur in any order. And steps 1002e and 1002c may be executed as a single step.

[0092] In the rendering sub-loop, the SRS 704 and SRC 702 may at step 1003 adapt the split of the rendering operations according to, among other possible factors, operating conditions at the SRS, SRC or the communication channel between SRS and SRC. The split adaptation may include one of the rendering entities in the SR session (the SRS or SRC) monitoring triggers to modify the split (e.g., monitoring for conditions that indicate to modify the split), determining a new split, negotiating the implementation of a desirable split with the other rendering entity. Further, the rendering entity may exchange state change information with the other rendering entity which may include information about changes in state machines associated with objects in the scene being split rendered or with the application as a whole or portions thereof.

[0093] Asset management may also be performed in the rendering loop to deliver assets for the SR session on demand to reduce startup latency of SR session setup and consequently the application and to optimize network resource usage. The signaling chart includes a first option 1004 (UE centric) and a second option 1005 (network centric), which may be used separately or together.

[0094] In the first option 1004, the SRC 702 at step 1004a proactively determines which (first) objects in the scene the SRC will render in the near future (within the certain upcoming time period), and if corresponding graphical assets are locally available at the SRC. If one or more of the corresponding graphical assets for the objects to be rendered in the near future are not locally available, the SRC at step 1004b sends a request to the SRS 704 to deliver the corresponding graphical asset(s). If a graphical asset is locally available, the SRC may still send a request for an updated version of the graphical asset, with higher texture resolution for example. In some examples, the request may include information about the position of the object relative to the camera object (virtual camera) in the scene, and the position may be used by the SRS to decide at which resolution the asset needs to be sent. Additionally or alternatively, for example, the request may include priority information enabling the SRS to send a more important asset first, with the higher resolution first. The SRS receives the request, and at step 1004c sends the requested graphical asset(s) to the SRC.

[0095] In the second option 1005, the SRS 704 at step 1005a proactively determines which (first) objects in the scene that the SRC 702 will render in the near future, and if corresponding graphical assets are locally available at the SRC (e.g., delivered initially or previously during the SR session). If one or more of the corresponding graphical assets for the objects to be rendered in the near future are not locally available at the SRC, the SRS at step 1005b delivers the corresponding graphical asset(s) to the SRC.

[0096] In some examples, depending on the position of the object relative to the camera object (virtual camera) in the scene, the SRS 704 may send again already available graphical assets to the SRC 702 with a higher resolution or quality. Additionally or alternatively, in some examples, depending on network capabilities and QoS policies, the SRS may decide which resolution of the graphical assets to send to the SRC, such as to meet the estimated delivery time of assets. The decision may be made based on the policies which can be defined and updated based on XR application requirements.

[0097] In examples in which both the first option 1004 and the second option 1005 are used in the SR asset management process, the SRC 702 and SRS 704 may negotiate the delivery of graphical assets. In some of these examples, the SRC at step 1004a proactively determines which (first) objects in the scene the SRC will render in the near future, and if corresponding graphical assets are locally available at the SRC; and the SRC at step 1004b sends a request to the SRS to deliver the corresponding graphical asset(s). The SRS at step 1005a decides which corresponding graphical assets to send to the SRC. The decision may be made based on the policies which can be defined and updated based on XR application requirements and the SRC request. And the SRS at step 1005b sends the determined graphical asset(s) to the SRC.

[0098] The SR session then continues.

[0099] In the main loop, the split adaptation and asset management sub-loops are shown in a particular order. It should be understood, however, that the split adaptation and asset management sub- loops may be executed in any order or in parallel. Also, for a given frame, the split adaptation and asset management sub-loops may be executed in advance, synchronously or asynchronously with each other.

[0100] In some examples, a third entity, within or outside the 5G network 706, may be involved in the asset delivery process. Examples of suitable third entities include an application provider’s AS, which may be a 5GS AS 314 or an IMS AS 612. In the latter case, the request from the SRC 702 may be sent to the third entity via a DCSF 616. Initial asset delivery during session establishment at step 1001 may include asset delivery from a third entity to the SRS 704, and asset delivery to the SRC from the SRS or directly from the third entity, for example with initial application.

[0101] In some examples, the SRS 704 may not have all the graphical assets needed for rendering at the start of an SR session. In some of these examples, the SRS may also at step 1005a consider graphical assets (corresponding to second objects) needed by the SRS. The SRS may then request the graphical assets needed by the SRS, the graphical assets needed by the SRC 702, or both from third entitydescribed above. On receiving the graphical assets needed by the SRC (either requested by the SRC or determined by the SRS), these graphical assets may be forwarded by the SRS to the SRC at step 1005b.

[0102] In some examples, the SRS 704 may send semantic information about the graphical assets instead of the graphical assets themselves, and expect the SRC 702 to locally generate the graphical assets. In some of these examples, the SRC may locally generate the graphical assets using a generation engine (may be artificial intelligence (AI)-based), or already available similar graphical asset(s). For example, the SRS may tell the SRC to generate a 15 meter high tree instead of directly sending the graphical asset for the tree. This may reduce the bandwidth in the 5G network 706.

[0103] In some examples, the request for graphical asset(s) (step 1004b), or the delivery of graphical asset(s) (step 1004c, step 1005b) may be via a transport channel different from the rendered media and metadata transport channel used for the rendering sub-loop (steps 1002a-1002e) and split adaptation sub- loop (step 1003). For example, the media transport may be over WebRTC or IMS media channel over RTP, or over IMS DC, while the asset delivery may be over HTTP1 / 2 / 3 or some other protocol.

[0104] FIG.11 is a signaling chart 1100 of adaptive split rendering in an IBACS split rendering architecture, according to some example implementations. A DC-MTSI client 618 providing a SRC 602 (at times referred to as a SR-DCMTSI client) or an MF 614 providing a SRS 604 may adapt the split of rendering operations due to change in operating conditions of the split rendering session. The operating conditions of the split rendering session may include operating conditions of the UE 208, the MF or changes in the application 620 or scene being rendered, for example, changes in the scene description.

[0105] As shown at steps 1101 and 1102, an IMS session is established between the DC-MTSI client 618 in UE 208 and a terminating DC-MTSI client, and a split rendering session is set up between the DC-MTSI client and a serving MF 614. Assets related to the application being split rendered may be delivered at step 1103 to participants of the split rendering session. The asset delivery may include javascript assets, scene descriptions, and graphical assets corresponding to objects needed for the session.

[0106] A rendering loop may be executed continuously during the duration of the split rendering session, for each frame. The DC-MTSI client 618 at step 1104 sends metadata required for rendering to the MF. The metadata may include pose, pose predictions, user inputs, etc. The DC-MTSI client and the MF 614 render the frame at steps 1105 and 1106. The frame rendered by the MF is transmitted at step 1107 to the DC-MTSI client as well as possible metadata. The DC-MTSI client at step 1108 composes a display frame from the received rendered media and media rendered locally. Note that although steps 1105, 1106 and 1107 are presented in a particular order, the steps may occur in any order. Step 1108 may include pose- correction. And steps 1108 and 1106 may be executed as a single step.

[0107] A split adaptation may occur within the rendering loop. In this regard, as shown at step 1109, a trigger to adapt the split occurs at the DC-MTSI client 618. The trigger may be, for example, a change in available UE resources, changes in QoE of the SR session, changes in the scene / application beingrendered, or the like. The DC-MTSI client at step 1110 decides if a new split of the rendering operations is needed, and determines the new split. In other cases, the MF 614 may trigger (e.g., initiate) the split adaptation.

[0108] The DC-MTSI client 618 at step 1111 sends a request to the MF 614 to adapt the split to the new split, and the DC-MTSI client at step 1112 sends a request for assets (including graphical assets) according to new split. The MF at step 1113 delivers the assets to the DC-MTSI client. The MF may acquire the assets from IMS AS 612 (e.g., via DCSF 616). The MF at step 1114 actuates the new split of the rendering operations, and the MF at step 1115 sends an acknowledgment of the new split to the DC-MTSI client. And as shown at step 1116, the rendering session (including the rendering loop) continues.

[0109] FIG.12 illustrates logic 1200 to determine and schedule graphical asset delivery, according to some example implementations. The logic may be implemented by a rendering entity in a SR session, such as the SRC 702 (first rendering entity) or the SRS 704 (second rendering entity). As shown at blocks 1202 and 1204, the rendering entity may traverse a scene graph and identify the next object in the scene graph. The rendering entity may determine whether graphical asset(s) for that object are available at the SRC, as shown at block 1206; and the rendering entity may determine whether the object is to be rendered at the SRC at a target display time 1208 (in the near future), as shown at block 1210.

[0110] The logic to determine whether the object is to be rendered at the SRC 702 at the target display time 1208 may be made based on an estimate of the split of rendering operations at the target display time, as shown at block 1212. In addition to the target display time, the estimate of the split of rendering operations at the target display time may take into account current pose or pose trajectory 1214, a current split of rendering operations 1216, split adaptation logic 1218, past user behavior 1220 (e.g., historic pose data), and / or application experience logic 1222. Additionally or alternatively, in some examples, the determination may also be made based on one or more factors such as network round trip time, size of assets, estimated delivery time of assets, scene content, user preferences, distance of the viewer from the objects in the scene, a user profile (to determine custom assets, e.g., for advertisement purpose), or the like. In a scene graph or scene description based split rendered application, the scene content may include nodes of the scene graph.

[0111] When the graphical asset(s) for the object are unavailable at the SRC, and the object is to be rendered at the SRC 702 at the target display time 1208, the rendering entity may also determine whether delivery of the graphical asset(s) is already scheduled, as shown at block 1224. As shown at block 1226, when delivery of the graphical asset(s) is not already scheduled, the rendering entity may estimate a download time and schedule delivery of the graphical asset(s) in scheduled deliveries 1228 for the SRC. The graphical asset(s) may therefore be scheduled, and the rendering entity may traverse to the next object in the scene graph, returning to block 1204.

[0112] Example implementations of the present disclosure therefore provide a solution in which graphical assets may be delivered on demand to rendering entities, such as a SRC 702 and SRS 704, during an SR session. In some examples, the solution may include signaling between the SRC and SRS to determine if a delayed, on-demand delivery of graphical assets to the SRC can be used during the SR session. The solution may also include delayed delivery if delayed delivery shall be used for one or more graphical assets, which may imply that if the respective graphical asset(s) are to be rendered by the SRS for now, then the respective graphical asset(s) are not delivered to the SRC but delivery is delayed.

[0113] The solution of some example implementations may include an indication that the SRC 702 expects the SRS 704 make the determination that first object(s) in the scene are to be rendered in the near future by the SRC, and trigger delivery of the corresponding graphical asset(s). The SRC may be prepared to receive the graphical asset(s) pushed by the SRS in this case. In case HTTP methods are used, a push stream may be set up between the SRC and SRS and appropriately indicated in signaling to both endpoints. If RTP assets are used, an RTP stream may be set up between the SRS and SRC and paused.

[0114] For example, the expectation regarding the SRS 704 making the determination regarding needed graphical asset(s) may be indicated during setup of the SR session, using for example, session description protocol (SDP) or MPEG-I-SD with appropriate extensions or a combination of SDP and MPEG-I-SD. The signaling may additionally or alternatively be in form of signaling messages over IMS, session initiation protocol (SIP), IMS DC, or signaling sub-channels defined over IMS data channels.

[0115] The signaling may additionally or alternatively be in the form of signaling messages over SR signaling messages compliant with message formats specified for SR MSE or IBACS split rendering.

[0116] The solution of some example implementations may further include an encoding of graphical aspects that facilitates their flexible delivery. In this regard, some graphical assets may be encoded with multiple coding layers referring to different qualities / resolution. In this encoding, a base layer asset may be quickly sent and refined later if a higher quality is needed, by sending an additional enhancement layer.

[0117] FIGS.13A – 13C are flowcharts illustrating various steps in a method 1300 performed by a first rendering entity on a user equipment, according to various example implementations. The method includes establishing a split rendering session for rendering a scene in which first objects in the scene are to be rendered by the first rendering entity, and second objects in the scene are to be rendered by a second rendering entity on a network device, as shown at block 1302 of FIG.13A. The method includes receiving graphical assets corresponding to the first objects, including receiving one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered, as shown at block 1304. The method includes rendering the first objects using the graphical assets corresponding to the first objects to produce first rendered media, as shown at block 1306. The method includes receiving, from the second rendering entity, second rendered media produced by the second rendering entity rendering the second objects, as shown at block 1308. And the methodincludes combining the first rendered media and the second rendered media into a view of the scene, as shown at block 1310.

[0118] In some examples, the method 1300 further includes carrying out an adaptation of the split rendering session that includes a change in the first objects to be rendered by the first rendering entity, and the second objects to be rendered by the second rendering entity, as shown at block 1312 of FIG.13B. In some of these examples, the graphical assets corresponding to the first objects are received at block 1304, and the first objects are rendered at block 1306 using the graphical assets corresponding to the first objects, in accordance with the change in the first objects and the second objects.

[0119] In some examples, the method 1300 further includes making a determination that the one or more of the first objects in the scene are to be rendered within a certain upcoming time period, as shown at block 1314 of FIG.13C. The method also includes sending, based on the determination, a request for the one or more of the graphical assets corresponding to the one or more of the first objects, as shown at block 1316. And in some of these examples, the one or more of the graphical assets are received at block 1304 in response to the request.

[0120] In some examples, making the determination at block 1314 includes making the determination that that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that that the one or more of the graphical assets are locally unavailable to the first rendering entity.

[0121] In some examples, making the determination at block 1314 includes making the determination that that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that at least one of the one or more of the graphical assets are locally available to the first rendering entity. In some of these examples, the request for the one or more of the graphical assets is a request for an updated version of the at least one of the one or more of the graphical assets.

[0122] In some examples, the request for the one or more of the graphical assets includes information that indicates a position of the one or more of the first objects relative to a camera object in the scene. In some of these examples, a quality of one or more of the graphical assets that are received at block 1304 depends on the position of the one or more of the first objects in the scene.

[0123] In some examples, the request for the one or more of the graphical assets includes information that indicates a priority of the one or more of the first objects. In some of these examples, the one or more of the graphical assets that are received at block 1304 are received in an order based on the priority of the one or more of the first objects.

[0124] In some examples, the graphical assets corresponding to the first objects are received at block 1304 from the second rendering entity, and the one or more of the graphical assets are received on demand based on the second rendering entity determining the one or more of the first objects in the scene are to be rendered within a certain upcoming time period.

[0125] FIGS.14A – 14D are flowcharts illustrating various steps in a method 1400 performed by a second rendering entity on a network device, according to various example implementations. The method includes establishing a split rendering session for rendering a scene in which first objects in the scene are to be rendered by a first rendering entity on a user equipment, and second objects in the scene are to be rendered by the second rendering entity, as shown at block 1402 of FIG.14A. The method includes sending, to the first rendering entity, graphical assets corresponding to the first objects for rendering the first objects using the graphical assets to produce first rendered media, sending the graphical assets including sending one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered, as shown at block 1404. The method includes rendering the second objects to produce second rendered media, as shown at block 1406. And the method includes sending the second rendered media to the first rendering entity for combining of the first rendered media and the second rendered media into a view of the scene, as shown at block 1408.

[0126] In some examples, the method 1400 further includes carrying out an adaptation of the split rendering session that includes a change in the first objects to be rendered by the first rendering entity, and the second objects to be rendered by the second rendering entity, as shown at block 1410 of FIG.14B. In some of these examples, the graphical assets corresponding to the first objects are sent at block 1404, and the second objects are rendered at block 1406, in accordance with the change in the first objects and the second object.

[0127] In some examples, the method 1400 further includes receiving, from the first rendering entity, a request for the one or more of the graphical assets based on the first rendering entity determining the one or more of the first objects in the scene are to be rendered within a certain upcoming time period, as shown at block 1412 of FIG.14C. In some of these examples, the one or more of the graphical assets are sent to the first rendering entity at block 1404 in response to the request.

[0128] In some examples, the method 1400 further includes making a determination that the one or more of the first objects in the scene are to be rendered within a certain upcoming time period, as shown at block 1414 of FIG.14D. In some of these examples, the one or more of the graphical assets are sent at block 1404 based on the determination.

[0129] In some examples, making the determination at block 1414 includes making a determination that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that the one or more of the graphical assets are locally unavailable to the first rendering entity.

[0130] In some examples, making the determination at block 1414 includes making a determination that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that at least one of the one or more of the graphical assets are locally available to the first renderingentity. In some of these examples, the one or more of the graphical assets that are sent at block 1404 include an updated version of the at least one of the one or more of the graphical assets.

[0131] In some examples, a quality of the one or more of the graphical assets that are sent at block 1404 depends on a position of the one or more of the first objects relative to a camera object in the scene.

[0132] In some examples, the method 1400 further includes determining a quality for the one or more of the graphical assets based on at least one of one or more network capabilities or quality of service policies. In some of these examples, the one or more of the graphical assets that are sent at block 1404 have the quality that depends on the at least one of the one or more network capabilities or the quality of service policies.

[0133] According to example implementations of the present disclosure, a telecommunications system 100 or PLMN 102, and its components such as a UE 110, CN 106, RAN 108, 5GC 202, NG-RAN 204, gNB 206, UE 208, AF 312, AS 314, SRC 502, SRS 504, SRC 602, SRS 604, MF 614, and / or DC-MTSI client 618, may be implemented by various means. Means for implementing the system and its components may include hardware, firmware, software, or combinations thereof. In some examples, one or more apparatuses may be configured to function as or otherwise implement the system and its components shown and described herein. In examples involving more than one apparatus, the respective apparatuses may be connected to or otherwise in communication with one another in a number of different manners, such as directly or indirectly via a wired or wireless network or the like.

[0134] According to some example implementations, at least some of the method 1300 described with respect to FIGS.13A-13C may be carried out by an apparatus comprising means for performing functions corresponding steps of the method. Similarly, at least some of the method 1400 described with respect to FIGS.14A-14D may be carried out by an apparatus comprising means for performing functions corresponding steps of the method. Examples of a suitable apparatus may include a network function, an application server, network function, media function, media resource function or any suitable apparatus, such as a server, host or node. Other examples of a suitable apparatus may include a user equipment, user device, user terminal or the like.

[0135] FIG.15 illustrates an apparatus 1500 in which means for performing various functions includes hardware, alone or under direction of one or more computer programs from a computer-readable storage medium or other memory, such as computer memory, according to some example implementations of the present disclosure. Generally, an apparatus of example implementations of the present disclosure may comprise, include or be embodied in one or more fixed or portable electronic devices. Examples of suitable electronic devices include a wearable computer, mobile phone, portable computer, desktop computer, workstation computer, server (server computer) or the like. The apparatus may include one or more of each of a number of components such as, for example, processing circuitry 1502 connected to computer- readable storage medium or other memory 1504.

[0136] The processing circuitry 1502 may be composed of one or more processors alone or in combination with one or more computer-readable storage media. The processing circuitry is generally any piece of computer hardware that is capable of processing information such as, for example, data, computer programs and / or other suitable electronic information. The processing circuitry is composed of a collection of electronic circuits some of which may be packaged as an integrated circuit or multiple interconnected integrated circuits (an integrated circuit at times more commonly referred to as a “chip”). The processing circuitry may be configured to execute computer programs, which may be stored onboard the processing circuitry or otherwise stored in the memory 1504 (of the same or another apparatus).

[0137] The processing circuitry 1502 may be a number of processors, a multi-core processor or some other type of processor, depending on the particular implementation. Further, the processing circuitry may be implemented using a number of heterogeneous processor systems in which a main processor is present with one or more secondary processors on a single chip. As another illustrative example, the processing circuitry may be a symmetric multi-processor system containing multiple processors of the same type. In yet another example, the processing circuitry may be embodied as or otherwise include one or more ASICs, FPGAs or the like. Thus, although the processing circuitry may be capable of executing a computer program to perform one or more functions, the processing circuitry of various examples may be capable of performing one or more functions without the aid of a computer program. In either instance, the processing circuitry may be appropriately programmed to perform functions or operations according to example implementations of the present disclosure.

[0138] The memory 1504 is generally any piece of computer hardware that is capable of storing information such as, for example, data, computer programs, instructions 1506 (e.g., computer-readable program code) and / or other suitable information either on a temporary basis and / or a permanent basis. The memory may include volatile and / or non-volatile memory, and may be fixed or removable. Examples of suitable memory include recording media, random access memory (RAM), read-only memory (ROM), a hard drive, a flash memory, a thumb drive, a removable computer diskette, an optical disk or some combination thereof.

[0139] The memory 1504 is a non-transitory device capable of storing information. One example of a suitable memory is a computer-readable storage medium, which is distinguishable from a computer- readable transmission medium capable of carrying information from one location to another. Examples of suitable computer-readable transmission media comprise electronic carrier signals, telecommunications signals, software distribution packages, or some combination thereof. As used herein, the term “non- transitory” is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM versus ROM). A computer-readable medium as described herein generally refers to a computer-readable storage medium or computer-readable transmission medium. A computer-readable medium is any entity or device capable in which information, such as one or more computer programs or portions thereof, may be stored and carried.

[0140] In addition to the memory 1504 (e.g., computer-readable storage medium), the processing circuitry 1502 may also be connected to one or more interfaces for displaying, transmitting and / or receiving information. The interfaces may include a communications interface 1508 and / or one or more user interfaces. The communications interface may be configured to transmit and / or receive information, such as to and / or from other apparatus(es), network(s) or the like. The communications interface may be configured to transmit and / or receive information by physical (wired) and / or wireless communications links. Examples of suitable communication interfaces include a network interface controller (NIC), wireless NIC (WNIC) or the like.

[0141] The user interfaces may include a display 1510 and / or one or more user input interfaces 1512. The display may be configured to present or otherwise display information to a user, suitable examples of which include a liquid crystal display (LCD), light-emitting diode (LED) display, organic LED (OLED) display, active-matrix OLED (AMOLED) or the like. The user input interfaces may be wired or wireless, and may be configured to receive information from a user into the apparatus, such as for processing, storage and / or display. Suitable examples of user input interfaces include a microphone, image or video capture device, keyboard or keypad, joystick, touch-sensitive surface (separate from or integrated into a touchscreen), biometric sensor or the like. The user interfaces may further include one or more interfaces for communicating with peripherals such as printers, scanners or the like.

[0142] Execution of the instructions 1506 by the processing circuitry 1502, or storage of the instructions in the memory 1504, supports combinations of operations for implementing example implementations of the present disclosure. In this manner, an apparatus 1500 may comprise at least one processing circuitry and at least one memory coupled to the at least one processing circuitry, where the at least one processing circuitry is configured to execute instructions stored in the at least one memory. It will also be understood that one or more functions, and combinations of functions, may be implemented by special purpose hardware-based computer systems and / or processing circuitry which perform the specified functions, or combinations of special purpose hardware and program code instructions.

[0143] Some example implementations of the present disclosure may also be carried out in the form of a computer process defined by one or more computer programs or portions thereof. Example implementations of the present disclosure may be carried out by executing at least one portion of a computer program comprising instructions. The computer program may be in source code form, object code form, or in some intermediate form. The computer program may be stored in a computer-readable medium that is readable by a computer, processing circuitry or other suitable apparatus. As indicated above, for example, the computer program may be stored in a memory, such as a computer-readable storage medium. Additionally or alternatively, for example, the computer program may be stored in acomputer-readable transmission medium. The coding of software for carrying out example implementations of the present disclosure is well within the scope of a person of ordinary skill in the art.

[0144] As will be appreciated, any suitable instructions may be loaded onto a computer, a processing circuitry or other programmable apparatus from a memory or a computer-readable medium (e.g., computer- readable storage medium, computer-readable transmission medium) to produce a particular machine, such that the particular machine becomes a means for implementing the functions specified herein. The instructions may also be stored in a computer-readable medium that can direct a computer, a processing circuitry or other programmable apparatus to function in a particular manner to thereby generate a particular machine or particular article of manufacture. In some examples, the instructions stored in the computer-readable medium may produce an article of manufacture, where the article of manufacture becomes a means for implementing functions described herein. The instructions may be retrieved from a computer-readable medium and loaded into a computer, processing circuitry or other programmable apparatus to configure the computer, processing circuitry or other programmable apparatus to execute operations to be performed on or by the computer, processing circuitry or other programmable apparatus.

[0145] Retrieval, loading and execution of instructions comprising program code instructions may be performed sequentially such that one instruction is retrieved, loaded and executed at a time. In some example implementations, retrieval, loading and / or execution may be performed in parallel such that multiple instructions are retrieved, loaded, and / or executed together. Execution of the program code instructions may produce a computer-implemented process such that the instructions executed by the computer, processing circuitry or other programmable apparatus provide operations for implementing functions described herein.

[0146] As explained above and reiterated below, the present disclosure includes, without limitation, the following example implementations.

[0147] Clause 1. An apparatus implemented by a user equipment including a first rendering entity, the apparatus comprising: at least one memory configured to store instructions; and at least one processing circuitry configured to access the at least one memory, and execute the instructions to cause the apparatus to at least: establish a split rendering session for rendering a scene in which first objects in the scene are to be rendered by the first rendering entity, and second objects in the scene are to be rendered by a second rendering entity on a network device; receive graphical assets corresponding to the first objects, including receiving one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; render the first objects using the graphical assets corresponding to the first objects to produce first rendered media; receive, from the second rendering entity, second rendered media produced by the second rendering entity rendering the second objects; and combine the first rendered media and the second rendered media into a view of the scene.

[0148] Clause 2. The apparatus of clause 1, wherein the at least one processing circuitry is configured to execute the instructions to cause the apparatus to further carry out an adaptation of the split rendering session that includes a change in the first objects to be rendered by the first rendering entity, and the second objects to be rendered by the second rendering entity, and wherein the graphical assets corresponding to the first objects are received, and the first objects are rendered using the graphical assets corresponding to the first objects, in accordance with the change in the first objects and the second objects.

[0149] Clause 3. The apparatus of clause 1 or clause 2, wherein the at least one processing circuitry is configured to execute the instructions to cause the apparatus to further at least: make a determination that the one or more of the first objects in the scene are to be rendered within a certain upcoming time period; and send, based on the determination, a request for the one or more of the graphical assets corresponding to the one or more of the first objects, and wherein the one or more of the graphical assets are received in response to the request.

[0150] Clause 4. The apparatus of clause 3, wherein the apparatus caused to make the determination comprises the apparatus caused to make the determination that that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that that the one or more of the graphical assets are locally unavailable to the first rendering entity.

[0151] Clause 5. The apparatus of clause 3 or clause 4, wherein the apparatus caused to make the determination comprises the apparatus caused to make the determination that that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that at least one of the one or more of the graphical assets are locally available to the first rendering entity, and wherein the request for the one or more of the graphical assets is a request for an updated version of the at least one of the one or more of the graphical assets.

[0152] Clause 6. The apparatus of any of clauses 3 to 5, wherein the request for the one or more of the graphical assets comprises information that indicates a position of the one or more of the first objects relative to a camera object in the scene, and wherein a quality of one or more of the graphical assets that are received depends on the position of the one or more of the first objects in the scene.

[0153] Clause 7. The apparatus of any of clauses 3 to 6, wherein the request for the one or more of the graphical assets comprises information that indicates a priority of the one or more of the first objects, and wherein the one or more of the graphical assets that are received are received in an order based on the priority of the one or more of the first objects.

[0154] Clause 8. The apparatus of any of clauses 1 to 7, wherein the graphical assets corresponding to the first objects are received from the second rendering entity, and the one or more of the graphical assets are received on demand based on the second rendering entity determining the one or more of the first objects in the scene are to be rendered within a certain upcoming time period.

[0155] Clause 9. An apparatus implemented by a first rendering entity on an user equipment, the apparatus comprising: means for establishing a split rendering session for rendering a scene in which first objects in the scene are to be rendered by the first rendering entity, and second objects in the scene are to be rendered by a second rendering entity on a network device; means for receiving graphical assets corresponding to the first objects, including receiving one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; means for rendering the first objects using the graphical assets corresponding to the first objects to produce first rendered media; means for receiving, from the second rendering entity, second rendered media produced by the second rendering entity rendering the second objects; and means for combining the first rendered media and the second rendered media into a view of the scene.

[0156] Clause 10. The apparatus of clause 9, wherein the apparatus further comprises means for carrying out an adaptation of the split rendering session that includes a change in the first objects to be rendered by the first rendering entity, and the second objects to be rendered by the second rendering entity, and wherein the graphical assets corresponding to the first objects are received, and the first objects are rendered using the graphical assets corresponding to the first objects, in accordance with the change in the first objects and the second objects.

[0157] Clause 11. The apparatus of clause 9 or clause 10, wherein the apparatus further comprises: means for making a determination that the one or more of the first objects in the scene are to be rendered within a certain upcoming time period; and means for sending, based on the determination, a request for the one or more of the graphical assets corresponding to the one or more of the first objects, and wherein the one or more of the graphical assets are received in response to the request.

[0158] Clause 12. The apparatus of clause 11, wherein the means for making the determination comprises means for making the determination that that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that that the one or more of the graphical assets are locally unavailable to the first rendering entity.

[0159] Clause 13. The apparatus of clause 11 or clause 12, wherein the means for making the determination comprises means for making the determination that that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that at least one of the one or more of the graphical assets are locally available to the first rendering entity, and wherein the request for the one or more of the graphical assets is a request for an updated version of the at least one of the one or more of the graphical assets.

[0160] Clause 14. The apparatus of any of clauses 11 to 13, wherein the request for the one or more of the graphical assets comprises information that indicates a position of the one or more of the first objects relative to a camera object in the scene, and wherein a quality of one or more of the graphical assets that are received depends on the position of the one or more of the first objects in the scene.

[0161] Clause 15. The apparatus of any of clauses 11 to 14, wherein the request for the one or more of the graphical assets comprises information that indicates a priority of the one or more of the first objects, and wherein the one or more of the graphical assets that are received are received in an order based on the priority of the one or more of the first objects.

[0162] Clause 16. The apparatus of any of clauses 9 to 15, wherein the graphical assets corresponding to the first objects are received from the second rendering entity, and the one or more of the graphical assets are received on demand based on the second rendering entity determining the one or more of the first objects in the scene are to be rendered within a certain upcoming time period.

[0163] Clause 17. A method performed by a first rendering entity on a user equipment, the method comprising: establishing a split rendering session for rendering a scene in which first objects in the scene are to be rendered by the first rendering entity, and second objects in the scene are to be rendered by a second rendering entity on a network device; receiving graphical assets corresponding to the first objects, including receiving one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; rendering the first objects using the graphical assets corresponding to the first objects to produce first rendered media; receiving, from the second rendering entity, second rendered media produced by the second rendering entity rendering the second objects; and combining the first rendered media and the second rendered media into a view of the scene.

[0164] Clause 18. The method of clause 17, wherein the method further comprises carrying out an adaptation of the split rendering session that includes a change in the first objects to be rendered by the first rendering entity, and the second objects to be rendered by the second rendering entity, and wherein the graphical assets corresponding to the first objects are received, and the first objects are rendered using the graphical assets corresponding to the first objects, in accordance with the change in the first objects and the second objects.

[0165] Clause 19. The method of clause 17 or clause 18, wherein the method further comprises: making a determination that the one or more of the first objects in the scene are to be rendered within a certain upcoming time period; and sending, based on the determination, a request for the one or more of the graphical assets corresponding to the one or more of the first objects, and wherein the one or more of the graphical assets are received in response to the request.

[0166] Clause 20. The method of clause 19, wherein making the determination comprises making the determination that that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that that the one or more of the graphical assets are locally unavailable to the first rendering entity.

[0167] Clause 21. The method of clause 19 or clause 20, wherein making the determination comprises making the determination that that the one or more of the first objects in the scene are to be renderedwithin the certain upcoming time period, and that at least one of the one or more of the graphical assets are locally available to the first rendering entity, and wherein the request for the one or more of the graphical assets is a request for an updated version of the at least one of the one or more of the graphical assets.

[0168] Clause 22. The method of any of clauses 19 to 21, wherein the request for the one or more of the graphical assets comprises information that indicates a position of the one or more of the first objects relative to a camera object in the scene, and wherein a quality of one or more of the graphical assets that are received depends on the position of the one or more of the first objects in the scene.

[0169] Clause 23. The method of any of clauses 19 to 22, wherein the request for the one or more of the graphical assets comprises information that indicates a priority of the one or more of the first objects, and wherein the one or more of the graphical assets that are received are received in an order based on the priority of the one or more of the first objects.

[0170] Clause 24. The method of any of clauses 17 to 23, wherein the graphical assets corresponding to the first objects are received from the second rendering entity, and the one or more of the graphical assets are received on demand based on the second rendering entity determining the one or more of the first objects in the scene are to be rendered within a certain upcoming time period.

[0171] Clause 25. A computer-readable storage medium that is non-transitory and has instructions stored therein that, in response to execution by at least one processing circuitry, causes a user equipment including a first rendering entity to at least: establish a split rendering session for rendering a scene in which first objects in the scene are to be rendered by the first rendering entity, and second objects in the scene are to be rendered by a second rendering entity on a network device; receive graphical assets corresponding to the first objects, including receiving one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; render the first objects using the graphical assets corresponding to the first objects to produce first rendered media; receive, from the second rendering entity, second rendered media produced by the second rendering entity rendering the second objects; and combine the first rendered media and the second rendered media into a view of the scene.

[0172] Clause 26. The computer-readable storage medium of clause 25, wherein the computer-readable storage medium has further instructions stored therein that, in response to execution by the at least one processing circuitry, causes the user equipment to further carry out an adaptation of the split rendering session that includes a change in the first objects to be rendered by the first rendering entity, and the second objects to be rendered by the second rendering entity, and wherein the graphical assets corresponding to the first objects are received, and the first objects are rendered using the graphical assets corresponding to the first objects, in accordance with the change in the first objects and the second objects.

[0173] Clause 27. The computer-readable storage medium of clause 25 or clause 26, wherein the computer-readable storage medium has further instructions stored therein that, in response to execution bythe at least one processing circuitry, causes the user equipment to further at least: make a determination that the one or more of the first objects in the scene are to be rendered within a certain upcoming time period; and send, based on the determination, a request for the one or more of the graphical assets corresponding to the one or more of the first objects, and wherein the one or more of the graphical assets are received in response to the request.

[0174] Clause 28. The computer-readable storage medium of clause 27, wherein the user equipment caused to make the determination comprises the user equipment caused to make the determination that that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that that the one or more of the graphical assets are locally unavailable to the first rendering entity.

[0175] Clause 29. The computer-readable storage medium of clause 27 or clause 28, wherein the user equipment caused to make the determination comprises the user equipment caused to make the determination that that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that at least one of the one or more of the graphical assets are locally available to the first rendering entity, and wherein the request for the one or more of the graphical assets is a request for an updated version of the at least one of the one or more of the graphical assets.

[0176] Clause 30. The computer-readable storage medium of any of clauses 27 to 29, wherein the request for the one or more of the graphical assets comprises information that indicates a position of the one or more of the first objects relative to a camera object in the scene, and wherein a quality of one or more of the graphical assets that are received depends on the position of the one or more of the first objects in the scene.

[0177] Clause 31. The computer-readable storage medium of any of clauses 27 to 30, wherein the request for the one or more of the graphical assets comprises information that indicates a priority of the one or more of the first objects, and wherein the one or more of the graphical assets that are received are received in an order based on the priority of the one or more of the first objects.

[0178] Clause 32. The computer-readable storage medium of any of clauses 25 to 31, wherein the graphical assets corresponding to the first objects are received from the second rendering entity, and the one or more of the graphical assets are received on demand based on the second rendering entity determining the one or more of the first objects in the scene are to be rendered within a certain upcoming time period.

[0179] Clause 33. An apparatus comprising means for performing the method of any of clauses 17 to 24.

[0180] Clause 34. A computer-readable medium comprising instructions that, in response to execution by at least one processing circuitry, causes an apparatus to perform the method of any of clauses 17 to 24.

[0181] Clause 35. A computer-readable storage medium comprising instructions that, in response to execution by at least one processing circuitry, causes an apparatus to perform the method of any of clauses 17 to 24.

[0182] Clause 36. A computer program comprising instructions that, in response to execution by at least one processing circuitry, causes an apparatus to perform the method of any of clauses 17 to 24.

[0183] Clause 37. An apparatus implemented by a network device including a second rendering entity, the apparatus comprising: at least one memory configured to store instructions; and at least one processing circuitry configured to access the at least one memory, and execute the instructions to cause the apparatus to at least: establish a split rendering session for rendering a scene in which first objects in the scene are to be rendered by a first rendering entity on a user equipment, and second objects in the scene are to be rendered by the second rendering entity; send, to the first rendering entity, graphical assets corresponding to the first objects for rendering the first objects using the graphical assets to produce first rendered media, sending the graphical assets including sending one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; render the second objects to produce second rendered media; and send the second rendered media to the first rendering entity for combining of the first rendered media and the second rendered media into a view of the scene.

[0184] Clause 38. The apparatus of clause 37, wherein the at least one processing circuitry is configured to execute the instructions to cause the apparatus to further carry out an adaptation of the split rendering session that includes a change in the first objects to be rendered by the first rendering entity, and the second objects to be rendered by the second rendering entity, and wherein the graphical assets corresponding to the first objects are sent, and the second objects are rendered, in accordance with the change in the first objects and the second object.

[0185] Clause 39. The apparatus of clause 37 or clause 38, wherein the at least one processing circuitry is configured to execute the instructions to cause the apparatus to further receive, from the first rendering entity, a request for the one or more of the graphical assets based on the first rendering entity determining the one or more of the first objects in the scene are to be rendered within a certain upcoming time period, and wherein the one or more of the graphical assets are sent to the first rendering entity in response to the request.

[0186] Clause 40. The apparatus of any of clauses 37 to 39, wherein the at least one processing circuitry is configured to execute the instructions to cause the apparatus to further make a determination that the one or more of the first objects in the scene are to be rendered within a certain upcoming time period, and wherein the one or more of the graphical assets are sent based on the determination.

[0187] Clause 41. The apparatus of clause 40, wherein the apparatus caused to make the determination comprises the apparatus caused to make a determination that the one or more of the first objects in thescene are to be rendered within the certain upcoming time period, and that the one or more of the graphical assets are locally unavailable to the first rendering entity.

[0188] Clause 42. The apparatus of clause 40 or clause 41, wherein the apparatus caused to make the determination comprises the apparatus caused to make a determination that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that at least one of the one or more of the graphical assets are locally available to the first rendering entity, and wherein the one or more of the graphical assets that are sent include an updated version of the at least one of the one or more of the graphical assets.

[0189] Clause 43. The apparatus of any of clauses 40 to 42, wherein a quality of the one or more of the graphical assets that are sent depends on a position of the one or more of the first objects relative to a camera object in the scene.

[0190] Clause 44. The apparatus of any of clauses 40 to 43, wherein the at least one processing circuitry is configured to execute the instructions to cause the apparatus to further determine a quality for the one or more of the graphical assets based on at least one of one or more network capabilities or quality of service policies, and wherein the one or more of the graphical assets that are sent have the quality that depends on the at least one of the one or more network capabilities or the quality of service policies.

[0191] Clause 45. An apparatus implemented by a second rendering entity on a network device, the apparatus comprising: means for establishing a split rendering session for rendering a scene in which first objects in the scene are to be rendered by a first rendering entity on an user equipment, and second objects in the scene are to be rendered by the second rendering entity; means for sending, to the first rendering entity, graphical assets corresponding to the first objects for rendering the first objects using the graphical assets to produce first rendered media, sending the graphical assets including sending one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; means for rendering the second objects to produce second rendered media; and means for sending the second rendered media to the first rendering entity for combining of the first rendered media and the second rendered media into a view of the scene.

[0192] Clause 46. The apparatus of clause 45, wherein the apparatus further comprises means for carrying out an adaptation of the split rendering session that includes a change in the first objects to be rendered by the first rendering entity, and the second objects to be rendered by the second rendering entity, and wherein the graphical assets corresponding to the first objects are sent, and the second objects are rendered, in accordance with the change in the first objects and the second object.

[0193] Clause 47. The apparatus of clause 45 or clause 46, wherein the apparatus further comprises means for receiving, from the first rendering entity, a request for the one or more of the graphical assets based on the first rendering entity determining the one or more of the first objects in the scene are to berendered within a certain upcoming time period, and wherein the one or more of the graphical assets are sent to the first rendering entity in response to the request.

[0194] Clause 48. The apparatus of any of clauses 45 to 47, wherein the apparatus further comprises means for making a determination that the one or more of the first objects in the scene are to be rendered within a certain upcoming time period, and wherein the one or more of the graphical assets are sent based on the determination.

[0195] Clause 49. The apparatus of clause 48, wherein the means for making the determination comprises means for making a determination that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that the one or more of the graphical assets are locally unavailable to the first rendering entity.

[0196] Clause 50. The apparatus of clause 48 or clause 49, wherein the means for making the determination comprises means for making a determination that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that at least one of the one or more of the graphical assets are locally available to the first rendering entity, and wherein the one or more of the graphical assets that are sent include an updated version of the at least one of the one or more of the graphical assets.

[0197] Clause 51. The apparatus of any of clauses 48 to 50, wherein a quality of the one or more of the graphical assets that are sent depends on a position of the one or more of the first objects relative to a camera object in the scene.

[0198] Clause 52. The apparatus of any of clauses 48 to 51, wherein the apparatus further comprises means for determining a quality for the one or more of the graphical assets based on at least one of one or more network capabilities or quality of service policies, and wherein the one or more of the graphical assets that are sent have the quality that depends on the at least one of the one or more network capabilities or the quality of service policies.

[0199] Clause 53. A method performed by a second rendering entity on a network device, the method comprising: establishing a split rendering session for rendering a scene in which first objects in the scene are to be rendered by a first rendering entity on a user equipment, and second objects in the scene are to be rendered by the second rendering entity; sending, to the first rendering entity, graphical assets corresponding to the first objects for rendering the first objects using the graphical assets to produce first rendered media, sending the graphical assets including sending one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; rendering the second objects to produce second rendered media; and sending the second rendered media to the first rendering entity for combining of the first rendered media and the second rendered media into a view of the scene.

[0200] Clause 54. The method of clause 53, wherein the method further comprises carrying out an adaptation of the split rendering session that includes a change in the first objects to be rendered by the first rendering entity, and the second objects to be rendered by the second rendering entity, and wherein the graphical assets corresponding to the first objects are sent, and the second objects are rendered, in accordance with the change in the first objects and the second object.

[0201] Clause 55. The method of clause 53 or clause 54, wherein the method further comprises receiving, from the first rendering entity, a request for the one or more of the graphical assets based on the first rendering entity determining the one or more of the first objects in the scene are to be rendered within a certain upcoming time period, and wherein the one or more of the graphical assets are sent to the first rendering entity in response to the request.

[0202] Clause 56. The method of any of clauses 53 to 55, wherein the method further comprises making a determination that the one or more of the first objects in the scene are to be rendered within a certain upcoming time period, and wherein the one or more of the graphical assets are sent based on the determination.

[0203] Clause 57. The method of clause 56, wherein making the determination comprises making a determination that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that the one or more of the graphical assets are locally unavailable to the first rendering entity.

[0204] Clause 58. The method of clause 56 or clause 57, wherein making the determination comprises making a determination that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that at least one of the one or more of the graphical assets are locally available to the first rendering entity, and wherein the one or more of the graphical assets that are sent include an updated version of the at least one of the one or more of the graphical assets.

[0205] Clause 59. The method of any of clauses 56 to 58, wherein a quality of the one or more of the graphical assets that are sent depends on a position of the one or more of the first objects relative to a camera object in the scene.

[0206] Clause 60. The method of any of clauses 56 to 59, wherein the method further comprises determining a quality for the one or more of the graphical assets based on at least one of one or more network capabilities or quality of service policies, and wherein the one or more of the graphical assets that are sent have the quality that depends on the at least one of the one or more network capabilities or the quality of service policies.

[0207] Clause 61. A computer-readable storage medium that is non-transitory and has instructions stored therein that, in response to execution by at least one processing circuitry, causes a network device including a second rendering entity to at least: establish a split rendering session for rendering a scene in which first objects in the scene are to be rendered by a first rendering entity on a user equipment, andsecond objects in the scene are to be rendered by the second rendering entity; send, to the first rendering entity, graphical assets corresponding to the first objects for rendering the first objects using the graphical assets to produce first rendered media, sending the graphical assets including sending one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; render the second objects to produce second rendered media; and send the second rendered media to the first rendering entity for combining of the first rendered media and the second rendered media into a view of the scene.

[0208] Clause 62. The computer-readable storage medium of clause 61, wherein the computer-readable storage medium has further instructions stored therein that, in response to execution by the at least one processing circuitry, causes the network device to further carry out an adaptation of the split rendering session that includes a change in the first objects to be rendered by the first rendering entity, and the second objects to be rendered by the second rendering entity, and wherein the graphical assets corresponding to the first objects are sent, and the second objects are rendered, in accordance with the change in the first objects and the second object.

[0209] Clause 63. The computer-readable storage medium of clause 61 or clause 62, wherein the computer-readable storage medium has further instructions stored therein that, in response to execution by the at least one processing circuitry, causes the network device to further receive, from the first rendering entity, a request for the one or more of the graphical assets based on the first rendering entity determining the one or more of the first objects in the scene are to be rendered within a certain upcoming time period, and wherein the one or more of the graphical assets are sent to the first rendering entity in response to the request.

[0210] Clause 64. The computer-readable storage medium of any of clauses 61 to 63, wherein the computer-readable storage medium has further instructions stored therein that, in response to execution by the at least one processing circuitry, causes the network device to further make a determination that the one or more of the first objects in the scene are to be rendered within a certain upcoming time period, and wherein the one or more of the graphical assets are sent based on the determination.

[0211] Clause 65. The computer-readable storage medium of clause 64, wherein the network device caused to make the determination comprises the network device caused to make a determination that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that the one or more of the graphical assets are locally unavailable to the first rendering entity.

[0212] Clause 66. The computer-readable storage medium of clause 64 or clause 65, wherein the network device caused to make the determination comprises the network device caused to make a determination that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that at least one of the one or more of the graphical assets are locally available to the firstrendering entity, and wherein the one or more of the graphical assets that are sent include an updated version of the at least one of the one or more of the graphical assets.

[0213] Clause 67. The computer-readable storage medium of any of clauses 64 to 66, wherein a quality of the one or more of the graphical assets that are sent depends on a position of the one or more of the first objects relative to a camera object in the scene.

[0214] Clause 68. The computer-readable storage medium of any of clauses 64 to 67, wherein the computer-readable storage medium has further instructions stored therein that, in response to execution by the at least one processing circuitry, causes the network device to further determine a quality for the one or more of the graphical assets based on at least one of one or more network capabilities or quality of service policies, and wherein the one or more of the graphical assets that are sent have the quality that depends on the at least one of the one or more network capabilities or the quality of service policies.

[0215] Clause 69. An apparatus comprising means for performing the method of any of clauses 53 to 60.

[0216] Clause 70. A computer-readable medium comprising instructions that, in response to execution by at least one processing circuitry, causes an apparatus to perform the method of any of clauses 53 to 60.

[0217] Clause 71. A computer-readable storage medium comprising instructions that, in response to execution by at least one processing circuitry, causes an apparatus to perform the method of any of clauses 53 to 60.

[0218] Clause 72. A computer program comprising instructions that, in response to execution by at least one processing circuitry, causes an apparatus to perform the method of any of clauses 53 to 60.

[0219] Many modifications and other implementations of the disclosure set forth herein will come to mind to one skilled in the art to which the disclosure pertains having the benefit of the teachings presented in the foregoing description and the associated figures. Therefore, it is to be understood that the disclosure is not to be limited to the specific implementations disclosed and that modifications and other implementations are intended to be included within the scope of the appended claims. Moreover, although the foregoing description and the associated figures describe example implementations in the context of certain example combinations of elements and / or functions, it should be appreciated that different combinations of elements and / or functions may be provided by alternative implementations without departing from the scope of the appended claims. In this regard, for example, different combinations of elements and / or functions than those explicitly described above are also contemplated as may be set forth in some of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

WHAT IS CLAIMED IS:

1. A method performed by a first rendering entity on a user equipment, the method comprising: establishing a split rendering session for rendering a scene in which first objects in the scene are to be rendered by the first rendering entity, and second objects in the scene are to be rendered by a second rendering entity on a network device; receiving, from the second rendering entity, graphical assets corresponding to the first objects, the receiving the graphical assets including receiving one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; rendering the first objects using the graphical assets corresponding to the first objects to produce first rendered media; receiving, from the second rendering entity, second rendered media produced by the second rendering entity rendering the second objects; and combining the first rendered media and the second rendered media into a view of the scene.

2. The method as claimed in claim 1, wherein the method further comprises carrying out an adaptation of the split rendering session that includes a change in the first objects to be rendered by the first rendering entity, and the second objects to be rendered by the second rendering entity, and wherein the graphical assets corresponding to the first objects are received, and the first objects are rendered using the graphical assets corresponding to the first objects, in accordance with the change in the first objects and the second objects.

3. The method as claimed in claim 1 or claim 2, wherein the method further comprises: making a determination that the one or more of the first objects in the scene are to be rendered within a certain upcoming time period; and sending, based on the determination, a request for the one or more of the graphical assets corresponding to the one or more of the first objects, and wherein the one or more of the graphical assets are received in response to the request.

4. The method as claimed in claim 3, wherein the making the determination comprises making a determination that that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that that the one or more of the graphical assets are locally unavailable to the first rendering entity.

5. The method as claimed in claim 3 or claim 4, wherein the making the determination comprises making a determination that that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that at least one of the one or more of the graphical assets are locally available to the first rendering entity, and wherein the request for the one or more of the graphical assets is a request for an updated version of the at least one of the one or more of the graphical assets.

6. The method as claimed in any of claims 3 to 5, wherein the request for the one or more of the graphical assets comprises information that indicates a position of the one or more of the first objects relative to a camera object in the scene, and wherein a quality of one or more of the graphical assets that are received depends on the position of the one or more of the first objects in the scene.

7. The method as claimed in any of claims 3 to 6, wherein the request for the one or more of the graphical assets comprises information that indicates a priority of the one or more of the first objects, and wherein the one or more of the graphical assets that are received are received in an order based on the priority of the one or more of the first objects.

8. The method as claimed in any of claims 1 to 7, wherein the graphical assets corresponding to the first objects are received from the second rendering entity, and the one or more of the graphical assets are received on demand based on the second rendering entity determining the one or more of the first objects in the scene are to be rendered within a certain upcoming time period.

9. An apparatus comprising means for performing the method as claimed in any of claims 1 to 8.

10. A computer-readable medium comprising instructions that, in response to execution by at least one processing circuitry, causes an apparatus to perform the method as claimed in any of claims 1 to 8.

11. A method performed by a second rendering entity on a network device, the method comprising:establishing a split rendering session for rendering a scene in which first objects in the scene are to be rendered by a first rendering entity on a user equipment, and second objects in the scene are to be rendered by the second rendering entity; sending, to the first rendering entity, graphical assets corresponding to the first objects for rendering the first objects using the graphical assets to produce first rendered media, the sending the graphical assets including sending one or more of the graphical assets corresponding to one or more of the first objects on demand relative to when the one or more of the first objects are to be rendered; rendering the second objects to produce second rendered media; and sending the second rendered media to the first rendering entity for combining of the first rendered media and the second rendered media into a view of the scene.

12. The method as claimed in claim 11, wherein the method further comprises carrying out an adaptation of the split rendering session that includes a change in the first objects to be rendered by the first rendering entity, and the second objects to be rendered by the second rendering entity, and wherein the graphical assets corresponding to the first objects are sent, and the second objects are rendered, in accordance with the change in the first objects and the second object.

13. The method as claimed in claim 11 or claim 12, wherein the method further comprises receiving, from the first rendering entity, a request for the one or more of the graphical assets based on the first rendering entity determining the one or more of the first objects in the scene are to be rendered within a certain upcoming time period, and wherein the one or more of the graphical assets are sent to the first rendering entity in response to the request.

14. The method as claimed in any of claims 11 to 13, wherein the method further comprises making a determination that the one or more of the first objects in the scene are to be rendered within a certain upcoming time period, and wherein the one or more of the graphical assets are sent based on the determination.

15. The method as claimed in claim 14, wherein the making the determination comprises making a determination that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that the one or more of the graphical assets are locally unavailable to the first rendering entity.

16. The method as claimed in claim 14 or claim 15, wherein the making the determination comprises making a determination that the one or more of the first objects in the scene are to be rendered within the certain upcoming time period, and that at least one of the one or more of the graphical assets are locally available to the first rendering entity, and wherein the one or more of the graphical assets that are sent include an updated version of the at least one of the one or more of the graphical assets.

17. The method as claimed in any of claims 14 to 16, wherein a quality of the one or more of the graphical assets that are sent depends on a position of the one or more of the first objects relative to a camera object in the scene.

18. The method as claimed in any of claims 14 to 17, wherein the method further comprises determining a quality for the one or more of the graphical assets based on at least one of one or more network capabilities or quality of service policies, and wherein the one or more of the graphical assets that are sent have the quality that depends on the at least one of the one or more network capabilities or the quality of service policies.

19. An apparatus comprising means for performing the method as claimed in any of claims 11 to 18.

20. A computer-readable medium comprising instructions that, in response to execution by at least one processing circuitry, causes an apparatus to perform the method as claimed in any of claims 11 to 18.

Images