Signaling of scene description for multimedia conferencing

Abstract

To provide systems and methods for rendering an immersive three-dimensional group session.SOLUTION: A scene graph is signaled through a session description protocol (SDP) during a session initiation protocol (SIP) session setup. The scene graph includes respective graphical output nodes assigned to be controlled by a plurality of participant computing devices in an immersive three-dimensional group session.SELECTED DRAWING: Figure 4

Description

Claim of Priority

[0001] Related Applications

[0001] This application claims the benefit of priority of U.S. Provisional Application No. 62 / 990,895, filed on March 17, 2020, entitled "Signaling of Scene Description For Multimedia Conferencing", the entire content of which is incorporated herein by reference for all purposes.

Technical Field

[0002] Relates to systems and methods for providing an immersive three-dimensional group session.

Background Art

[0003]

[0002] Long Term Evolution (LTE (registered trademark)), 5th Generation (5G) New Radio (NR), and other recently developed communication technologies enable wireless devices to communicate information at data rates that are orders of magnitude greater than those available just a few years ago (e.g., regarding gigabits per second).

[0004]

[0003] Today's communication networks are also more secure, resistant to multipath fading, enable lower network traffic latency, and provide better communication efficiency (e.g., regarding bits per second per unit of bandwidth used). These and other recent improvements have facilitated the emergence of technologies that rely on the Internet of Things (IoT), large-scale machine-to-machine (M2M) communication systems, autonomous vehicles, and other technologies that rely on consistent and secure communication.

[0005]

[0004] In recent years, augmented reality software applications that combine real-world images from a user's physical environment with computer-generated images or virtual objects (VOs) have grown in popularity and use. Augmented reality software applications can add graphics, sound, and / or haptic feedback to the natural world surrounding the application's user. Images, video streams, and information about people and / or objects can be presented to the user by being superimposed onto the visual world as an augmented scene on a wearable electronic display or a head-mounted device (e.g., smart glasses, augmented reality glasses, etc.).

Summary of the Invention

[0006]

[0005] Various aspects include systems and methods for providing an immersive three - dimensional group session. Various embodiments potentially include methods and devices for signaling a description of a scene having media components from different parties. In various aspects, a scene graph can be signaled through the Session Description Protocol (SDP) during Session Initiation Protocol (SIP) session setup. In various aspects, a scene graph can include respective graphical output nodes assigned to be controlled by each of a plurality of participant computing devices in an immersive three - dimensional group session. Various aspects can be implemented by a processor of a wireless device that is one of a plurality of participant computing devices operating in an immersive three - dimensional group session. Various aspects include receiving a scene graph for an immersive three - dimensional group session, where the scene graph includes at least an own graphical output node assigned to be controlled by the wireless device and respective other graphical output nodes assigned to be controlled by each of the other ones of the plurality of participant computing devices, and controlling components of the own graphical output node with respect to the three - dimensional space of the immersive three - dimensional group session, sending components of the own graphical output node in a first media stream to the other ones of the plurality of participant computing devices, receiving components of the other graphical output nodes in a media stream from each of the other ones of the plurality of participant computing devices, and rendering an immersive three - dimensional group session on a display of the wireless device based at least in part on components of the own graphical output node and components of the other graphical output nodes.

[0007]

[0006] Some embodiments may further include receiving a scene graph update that includes instructions for a new participant computing device for an immersive 3D group session and instructions for a new graphical output node assigned to be controlled by the new participant computing device, receiving components of the new graphical output node in a second media stream from the new participant computing device, and rendering an immersive 3D group session on a display of a wireless device based at least in part on components of the unique graphical output node, components of other graphical output nodes, and components of the new graphical output node.

[0008]

[0007] Some embodiments may further include receiving a session description protocol (SDP) for an immersive 3D group session that indicates an address of a data channel over which the scene graph is to be shared, where receiving the scene graph includes downloading the scene graph over the data channel.

[0009]

[0008] Some embodiments may further include sending an offer to send or receive a scene graph to other ones of a plurality of participant computing devices as part of a session initiation protocol (SIP) setup for an immersive 3D group session.

[0010]

[0009] Some embodiments may further include sending an offer to send or receive a scene graph to other ones of a plurality of participant computing devices, where the offer indicates a unique graphical output node.

[0011]

[0010] In some embodiments, the immersive 3D group session is a web real-time communication (WebRTC) session.

[0012]

[0011] In some embodiments, controlling components of a unique graphical output node with respect to the three-dimensional space of an immersive three-dimensional group session may include controlling the components of the unique graphical output node based at least in part on a determined position of a wireless device with respect to the three-dimensional space of the immersive three-dimensional group session.

[0013]

[0012] In some embodiments, controlling components of a unique graphical output node based at least in part on a determined position of a wireless device with respect to the three-dimensional space of an immersive three-dimensional group session may include controlling the components of the unique graphical output node based at least in part on a determined position of the wireless device with respect to the three-dimensional space of the immersive three-dimensional group session and a determined orientation of the wireless device with respect to the three-dimensional space of the immersive three-dimensional group session.

[0014]

[0013] Further embodiments may include a wireless device having a processor configured to perform one or more operations of any of the methods summarized above. Further embodiments may include a non-transitory processor-readable storage medium storing processor-executable instructions configured to cause a processor of a wireless device to perform an operation of any of the methods summarized above. Further embodiments may include a wireless device having means for performing any of the functions of the methods summarized above. Further embodiments may include a system-on-chip for use in a wireless device including a processor configured to perform one or more operations of any of the methods summarized above. Further embodiments may include a system in a package including two system-on-chips for use in a wireless device including a processor configured to perform one or more operations of any of the methods summarized above.

[0015] [

[0014] ]The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the claims and, together with the general description given above and the detailed description given below, serve to explain the features of the claims. [

[0015] ]Brief Description of the Drawings

[0016]

Figure 1A

[0015] ]A system block diagram showing an exemplary communication system suitable for implementing various embodiments.

Figure 1B

[0016] ]A diagram showing a head-mounted device (e.g., augmented reality glasses) that can implement various embodiments.

Figure 2

[0017] A component block diagram showing an exemplary computing and wireless modem system suitable for implementing various embodiments.

Figure 3

[0018] A diagram showing an example of a software architecture including a wireless protocol stack for a user plane and a control plane in wireless communication according to various embodiments.

Figure 4

[0019] A process flow diagram showing a method for supporting an immersive experience in a teleconference or telepresence session according to various embodiments.

Figure 5

[0020] A process flow diagram showing a method for supporting an immersive experience in a teleconference or telepresence session according to various embodiments.

Figure 6

[0021] A diagram showing the configuration of a scene graph document in glTF2.0.

Figure 7

[0022] A diagram showing the structure of a scene graph.

Figure 8

[0023] A call flow diagram showing operations for supporting an immersive experience in a teleconference or telepresence session according to various embodiments.

Figure 9

[0024] A process flow diagram showing a method for providing an immersive three-dimensional group session according to various embodiments.

Figure 10

[0025] A process flow diagram showing a method for providing an immersive three-dimensional group session according to various embodiments.

Figure 11

[0026] An exemplary server component block diagram suitable for implementing various embodiments.

Figure 12

[0027] A wireless device component block diagram suitable for implementing various embodiments.

DETAILED DESCRIPTION OF THE INVENTION

[0017]

[0028] Various embodiments are described in detail with reference to the accompanying drawings. Whenever possible, the same reference numbers are used throughout the drawings to refer to the same or similar parts. References made to specific examples and implementations are for illustrative purposes and are not intended to limit the scope of the claims.

[0018]

[0029] Various embodiments may enable an immersive 3D group session for a plurality of participant computing devices, where the scene graph includes respective graphical output nodes assigned to be controlled by each of the plurality of participant computing devices in the immersive 3D group session. Various embodiments may enable the plurality of participant computing devices to share media streams of components of their respective assigned graphical output nodes with each other in the immersive 3D group session. By assigning control of its own respective graphical output node to each participant computing device in the immersive 3D group session and sharing media streams of components of the graphical output nodes among the participant computing devices, various embodiments may support rendering an immersive 3D group session with a shared 3D space where each participant computing device controls its own respective 3D object in the shared 3D space.

[0019]

[0030] The term "wireless device" is used herein to refer to any one or all of a wireless router device, a wireless appliance, a cellular phone, a smartphone, a portable computing device, a personal or mobile multimedia player, a laptop computer, a tablet computer, a smartbook, an ultrabook, a palmtop computer, a wireless email receiver, a multimedia Internet-enabled cellular phone, a medical device and instrument, a biosensor / device, a smartwatch, a smart closing, smart glasses, a smart list band, wearable devices including smart jewelry (e.g., smart rings, smart bracelets, etc.), entertainment devices (e.g., wireless gaming controllers, music and video players, satellite radios, etc.), Internet of Things (IoT) devices with wireless network connectivity including smart meters / sensors, industrial manufacturing equipment, large and small mechanical appliances for home or enterprise use, wireless communication elements within autonomous and semi-autonomous vehicles, wireless devices fixed or incorporated into various mobile platforms, global positioning system devices, as well as similar electronic devices including memory, wireless communication components, and programmable processors.

[0020]

[0031] Various embodiments can be implemented in a device capable of transmitting and receiving RF signals according to any of the wireless communication standards including IEEE802.15.4 protocols (e.g., Thread, ZigBee®, and Z-Wave), 6LoWPAN, Bluetooth® Low Energy (BLE), LTE Machine Type Communication (LTE MTC), NarrowBand LTE (NB-LTE), Cellular IoT (CIoT), NarrowBand IoT (NB-IoT), BT Smart, Wi-Fi® (e.g., Wi-Fi NAN, etc.), LTE-U, LTE-Direct, MuLTEfire, and relatively extended range wide area physical layer interfaces (PHYs) such as Random Phase Multiple Access (RPMA), Ultra NarrowBand (UNB), Low Power Long Range (LoRa), Low Power Long Range Wide Area Network (LoRaWAN), Weightless, or systems using 3G, 4G or 5G, Cellular V2X or further implementations, technologies, etc., any of the Institute of Electrical and Electronics Engineers (IEEE) 16.11 standards, or any of the IEEE802.11 standards, Bluetooth standards (e.g., Bluetooth4, Bluetooth5, etc.), Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Global System for Mobile Communications (GSM®), GSM / General Packet Radio Service (GPRS), Enhanced Data GSM Environment (EDGE), Terrestrial Trunked Radio (TETRA), Wideband CDMA (W-CDMA®), Evolution-Data Optimized (EV-DO), 1xEV-DO, EV-DO Rev A, EV-DO Rev B, High Speed Packet Access (HSPA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Evolved High Speed Packet Access (HSPA+), Long Term Evolution (LTE), AMPS, or any other known signals.

[0021]

[0032] The term "system on a chip" (SOC) is used herein to refer to a single integrated circuit (IC) chip that includes multiple resources and / or processors incorporated on a single substrate. A single SOC may include circuitry for digital, analog, mixed signal, and radio frequency functions. A single SOC may also include any number of general-purpose and / or dedicated processors (such as digital signal processors, modem processors, video processors, etc.), memory blocks (such as ROM, RAM, flash, etc.), and resources (such as timers, voltage regulators, oscillators, etc.). An SOC may also include software for controlling the integrated resources and processors, as well as for controlling peripheral devices.

[0022]

[0033] The term "system in package" (SIP) may be used herein to refer to a single module or package that includes two or more IC chips, substrates, or multiple resources, computing units, cores, and / or processors on an SOC. For example, an SIP may include a single substrate on which multiple IC chips or semiconductor dies are stacked in a vertical configuration. Similarly, an SIP may include one or more multi-chip modules (MCMs) on which multiple ICs or semiconductor dies are packaged on an integrated substrate. An SIP may also include multiple independent SOCs that are coupled to each other via high-speed communication circuits and packaged in close proximity, such as on a single motherboard or within a single wireless device. The proximity of the SOCs facilitates high-speed communication and the sharing of memory and resources.

[0023]

[0034] In this specification, various embodiments are described using the term "server" to refer to any computing device capable of functioning as a server, such as a master exchange server, a web server, a mail server, a document server, a content server, or any other type of server. The server can be a dedicated computing device or a computing device that includes a server module (e.g., an application that can operate the computing device as a server). The server module (e.g., a server application) can be a full-functional server module or a simplified or secondary server module (e.g., a simplified server application or a secondary server application) configured to provide synchronization services between a dynamic database on a receiver device. A simplified server or a secondary server is a miniaturized version of the server-type functionality that can be implemented on a receiver device, thereby enabling it to function as an Internet server (e.g., a corporate email server) only to the extent necessary to provide the functionality described herein.

[0024]

[0035] The term "head-mounted device" and its acronym (HMD) are used herein to refer to an electronic display system that is wearable and presents at least partially computer-generated imagery to a user. The HMD can present either computer-generated imagery only, or a combination of computer-generated imagery and real-world imagery from the user's physical environment (i.e., what the user would see without glasses). The HMD can enable the user to observe the generated imagery in the context of a real-world scene. Non-limiting examples of head-mounted devices can include, or be included in, helmets, glasses, virtual reality glasses, augmented reality glasses, electronic goggles, and other similar technologies / devices. A head-mounted device can include various hardware elements such as a processor, memory, display, one or more cameras (e.g., a world-view camera, a gaze-view camera, etc.), and a wireless interface for connecting to the Internet, a network, or another computing device. In some embodiments, the head-mounted device processor can be configured to implement or execute an augmented reality software application.

[0025]

[0036] In some embodiments, the head-mounted device is an accessory for a wireless device (e.g., a desktop, laptop, smartphone, tablet computer, etc.) and / or can receive information from the wireless device, and all or part of the processing is performed on the processor of the wireless device. Thus, in various embodiments, the head-mounted device can be configured to perform all processing locally on a processor in the head-mounted device, offload all of the main processing to a processor in another computing device (e.g., a laptop present in the same space as the head-mounted device), or split the main processing operations between a processor in the head-mounted device and a processor in another computing device. In some embodiments, the processor in another computing device can be a server in the "cloud" with which the processor in the head-mounted device or the associated wireless device communicates via a network connection (e.g., a cellular network connection to the Internet).

[0026]

[0037] Telepresence services are becoming more capable, enabling the composition of several objects into a single immersive environment where conference participants can navigate and interact more freely. The current Session Description Protocol (SDP) is limited in its support for describing rich composition and does not provide tools for supporting immersive telepresence environments. One particular type of telepresence or teleconference implementation is the immersive three-dimensional (3D) group session. In an immersive 3D group session, each participant computing device in the session can render a 3D graphical display of the session on its respective display so that a virtual reality (VR) view of the 3D space of the immersive 3D group session is presented to each participant user in the immersive 3D group session. Three-dimensional objects such as avatars, characters, etc., representing participants in the immersive 3D group session can be observed by each participant and appear to move within the 3D space of the immersive 3D group session.

[0027]

[0038] Support for Immersive Teleconferencing and Telepresence for Remote Terminals (ITT4RT) is a standard developed to facilitate immersive multi-computing device virtual reality (VR) video conferencing such as immersive 3D group sessions. ITT4RT use cases include, for example, the composition of VR video captured from a conference room and other content such as two-dimensional (2D) video slides. Work item descriptions related to ITT4RT indicate that ITT4RT is working towards enabling scenarios with two-way audio and one-way immersive video, for example, a single remote user wearing an HMD and participating in a conference sending audio and optionally 2D video (e.g., of the user's own presentation, screen sharing, and / or capture).

[0028]

[0039] The complexity of VR scenes presents challenges for traditional Session Description Protocol (SDP) signaling and can become extremely difficult to handle very quickly, such as when many remote users share their own content and synthesize it into the scene. SDP is simply not designed to carry scene description information. Various embodiments provide a solution to such problems by providing a scene description-based embodiment solution for supporting composition and overlay. Various embodiments potentially provide methods and devices for signaling the description of a scene having media components from different parties. In various embodiments, the scene description may be signaled through SDP during Session Initiation Protocol (SIP) session setup. In various embodiments, the scene description may be linked to other media streams during the session, for example, as textures for overlays in immersive conference scenes and the like, for the purpose of leveraging them.

[0029]

[0040] A scene graph is a directed acyclic graph, usually simply a plain tree structure, which represents an object-based hierarchy of the geometry of a scene. The leaf nodes of the graph represent geometric primitives such as polygons. Each node in the graph holds pointers to its children. The child nodes can be, in particular, groups of other nodes, geometry elements, transformation matrices, etc. Spatial transformations are attached to the nodes of the graph and are represented by transformation matrices. This structure of the scene graph has the advantage of reduced processing complexity, such as when traversing the graph for rendering. An exemplary operation simplified by the graph representation is the culling operation where branches of the graph are dropped if the space of a parent node is not visible or is considered not relevant to the rendering of the current view frustum (referred to as the level of detail culling). A scene graph can include various types of nodes, such as visual output nodes, audio source nodes, graphical output nodes, shared content nodes, etc. As a specific example, a graphical output node can define a three-dimensional object to be output in the three-dimensional space defined by the scene graph.

[0030]

[0041] The Graphics Library (GL) Transmission Format (TF) (glTF) 2.0 (glTF2.0) is a new standard developed by Khronos to enable physically based rendering. glTF2.0 provides a compact and low-level representation of a scene graph. glTF2.0 provides a flat hierarchy of the scene graph representation to simplify processing. The glTF2.0 scene graph is represented in JavaScript (R) Object Notation (JSON) to ease integration in web environments. The glTF2.0 specification is designed to eliminate redundancy in the representation and provide efficient indexing of different objects in the scene graph. The Moving Picture Experts Group (MPEG) is working on extensions to glTF2.0 to add support for real-time media, scene updates, and other features.

[0031]

[0042] In various embodiments, a scene graph can enable the composition of a scene (also referred to as a space, such as a three-dimensional space) for an immersive presentation, such as an immersive three-dimensional group session. In some embodiments, the composition can be performed at a call server, such as a multimedia resource function (MRF), a multipoint communication unit (MCU), a telepresence application server, etc. Alternatively, in some embodiments, a designated computing device participating in the conference can be responsible for creating an initial scene graph and sharing the scene graph with all other parties in the call (such as an immersive three-dimensional group session). This computing device can be one that creates major VR content, such as a computing device in a conference room using VR capture. In some embodiments, each computing device participating in the conference (such as an immersive three-dimensional group session) can provide one or more nodes to the scene graph. In some embodiments, each node can identify or be assigned its associated transformation (such as in the form of a matrix, or individual translation and rotation operations) to properly position the node in the scene (or space), such as in a three-dimensional space.

[0032]

[0043] In some embodiments, each computing device participating in a conference call (e.g., an immersive 3D group session) may make an offer to send and receive a scene graph through session-level attributes. In some embodiments, the offer may indicate one or more unique graphical output nodes owned by the computing device sending the offer. As an example, each computing device participating in a conference call (e.g., an immersive 3D group session) may make an offer to send and receive a scene graph through the following session-level attributes given in augmented Backus-Naur Form (ABNF) syntax. Session-Description=“a=scene-description:” SP mime-type [SP uri] [SP sent-nodes] CRLF mime-type=“mime-type:” byte-string sent-nodes=“nodes-owned=1*(byte-string “;”) uri=“websocket-uri:” URI.

[0044] In some embodiments, a Uniform Resource Indicator (URI) parameter may be a WebSocket URI for a data channel through which the scene graph will be shared and updated. Alternatively, an application media session may be used with a protocol identifier such as the following protocol identifiers TCP / WSS / SD (Transmission Control Protocol / Websocket Secure / Session Description).

[0033]

[0045] In some embodiments, the WebSocket URI may be provided according to the syntax and offer / answer negotiation defined in Internet Engineering Task Force (IETF) Request for Comments (RFC) 8124. An exemplary schema for such WebSocket URI provisioning may be as follows. m=application 50000 TCP / WSS / SD *a=setup:passive a=connection:newa=websocket-uri:wss: / / mrf.operator.com / call / 21323asd23a=mime-type:model / gltf+jsona=nodes-owned:node{12,node13,node14

[0046] In various embodiments, the scene graph refers to media streams from a conference session that are used as components of nodes in a scene (e.g., a three-dimensional space). An example is the video stream of a conference participant that will be displayed within a rectangular region in a 3D scene (also called a three-dimensional (3D) space). For example, the following URI format may be used.

[0034] url = “rtp: / / ”fqdn_or_ip“ / ”call_id“ / ”ssrc“ / ”mid

[0047] In this URI format, "fqdn_or_ip" represents the domain name or Internet Protocol (IP) address of the MRF or SIP proxy that manages the call (e.g., an immersive 3D group session). If the MRF or SIP proxy does not manage the call (e.g., an immersive 3D group session), "fqdn_or_ip" may represent the domain name or IP address of the SIP address of the host of the call (e.g., an immersive 3D group session). "call_id" provides a unique identifier for the current call or conference (e.g., the current immersive 3D group session). "ssrc" represents the synchronization source of the owner / sending participant of the media stream. Finally, "mid" represents the media session identifier provided in the SDP. Other forms of addressing may be defined, for example, as a Uniform Resource Name (URN).

[0035]

[0048] In some embodiments, when using Web Real-Time Communication (WebRTC), the session setup protocol may be delegated to the application. Some implementations rely on SIP over WebSocket for this purpose. However, other protocols may be used to set up and describe the call (e.g., an immersive 3D group session).

[0036]

[0049] Some embodiments may use a scene graph as an entry point into a conference call (e.g., an immersive 3D group session). In such embodiments, all participants share a scene graph document that sets up a 3D scene (or 3D space) at the start of the call (e.g., an immersive 3D group session). The scene graph defines graph nodes for each participant and identifies the components for which each participant needs to provide a media stream. This can be done by a central call server, such as an MRF, to which all conference participants connect. The MRF can be configured to update the scene graph during the call (e.g., an immersive 3D group session) to add new nodes for newly joining call participants or participants who have left the call, or to remove nodes, etc.

[0037]

[0050] In some embodiments, a participant computing device may use a link to a web page provided by a call server to join a WebRTC conference. The call server may provide the participant computing device with the web page, along with a scene graph file that sets up an initial / default configuration in the 3D space of the call participants and materials (e.g., each participant will be assigned a visual node, an audio source node, and potentially also nodes for graphics and other shared content). Each participant computing device may add or modify the nodes it owns in the scene graph. Media streams that provide components for the nodes in the scene graph may be streamed using WebRTC. These streams may be exchanged directly or through a server such as a media proxy server.

[0038]

[0051] In some embodiments, the computing devices participating in the ITT4RT session may establish direct peer-to-peer WebSocket channels with each other, or the connections may be offered to all parties by the MRF. In such embodiments, the WebSocket channels may use a text frame format. In a scene (e.g., in a three-dimensional space), the node names may be unique and may be declared in the SDP to ensure that there are no naming conflicts at the nodes provided by different computing devices during a call. In such embodiments, the nodes in the scene graph may refer to external media streams, such as other media streams declared in the SDP. In such embodiments, the receiver may mask nodes from several computing devices in the rendering process, such as based on user input.

[0039]

[0052] In some embodiments, the MRF may, by default, be the owner of the master scene graph, which is a computing device that sets the coordinate system and to which all other nodes are synthesized. In some embodiments, the MRF may also be a computing device that defines the primary camera in the scene (or space). In some embodiments, in the absence of a centralized MRF, the computing device in a call may select one computing device to provide the primary scene graph, for example, by selecting a computing device that provides VR content or the organizer of the call. In various embodiments, an overlay may be a 2D or 3D object placed within the scene (or space). In some embodiments, the geometry of the overlay and its texture may be defined by the node corresponding to that overlay object. A simple example is a set of slides that are played within a rectangular area shown within a VR scene (or VR space). In this example, the geometry may be rectangular and the texture may be derived from a video media stream. The rectangle may be placed within the scene (or space). In a viewport-dependent overlay, the position of the rectangle may be locked to the camera direction.

[0040]

[0053] Figure 1A shows an example of a communication system 100 suitable for implementing various embodiments. The communication system 100 may be a 5G NR network or any other suitable network such as an LTE network.

[0041]

[0054] The communication system 100 may include a heterogeneous network architecture including a core network 140 and various wireless devices (shown as wireless devices 120a - 120e in FIG. 1) (also referred to as user equipment (UE) computing devices). The communication system 100 may also include several base stations (shown as BS110a, BS110b, BS110c, and BS110d) and other network entities. A base station is an entity that communicates with wireless devices (wireless devices or UE computing devices), and may also be referred to as Node B, Node B, Long Term Evolution evolved Node B (eNB), access point (AP), radio head, transmission and reception point (TRP), new radio base station (NR BS), 5G Node B (NB), next generation Node B (gNB), etc. Each base station may provide communication coverage for a specific geographical area. In 3GPP (registered trademark), the term "cell" can refer to the coverage area of a base station, the coverage area of a base station subsystem serving this coverage area, or a combination thereof, depending on the context in which the term is used.

[0042]

[0055] Base stations 110a to 110d may provide communication coverage for macrocells, picocells, femtocells, another type of cell, or a combination thereof. A macrocell may cover a relatively large geographical area (e.g., several kilometers in radius) and may enable unrestricted access by wireless devices subscribed to the service. A picocell may cover a relatively small geographical area and may enable unrestricted access by wireless devices subscribed to the service. A femtocell may cover a relatively small geographical area (e.g., a home) and may enable restricted access by wireless devices associated with the femtocell (e.g., wireless devices in a closed subscriber group (CSG)). The base station for a macrocell may sometimes be referred to as a macro BS. The base station for a picocell may sometimes be referred to as a pico BS. The base station for a femtocell may sometimes be referred to as a femto BS or a home BS. In the example shown in FIG. 1A, base station 110a may be a macro BS for macrocell 102a, base station 110b may be a pico BS for picocell 102b, and base station 110c may be a femto BS for femtocell 102c. Base stations 110a to 110d may support one or more (e.g., three) cells. The terms “eNB”, “base station”, “NR BS”, “gNB”, “TRP”, “AP”, “Node B”, “5G NB”, and “cell” may be used interchangeably herein.

[0043]

[0056] In some examples, the cell may not be fixed, and the geographical area of the cell may move according to the location of the mobile base station. In some examples, base stations 110a to 110d may be interconnected with each other and with one or more other base stations or network nodes (not shown) in communication system 100 through various types of backhaul interfaces, such as direct physical connections, virtual networks, or combinations thereof, using any suitable transport network.

[0044]

[0057] Base stations 110a - 110d can communicate with the core network 140 via a wired or wireless communication link 126. Wireless devices 120a - 120e (UE computing devices) can communicate with base stations 110a - 110d via a wireless communication link 122.

[0045]

[0058] The wired communication link 126 can use one or more wired communication protocols such as Ethernet (registered trademark), Point - to - Point Protocol, High - Level Data Link Control (HDLC), Advanced Data Communication Control Protocol (ADCCP), and Transmission Control Protocol / Internet Protocol (TCP / IP), and can use various wired networks (e.g., Ethernet, TV cable, telephone, optical fiber, and other forms of physical network connections).

[0046]

[0059] The communication system 100 can also include a relay station (e.g., relay BS110d). A relay station is an entity that can receive the transmission of data from an upstream station (e.g., a base station or a wireless device) and send the transmission of data to a downstream station (e.g., a wireless device or a base station). A relay station can also be a wireless device that can relay transmissions for other wireless devices. In the example shown in FIG. 1, the relay station 110d can communicate with the base station 110a and the wireless device 120d to facilitate communication between the macro base station 110a and the wireless device 120d. Relay stations may also be called relay base stations, relay stations, relays, etc.

[0047]

[0060] The communication system 100 can be a heterogeneous network including different types of base stations, such as macro base stations, pico base stations, femto base stations, relay base stations, etc. These different types of base stations can have different transmission power levels, different coverage areas, and different impacts on interference in the communication system 100. For example, a macro base station can have a high transmission power level (e.g., 5 - 40 watts), while pico base stations, femto base stations, and relay base stations can have lower transmission power levels (e.g., 0.1 - 2 watts).

[0048]

[0061] The network controller 130 can be coupled to a set of base stations and can perform the coordination and control of these base stations. The network controller 130 can communicate with the base stations via a backhaul. The base stations can also communicate with each other directly or indirectly, for example, via a wireless or wireline backhaul.

[0049]

[0062] Wireless devices (UE computing devices) 120a, 120b, 120c can be distributed throughout the communication system 100, and each wireless device can be fixed or mobile. Wireless devices may also be referred to as access terminals, UEs, terminals, mobile stations, subscriber units, stations, etc.

[0050]

[0063] Macro base station 110a can communicate with communication network 140 via a wired or wireless communication link 126. Wireless devices 120a, 120b, 120c can communicate with base stations 110a - 110d via wireless communication link 122. Core network 140 can be connected to other devices such as call server 150 (e.g., multimedia resource function (MRF), multipoint communication unit (MCU), telepresence application server, etc.). In this way, via the connection to core network 140, call server 150 can make available telepresence services, such as immersive teleconference and telepresence for remote terminals (ITT4RT) services, to wireless devices 120a, 120b, 120c, 120d (e.g., from core network 140 via link 126 and from base stations 110a - 110d via link 122). Although shown outside core network 140, call server 150 can be a part of core network 140 itself.

[0051]

[0064] Wireless communication links 122, 124 may include a plurality of carrier signals, frequencies, or frequency bands, each of which may include a plurality of logical channels. Wireless communication links 122 and 124 may utilize one or more radio access technologies (RATs). Examples of RATs that may be used in wireless wide area network (WWAN) wireless communication links 122, 124 within the distributed communication system 100 include 3GPP LTE, 3G, 4G, 5G (e.g., NR), GSM, code division multiple access (CDMA), wideband code division multiple access (WCDMA (registered trademark)), worldwide interoperability for microwave access (WiMAX (registered trademark)), time division multiple access (TDMA), and other mobile telephone communication technology cellular RATs. Examples of RATs that may be used in wireless local area network (WLAN) wireless communication links 122, 124 within the local communication system 100 include medium-range wireless protocols such as Wi-Fi, LTE-U, LTE-Direct, LAA, MuLTEfire, as well as relatively short-range RATs such as ZigBee, Bluetooth, and Bluetooth Low Energy (LE).

[0052]

[0065] Some wireless networks (e.g., LTE) utilize orthogonal frequency division multiplexing (OFDM) on the downlink and single carrier frequency division multiplexing (SC-FDM) on the uplink. OFDM and SC-FDM divide the system bandwidth into a plurality (K) of orthogonal subcarriers, also commonly referred to as tones, bins, etc. Each subcarrier can be modulated with data. Generally, the modulation symbols are sent in the frequency domain for OFDM and in the time domain for SC-FDM. The spacing between adjacent subcarriers can be fixed, and the total number (K) of subcarriers can depend on the system bandwidth. For example, the subcarrier spacing can be 15 kHz, and the minimum resource allocation (referred to as a "resource block") can be 12 subcarriers (or 180 kHz). Thus, the nominal fast Fourier transform (FFT) size can be equal to 128, 256, 512, 1024, or 2048 for system bandwidths of 1.25, 2.5, 5, 10, or 20 megahertz (MHz), respectively. The system bandwidth can also be divided into sub-bands. For example, a sub-band can cover 1.08 MHz (i.e., 6 resource blocks), and there can be 1, 2, 4, 8, or 16 sub-bands for system bandwidths of 1.25, 2.5, 5, 10, or 20 MHz, respectively.

[0053]

[0066] The description of some embodiments may use terms and examples related to LTE technology, but the various embodiments may be applicable to other wireless communication systems such as New Radio (NR) or 5G networks. NR may utilize OFDM with cyclic prefix (CP) on both the uplink (UL) and downlink (DL), and may support half-duplex operation using time division duplexing (TDD). A single component carrier bandwidth of 100 MHz may be supported. The NR resource block may span 12 subcarriers with a subcarrier bandwidth of 75 kHz over a duration of 0.1 ms. Each radio frame may consist of 50 subframes with a length of 10 ms. Thus, each subframe may have a length of 0.2 ms. Each subframe may indicate the link direction for data transmission (i.e., DL or UL), and the link direction for each subframe may be dynamically switched. Each subframe may include DL / UL data as well as DL / UL control data. Beamforming may be supported and the beam direction may be dynamically configured. Multiple-input multiple-output (MIMO) transmission using precoding may also be supported. The MIMO configuration in the DL may support up to 8 transmit antennas using multi-layer DL transmission with up to 8 streams and up to 2 streams per wireless device. Multi-layer transmission using up to 2 streams per wireless device may be supported. Aggregation of multiple cells may be supported using up to 8 serving cells. Alternatively, NR may support different air interfaces other than the OFDM-based air interface.

[0054]

[0067] Some wireless devices may be considered machine type communication (MTC) wireless devices or enhanced or extended machine type communication (eMTC) wireless devices. MTC wireless devices and eMTC wireless devices can communicate with, for example, a base station, another device (e.g., a remote device), or some other entity, including robots, drones, remote devices, sensors, meters, monitors, location tags, etc. A wireless node can provide connectivity for or to a network (e.g., a wide area network such as the Internet or a cellular network) via, for example, a wired or wireless communication link. Some wireless devices may be considered Internet of Things (IoT) devices or may be implemented as narrowband Internet of Things (NB-IoT) devices. Wireless devices 120a - e may be included within a housing that stores components of the wireless device, such as processor components, memory components, similar components, or combinations thereof.

[0055]

[0068] Generally, any number of communication systems and any number of wireless networks may be deployed within a given geographical area. Each communication system and wireless network may support a specific radio access technology (RAT) and may operate on one or more frequencies. RATs may also be referred to as wireless technologies, air interfaces, etc. Frequencies may also be referred to as carriers, frequency channels, etc. Each frequency may support a single RAT within a given geographical area to avoid interference between communication systems of different RATs. In some cases, an NR or 5G RAT network may be deployed.

[0056]

[0069] In some implementations, two or more wireless devices 120a - e (such as shown as wireless devices 120a and 120e) can communicate directly using one or more sidelink channels 124 (for example, without using base stations 110a - 110d as a medium to communicate with each other). For example, wireless devices 120a - e can communicate using peer - to - peer (P2P) communication, device - to - device (D2D) communication, vehicle - to - vehicle (V2V) protocols, vehicle - to - infrastructure (V2I) protocols, or vehicle - to - everything (V2X) protocols (which may include similar protocols), mesh networks, or similar networks, or combinations thereof. In this case, wireless devices 120a - e can perform scheduling operations, resource selection operations, and other operations described elsewhere in this specification as being performed by base station 110a.

[0057]

[0070] Figure 1B shows a head-mounted device 172 that can be configured according to various embodiments. With respect to FIGS. 1A and 1B, in the example shown in FIG. 1A, the head-mounted device 172 can be a particular implementation of a user equipment computing device (e.g., UE120c, 120d, 120e). The head-mounted device 172 includes a frame 152, two optical lenses 154, an outward-facing world view image sensor / camera 158, an inward-facing line-of-sight view sensor / camera 160, a sensor array 162, a memory 164, and a processor 156 communicatively coupled to a communication circuit 166. In various embodiments, the communication circuit 166 can support one or more RATs to support communication between various devices as described in system 100 with respect to FIG. 1A. In some embodiments, the head-mounted device 172 can include a capacitance touch sensing circuit along the arm 180 of the frame or within the nose bridge 182 of the head-mounted device 172. In some embodiments, the head-mounted device 172 can also include sensors for monitoring physical states (e.g., location, movement, acceleration, orientation, altitude, etc.). The sensors can include any or all of a gyroscope, an accelerometer, a magnetometer, a magnetic compass, an altimeter, an odometer, and a pressure sensor. The sensors can also include various biosensors (e.g., a heart rate monitor, a body temperature sensor, a carbon sensor, an oxygen sensor, etc.) for collecting information related to the environment and / or user state. The sensors can also be external to the head-mounted device 172 and can be paired or grouped with the head-mounted device 172 via a wired or wireless connection (e.g., Bluetooth, etc.).

[0058]

[0071] In some embodiments, the processor 156 may also be communicatively coupled to an image rendering device 168 (e.g., an image projector) that may be embedded within the arm portion 180 of the frame 152 and configured to project an image onto the optical lens 154. In some embodiments, the image rendering device 168 may include a light emitting diode (LED) module, a light tunnel, a homogenizing lens, an optical display, a fold mirror, or other components well-known projectors or head-mounted displays. In some embodiments (e.g., embodiments where the image rendering device 168 is not included or not used), the optical lens 154 may be or include a see-through or partially see-through electronic display. In some embodiments, the optical lens 154 includes image generating elements, such as a see-through organic light emitting diode (OLED) display element or a liquid crystal on silicon (LCOS) display element. In some embodiments, the optical lens 154 may include separate left and right eye display elements. In some embodiments, the optical lens 154 may include or operate as a light guide for delivering light from the display element to the wearer's eye.

[0059]

[0072] The outward or world view image sensor / camera 158 may be configured to capture a real-world image from the user's physical environment and send the corresponding image data to the processor 156. The processor 156 may combine the real-world image with computer-generated images or virtual objects (VOs) to generate an augmented scene (or space) and render the augmented scene (or space) on the electronic display of the head-mounted device 172 or on the optical lens 154.

[0060]

[0073] The inward or line of sight view sensor / camera 160 may be configured to collect image data from the user's eye or the facial structure around the user's eye.

[0061]

[0074] Various embodiments may be implemented on several single-processor and multi-processor computer systems, including a system-on-chip (SOC) or a system-in-package (SIP). FIG. 2 shows an exemplary computing system or SIP200 architecture that may be used in a wireless device (UE computing device) implementing various embodiments.

[0062]

[0075] Referring to FIGS. 1A, 1B, and 2, the illustrated exemplary SIP200 includes two SOCs 202, 204, a clock 206, a voltage regulator 208, and one or more wireless transceivers 266 configured to wirelessly communicate via an antenna (not shown) with a base station 110a and / or other network wireless devices such as other wireless devices 120a - e. In some embodiments, the first SOC 202 operates as a central processing unit (CPU) of a wireless device that executes instructions of a software application program by performing arithmetic, logical, control, and input / output (I / O) operations specified by the instructions. In some embodiments, the second SOC 204 may operate as a dedicated processing unit. For example, the second SOC 204 may operate as a dedicated 5G processing unit responsible for managing high volume, high speed (e.g., 5 Gbps, etc.), and / or ultra-high frequency short wavelength (e.g., 28 GHz mmWave spectrum, etc.) communications. In some embodiments, the wireless transceiver 266 may be a wireless transceiver configured to support peer-to-peer (P2P) communication, device-to-device (D2D) communication, vehicle-to-everything (V2X) protocols (which may include vehicle-to-vehicle (V2V) protocols, vehicle-to-infrastructure (V2I) protocols, or similar protocols), Bluetooth communication, Wi-Fi communication, etc. In some embodiments, each of the wireless transceivers 266 may be connected to the first SOC 202 and / or the second SOC 204 may be connected to each of the one or more wireless transceivers 266 by various physical connections 267 such as a peripheral component interconnect express (PCIe) connection, a universal serial bus (USB) connection, a high-speed inter-chip (HSIC) connection, an Ethernet connection, etc. (also referred to as interconnects, buses, etc.).

[0063]

[0076] The first SOC 202 may include a digital signal processor (DSP) 210, a modem processor 212, a graphics processor 214, an application processor 216, one or more coprocessors 218 (e.g., vector coprocessors) connected to one or more of the processors, a memory 220, a custom circuit 222, system components and resources 224, an interconnect / bus module 226, one or more temperature sensors 230, a thermal management unit 232, and a thermal power envelope (TPE) component 234. The second SOC 204 may include a 5G modem processor 252, a power management unit 254, an interconnect / bus module 264, a plurality of mmWave transceivers 256, a memory 258, and various additional processors 260 such as an application processor, a packet processor, etc.

[0064]

[0077] Each of the processors 210, 212, 214, 216, 218, 252, 260 may include one or more cores, and each processor / core may perform operations independently of other processors / cores. For example, the first SOC 202 may include a processor that executes a first type of operating system (e.g., FreeBSD, LINUX®, OS X, etc.) and a processor that executes a second type of operating system (e.g., MICROSOFT WINDOWS® 10). Additionally, any or all of the processors 210, 212, 214, 216, 218, 252, 260 may be included as part of a processor cluster architecture (e.g., a synchronous processor cluster architecture, an asynchronous or heterogeneous processor cluster architecture, etc.).

[0065]

[0078] The first SOC 202 and the second SOC 204 may include various system components, resources, and custom circuits for managing sensor data, analog-to-digital conversion, wireless data transmission, and performing other special operations such as decrypting data packets and processing encoded audio and video signals for rendering in a web browser. For example, the system components and resources 224 of the first SOC 202 may include a power amplifier, a voltage regulator, an oscillator, a phase-locked loop, a peripheral bridge, a data controller, a memory controller, a system controller, an access port, a timer, and other similar components used to support a processor and software client running on a wireless device. The system components and resources 224 and / or the custom circuit 222 may also include circuits for interfacing with peripheral devices such as a camera, an electronic display, a wireless communication device, an external memory chip, etc.

[0066]

[0079] The first SOC 202 and the second SOC 204 may communicate via an interconnect / bus module 250. The various processors 210, 212, 214, 216, 218 may be interconnected via an interconnect / bus module 226 to one or more memory elements 220, system components and resources 224, custom circuit 222, and thermal management unit 232. Similarly, the processor 252 may be interconnected via an interconnect / bus module 264 to a power management unit 254, a mmWave transceiver 256, a memory 258, and various additional processors 260. The interconnect / bus modules 226, 250, 264 may include an array of reconfigurable logic gates and / or implement a bus architecture (e.g., CoreConnect, AMBA, etc.). The communication may be provided by a high-performance network-on-chip (NoC) or other advanced interconnect.

[0067]

[0080] The first SOC 202 and / or the second SOC 204 may further include an input / output module (not shown) for communicating with external resources of the SOC, such as a clock 206 and a voltage regulator 208. External resources of the SOC (e.g., clock 206, voltage regulator 208) may be shared by two or more of the internal SOC processors / cores.

[0068]

[0081] In addition to the exemplary SIP 200 described above, various embodiments may be implemented in a wide variety of computing systems that may include a single processor, multiple processors, a multi-core processor, or any combination thereof.

[0069]

[0082] FIG. 3 shows an example of a software architecture 300 that includes a radio protocol stack for the user plane and the control plane in wireless communication between a base station 350 (e.g., base station 110a) and a wireless device (UE computing device) 320 (e.g., wireless devices 120a-120e, 172, 200). Referring to FIGS. 1A-3, wireless device 320 may implement software architecture 300 to communicate with base station 350 of a communication system (e.g., 100). In various embodiments, layers in software architecture 300 may form logical connections with corresponding layers in the software of base station 350. Software architecture 300 may be distributed among one or more processors (e.g., processors 212, 214, 216, 218, 252, 260). Although shown with respect to one radio protocol stack, in a multi-SIM (Subscriber Identity Module) wireless device, software architecture 300 may include multiple protocol stacks, each of which may be associated with a different SIM (e.g., in a dual-SIM wireless communication device, two protocol stacks, each associated with one of the two SIMs). Although described below with respect to LTE communication layers, software architecture 300 may support any of various standards and protocols for wireless communication and / or may include additional protocol stacks that support any of various standards and protocols for wireless communication.

[0070]

[0083] The software architecture 300 may include a non-access stratum (NAS) 302 and an access stratum (AS) 304. The NAS 302 may include functions and protocols for packet filtering, security management, mobility control, session management, traffic and signaling between the (one or more) SIMs (e.g., (one or more) SIM 204) of the wireless device and its core network 140. The AS 304 may include functions and protocols for supporting communication between the (one or more) SIMs (e.g., (one or more) SIM 204) and an entity (e.g., a base station) of the supported access network. In particular, the AS 304 may include at least three layers (layer 1, layer 2, and layer 3), and each of these layers may include various sub-layers.

[0071]

[0084] In the user plane and the control plane, layer 1 (L1) of the AS 304 may be a physical layer (PHY) 306 that supervises functions enabling transmission and / or reception via an air interface. Examples of such physical layer 306 functions may include cyclic redundancy check (CRC) attachment, coding blocks, scrambling and descrambling, modulation and demodulation, signal measurement, MIMO, etc. The physical layer may include various logical channels, including a physical downlink control channel (PDCCH) and a physical downlink shared channel (PDSCH).

[0072]

[0085] In the user plane and the control plane, layer 2 (L2) of the AS 304 may be responsible for the link between the wireless device 320 and the base station 350 on the physical layer 306. In various embodiments, layer 2 may include a media access control (MAC) sub-layer 308, a radio link control (RLC) sub-layer 310, and a packet data convergence protocol (PDCP) 312 sub-layer, and each of these sub-layers forms a logical connection terminated at the base station 350.

[0073]

[0086] In the control plane, the layer 3 (L3) of AS304 may include a radio resource control (RRC) sub-layer 3. Although not shown, the software architecture 300 may include additional layer 3 sub-layers, as well as various upper layers above layer 3. In various embodiments, the RRC sub-layer 313 may provide functions including broadcasting system information, paging, and establishing and releasing an RRC signaling connection between the wireless device 320 and the base station 350.

[0074]

[0087] In various embodiments, the PDCP sub-layer 312 may provide uplink functions including multiplexing between different radio bearers and logical channels, adding sequence numbers, handover data processing, integrity protection, encryption, and header compression. On the downlink, the PDCP sub-layer 312 may provide functions including in-order delivery of data packets, duplicate data packet detection, integrity verification, decoding, and header restoration.

[0075]

[0088] On the uplink, the RLC sub-layer 310 may provide segmentation and concatenation of upper layer data packets, retransmission of lost data packets, and automatic repeat request (ARQ). On the other hand, on the downlink, the RLC sub-layer 310 functions may include reordering of data packets to compensate for out-of-order reception, reassembly of upper layer data packets, and ARQ.

[0076]

[0089] On the uplink, the MAC sub-layer 308 may provide functions including multiplexing between logical channels and transport channels, random access procedures, logical channel prioritization, and hybrid ARQ (HARQ) operations. On the downlink, the MAC layer functions may include channel mapping within the cell, demultiplexing, discontinuous reception (DRX), and HARQ operations.

[0077]

[0090] The software architecture 300 may provide a function for transmitting data through a physical medium. However, the software architecture 300 may further include at least one host layer 314 to provide a data transfer service to various applications in the wireless device 320. In some embodiments, the application-specific functions provided by at least one host layer 314 may provide an interface between the software architecture and the general-purpose processor 206.

[0078]

[0091] In other embodiments, the software architecture 300 may include one or more upper logical layers (e.g., transport, session, presentation, application, etc.) that provide host layer functions. For example, in some embodiments, the software architecture 300 may include a network layer (e.g., IP layer) where the logical connection terminates at a packet data network (PDN) gateway (PGW). In some embodiments, the software architecture 300 may include an application layer where the logical connection terminates at another device (e.g., an end-user device, a server, etc.). In some embodiments, the software architecture 300 may further include a hardware interface 316 between the physical layer 306 and communication hardware (e.g., one or more radio frequency (RF) transceivers) in the AS 304.

[0079]

[0092] Figure 4 shows a process flow diagram of an exemplary method 400 for supporting an immersive experience in a teleconference or telepresence session according to various embodiments. Referring to FIGS. 1A-4, method 400 may be implemented by a processor of a wireless device (such as wireless devices 120a-120e, 172, 200, 320, etc.) (such as 156, 212, 216, 252, or 260, etc.). In various embodiments, the operations of method 400 may be performed by a processor of a wireless device that is one of a plurality of participant computing devices in a teleconference or telepresence session, such as an immersive 3D group session.

[0080]

[0093] In block 402, the process performs operations for instructing an offer to send and / or receive a scene graph as part of a session initiation protocol (SIP) setup for the session. In some embodiments, the offer may indicate a graphical output node owned by the wireless device. In some embodiments, the session may be a WebRTC session.

[0081]

[0094] In block 404, the processor may perform operations for receiving a session description protocol (SDP) for the session that indicates an address of a data channel over which a scene graph for the session is to be shared. In some embodiments, the scene graph may define one or more nodes assigned to each computing device participating in the session. In various embodiments, a node may reference other media streams from other computing devices participating in the session, and the other media streams may be overlaid in the session. In various embodiments, one or more nodes assigned to each computing device participating in the session may include one or more visual nodes, audio source nodes, graphics nodes, or shared content nodes.

[0082]

[0095] In block 406, the processor may perform operations to download a scene graph via a data channel.

[0083]

[0096] In block 408, the processor may perform operations to receive and render a session according to the scene graph for rendering on an image rendering device (e.g., 168). Receiving and rendering a session may include receiving a streaming service of the session according to the scene graph and rendering the session on a display. In some embodiments, the display of the session may be rendered on an HMD (e.g., 172), on a video conference room, on a volumetric display, or on any other image and sound rendering device, and receiving and rendering may include outputting the session to the user via the image and sound rendering device.

[0084]

[0097] In block 410, the processor may perform operations to add a node assigned to a wireless device to the scene graph or to modify a node among one or more nodes assigned to the wireless device in the scene graph.

[0085]

[0098] FIG. 5 shows a process flow diagram of an exemplary method 500 for supporting an immersive experience in a teleconference or telepresence session according to various embodiments. Referring to FIGS. 1A - 5, method 500 may be implemented by a processor (such as processors 156, 212, 216, 252, or 260) of a wireless device (such as wireless devices 120a - 120e, 172, 200, 320, etc.) and / or a call server (such as call server 150). In various embodiments, the operations of method 500 may be performed by a processor of a host computing device hosting a teleconference or telepresence session. In some embodiments, the host computing device may be a separate call server, such as an MRF, an MCU, a teleconference application server, etc. In some embodiments, the host computing device may be a wireless device that is one of a plurality of computing devices participating in a teleconference or telepresence session, such as an immersive 3D group session. In various embodiments, the operations of method 500 may be performed in conjunction with the operations of method 400.

[0086]

[0099] At block 502, a processor of a host computing device may perform operations to receive an indication for sending and / or receiving a scene graph as part of a session initiation protocol (SIP) setup for the session from a computing device participating in the session. In some embodiments, the session may be a WebRTC session. In some embodiments, the received indication may indicate a graphical output node owned by a wireless device that sends the offer indication.

[0087]

[0100] In block 504, the processor of the host computing device may perform operations to generate a Session Description Protocol (SDP) that indicates the address of a data channel for which a scene graph for the session is to be shared. In some embodiments, the scene graph may define one or more nodes assigned to each computing device participating in the session.

[0088]

[0101] In block 506, the processor of the host computing device may perform operations to send the SDP to the computing devices participating in the session.

[0089]

[0102] In block 508, the processor of the host computing device may perform operations to send the session and the scene graph to the computing devices participating in the session. As an example, the host computing device may perform operations to stream the session and the scene graph to the computing devices participating in the session. In various embodiments, the computing devices participating in the session may be an HMD (e.g., 172), a TV in a conference room, a volumetric display, or other image and sound rendering devices, and sending the session and / or the scene graph may include sending the session and / or the scene graph so that the computing devices participating in the session can output the session to the user on an image and sound rendering device.

[0090]

[0103] FIG. 6 shows the configuration of a scene graph document 600 in glTF 2.0 suitable for use in various embodiments. Referring to FIGS. 1A-6, in various embodiments, the scene graph document 600 can be part of a scene graph. The scene graph document 600 can include JSON elements that define a node hierarchy, material descriptions, lighting information, camera information, etc. The scene graph document 600 can include a binary file (BIN) that defines geometry information such as vertices and indices, animation information such as keyframes, skin information such as inverse bind matrices, etc. The scene graph document 600 can include a GL shading language (glSL) file that defines shader information. The scene graph document 600 can include various other types of files such as a Portable Network Graphics (PNG) file, a Joint Photographic Experts Group (JPEG) file, etc., that define other information for the scene graph, such as textures.

[0091]

[0104] FIG. 7 shows the structure of a scene graph suitable for use in various embodiments. Referring to FIGS. 1A-7, in various embodiments, the scene graph can include a plurality of nodes. Each node can include child nodes that describe various components of the node, such as a camera view, mesh information, lighting information, etc. The scene graph can define a hierarchical relationship between attributes for rendering the mesh of the node, such as an accessory, skin, buffer view, buffer information, material, technique, program, shader, texture, image, and sampler, as shown in FIG. 7.

[0092]

[0105] FIG. 8 is a call flow diagram showing operations for supporting an immersive experience in a teleconference or telepresence session according to various embodiments. Referring to FIGS. 1A-8, the operations between call participants (e.g., call participants #1 and #2) and a call server may include, in operation 1), the participants (e.g., call participants #1 and #2) using a provided link to a web page to join a WebRTC conference. In operation 2), the call server provides the participants (e.g., call participants #1 and #2) with the web page along with a scene graph file that sets up an initial / default configuration in a 3D space between the call participants (e.g., call participants #1 and #2) and materials. For example, each participant may be assigned a visual node, an audio source node, and potentially also nodes for graphics and other shared content. In operation 3), each participant (e.g., call participants #1 and #2) may add or modify the nodes it owns in the scene graph. In operation 4), media streams that provide components for the nodes in the scene graph may be streamed using WebRTC. These streams may be exchanged directly or through a server, such as a media proxy server.

[0093]

[0106] FIG. 9 is a process flow diagram showing a method 900 for providing an immersive 3D group session according to various embodiments. Referring to FIGS. 1A-9, method 900 may be implemented by a processor (such as 156, 212, 216, 252, or 260) of a wireless device (such as wireless devices 120a-120e, 172, 200, 320). In various embodiments, the operations of method 900 may be performed by a processor of a wireless device that is one of a plurality of participant computing devices in a teleconference or telepresence session, such as an immersive 3D group session. In various embodiments, the operations of method 900 may be performed in conjunction with any one or more of the operations of methods 400 (FIG. 4) and / or 500 (FIG. 5).

[0094]

[0107] In block 902, the processor may perform operations including receiving a scene graph for an immersive 3D group session, where the scene graph includes at least a unique graphical output node assigned to be controlled by the wireless device and respective other graphical output nodes assigned to be controlled by each of the other ones of the plurality of participant computing devices. For example, the scene graph may be a scene graph as shown in FIG. 8.

[0095]

[0108] The portion of the scene graph received at block 902 can be assigned to each respective participant computing device, such as one or more nodes, in an immersive 3D group session. By parsing the scene graph, a processor of the wireless device can determine the nodes to be controlled by the wireless device. Assigning a graphical output node for each participant computing device can enable each participant computing device to control at least one graphical output node. The graphical output node can include components that define a 3D object to be output in the 3D space defined by the scene graph. For example, the 3D object can include an avatar, a character, or other representation, and the components of the graphical output node can define how the 3D object is to be rendered in the 3D space of the immersive 3D group session. In this way, by controlling the components of each respective graphical output node assigned thereto, the processor of the wireless device can control how other participant computing devices observe one or more 3D objects associated with the wireless device, such as an avatar, a character, or other representation selected by the user of the wireless device in the immersive 3D group session. Similarly, by controlling the components of each respective graphical output node assigned thereto, other participant computing devices can control how the user of the wireless device observes one or more 3D objects associated with them, such as an avatar, a character, or other representation in the immersive 3D group session.

[0096]

[0109] In block 904, the processor may perform operations including controlling components of a unique graphical output node for the three-dimensional space of an immersive three-dimensional group session. In some embodiments, the user may adjust the unique graphical node, such as moving the placement of the unique graphical node in the scene, regardless of the position of the wireless device. For example, the user may select a position in the scene to display the user's three-dimensional object (e.g., avatar, character, etc.) in the scene. In some embodiments, controlling the components of the unique graphical output node may include controlling the components of the unique graphical output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session. In some embodiments, the position of the wireless device may be determined in the three-dimensional space relative to a central point, grid coordinates, or other reference for the three-dimensional space. Since some immersive three-dimensional group sessions may support the movement of participants within the three-dimensional space for the immersive three-dimensional group session, determining the position of the wireless device may support rendering an object for the graphical output node of the wireless device in the correct relative position. As a specific example, the lighting and / or camera components of the assigned unique graphical output node of the wireless device may be controlled to reflect the lighting and / or camera components of the current position of the wireless device in the three-dimensional space for the immersive three-dimensional group session.

[0097]

[0110] In block 904, in addition to controlling the components of the unique graphical output node based on position, the components of the unique graphical output node can be controlled based on other metrics, such as the orientation of the wireless device. For example, changes in the position and orientation of the movement with respect to the common reference point and common reference plane indicated by the accelerometer of the wireless device can be used to determine the position and orientation of the wireless device with respect to the three-dimensional space of the immersive three-dimensional group session. As a specific example, the lighting and / or camera components of the uniquely assigned graphical output node of the wireless device can be controlled to reflect the lighting and / or camera components of the current position and current orientation of the wireless device in the three-dimensional space for the immersive three-dimensional group session. Controlling the components of the unique graphical output node based on the current position and / or current orientation can be useful in an implementation where the wireless device is a head-mounted device that enables the movement of the user of the head-mounted device to be visually presented to other participants in the immersive three-dimensional group session.

[0098]

[0111] In block 906, the processor may perform operations including sending the components of the unique graphical output node to other ones of the plurality of participant computing devices in a first media stream. Those components can be sent in the media stream directly to other participant computing devices and / or via a call server (such as, for example, 150), such as an MRF, an MCU, a teleconference application server. In this way, the processor of the wireless device provides a media stream for its respective unique graphical output node to other participant computing devices, thereby enabling the other participant computing devices to control how to render that graphical output node.

[0099]

[0112] In block 908, the processor may perform operations including receiving, in a media stream, components of other graphical output nodes from each of the other ones of the plurality of participant computing devices. Those components may be received directly from each of the other participant computing devices and / or via a call server (e.g., 150) such as an MRF, an MCU, a teleconference application server, etc. in the media stream. In this way, the processor of the wireless device may receive the media stream of the graphical output nodes assigned to other participant computing devices.

[0100]

[0113] In block 910, the processor may perform operations including rendering an immersive three-dimensional group session on a display of the wireless device, at least partially based on components of its own graphical output node and components of other graphical output nodes. For example, components of the media stream of the wireless device and components of the media stream of other participant computing devices may be overlaid with components of other media streams to generate a displayed output in the three-dimensional space of the immersive three-dimensional group session.

[0101]

[0114] The processor may continuously perform the operations of blocks 904, 906, 908, and 910 within the immersive three-dimensional group session to render the immersive three-dimensional group session.

[0102]

[0115] FIG. 10 is a process flow diagram showing a method 1000 for providing an immersive 3D group session according to various embodiments. Referring to FIGS. 1A - 10, method 1000 may be implemented by a processor (such as 156, 212, 216, 252, or 260) of a wireless device (such as wireless devices 120a - 120e, 172, 200, 320). In various embodiments, the operations of method 1000 may be performed by a processor of a wireless device that is one of a plurality of participant computing devices in a teleconference or telepresence session, such as an immersive 3D group session. In various embodiments, the operations of method 1000 may be performed in conjunction with any one or more of the operations of method 400 (FIG. 4), method 500 (FIG. 5), and / or method 900 (FIG. 9). As a specific example, the operations of method 1000 may be performed as part of the operations of block 910 of method 900 to render an immersive 3D group session.

[0103]

[0116] In block 1002, the processor may perform operations including receiving a scene graph update that includes an indication of a new participant computing device for an immersive 3D group session and an indication of a new graphical output node assigned to be controlled by the new participant computing device. The scene graph update may be sent by a host computing device in response to a new participant joining the immersive 3D group session. In some embodiments, the scene graph update may be received directly from another participant computing device and / or via a call server (such as 150), such as an MRF, MCU, teleconference application server.

[0104]

[0117] In block 1004, the processor may perform operations including receiving components of a new graphical output node in a second media stream from a new participant computing device. Those components may be received in the media stream directly from the new participant computing device and / or via a call server (such as 150), such as an MRF, MCU, teleconference application server, etc. In this way, the processor of the wireless device may receive the media stream of the graphical output node assigned to the newly added participant computing device.

[0105]

[0118] In block 1006, the processor may perform operations including rendering an immersive 3D group session on the display of the wireless device, at least partially based on components of its own graphical output node, components of other graphical output nodes, and components of the new graphical output node. For example, components of the media stream of the wireless device and components of the media streams of other participant computing devices, including the second media stream of the newly added participant computing device, may be overlaid with components of other media streams to generate the displayed output in the 3D space of the immersive 3D group session.

[0106]

[0119] Various embodiments may be implemented on various wireless network devices, an example of which is shown in FIG. 11 in the form of a wireless network computing device 1100 that functions as a network element of a communication network, such as a call server (e.g., call server 150). Such a network computing device may include at least the components shown in FIG. 11. Referring to FIGS. 1A-11, network computing device 1100 may typically include a processor 1101 coupled to a volatile memory 1102 and a mass non-volatile memory such as a disk drive 1103. Network computing device 1100 may also include a peripheral memory access device such as a floppy (registered trademark) disk drive, a compact disk (CD), or a digital video disk (DVD) drive 1106 coupled to processor 1101. Network computing device 1100 may also include a network access port 1104 (or interface) coupled to processor 1101 for establishing a data connection to a network such as the Internet and / or a local area network coupled to other system computers and servers. Network computing device 1100 may include one or more antennas 1107 for sending and receiving electromagnetic radiation that may be connected to a wireless communication link. Network computing device 1100 may include additional access ports such as USB, Firewire, Thunderbolt for coupling to peripheral devices, external memory, or other devices.

[0107]

[0120] Various embodiments may be implemented on various wireless devices (e.g., wireless devices 120a - 120e, 172, 200, 320), an example of which is shown in FIG. 12 in the form of a smartphone 1200. Referring to FIGS. 1A - 12, the smartphone 1200 may include a first SOC 202 (e.g., SOC - CPU) coupled to a second SOC 204 (e.g., 5G - capable SOC). The first SOC 202 and the second SOC 204 may be coupled to internal memories 1206, 1216, a display 1212, and a speaker 1214. Further, the smartphone 1200 may include an antenna 1204 for transmitting and receiving electromagnetic radiation that may be connected to a wireless data link, and / or a cellular phone transceiver 266 coupled to one or more processors in the first SOC 202 and / or the second SOC 204. The smartphone 1200 also typically includes a menu selection button or rocker switch 1220 for receiving user input.

[0108]

[0121] A typical smartphone 1200 also includes an audio encoding / decoding (codec) circuit 1210 that digitizes sound received from a microphone into data packets suitable for wireless transmission, decodes the received audio data packets, and generates an analog signal provided to a speaker to generate sound. Also, one or more of the processors in the first SOC 202 and the second SOC 204, the wireless transceiver 266, and the codec 1210 may include a digital signal processor (DSP) circuit (not shown separately).

[0109]

[0122] The processors of the wireless network computing device 1100 and the smartphone 1200 can be any programmable microprocessor, microcomputer, or one or more multiple processor chips configured by software instructions (applications) to perform various functions, including the functions of the various embodiments described below. In some wireless devices, multiple processors may be provided, such as one processor within the SOC 204 dedicated to wireless communication functions and one processor within the SOC 202 dedicated to executing other applications. Typically, software applications can be stored in memories 1206 and 1216 before they are accessed and loaded into the processor. The processor may include sufficient internal memory to store application software instructions.

[0110]

[0123] As used in this application, terms such as "component", "module", "system", etc., although not limited, are intended to include computer-related entities such as hardware, firmware, a combination of hardware and software, software, or software in execution, which are configured to perform a particular operation or function. For example, a component can be, although not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As an example, both an application running on a wireless device and the wireless device may be referred to as components. One or more components can exist within a process and / or an execution thread, and a component can be localized on one processor or core and / or distributed between two or more processors or cores. Further, these components can execute from various non-transitory computer-readable media storing various instructions and / or data structures. Components can communicate via local and / or remote processes, function or procedure calls, electronic signals, data packets, memory reads / writes, as well as other known network, computer, processor, and / or process-related communication methods.

[0111]

[0124] Several different cellular and mobile communication services and standards will be available or are contemplated in the future, and all of them can implement and benefit from various embodiments. Such services and standards include, for example, the 3rd Generation Partnership Project (3GPP), the Long Term Evolution (LTE) system, the 3rd Generation Wireless Mobile Communication Technology (3G), the 4th Generation Wireless Mobile Communication Technology (4G), the 5th Generation Wireless Mobile Communication Technology (5G), the Global System for Mobile Communications (GSM), the Universal Mobile Telecommunications System (UMTS), 3GSM, the General Packet Radio Service (GPRS), the Code Division Multiple Access (CDMA) system (e.g., cdmaOne, CDMA1020 (registered trademark)), the GSM Evolution High Speed Data Rate (EDGE), the Advanced Mobile Phone System (AMPS), the Digital AMPS (IS-136 / TDMA), the Evolution Data Optimized (EV-DO), the Digital Enhanced Cordless Telecommunications (DECT), the Worldwide Interoperability for Microwave Access (WiMAX), the Wireless Local Area Network (WLAN), the Wi-Fi Protected Access I and II (WPA, WPA2), and the Integrated Digital Enhanced Network (iDEN). Each of these technologies involves, for example, the transmission and reception of voice, data, signaling, and / or content messages. It should be understood that any reference to terms and / or technical details related to individual telecommunications standards or technologies is for illustrative purposes only and is not intended to limit the claims to a particular communication system or technology unless specified in the claim language.

[0112]

[0125] The various embodiments illustrated and described are provided only as examples to illustrate various features of the claims. However, the features illustrated and described with respect to any given embodiment are not necessarily limited to the related embodiment and can be used with or combined with other illustrated and described embodiments. Further, the claims are not intended to be limited by any one exemplary embodiment. For example, one or more of the operations of methods 400, 500, 900, and / or 1000 can be replaced with or combined with one or more of the operations of methods 400, 500, 900, and / or 1000.

[0113]

[0126] The foregoing description of the methods and process flow diagrams are provided only as exemplary examples and are not intended to require or imply that the operations of the various embodiments must be performed in the order presented. As will be understood by those skilled in the art, the order of operations in the foregoing embodiments can be performed in any order. Words such as "thereafter," "then," "next," etc. are not intended to limit the order of operations and are used to guide the reader through the description of the method. Further, references to elements of singular claims using, for example, the articles "a," "an," or "the" should not be construed as limiting the element to the singular form.

[0114]

[0127] With respect to the embodiments disclosed herein, the various illustrative logical blocks, modules, components, circuits, and algorithmic operations described may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and operations have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementations should not be construed as causing a departure from the scope of the claims.

[0115]

[0128] The hardware used to implement the various illustrative logics, logical blocks, modules, and circuits described with respect to the embodiments disclosed herein may be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gates or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of receiver smart objects, such as, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors working in conjunction with a DSP core, or any other such configuration. Alternatively, some operations or methods may be implemented by circuitry that is specific to a given function.

[0116]

[0129] In one or more embodiments, the described functions may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable storage medium or a non-transitory processor-readable storage medium. The operations of the methods or algorithms disclosed herein may be implemented in processor-executable software modules or processor-executable instructions present on a non-transitory computer-readable storage medium or a non-transitory processor-readable storage medium. A non-transitory computer-readable storage medium or a non-transitory processor-readable storage medium may be any storage medium that can be accessed by a computer or a processor. By way of example and not limitation, such a non-transitory computer-readable storage medium or a non-transitory processor-readable storage medium can include RAM, ROM, EEPROM®, FLASH® memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer. As used herein, disk and disc include compact disc (CD), laser disc®, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray® disc, where disk typically magnetically reproduces data and disc optically reproduces data with a laser. Combinations of the above are also included within the scope of non-transitory computer-readable media and non-transitory processor-readable media. Further, the operations of a method or algorithm may exist as code and / or instructions, or any combination or set thereof, on a non-transitory processor-readable storage medium and / or a non-transitory computer-readable storage medium that can be incorporated into a computer program product.

[0117]

[0130] The foregoing description of the disclosed embodiments is provided to enable a person of ordinary skill in the art to make or use the claimed subject matter. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the claimed subject matter. Accordingly, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein.

Claims

Claim 1 A method implemented by a processor of a wireless device that is one of a plurality of participant computing devices operating in an immersive three-dimensional group session, the method comprising: receiving a scene graph for the immersive three-dimensional group session, wherein the scene graph comprises at least a unique graphical output node assigned to be controlled by the wireless device and respective other graphical output nodes assigned to be controlled by each of the other ones of the plurality of participant computing devices; controlling components of the unique graphical output node with respect to a three-dimensional space of the immersive three-dimensional group session; sending the components of the unique graphical output node to the other ones of the plurality of participant computing devices in a first media stream; receiving components of the other graphical output nodes from each of the other ones of the plurality of participant computing devices in a media stream; rendering the immersive three-dimensional group session on a display of the wireless device based at least in part on the components of the unique graphical output node and the components of the other graphical output nodes; A method comprising the above steps. Claim 2 receiving a scene graph update including an indication of a new participant computing device for the immersive three-dimensional group session and an indication of a new graphical output node assigned to be controlled by the new participant computing device; receiving components of the new graphical output node from the new participant computing device in a second media stream; rendering the immersive three-dimensional group session on the display of the wireless device based at least in part on the components of the unique graphical output node, the components of the other graphical output nodes, and the components of the new graphical output node; The method according to claim 1, further comprising the above steps. Claim 3 Receiving a session description protocol (SDP) for the immersive three-dimensional group session that indicates an address of a data channel through which the scene graph will be shared; further comprising; The method according to claim 1, wherein receiving the scene graph comprises downloading the scene graph via the data channel.

4. Sending an offer to send or receive the scene graph to the other of the plurality of participant computing devices as part of a session initiation protocol (SIP) setup for the immersive three-dimensional group session; The method according to claim 1, further comprising;

5. further comprising sending an offer to send or receive the scene graph to the other of the plurality of participant computing devices, wherein the offer indicates the unique graphical output node; The method according to claim 1.

6. The method according to claim 1, wherein the immersive three-dimensional group session is a Web Real-Time Communication (WebRTC) session.

7. Controlling components of the unique graphical output node with respect to the three-dimensional space of the immersive three-dimensional group session comprises controlling components of the unique graphical output node at least in part based on a determined position of the wireless device with respect to the three-dimensional space of the immersive three-dimensional group session. The method according to claim 1.

8. Controlling components of the unique graphical output node at least in part based on the determined position of the wireless device with respect to the three-dimensional space of the immersive three-dimensional group session comprises controlling components of the unique graphical output node at least in part based on the determined position of the wireless device with respect to the three-dimensional space of the immersive three-dimensional group session and a determined orientation of the wireless device with respect to the three-dimensional space of the immersive three-dimensional group session. The method according to claim 7.

9. A display; a processor coupled to the display; A wireless device comprising, wherein the processor; Receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph comprises at least a unique graphical output node assigned to be controlled by the wireless device and respective other graphical output nodes assigned to be controlled by each of a plurality of participant computing devices, Controlling components of the unique graphical output node with respect to a three-dimensional space of the immersive three-dimensional group session, Sending, in a first media stream, the components of the unique graphical output node to other ones of the plurality of participant computing devices, Receiving, in a media stream, components of the other graphical output nodes from each of the other ones of the plurality of participant computing devices, Rendering the immersive three-dimensional group session on the display of the wireless device based at least in part on the components of the unique graphical output node and the components of the other graphical output nodes, A wireless device configured with processor-executable instructions for performing the above.

10. The processor is Receiving a scene graph update including an indication of a new participant computing device for the immersive three-dimensional group session and an indication of a new graphical output node assigned to be controlled by the new participant computing device, Receiving, in a second media stream, components of the new graphical output node from the new participant computing device, Rendering the immersive three-dimensional group session on the display of the wireless device based at least in part on the components of the unique graphical output node, the components of the other graphical output nodes, and the components of the new graphical output node, The wireless device according to claim 9, further configured with processor-executable instructions for performing the above.

11. The processor is Receiving a session description protocol (SDP) for the immersive 3D group session that indicates an address of a data channel through which the scene graph is to be shared; further configured with processor-executable instructions for doing so; The processor is further configured with processor-executable instructions for receiving the scene graph by downloading the scene graph over the data channel, the wireless device of claim 9. **Claim 12** The processor is sending an offer to send or receive the scene graph to the other of the plurality of participant computing devices as part of a session initiation protocol (SIP) setup for the immersive 3D group session; the wireless device of claim 9, further configured with processor-executable instructions for doing so. **Claim 13** The processor is further configured with processor-executable instructions for sending an offer to send or receive the scene graph to the other of the plurality of participant computing devices, wherein the offer indicates the unique graphical output node; the wireless device of claim 9. **Claim 14** The immersive 3D group session is a web real-time communication (WebRTC) session, the wireless device of claim 9. **Claim 15** The processor is further configured with processor-executable instructions for controlling components of the unique graphical output node with respect to the 3D space of the immersive 3D group session by at least partially basing on a determined position of the wireless device with respect to the 3D space of the immersive 3D group session, the method of claim 9. **Claim 16** The processor is further configured with processor-executable instructions for controlling components of the unique graphical output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session and the determined orientation of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session, to control components of the unique graphical output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session, the wireless device according to claim 15. **Claim 17** A non-transitory processor-readable medium storing processor-executable instructions, the processor-executable instructions causing a processor of a wireless device to receive a scene graph for an immersive three-dimensional group session, wherein the scene graph comprises at least a unique graphical output node assigned to be controlled by the wireless device and respective other graphical output nodes assigned to be controlled by each of a plurality of participant computing devices, control components of the unique graphical output node relative to the three-dimensional space of the immersive three-dimensional group session, send, in a first media stream, the components of the unique graphical output node to other ones of the plurality of participant computing devices, receive, in a media stream, components of the other graphical output nodes from each of the other ones of the plurality of participant computing devices, render the immersive three-dimensional group session on a display of the wireless device based at least in part on the components of the unique graphical output node and the components of the other graphical output nodes, A non-transitory processor-readable medium configured to cause the operations described above. **Claim 18** The stored processor-executable instructions cause a processor of a wireless device to Receiving a scene graph update that includes instructions for a new participant computing device for the immersive 3D group session and instructions for a new graphical output node assigned to be controlled by the new participant computing device; Receiving, from the new participant computing device, components of the new graphical output node in a second media stream; Rendering the immersive 3D group session on the display of the wireless device, at least in part based on the components of the unique graphical output node, the components of the other graphical output nodes, and the components of the new graphical output node; The non-transitory processor-readable medium of claim 17, further configured to cause an operation to be performed that further comprises:

19. The stored processor-executable instructions cause a processor of a wireless device to: Receive a session description protocol (SDP) for the immersive 3D group session that indicates an address of a data channel over which the scene graph is to be shared; The non-transitory processor-readable medium of claim 17, further configured to cause an operation to be performed that further comprises: The processor-executable instructions cause a processor of a wireless device to perform an operation that comprises downloading the scene graph over the data channel, wherein receiving the scene graph is enabled; The non-transitory processor-readable medium of claim 17.

20. The stored processor-executable instructions cause a processor of a wireless device to: Send an offer to send or receive the scene graph to the other of the plurality of participant computing devices as part of a session initiation protocol (SIP) setup for the immersive 3D group session; The non-transitory processor-readable medium of claim 17, further configured to cause an operation to be performed that further comprises:

21. The stored processor-executable instructions cause a processor of a wireless device to: configured to perform an operation further comprising sending an offer to send or receive the scene graph to the other of the plurality of participant computing devices, wherein the offer indicates the unique graphical output node The non-transitory processor-readable medium of claim 17. **Claim 22** The non-transitory processor-readable medium of claim 17, wherein the immersive three-dimensional group session is a Web Real-Time Communication (WebRTC) session. **Claim 23** The non-transitory processor-readable medium of claim 17, wherein the stored processor-executable instructions are configured to cause a processor of a wireless device to control components of the unique graphical output node with respect to the three-dimensional space of the immersive three-dimensional group session, at least in part based on a determined position of the wireless device with respect to the three-dimensional space of the immersive three-dimensional group session, such that the operation comprises controlling components of the unique graphical output node. **Claim 24** The non-transitory processor-readable medium of claim 17, wherein the stored processor-executable instructions are configured to cause a processor of a wireless device to control components of the unique graphical output node with respect to the three-dimensional space of the immersive three-dimensional group session, at least in part based on the determined position of the wireless device with respect to the three-dimensional space of the immersive three-dimensional group session and the determined orientation of the wireless device with respect to the three-dimensional space of the immersive three-dimensional group session, such that the operation comprises controlling components of the unique graphical output node. **Claim 25** A wireless device, means for receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph comprises at least a unique graphical output node assigned to be controlled by the wireless device and respective other graphical output nodes assigned to be controlled by each of a plurality of participant computing devices Means for controlling components of the unique graphical output node with respect to the three-dimensional space of the immersive three-dimensional group session; Means for sending the components of the unique graphical output node in a first media stream to other ones of the plurality of participant computing devices; Means for receiving components of other graphical output nodes in a media stream from each of the other ones of the plurality of participant computing devices; Means for rendering the immersive three-dimensional group session on the display of the wireless device, based at least in part on the components of the unique graphical output node and the components of the other graphical output nodes; A wireless device comprising the above.

26. Means for receiving a scene graph update including an indication of a new participant computing device for the immersive three-dimensional group session and an indication of a new graphical output node assigned to be controlled by the new participant computing device; Means for receiving components of the new graphical output node in a second media stream from the new participant computing device; Means for rendering the immersive three-dimensional group session on the display of the wireless device, based at least in part on the components of the unique graphical output node, the components of the other graphical output nodes, and the components of the new graphical output node; The wireless device according to claim 25, further comprising the above.

27. Further comprising means for receiving a session description protocol (SDP) for the immersive three-dimensional group session indicating an address of a data channel through which the scene graph is to be shared; The wireless device according to claim 25, wherein the means for receiving the scene graph comprises means for downloading the scene graph via the data channel.

28. As part of the session initiation protocol (SIP) setup for the immersive 3D group session, means for sending an offer to send or receive the scene graph to the other of the plurality of participant computing devices. The wireless device according to claim 25, further comprising. **Claim 29** The wireless device according to claim 25, further comprising means for sending an offer to send or receive the scene graph to the other of the plurality of participant computing devices, wherein the offer indicates the unique graphical output node. The wireless device according to claim 25. **Claim 30** The wireless device according to claim 25, wherein the immersive 3D group session is a Web Real-Time Communication (WebRTC) session.

Images