Method, wireless device and medium for supporting immersive three-dimensional group session

Abstract

A method performed by a processor of a wireless device for supporting an immersive three-dimensional group session, a wireless device, and a medium are provided. The method includes receiving a scene graph for an immersive three-dimensional group session, where the scene graph indicates that at least an own graphical output node is controlled by the wireless device and that a respective one or more other graphical output nodes are controlled by each of one or more participant computing devices; and outputting the immersive three-dimensional group session for presentation on a display of the wireless device based at least in part on one or more components of the own graphical output node and one or more other components of the one or more other graphical output nodes.

Description

[0001] This application is a divisional application of the invention patent application with the application date of March 11, 2021, application number 202180020532.X, and invention name “Signaling for scene description for multimedia conferencing”.

[0002] Related Applications

[0003] This application claims priority to U.S. Provisional Application No. 62 / 990,895, filed on March 17, 2020, entitled “Signaling of Scene Description For Multimedia Conferencing,” the entire contents of which are hereby incorporated by reference herein for all purposes. Background Art

[0004] Long Term Evolution (LTE), fifth generation (5G) New Radio (NR), and other recently developed communications technologies allow wireless devices to transmit information at data rates that are orders of magnitude higher (e.g., measured in gigabits per second, etc.) than were available just a few years ago.

[0005] Today’s communications networks are also more secure, resilient to multipath fading, allow for lower latency for network traffic, and provide better communications efficiency (e.g., in terms of bits per second per unit of bandwidth used). These and other recent improvements have facilitated the emergence of the Internet of Things (IoT), massive machine-to-machine (M2M) communication systems, autonomous vehicles, and other technologies that rely on consistent and secure communications.

[0006] In recent years, augmented reality software applications that combine real-world images from a user's physical environment with computer-generated images or virtual objects (VOs) have become increasingly popular and used. Augmented reality software applications can add graphics, sounds, and / or tactile feedback to the natural world around the application user. Information about images, video streams, and images of people and / or objects can be presented to the user as an augmented scene on a wearable electronic display or head-mounted device (e.g., smart glasses, augmented reality glasses, etc.), superimposed on the visual world. Summary of the invention

[0007] Various aspects include systems and methods for providing an immersive three-dimensional group session. Various embodiments include methods and apparatus for signaling a description of a scene using media components that may be from different parties. In various aspects, a scene graph may be signaled via a session description protocol (SDP) during a session initiation protocol (SIP) session establishment. In various aspects, the scene graph may include respective graphical output nodes assigned to be controlled by each of a plurality of participant computing devices in an immersive three-dimensional group session. Various aspects may be performed by a processor of a wireless device that is one of a plurality of participant computing devices operating in an immersive three-dimensional group session. Various aspects may include: receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph includes at least an owned graphics output node assigned to be controlled by a wireless device and corresponding other graphics output nodes assigned to be controlled by each of other participant computing devices in a plurality of participant computing devices; a component for controlling the owned graphics output node relative to a three-dimensional space of the immersive three-dimensional group session; a component for sending the owned graphics output node to other participant computing devices in a plurality of participant computing devices in a first media stream; a component for receiving the other graphics output nodes in a media stream from each of the other participant computing devices in the plurality of participant computing devices; and presenting the immersive three-dimensional group session on a display of the wireless device based at least in part on the component of the owned graphics output node and the component of the other graphics output nodes.

[0008] Some aspects may also include: receiving a scene graph update that includes an indication of a new participant computing device for the immersive three-dimensional group session and an indication of a new graphics output node assigned to be controlled by the new participant computing device; receiving components of the new graphics output node in a second media stream from the new participant computing device; and presenting the immersive three-dimensional group session on a display of the wireless device based at least in part on components of the own graphics output node, components of other graphics output nodes, and components of the new graphics output node.

[0009] Some aspects may also include receiving a session description protocol (SDP) for the immersive 3D group session, the session description protocol (SDP) indicating an address of a data channel over which the scene graph is to be shared, wherein receiving the scene graph includes downloading the scene graph via the data channel.

[0010] Some aspects may also include sending a proposal to send or receive a scene graph to other participant computing devices of the plurality of participant computing devices as part of a session initiation protocol (SIP) setup for the immersive three-dimensional group session.

[0011] Some aspects may also include sending a proposal to send or receive a scene graph to other participant computing devices of the plurality of participant computing devices, wherein the proposal indicates an own graph output node.

[0012] In some aspects, the immersive 3D group session is a Web Real-Time Communication (WebRTC) session.

[0013] In some aspects, the component controlling the own graphics output node relative to the three-dimensional space of the immersive three-dimensional group session may include a component controlling the own graphics output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session.

[0014] In some aspects, components that control an owned graphics output node based at least in part on a determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session may include components that control the owned graphics output node based at least in part on a determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session and a determined orientation of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session.

[0015] Further aspects may include a wireless device having a processor configured to perform one or more operations of any of the methods outlined above. Further aspects may include a non-transitory processor-readable storage medium having processor-executable instructions stored thereon, the processor-executable instructions configured to cause a processor of the wireless device to perform the operations of any of the methods outlined above. Further aspects include a wireless device having components for performing the functions of any of the methods outlined above. Further aspects include a system-on-chip for use in a wireless device, the wireless device including a processor configured to perform one or more operations of any of the methods outlined above. Further aspects include a system-in-package including two systems-on-chip for use in a wireless device, the wireless device including a processor configured to perform one or more operations of any of the methods outlined above.

[0016] Further aspects may include a method for supporting an immersive three-dimensional group session executed by a processor of a wireless device, comprising: receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph indicates that at least an own graphics output node is controlled by the wireless device, and indicates that corresponding one or more other graphics output nodes are controlled by each of one or more participant computing devices; and presenting the immersive three-dimensional group session on a display of the wireless device based at least in part on one or more components of the own graphics output node and one or more other components of the one or more other graphics output nodes.

[0017] A further aspect may include a wireless device comprising: a processor configured with processor-executable instructions to receive a scene graph for an immersive three-dimensional group session, wherein the scene graph indicates that at least an own graphics output node is controlled by the wireless device and indicates that corresponding one or more other graphics output nodes are controlled by each of one or more participant computing devices; and the immersive three-dimensional group session is presented on a display of the wireless device based at least in part on one or more components of the own graphics output node and one or more other components of the one or more other graphics output nodes.

[0018] Further aspects may include a non-transitory processor-readable medium having processor-executable instructions stored thereon, the processor-executable instructions being configured to cause a processor of the wireless device to perform operations including: receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph indicates that at least an own graphics output node is controlled by the wireless device and indicates that corresponding one or more other graphics output nodes are controlled by each of one or more participant computing devices; and presenting the immersive three-dimensional group session on a display of the wireless device based at least in part on one or more components of the own graphics output node and one or more other components of the one or more other graphics output nodes.

[0019] A further aspect may include a wireless device comprising: a device for receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph indicates that at least an own graphics output node is controlled by the wireless device and indicates that corresponding one or more other graphics output nodes are controlled by each of one or more participant computing devices; and a device for presenting the immersive three-dimensional group session on a display of the wireless device based in part on one or more components of the own graphics output node and one or more other components of the one or more other graphics output nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated herein and constitute a part of this specification, illustrate exemplary embodiments of the claims and, together with the general description given above and the detailed description given below, serve to explain features of the claims.

[0021] Figure 1A is a system block diagram illustrating an exemplary communication system suitable for implementing various embodiments.

[0022] Figure 1B is an illustration of a head mounted device (e.g., augmented reality glasses) in which various embodiments may be implemented.

[0023] Figure 2is a block diagram illustrating components of an example computing and wireless modem system suitable for implementing the various embodiments.

[0024] Figure 3 is a diagram illustrating an example of a software architecture including a radio protocol stack for user and control planes in wireless communications according to various embodiments.

[0025] Figure 4 is a process flow diagram illustrating a method for supporting an immersive experience in a teleconference or telepresence session in accordance with various embodiments.

[0026] Figure 5 is a process flow diagram illustrating a method for supporting an immersive experience in a teleconference or telepresence session in accordance with various embodiments.

[0027] Figure 6 Shows the layout of a scene graph document in glTF 2.0.

[0028] Figure 7 shows the structure of the scene graph.

[0029] Figure 8 is a call flow diagram illustrating operations for supporting an immersive experience in a teleconference or telepresence session in accordance with various embodiments.

[0030] Fig. 9 is a process flow diagram illustrating a method for providing an immersive three-dimensional group session according to various embodiments.

[0031] Fig.10 is a process flow diagram illustrating a method for providing an immersive three-dimensional group session according to various embodiments.

[0032] Fig.11 is a block diagram of components of an example server suitable for implementing various embodiments.

[0033] Fig.12 is a block diagram of components of a wireless device suitable for implementing various embodiments. DETAILED DESCRIPTION

[0034] Various embodiments will be described in detail with reference to the accompanying drawings. Whenever possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. References to specific examples and implementations are for illustrative purposes and are not intended to limit the scope of the claims.

[0035] Various embodiments may implement an immersive 3D group session for multiple participant computing devices, wherein a scene graph may include a respective graphics output node assigned to be controlled by each of the multiple participant computing devices in the immersive 3D group session. Various embodiments may enable multiple participant computing devices to share media streams of components of their respective assigned graphics output nodes with each other in the immersive 3D group session. By assigning each participant computing device control of its own respective graphics output node in the immersive 3D group session and sharing media streams of components of the graphics output nodes between the participant computing devices, various embodiments may support rendering of an immersive 3D group session with a shared 3D space, wherein each participant computing device controls its respective 3D object in the shared 3D space.

[0036] The term "wireless device" as used herein refers to wireless router devices, wireless appliances, cellular phones, smart phones, portable computing devices, personal or mobile multimedia players, laptop computers, tablet computers, smartbooks, ultrabooks, PDAs, wireless email receivers, multimedia Internet-enabled cellular phones, medical devices and equipment, biometric sensors / devices, wearable devices (including smart watches, smart clothing, smart glasses, smart wristbands, smart jewelry (e.g., smart rings and smart bracelets, etc.)), entertainment devices (e.g., wireless game controllers, music and video players, satellite radios, etc.), wireless network-enabled Internet of Things (IoT) devices (including smart meters / sensors), industrial manufacturing equipment, large and small machinery and appliances used in homes or businesses, wireless communication elements within autonomous and semi-autonomous vehicles, wireless devices attached to or incorporated into various mobile platforms, global positioning system devices, and similar electronic devices that include memory, wireless communication components, and programmable processors.

[0037] Various embodiments may be implemented on a network capable of transmitting and receiving RF signals in accordance with the Institute of Electrical and Electronics Engineers (IEEE) 16.11 standard or any IEEE 802.11 standard, a Bluetooth® standard (e.g., Bluetooth 4, Bluetooth 5, etc.), Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Global System for Mobile Communications (GSM), GSM / General Packet Radio Service (GPRS), Enhanced Data GSM Environment (EDGE), Terrestrial Trunked Radio (TETRA), Wideband-CDMA (W-CDMA), Evolution-Data Optimized (EV-DO), 1xEV-DO, EV-DO RevA, EV-DO RevB, High Speed ​​Packet Access (HSPA), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolved High Speed ​​Packet Access (HSPA+), Long Term Evolution (LTE), AMPS, or other known signals for communicating in a wireless, cellular, or Internet of Things (IoT) network (such as IEEE 802.15.4 protocols (such as Thread, ZigBee and Z-Wave), 6LoWPAN, Bluetooth Low Energy (BLE), LTE Machine Type Communication (LTE MTC), Narrowband LTE (NB-LTE), Cellular Internet of Things (CIoT), Narrowband Internet of Things (NB-IoT), BT Smart, Wi-Fi (such as Wi-Fi NAN, etc.), LTE-U, LTE-Direct, MuLTEfire, and relatively extended range wide area physical layer interface (PHY) such as Random Phase Multiple Access (RPMA), Ultra Narrow Band (UNB), Low Power Long Range (LoRa), Low Power Long Range Wide Area Network (LoRaWAN), Weightless) devices, or implemented in systems using 3G, 4G or 5G, cellular V2X or their further specific implementations and technologies.

[0038] The term "system on a chip" (SOC) is used herein to refer to a single integrated circuit (IC) chip that contains multiple resources and / or processors integrated on a single substrate. A single SOC may contain circuits for digital, analog, mixed-signal, and radio frequency functions. A single SOC may also include any number of general-purpose and / or special-purpose processors (digital signal processors, modem processors, video processors, etc.), memory blocks (e.g., ROM, RAM, flash memory, etc.), and resources (e.g., timers, voltage regulators, oscillators, etc.). The SOC may also include software for controlling the integrated resources and processors and for controlling peripheral devices.

[0039] The term "system-in-package" (SIP) may be used herein to refer to a single module or package that contains multiple resources, computing units, cores, and / or processors on two or more IC chips, substrates, or SOCs. For example, a SIP may include a single substrate on which multiple IC chips or semiconductor dies are stacked in a vertical configuration. Similarly, a SIP may include one or more multi-chip modules (MCMs) on which multiple ICs or semiconductor dies are packaged into a unified substrate. A SIP may also include multiple independent SOCs that are coupled together via high-speed communication circuits and tightly packaged, such as on a single motherboard or in a single wireless device. The proximity of the SOCs facilitates high-speed communication and the sharing of memory and resources.

[0040] Various embodiments described herein use the term "server" to refer to any computing device capable of functioning as a server, such as a primary exchange server, a web server, a mail server, a document server, a content server, or any other type of server. A server may be a dedicated computing device or a computing device that includes a server module (e.g., running an application that enables the computing device to operate as a server). A server module (e.g., a server application) may be a full-featured server module, or a lightweight or auxiliary server module (e.g., a lightweight or auxiliary server application) configured to provide synchronization services between dynamic databases on a receiver device. A lightweight server or auxiliary server may be a stripped-down version of server-type functionality that may be implemented on a receiver device to enable it to function as an Internet server (e.g., an enterprise email server) only to the extent necessary to provide the functionality described herein.

[0041] The phrase "head mounted device" and the acronym (HMD) are used herein to refer to any electronic display system that is wearable and presents at least some computer-generated images to a user. The HMD may present only computer-generated images, or a combination of computer-generated images and real-world images from the user's physical environment (i.e., what the user would see without wearing glasses). The HMD may enable the user to view the generated images in the context of a real-world scene. Non-limiting examples of head mounted devices include or may be included in helmets, glasses, virtual reality glasses, augmented reality glasses, electronic goggles, and other similar technologies / devices. The head mounted device may include various hardware elements, such as a processor, a memory, a display, one or more cameras (e.g., a world view camera, a gaze view camera, etc.), and a wireless interface for connecting to the Internet, a network, or other computing devices. In some embodiments, the head mounted device processor may be configured to execute or run an augmented reality software application.

[0042] In some embodiments, the head-mounted device may be an accessory for and / or receive information from a wireless device (e.g., a desktop computer, laptop, smartphone, tablet, etc.) with all or part of the processing being performed on a processor of the wireless device. Thus, in various embodiments, the head-mounted device may be configured to perform all processing locally on a processor in the head-mounted device, offload all primary processing to a processor in another computing device (e.g., a laptop, etc., that is present in the same room as the head-mounted device), or split the primary processing operations between a processor in the head-mounted device and a processor in the other computing device. In some embodiments, the processor in the other computing device may be a server in the "cloud" with which the processor in the head-mounted device or associated wireless device communicates via a network connection (e.g., a cellular network connection to the Internet).

[0043] Telepresence services are becoming increasingly powerful, allowing multiple objects to be composited into a single immersive environment so that participants in a meeting can navigate and interact more freely. The current Session Description Protocol (SDP) is limited in supporting rich description compositions and does not provide any tools to support immersive telepresence environments. A specific type of telepresence or teleconferencing implementation is an immersive three-dimensional group session. In an immersive three-dimensional group session, each participant computing device in the session can present a three-dimensional graphical display of the session on its respective display, so that a virtual reality (VR) view of the three-dimensional space of the immersive three-dimensional group session is presented to each participant user in the immersive three-dimensional group session. Three-dimensional objects (such as avatars, characters, etc.) representing participants of the immersive three-dimensional group session can be viewed by each participant and appear to move within the three-dimensional space of the immersive three-dimensional group session.

[0044] Immersive Teleconferencing and Remote Terminal Telepresence (ITT4RT) is a standard being developed to facilitate immersive multi-computing device virtual reality (VR) videoconferencing, such as immersive 3D group sessions. ITT4RT use cases include compositing of captured VR video, e.g. from a conference room, with other content, such as two-dimensional (2D) video slides. The work item description associated with ITT4RT indicates that ITT4RT is working towards enabling scenarios with two-way audio and one-way immersive video, e.g., a remote single user wearing an HMD participating in a conference will send audio and optionally 2D video (e.g., presentations, screen sharing, and / or capture of the user itself).

[0045] The complexity of VR scenes presents challenges to traditional Session Description Protocol (SDP) signaling and can quickly become unmanageable, such as when many remote users are sharing and compositing their own content in a scene. SDP is simply not designed to carry scene description information. Various embodiments provide solutions to these problems by providing an embodiment solution based on scene description to support compositing and overlays. Various embodiments provide methods and devices for signaling a description of a scene using media components that may come from different parties. In various embodiments, the scene description may be signaled via SDP during a Session Initiation Protocol (SIP) session establishment. In various embodiments, the scene description may be linked to other media streams in the session, for example, to utilize them as textures, overlays, etc. in an immersive conference scene.

[0046] A scene graph is a directed acyclic graph, usually just a simple tree structure, that represents an object-based hierarchy of scene geometry. The leaf nodes of the graph represent geometric primitives, such as polygons. Each node in the graph contains a pointer to its child nodes. Among other things, a child node can be a set of other nodes, geometric elements, transformation matrices, etc. Spatial transformations are attached to the nodes of the graph and represented by transformation matrices. This scene graph structure has the advantage of reducing processing complexity, such as when traversing the graph for rendering. An example operation simplified by the graph representation is a culling operation, in which the branch of the graph is removed from the processing (called detail level culling) if the space of the parent node is considered invisible or irrelevant to the rendering of the current view frustum. The scene graph can include various types of nodes, such as visual output nodes, audio source nodes, graphics output nodes, shared content nodes, etc. As a specific example, a graphics output node can define a three-dimensional object to be output in the three-dimensional space defined by the scene graph.

[0047] Graphics Library (GL) Transmission Format (TF) (glTF) 2.0 (glTF 2.0) is a new standard developed by Khronos for implementing physically based rendering. glTF 2.0 provides a compact and low-level representation of a scene graph. glTF 2.0 provides a flat hierarchy of scene graph representations to simplify processing. glTF 2.0 scene graphs are represented in JavaScript Object Notation (JSON) to facilitate integration in network environments. The glTF 2.0 specification aims to eliminate redundancy in the representation and provide efficient indexing of different objects in the scene graph. The Moving Picture Experts Group (MPEG) is developing extensions to glTF 2.0 to add support for real-time media, scene updates, and other features.

[0048] In various embodiments, a scene graph may enable synthesis of a scene (also referred to as a space, such as a three-dimensional space) for an immersive presentation (e.g., an immersive three-dimensional group session). In some embodiments, synthesis may be performed at a call server, a main picture multimedia resource function (MRF), a multipoint communication unit (MCU), a remote presentation application server, etc. Alternatively, in some embodiments, a designated computing device participating in a conference may be responsible for creating an initial scene graph and sharing the scene graph with all other parties in the call (e.g., an immersive three-dimensional group session). The computing device may be a device that creates the primary VR content, such as a computing device that performs VR capture in a conference room. In some embodiments, each computing device participating in a conference (e.g., an immersive three-dimensional group session) may contribute one or more nodes to the scene graph. In some embodiments, each node may identify or be assigned its associated transformation (e.g., in the form of a matrix, or separate translation and rotation operations) to appropriately place the node in the scene (or space), for example, to appropriately place the node in a three-dimensional space.

[0049] In some embodiments, each computing device participating in a conference call (e.g., an immersive 3D group session) may offer to send and receive a scene graph via a session-level attribute. In some embodiments, the offer may indicate one or more owned graph output nodes owned by the computing device sending the offer. As an example, each computing device participating in a conference call (e.g., an immersive 3D group session) may offer to send and receive a scene graph via the following session-level attribute, which is given in an Augmented Backus-Naur Form (ABNF) grammar:

[0050] Session-Description = “a=scene-description:” SP mime-type [SP uri][SP sent-nodes] CRLF

[0051] mime-type = "mime-type:" byte-string

[0052] sent-nodes = “nodes-owned=1*(byte-string “;”)

[0053] uri = "websocket-uri:" URI.

[0054] In some embodiments, the uniform resource indicator (URI) parameter may be a WebSocket URI for a data channel over which the scene graph will be shared and updated. Alternatively, the application media session may be used with a protocol identifier, such as the following protocol identifier TCP / WSS / SD (Transmission Control Protocol / Websocket Security / Session Description).

[0055] In some embodiments, a WebSocket URI may be provided according to the syntax and offer / answer negotiation defined in Internet Engineering Task Force (IETF) Request for Comments (RFC) 8124. An example pattern for such a WebSocket URI configuration may be:

[0056] m=application 50000 TCP / WSS / SD *

[0057] a=setup:passive

[0058] a=connection:new

[0059] a=websocket-uri:wss: / / mrf.operator.com / call / 21323asd23

[0060] a=mime-type:model / gltf+json

[0061] a=nodes-owned: node12,node13,node14

[0062] In various embodiments, the scene graph references media streams from a conference session that are used as components of nodes in a scene (e.g., a three-dimensional space). An example is a video stream of a conference participant to be displayed in a rectangular area in a three-dimensional (3D) scene (also referred to as 3D space). For example, the following URI format can be used:

[0063] url=”rtp: / / ” fqdn_or_ip “ / ” call_id “ / ” ssrc “ / ” mid

[0064] In this URI format, "fqdn_or_ip" represents the domain name or Internet Protocol (IP) address of the MRF or SIP proxy that manages the call (e.g., an immersive 3D group session). If no MRF or SIP proxy manages the call (e.g., an immersive 3D group session), "fqdn_or_ip" can represent the domain name or IP address of the SIP address of the host of the call (e.g., an immersive 3D group session). "Call_id" provides a unique identifier for the current call or conference (e.g., the current immersive 3D group session). "Ssrc" represents the synchronization source of the owner / sending participant of the media stream. Finally, "mid" represents the media session identifier provided in the SDP. Other forms of addressing may be defined, such as a Uniform Resource Name (URN).

[0065] In some embodiments, when WebRTC is used, the session establishment protocol can be left to the application. To this end, some implementations rely on SIP over WebSocket. However, other protocols can be used to establish and describe calls (e.g., immersive 3D group sessions).

[0066] Some embodiments may use a scene graph as an entry point for a conference call (e.g., an immersive 3D group session). In such an embodiment, all participants share a scene graph document that sets up the 3D scene (or 3D space) at the start of the call (e.g., an immersive 3D group session). The scene graph will define graph nodes for each participant and identify the components that each participant needs to have media streams for. This can be done through a central call server (such as an MRF) to which all conference participants are connected. The MRF can be configured to update the scene graph during the call (e.g., an immersive 3D group session), such as adding new nodes or removing nodes for newly joined call participants or participants leaving the call.

[0067] In some embodiments, a participant computing device can join a WebRTC conference using a link to a web page provided by a call server. The call server can provide the participant computing device with a web page and a scene graph file that sets the initial / default arrangement of call participants and materials in 3D space (e.g., each participant will be assigned a visual node, an audio source node, and possibly a node for graphics and other shared content). Each participant computing device can add or modify the nodes it owns in the scene graph. Media streams that provide components for nodes in the scene graph can be streamed using WebRTC. These streams can be exchanged directly or through a server, such as a media proxy server.

[0068] In some embodiments, computing devices participating in an ITT4RT conference can establish direct peer-to-peer WebSocket channels with each other, or connections can be provided to all parties by the MRF. In such an embodiment, the WebSocket channel can use a text box format. In a scene (e.g., in three-dimensional space), node names can be unique and can be declared in the SDP to ensure that there are no naming conflicts for nodes provided by different computing devices in the call. In such an embodiment, nodes in the scene graph can reference external media streams, such as other media streams declared in the SDP. In such an embodiment, the receiver can shield nodes from certain computing devices during the rendering process, such as based on user input.

[0069] In some embodiments, by default, the MRF can be the owner of the master scene graph, which is the computing device that sets the coordinate system and synthesizes all other nodes in it. In some embodiments, the MRF can also be the computing device that defines the main camera in the scene (or space). In some embodiments, in the absence of a centralized MRF, the computing device in the call can select a computing device to provide the master scene graph, such as by selecting a computing device that provides VR content or the organizer of the call. In various embodiments, the overlay can be a 2D or 3D object placed in the scene (or space). In some embodiments, the geometry of the overlay and its texture can be defined by the node corresponding to the overlay object. A simple example is a set of slides played in a rectangular area displayed in a VR scene (or VR space). In this example, the geometry can be a rectangle and the texture can come from a video media stream. The rectangle can be placed in the scene (or space). For viewport-related overlays, the position of the rectangle can be locked relative to the camera direction.

[0070] Figure 1A An example of a communication system 100 suitable for implementing various embodiments is shown. The communication system 100 may be a 5G NR network, or any other suitable network, such as an LTE network.

[0071] The communication system 100 may include a heterogeneous network architecture including a core network 140 and various wireless devices (also referred to as user equipment (UE) computing devices) (shown as wireless devices 120a-120e in FIG. 1 ). The communication system 100 may also include multiple base stations (shown as BS 110a, BS 110b, BS 110c, and BS 110d) and other network entities. A base station is an entity that communicates with a wireless device (wireless device or UE computing device) and may also be referred to as a NodeB, NodeB, LTE Evolved NodeB (eNB), Access Point (AP), Radio Head, Transmit Receive Point (TRP), New Radio Base Station (NR BS), 5G NodeB (NB), Next Generation NodeB (gNB), etc. Each base station may provide communication coverage for a specific geographic area. In 3GPP, the term "cell" may refer to a coverage area of ​​a base station, a base station subsystem serving the coverage area, or a combination thereof, depending on the context in which the term is used.

[0072] The base stations 110a-110d may provide communication coverage for macro cells, pico cells, femto cells, other types of cells, or a combination thereof. A macro cell may cover a relatively large geographic area (e.g., several kilometers in radius) and may allow unrestricted access by wireless devices with service subscriptions. A pico cell may cover a relatively small geographic area and may allow unrestricted access by wireless devices with service subscriptions. A femto cell may cover a relatively small geographic area (e.g., a home) and may allow restricted access by wireless devices associated with the femto cell (e.g., wireless devices in a closed subscriber group (CSG)). A base station for a macro cell may be referred to as a macro BS. A base station for a pico cell may be referred to as a pico BS. A base station for a femto cell may be referred to as a femto BS or a home BS. In Figure 1A In the example shown, base station 110a may be a macro BS for macro cell 102a, base station 110b may be a pico BS for pico cell 102b, and base station 110c may be a femto BS for femto cell 102c. Base stations 110a-110d may support one or more (e.g., three) cells. The terms “eNB”, “base station”, “NR BS”, “gNB”, “TRP”, “AP”, “Node B”, “5G NB”, and “cell” may be used interchangeably herein.

[0073] In some examples, the cell may not be fixed, and the geographic area of ​​the cell may move depending on the location of the mobile base station. In some examples, the base stations 110a-110d may be interconnected with each other and with one or more other base stations or network nodes (not shown) in the communication system 100 through various types of backhaul interfaces (such as direct physical connections, virtual networks, or combinations thereof) using any suitable transport network.

[0074] The base stations 110a-110d may communicate with the core network 140 via a wired or wireless communication link 126. The wireless devices 120a-120e (UE computing devices) may communicate with the base stations 110a-110d via a wireless communication link 122.

[0075] The wired communication link 126 can use a variety of wired networks (e.g., Ethernet, television cable, telephone, fiber optic, and other forms of physical network connections) that can use one or more wired communication protocols, such as Ethernet, Point-to-Point Protocol, High-Level Data Link Control (HDLC), Advanced Data Communications Control Protocol (ADCCP), and Transmission Control Protocol / Internet Protocol (TCP / IP).

[0076] The communication system 100 may also include a relay station (e.g., relay BS 110d). A relay station is an entity that can receive data transmissions from an upstream station (e.g., a base station or a wireless device) and send data transmissions to a downstream station (e.g., a wireless device or a base station). A relay station may also be a wireless device that can relay transmissions of other wireless devices. In the example shown in FIG. 1 , a relay station 110d may communicate with a macro base station 110a and a wireless device 120d to facilitate communication between the base station 110a and the wireless device 120d. A relay station may also be referred to as a relay base station, a relay base station, a relay, etc.

[0077] The communication system 100 may be a heterogeneous network including different types of base stations, such as macro base stations, pico base stations, femto base stations, relay base stations, etc. These different types of base stations may have different transmit power levels, different coverage areas, and different impacts on interference in the communication system 100. For example, a macro base station may have a higher transmit power level (e.g., 5 watts to 40 watts), while a pico base station, a femto base station, and a relay base station may have a lower transmit power level (e.g., 0.1 watt to 2 watts).

[0078] The network controller 130 may be coupled to a set of base stations and may provide coordination and control for these base stations. The network controller 130 may communicate with the base stations via a backhaul. The base stations may also communicate with each other directly or indirectly, for example, via a wireless or wired backhaul.

[0079] Wireless devices (UE computing devices) 120a, 120b, 120c may be dispersed throughout the communication system 100, and each wireless device may be fixed or mobile. A wireless device may also be referred to as an access terminal, UE, terminal, mobile station, subscriber unit, station, etc.

[0080] The macro base station 110a may communicate with the communication network 140 via a wired or wireless communication link 126. The wireless devices 120a, 120b, 120c may communicate with the base stations 110a-110d via wireless communication links 122. The core network 140 may be connected to other devices, such as a call server 150 (e.g., a multimedia resource function (MRF), a multipoint communication unit (MCU), a telepresence application server, etc.). In this manner, via connection to the core network 140, the call server 150 may make telepresence services such as immersive teleconferencing and telepresence for remote terminals (ITT4RT) services available to the wireless devices 120a, 120b, 120c, 120d (e.g., from the core network 140 via link 126 and from the base stations 110a-110d via link 122). Although shown as being outside the core network 140, the call server 150 may be part of the core network 140 itself.

[0081] The wireless communication links 122, 124 may include multiple carrier signals, frequencies or frequency bands, each of which may include multiple logical channels. The wireless communication links 122 and 124 may utilize one or more radio access technologies (RATs). Examples of RATs that may be used in wireless wide area network (WWAN) wireless communication links 122, 124 within the paid communication system 100 include 3GPP LTE, 3G, 4G, 5G (e.g., NR), GSM, code division multiple access (CDMA), wideband code division multiple access (WCDMA), microwave world interoperability access (WiMAX), time division multiple access (TDMA), and other mobile phone communication technology cellular RATs. Examples of RATs that may be used in wireless local area network (WLAN) wireless communication links 122, 124 within the local communication system 100 include medium-range wireless protocols such as Wi-Fi, LTE-U, LTE-Direct, LAA, MuLTEfire, and relatively short-range RATs such as ZigBee, Bluetooth, and Bluetooth Low Energy (LE).

[0082] Some wireless networks (e.g., LTE) use orthogonal frequency division multiplexing (OFDM) on the downlink and single carrier frequency division multiplexing (SC-FDM) on the uplink. OFDM and SC-FDM divide the system bandwidth into multiple (K) orthogonal subcarriers, which are also commonly referred to as tones, bins, etc. Each subcarrier can be modulated with data. In general, modulation symbols are sent in the frequency domain using OFDM and in the time domain using SC-FDM. The spacing between adjacent subcarriers can be fixed, and the total number of subcarriers (K) can depend on the system bandwidth. For example, the spacing of the subcarriers can be 15 kHz, and the minimum resource allocation (called a "resource block") can be 12 subcarriers (or 180 kHz). Therefore, the nominal fast Fourier transform (FFT) size can be equal to 128, 256, 512, 1024, or 2048 for system bandwidths of 1.25, 2.5, 5, 10, or 20 megahertz (MHz), respectively. The system bandwidth can also be divided into subbands. For example, one subband may cover 1.08 MHz (eg, 6 resource blocks), and there may be 1, 2, 4, 8, or 16 subbands for system bandwidths of 1.25, 2.5, 5, 10, or 20 MHz, respectively.

[0083] Although the description of some embodiments may use terms and examples associated with LTE technology, various embodiments may be applicable to other wireless communication systems, such as new radio (NR) or 5G networks. NR may use OFDM with a cyclic prefix (CP) on the uplink (UL) and downlink (DL), and include support for half-duplex operation using time division duplex (TDD). A single component carrier bandwidth of 100 MHz may be supported. An NR resource block may span 12 subcarriers with a subcarrier bandwidth of 75 kHz in a duration of 0.1 ms. Each radio frame may consist of 50 subframes with a length of 10 ms. Thus, each subframe may have a length of 0.2 ms. Each subframe may indicate a link direction (ie, DL or UL) for data transmission, and the link direction of each subframe may be switched dynamically. Each subframe may include DL / UL data and DL / UL control data. Beamforming may be supported, and the beam direction may be dynamically configured. Multiple-input multiple-output (MIMO) transmission with precoding may also be supported. MIMO configurations in the DL can support up to eight transmit antennas with multi-layer DL transmissions of up to eight streams and up to two streams per wireless device. Multi-layer transmissions with up to 2 streams per wireless device can be supported. Aggregation of multiple cells can be supported with up to eight serving cells. Alternatively, NR can support different air interfaces in addition to OFDM-based air interfaces.

[0084] Some wireless devices may be considered machine type communication (MTC) or evolved or enhanced machine type communication (eMTC) wireless devices. MTC and eMTC wireless devices include, for example, robots, drones, remote devices, sensors, meters, monitors, location tags, etc. that can communicate with a base station, another device (e.g., a remote device), or some other entity. A wireless node may provide, for example, a connection to or to a network (e.g., a wide area network such as the Internet or a cellular network) via a wired or wireless communication link. Some wireless devices may be considered Internet of Things (IoT) devices or may be implemented as NB-IoT (narrowband Internet of Things) devices. The wireless devices 120a-e may be included inside a housing that houses components of the wireless device (such as a processor component, a memory component, a similar component, or a combination thereof).

[0085] Generally, any number of communication systems and any number of wireless networks can be deployed in a given geographic area. Each communication system and wireless network can support a specific radio access technology (RAT) and can operate on one or more frequencies. RAT can also be referred to as radio technology, air interface, etc. Frequency can also be referred to as carrier, frequency channel, etc. Each frequency can support a single RAT in a given geographic area to avoid interference between communication systems of different RATs. In some cases, NR or 5G RAT networks can be deployed.

[0086] In some implementations, two or more wireless devices 120a-e (e.g., shown as wireless device 120a and wireless device 120e) may communicate directly (e.g., without using base stations 110a-110d as an intermediary to communicate with each other) using one or more sidelink channels 124. For example, the wireless devices 120a-e may communicate using peer-to-peer (P2P) communication, device-to-device (D2D) communication, vehicle-to-everything (V2X) protocols (which may include vehicle-to-vehicle (V2V) protocols, vehicle-to-infrastructure (V2I) protocols, or similar protocols), mesh networks, or similar networks, or combinations thereof. In this case, the wireless devices 120a-e may perform scheduling operations, resource selection operations, and other operations described elsewhere herein as being performed by the base station 110a.

[0087] Figure 1B A head mounted device 172 is shown which may be configured in accordance with various embodiments. Figure 1A and Figure 1B ,exist Figure 1AIn the example shown, the head mounted device 172 may be a specific implementation of a user equipment computing device (e.g., UE 120c, 120d, 120e). The head mounted device 172 includes a frame 152, two optical lenses 154, and a processor 156, which is communicatively coupled to an outward-facing world view image sensor / camera 158, an inward-facing gaze view sensor / camera 160, a sensor array 162, a memory 164, and a communication circuit 166. In various embodiments, the communication circuit 166 may support one or more RATs to support the referenced Figure 1A Communication between various devices described in system 100. In some embodiments, head mounted device 172 may include capacitive touch sensing circuits along arms 180 of the frame or in nose bridge 182 of head mounted device 172. In some embodiments, head mounted device 172 may also include sensors for monitoring physical conditions (e.g., location, motion, acceleration, orientation, altitude, etc.). Sensors may include any or all of gyroscopes, accelerometers, magnetometers, magnetic compasses, altimeters, odometers, and pressure sensors. Sensors may also include various biosensors (e.g., heart rate monitors, body temperature sensors, carbon sensors, oxygen sensors, etc.) for collecting information related to the environment and / or user conditions. Sensors may also be external to head mounted device 172 and paired or grouped with head mounted device 172 via a wired or wireless connection (e.g., Bluetooth®, etc.).

[0088] In some embodiments, the processor 156 may also be communicatively coupled to an image presentation device 168 (e.g., an image projector), which may be embedded in the arm portion 180 of the frame 152 and configured to project an image onto the optical lens 154. In some embodiments, the image presentation device 168 may include a light emitting diode (LED) module, an optical channel, an averaging lens, an optical display, a folding mirror, or other well-known projector or head-mounted display components. In some embodiments (e.g., those embodiments in which the image presentation device 168 is not included or used), the optical lens 154 may be or may include a see-through or partially see-through electronic display. In some embodiments, the optical lens 154 includes an image generating element, such as a see-through organic light emitting diode (OLED) display element or a liquid crystal on silicon (LCOS) display element. In some embodiments, the optical lens 154 may include independent left-eye display elements and right-eye display elements. In some embodiments, the optical lens 154 may include or serve as a light guide for transmitting light from the display element to the wearer's eyes.

[0089] The outward-facing or world-view image sensor / camera 158 may be configured to capture images of the real world from the user's physical environment and send corresponding image data to the processor 156. The processor 156 may combine the real-world images with computer-generated images or virtual objects (VOs) to generate an augmented scene (or space) and present the augmented scene (or space) on the electronic display or optical lens 154 of the head mounted device 172.

[0090] Inward-facing or gaze perspective sensor / camera 160 may be configured to acquire image data from the user's eyes or facial structures surrounding the user's eyes.

[0091] Various embodiments may be implemented on a number of single-processor and multi-processor computer systems, including a system-on-chip (SOC) or a system-in-package (SIP). Figure 2 An example computing system or SIP 200 architecture is shown that may be used in a wireless device (UE computing device) implementing various embodiments.

[0092] refer to Figure 1A , Figure 1B and Figure 2, the illustrated example SIP 200 includes two SOCs 202 and 204, a clock 206, a voltage regulator 208, and one or more wireless transceivers 266, which are configured to send and receive wireless communications to and from network wireless devices (such as base station 110a) and / or other wireless devices (e.g., wireless devices 120a-e) via antennas (not shown). In some embodiments, the first SOC 202 can operate as a central processing unit (CPU) of a wireless device, which executes instructions of a software application by performing arithmetic, logic, control, and input / output (I / O) operations specified by the instructions. In some embodiments, the second SOC 204 can operate as a dedicated processing unit. For example, the second SOC 204 can operate as a dedicated 5G processing unit, which is responsible for managing high-capacity, high-speed (e.g., 5 Gbps, etc.) or very high frequency short wavelength (e.g., 28 GHz millimeter wave spectrum, etc.) communications. In some embodiments, the wireless transceiver 266 may be a wireless transceiver configured to support peer-to-peer (P2P) communication, device-to-device (D2D) communication, vehicle-to-everything (V2X) protocol (which may include vehicle-to-vehicle (V2V) protocol, vehicle-to-infrastructure (V2I) protocol, or similar protocol), Bluetooth communication, Wi-Fi communication, etc. In some embodiments, the wireless transceiver 266 may each be connected to the first SOC 202 and / or the second SOC 204 may be connected to each of the one or more wireless transceivers 266 through various physical connections 267 (also referred to as interconnects, buses, etc.) (such as peripheral component interconnect express (PCIe) connections, universal serial bus (USB) connections, high-speed inter-chip (HSIC) connections, Ethernet connections, etc.).

[0093] The first SOC 202 may include a digital signal processor (DSP) 210, a modem processor 212, a graphics processor 214, an application processor 216, one or more coprocessors 218 (e.g., vector coprocessors) connected to one or more processors, memory 220, custom circuits 222, system components and resources 224, an interconnect / bus module 226, one or more temperature sensors 230, a thermal management unit 232, and a thermal power envelope (TPE) component 234. The second SOC 204 may include a 5G modem processor 252, a power management unit 254, an interconnect / bus module 264, a plurality of millimeter wave transceivers 256, a memory 258, and various additional processors 260, such as an application processor, a packet processor, etc.

[0094] Each processor 210, 212, 214, 216, 218, 252, 260 may include one or more cores, and each processor / core may perform operations independently of other processors / cores. For example, the first SOC 202 may include a processor that executes a first type of operating system (such as FreeBSD, LINUX, OS X, etc.) and a processor that executes a second type of operating system (such as Microsoft WINDOWS 10). In addition, any or all of the processors 210, 212, 214, 216, 218, 252, 260 may be included as part of a processor cluster architecture (such as a synchronous processor cluster architecture, an asynchronous or heterogeneous processor cluster architecture, etc.).

[0095] The first SOC 202 and the second SOC 204 may include various system components, resources, and custom circuits for managing sensor data, analog-to-digital conversion, wireless data transmission, and for performing other specialized operations, such as decoding data packets and processing encoded audio and video signals for presentation in a Web browser. For example, the system components and resources 224 of the first SOC 202 may include power amplifiers, voltage regulators, oscillators, phase-locked loops, peripheral bridges, data controllers, memory controllers, system controllers, access ports, timers, and other similar components for supporting processors and software clients running on wireless devices. The system components and resources 224 and / or custom circuits 222 may also include circuits for interfacing with peripheral devices, such as cameras, electronic displays, wireless communication devices, external memory chips, and the like.

[0096] The first SOC 202 and the second SOC 204 can communicate via an interconnect / bus module 250. The various processors 210, 212, 214, 216, 218 can be interconnected to one or more memory elements 220, system components and resources 224, and custom circuits 222 and thermal management units 232 via an interconnect / bus module 226. Similarly, the processor 252 can be interconnected to a power management unit 254, a millimeter wave transceiver 256, a memory 258, and various additional processors 260 via an interconnect / bus module 264. The interconnect / bus modules 226, 250, 264 can include reconfigurable logic gate arrays and / or implement bus architectures (e.g., CoreConnect, AMBA, etc.). Communications can be provided by advanced interconnects, such as high-performance networks on chip (NoC).

[0097] The first SOC 202 and / or the second SOC 204 may also include input / output modules (not shown) for communicating with resources external to the SOC, such as a clock 206 and a voltage regulator 208. Resources external to the SOC (e.g., clock 206, voltage regulator 208) may be shared by two or more internal SOC processors / cores.

[0098] In addition to the example SIP 200 discussed above, various embodiments may be implemented in a wide variety of computing systems that may include a single processor, multiple processors, multi-core processors, or any combination thereof.

[0099] Figure 3 An example of a software architecture 300 is shown, which includes a radio protocol stack for user and control planes in wireless communications between a base station 350 (e.g., base station 110a) and a wireless device (UE computing device) 320 (e.g., wireless devices 120a-120e, 172, 200). Figures 1A to 3 , the wireless device 320 may implement the software architecture 300 to communicate with a base station 350 of a communication system (e.g., 100). In various embodiments, the layers in the software architecture 300 may form a logical connection with the corresponding layers in the software of the base station 350. The software architecture 300 may be distributed in one or more processors (e.g., processors 212, 214, 216, 218, 252, 260). Although illustrated with respect to one radio protocol stack, in a multi-SIM (subscriber identity module) wireless device, the software architecture 300 may include multiple protocol stacks, each of which may be associated with a different SIM (e.g., two protocol stacks are associated with two SIMs, respectively, in a dual SIM wireless communication device). Although described below with reference to LTE communication layers, the software architecture 300 may support any of a variety of standards and protocols for wireless communication, and / or may include additional protocol stacks that support any of a variety of standard and protocol wireless communications.

[0100] The software architecture 300 may include a non-access stratum (NAS) 302 and an access stratum (AS) 304. The NAS 302 may include functions and protocols to support packet filtering, security management, mobility control, session management, and services and signaling between a SIM of a wireless device (such as SIM 204) and its core network 140. The AS 304 may include functions and protocols to support communication between a SIM (e.g., SIM 204) and an entity (e.g., a base station) of a supported access network. Specifically, the AS 304 may include at least three layers (Layer 1, Layer 2, and Layer 3), each of which may include various sublayers.

[0101] In the user and control planes, layer 1 (L1) of the AS 304 may be a physical layer (PHY) 306, which may oversee functions that enable transmission and / or reception over an air interface. Examples of such physical layer 306 functions may include cyclic redundancy check (CRC) attachment, codec blocks, scrambling and descrambling, modulation and demodulation, signal measurement, MIMO, etc. The physical layer may include various logical channels, including a physical downlink control channel (PDCCH) and a physical downlink shared channel (PDSCH).

[0102] In the user and control planes, Layer 2 (L2) of AS 304 may be responsible for the link between wireless device 320 and base station 350 through physical layer 306. In various embodiments, Layer 2 may include a medium access control (MAC) sublayer 308, a radio link control (RLC) sublayer 310, and a packet data convergence protocol (PDCP) 312 sublayer, each of which forms a logical connection that terminates at base station 350.

[0103] In the control plane, Layer 3 (L3) of AS 304 may include a radio resource control (RRC) sublayer 3. Although not shown, software architecture 300 may include additional Layer 3 sublayers, as well as various upper layers above Layer 3. In various embodiments, RRC sublayer 313 may provide functionality including broadcasting system information, paging, and establishing and releasing RRC signaling connections between wireless devices 320 and base stations 350.

[0104] In various embodiments, the PDCP sublayer 312 may provide uplink functions including multiplexing between different radio bearers and logical channels, sequence number addition, handover data processing, integrity protection, encryption, and header compression. In the downlink, the PDCP sublayer 312 may provide functions including in-sequence delivery of data packets, duplicate data packet detection, integrity verification, decryption, and header decompression.

[0105] In the uplink, the RLC sublayer 310 may provide segmentation and concatenation of upper layer data packets, retransmission of lost data packets, and automatic repeat request (ARQ). In the downlink, while the functions of the RLC sublayer 310 may include reordering data packets to compensate for out-of-order reception, reassembling upper layer data packets, and ARQ.

[0106] In the uplink, the MAC sublayer 308 may provide functions including multiplexing between logical and transport channels, random access procedures, logical channel priority, and hybrid ARQ (HARQ) operations. In the downlink, MAC layer functions may include intra-cell channel mapping, demultiplexing, discontinuous reception (DRX), and HARQ operations.

[0107] While the software architecture 300 may provide functionality for sending data over a physical medium, the software architecture 300 may also include at least one host layer 314 to provide data transmission services to various applications in the wireless device 320. In some embodiments, the application-specific functionality provided by the at least one host layer 314 may provide an interface between the software architecture and the general purpose processor 206.

[0108] In other embodiments, the software architecture 300 may include one or more higher logical layers (e.g., transport layer, session layer, presentation layer, application layer, etc.) that provide host layer functionality. For example, in some embodiments, the software architecture 300 may include a network layer (e.g., IP layer) where the logical connection terminates at a packet data network (PDN) gateway (PGW). In some embodiments, the software architecture 300 may include an application layer where the logical connection terminates at another device (e.g., an end-user device, a server, etc.). In some embodiments, the software architecture 300 may also include a hardware interface 316 between the physical layer 306 and communication hardware (e.g., one or more radio frequency (RF) transceivers) in the AS 304.

[0109] Figure 4 A process flow diagram of an example method 400 for supporting an immersive experience in a teleconference or telepresence session according to various embodiments is shown. Figures 1A to 4 , method 400 may be implemented by a processor (such as 156, 212, 216, 252, or 260) of a wireless device (such as wireless devices 120a-120e, 172, 200, 320). In various embodiments, the operations of method 400 may be performed by a processor of a wireless device that is one of a plurality of participant computing devices in a teleconferencing or telepresence session (such as an immersive 3D group session).

[0110] In block 402, the process may perform operations to indicate a proposal to send and / or receive a scene graph as part of a Session Initiation Protocol (SIP) setup for a session. In some embodiments, the proposal may indicate a graph output node owned by the wireless device. In some embodiments, the session may be a WebRTC session.

[0111] In block 404, the processor may perform operations to receive a session description protocol (SDP) for the session, the session description protocol (SDP) indicating an address of a data channel on which a scene graph of the session will be shared. In some embodiments, the scene graph may define one or more nodes assigned to each computing device participating in the session. In various embodiments, the nodes may reference other media streams from other computing devices participating in the session, and the other media streams may be overlaid in the session. In various embodiments, the one or more nodes assigned to each computing device participating in the session may include one or more visual nodes, audio source nodes, graphics nodes, or shared content nodes.

[0112] In block 406 , the processor may perform operations to download the scene graph via a data channel.

[0113] In block 408, the processor may perform operations to receive and present the session according to the scene graph for presentation on an image presentation device (e.g., 168). Receiving and presenting the session may include receiving a streaming service of the session according to the scene graph and presenting the session on a display. In some embodiments, the display of the session may be presented on an HMD (e.g., 172), a video conference room, a stereo display, or any other image and sound presentation device, and receiving and presenting may include outputting the session to a user via the image and sound presentation device.

[0114] In block 410, the processor may perform operations to add a node assigned to the wireless device to a scene graph or to modify a node among one or more nodes assigned to the wireless device in the scene graph.

[0115] Figure 5 A process flow diagram of an example method 500 for supporting an immersive experience in a teleconference or telepresence session according to various embodiments is shown. Figures 1A to 5 , method 500 may be implemented by a processor (such as 156, 212, 216, 252, or 260) of a wireless device (such as wireless devices 120a-120e, 172, 200, 320) and / or a call server (such as call server 150). In various embodiments, the operations of method 500 may be performed by a processor of a host computing device that hosts a teleconference or telepresence session. In some embodiments, the host computing device may be a separate call server, such as an MRF, an MCU, a teleconferencing application server, etc. In some embodiments, the host computing device may be a wireless device that is one of a plurality of computing devices participating in a teleconference or telepresence session (such as an immersive three-dimensional group session). In various embodiments, the operations of method 500 may be performed in conjunction with the operations of method 400.

[0116] In block 502, a processor of a host computing device may perform operations to receive an indication of a proposal to send and / or receive a scene graph as part of a session initiation protocol (SIP) setup for a session from a computing device participating in the session. In some embodiments, the session may be a WebRTC session. In some embodiments, the received indication may indicate a graph output node owned by the wireless device sending the proposed indication.

[0117] In block 504, the processor of the host computing device may perform operations to generate a session description protocol (SDP) for the session indicating an address of a data channel over which a scene graph for the session will be shared. In some embodiments, the scene graph may define one or more nodes assigned to each computing device participating in the session.

[0118] In block 506 , the processor of the host computing device may perform operations to send the SDP to computing devices participating in the session.

[0119] In block 508, the processor of the host computing device may perform operations to send the session and scene graph to the computing devices participating in the session. As an example, the host computing device may perform operations to stream the session and scene graph to the computing devices participating in the session. In various embodiments, the computing devices participating in the session may be an HMD (e.g., 172), a television in a conference room, a stereo display, or any other image and sound presentation device, and sending the session and / or scene graph may include sending the session and / or scene graph so that the computing devices participating in the session can output the session to a user on the image and sound presentation device.

[0120] Figure 6 FIG. 6 shows an arrangement of a scene graph document 600 in glTF 2.0 suitable for various embodiments. Figures 1A to 6 In various embodiments, the scene graph document 600 may be part of a scene graph. The scene graph document 600 may include JSON elements that define node hierarchies, material descriptions, lighting information, camera information, and the like. The scene graph document 600 may include a binary file (BIN) that defines geometry information (such as vertices and indices), animation information (such as keyframes), skin information (such as inverse binding matrices, and the like). The scene graph document 600 may include a GL shading library (glSL) file that defines shader information. The scene graph document 600 may include various other types of files, such as portable network graphics (PNG) files, joint photographic experts group (JPEG) files, and the like, thereby defining other information of the scene graph, such as textures, and the like.

[0121] Figure 7 The structure of a scene graph suitable for use in various embodiments is shown. Figures 1A to 7In various embodiments, a scene graph may include a plurality of nodes. Each node may include child nodes that describe various components of the node, such as camera views, mesh information, lighting information, etc. A scene graph may define a hierarchical relationship between attributes to render a mesh of nodes, such as attachments, skins, buffer views, buffer information, materials, techniques, programs, shaders, textures, images, and samplers, such as Figure 7 shown.

[0122] Figure 8 is a call flow diagram illustrating operations for supporting an immersive experience in a teleconference or telepresence session according to various embodiments. Figures 1A to 8 , operations between call participants (e.g., call participants #1 and #2) and a call server may include, in operation 1), the participants (e.g., call participants #1 and #2) join a WebRTC conference using a provided web page link. In operation 2), the call server provides the participants (e.g., call participants #1 and #2) with a web page and a scene graph file that sets an initial / default arrangement of call participants (e.g., call participants #1 and #2) and materials in a 3D space. For example, each participant will be assigned a visual node, an audio source node, and possibly also nodes for graphics and other shared content. In operation 3), each participant (e.g., call participants #1 and #2) can add or modify the nodes it owns in the scene graph. In operation 4), media streams that provide components for nodes in the scene graph can be streamed using WebRTC. These streams can be exchanged directly or through a server, such as a media proxy server.

[0123] Fig. 9 is a process flow diagram illustrating a method 900 for providing an immersive three-dimensional group session according to various embodiments. Figures 1A to 9 , method 900 may be implemented by a processor (such as 156, 212, 216, 252, or 260) of a wireless device (such as wireless devices 120a-120e, 172, 200, 320). In various embodiments, the operations of method 900 may be performed by a processor of a wireless device that is one of a plurality of participant computing devices in a teleconference or telepresence session (such as an immersive three-dimensional group session). In various embodiments, the operations of method 900 may be combined with the operations of method 400 ( Figure 4 ) and / or 500 ( Figure 5 ) to perform any one or more of the operations.

[0124] In block 902, a processor may perform operations including receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph includes at least an own graphics output node assigned to be controlled by a wireless device and corresponding other graphics output nodes assigned to be controlled by each of the other participant computing devices in a plurality of participant computing devices. For example, the scene graph may be as follows: Figure 8 The scene diagram shown.

[0125] Portions of the scene graph received in block 902 may be assigned to each participant computing device in the immersive 3D group session, such as to one or more nodes of each respective participant computing device. By parsing the scene graph, the processor of the wireless device may determine the nodes to be controlled by the wireless device. Allocating the graphics output nodes on a per-participant computing device basis may enable each participant computing device to control at least one graphics output node. The graphics output node may include components that define a 3D object to be output in a 3D space defined by the scene graph. For example, the 3D object may include an avatar, a character, or other representation, and the components of the graphics output node may define how the 3D object is presented in the 3D space of the immersive 3D group session. In this manner, by controlling the components of its assigned respective graphics output node, the processor of the wireless device may control how other participant computing devices view one or more 3D objects associated with the wireless device, such as an avatar, a character, or other representation selected by a user of the wireless device in the immersive 3D group session. Similarly, by controlling the components of their respective assigned graphics output nodes, other participant computing devices may control how the user of the wireless device views one or more 3D objects associated with each of them, such as an avatar, a character, or other representation in the immersive 3D group session.

[0126] In block 904, the processor may perform operations including controlling a component of an own graphics output node relative to a three-dimensional space of an immersive three-dimensional group session. In some embodiments, a user may adjust the own graphics node, such as moving the placement of the own graphics node in a scene independently of the position of the wireless device. For example, a user may select a location in a scene where his or her three-dimensional object (e.g., avatar, character, etc.) is displayed in the scene. In some embodiments, the component controlling the own graphics output node may include a component controlling the own graphics output node based at least in part on the determined location of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session. In some embodiments, the location of the wireless device may be determined in the three-dimensional space, such as a location relative to a center point, a grid coordinate, or other reference to the three-dimensional space. Since some immersive three-dimensional group sessions may support movement of participants within the three-dimensional space of the immersive three-dimensional group session, determining the location of the wireless device may support presenting an object of the graphics output node of the wireless device in a correct relative position. As a specific example, the lighting and / or camera components of the own graphics output node assigned to the wireless device may be controlled to reflect the lighting and / or camera components of the current location of the wireless device in the three-dimensional space of the immersive three-dimensional group session.

[0127] In addition to controlling components of the owned graphics output node based on position in box 904, components of the owned graphics output node can also be controlled based on other metrics such as the orientation of the wireless device. For example, position movement and orientation changes relative to a common reference point and a common reference plane as indicated by an accelerometer of the wireless device can be used to determine the position and orientation of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session. As a specific example, the lighting and / or camera components of the owned graphics output node assigned to the wireless device can be controlled to reflect the current position and current orientation of the wireless device in the three-dimensional space of the immersive three-dimensional group session. Controlling components of the owned graphics output node based on the current position and / or current orientation may be useful in a specific implementation in which the wireless device is a head-mounted device so that the movements of the user of the head-mounted device can be visually conveyed to other participants in the immersive three-dimensional group session.

[0128] In block 906, the processor may perform operations including sending a component of an owned graphics output node in a first media stream to other participant computing devices in the plurality of participant computing devices. The component may be sent directly to the other participant computing devices in the media stream and / or sent to the other participant computing devices via a call server (e.g., 150), such as an MRF, an MCU, a teleconferencing application server, etc. In this manner, the processor of the wireless device may provide a media stream for its corresponding owned graphics output node to the other participant computing devices and thereby control how the other participant computing devices present the graphics output node.

[0129] In block 908, the processor may perform operations including a component that receives other graphical output nodes in a media stream from each of the other computing devices in the plurality of participant computing devices. The component may receive in a media stream directly from each of the other participant computing devices and / or via a call server (e.g., 150), such as an MRF, an MCU, a teleconferencing application server, etc. In this manner, the processor of the wireless device may receive media streams assigned to graphical output nodes of the other participant computing devices.

[0130] In block 910, the processor may perform operations including presenting an immersive three-dimensional group session on a display of the wireless device based at least in part on components of the own graphics output node and components of other graphics output nodes. For example, components of the media stream of the wireless device and components of the media streams of other participant computing devices may be overlaid with components of the other media streams to generate a display output of a three-dimensional space of the immersive three-dimensional group session.

[0131] The processor may continuously perform the operations of blocks 904 , 906 , 908 , and 910 during the immersive three-dimensional group session to present the immersive three-dimensional group session.

[0132] Fig.10 is a process flow diagram illustrating a method 1000 for providing an immersive three-dimensional group session according to various embodiments. Figures 1A to 10 , method 1000 may be implemented by a processor (such as 156, 212, 216, 252, or 260) of a wireless device (such as wireless devices 120a-120e, 172, 200, 320). In various embodiments, the operations of method 1000 may be performed by a processor of a wireless device that is one of a plurality of participant computing devices in a teleconference or telepresence session (such as an immersive three-dimensional group session). In various embodiments, the operations of method 1000 may be combined with method 400 ( Figure 4 )、500( Figure 5 ) and / or 900 ( Fig. 9As a specific example, the operations of method 1000 may be performed as part of the operations of block 910 of method 900 to present an immersive three-dimensional group session.

[0133] In block 1002, a processor may perform operations including receiving a scene graph update including an indication of a new participant computing device for an immersive 3D group session and an indication of a new graphical output node assigned to be controlled by the new participant computing device. In response to a new participant joining the immersive 3D group session, the scene graph update may be sent by the host computing device. In some embodiments, the scene graph update may be received directly from another participant computing device and / or via a call server (e.g., 150), such as an MRF, MCU, teleconferencing application server, etc.

[0134] In block 1004, the processor may perform operations including a component that receives the new graphical output node in a second media stream from the new participant computing device. The component may be received in the media stream directly from the new participant computing device and / or via a call server (e.g., 150), such as an MRF, MCU, teleconferencing application server, etc. In this manner, the processor of the wireless device may receive a media stream assigned to the graphical output node of the newly added participant computing device.

[0135] In block 1006, the processor may perform operations including presenting the immersive 3D group session on a display of the wireless device based at least in part on components of the own graphics output node, components of other graphics output nodes, and components of the new graphics output node. For example, components of the media stream of the wireless device and components of the media streams of other participant computing devices, including a second media stream of the newly added participant computing device, may be superimposed with components of the other media streams to generate a display output of the 3D space of the immersive 3D group session.

[0136] Various embodiments may be implemented on various wireless network devices, examples of which are Fig.11 1100, which is used as a network element of a communication network, such as a call server (e.g., call server 150). Such a network computing device may include at least Fig.11 Reference Figures 1A to 11, the network computing device 1100 may generally include a processor 1101 coupled to a volatile memory 1102 and a large capacity non-volatile memory (such as a disk drive 1103). The network computing device 1100 may also include a peripheral memory access device, such as a floppy disk drive, a compact disk (CD) or a digital video disk (DVD) drive 1106 coupled to the processor 1101. The network computing device 1100 may also include a network access port 1104 (or interface) coupled to the processor 1101 for establishing a data connection with a network such as the Internet and / or a local area network coupled to other system computers and servers. The network computing device 1100 may include one or more antennas 1107 for sending and receiving electromagnetic radiation, which may be connected to a wireless communication link. The network computing device 1100 may include additional access ports, such as USB, Firewire, Thunderbolt, etc., for coupling to peripheral devices, external memory or other devices.

[0137] Various embodiments may be implemented on various wireless devices (eg, wireless devices 120a-120e, 172, 200, 320), examples of which are described in Fig.12 1200 is shown in the form of a smartphone 1200. Figures 1A to 12 , the smartphone 1200 may include a first SOC 202 (e.g., a SOC-CPU) coupled to a second SOC 204 (e.g., a 5G-capable SOC). The first SOC 202 and the second SOC 204 may be coupled to internal memories 1206 and 1216, a display 1212, and a speaker 1214. In addition, the smartphone 1200 may include an antenna 1204 for sending and receiving electromagnetic radiation, which may be connected to a wireless data link and / or a cellular telephone transceiver 266 coupled to one or more processors in the first SOC 202 and / or the second SOC 204. The smartphone 1200 also typically includes a menu selection button or rocker switch 1220 for receiving user input.

[0138] The typical smartphone 1200 also includes a sound coding / decoding (CODEC) circuit 1210, which digitizes the sound received from the microphone into data packets suitable for wireless transmission and decodes the received sound data packets to generate analog signals provided to the speaker to generate sound. In addition, one or more processors in the first SOC 202 and the second SOC 204, the wireless transceiver 266 and the CODEC 1210 may include a digital signal processor (DSP) circuit (not shown separately).

[0139] The processors of the wireless network computing device 1100 and the smartphone 1200 may be any programmable microprocessor, microcomputer or multi-processor chip or chips that can be configured by software instructions (applications) to perform various functions, including the functions of the various embodiments described below. In some wireless devices, multiple processors may be provided, such as one processor within the SOC 204 dedicated to wireless communication functions and one processor within the SOC 202 dedicated to running other applications. Typically, software applications may be stored in the memory 1206, 1216 before they are accessed and loaded into the processor. The processor may include internal memory sufficient to store the application software instructions.

[0140] As used in this application, the terms "component", "module", "system", etc. are intended to include computer-related entities, such as but not limited to hardware, firmware, a combination of hardware and software, software, or software in execution, which are configured to perform specific operations or functions. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As an example, both an application running on a wireless device and a wireless device can be referred to as a component. One or more components can reside in a process and / or an execution thread, and a component can be located on a processor or core and / or distributed between two or more processors or cores. In addition, these components can be executed by various non-temporary computer-readable media having various instructions and / or data structures stored thereon. Components can communicate through local and / or remote processes, function or procedure calls, electronic signals, data packets, memory read / writes, and other known network, computer, processor, and / or process-related communication methods.

[0141] Many different cellular and mobile communication services and standards are available or expected in the future, all of which can be implemented and benefit from various embodiments. Such services and standards include, for example, the Third Generation Partnership Project (3GPP), Long Term Evolution (LTE) system, third generation wireless mobile communication technology (3G), fourth generation wireless mobile communication technology (4G), fifth generation wireless mobile communication technology (5G), Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), 3GSM, General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA) system (e.g., cdmaOne, CDMA1020TM), Enhanced Data Rates for GSM Evolution (EDGE), Advanced Mobile Phone System (AMPS), Digital AMPS (IS-136 / TDMA), Evolution Data Optimized (EV-DO), Digital Enhanced Cordless Telecommunications (DECT), Worldwide Interoperability for Microwave Access (WiMAX)), Wireless Local Area Network (WLAN), Wi-Fi Protected Access I and II (WPA, WPA2), and Integrated Digital Enhanced Network (iDEN). For example, each of these technologies involves the transmission and reception of voice, data, signaling and / or content messages. It should be understood that any reference to terms and / or technical details related to a single telecommunication standard or technology is only for illustrative purposes and is not intended to limit the scope of the claims to a specific communication system or technology unless specifically stated in the claims.

[0142] The various embodiments shown and described are provided as examples only to illustrate various features of the claims. However, the features shown and described with respect to any given embodiment are not necessarily limited to the associated embodiment and can be used or combined with other embodiments shown and described. In addition, the claims are not intended to be limited to any one exemplary embodiment. For example, one or more operations of methods 400, 500, 900, and / or 1000 can be replaced or combined with one or more operations of methods 400, 500, 900, and / or 1000.

[0143] The above method description and process flow chart are only provided as illustrative examples, and are not intended to require or imply that the operation of various embodiments must be performed in the proposed order. As will be appreciated by those skilled in the art, the order of operations in the aforementioned embodiments can be performed in any order. For example, the words "afterwards", "subsequently", "next", etc. are not intended to limit the order of operations, and these words are used to guide the reader to complete the description of the method. In addition, any reference to a claim element in the singular form (for example, by using the article "one", "an" or "the") should not be interpreted as limiting the element to the singular.

[0144] Various illustrative logic blocks, modules, components, circuits and algorithmic operations described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits and operations have been described above generally according to their functions. Whether such functions are implemented as hardware or software depends on specific applications and on the design constraints imposed by the overall system. Technicians can implement the functions in different ways for each specific application, but such embodiment decisions should not be interpreted as causing a departure from the scope of the claims.

[0145] The hardware for implementing the various exemplary logics, logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein can be implemented or executed with a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. The general-purpose processor can be a microprocessor, but alternatively, the processor can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of receiver smart objects, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, a combination of one or more microprocessors and a DSP core, or any other such configuration. Alternatively, some operations or methods can be performed by circuits specific to a given function.

[0146] In one or more embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a non-transitory computer-readable storage medium or a non-transitory processor-readable storage medium. The operation of the method or algorithm disclosed herein may be embodied in a processor-executable software module or processor-executable instruction, which may reside on a non-transitory computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable storage medium may be any storage medium accessible by a computer or processor. By way of example and not limitation, such non-transitory computer-readable or processor-readable storage media may include RAM, ROM, EEPROM, flash memory, CD-ROM or other optical disk storage device, disk storage device or other magnetic storage smart object, or any other medium that can be used to store the required program code in the form of an instruction or data structure and can be accessed by a computer. As used herein, disks and optical disks include compact disks (CDs), laser optical disks, optical optical disks, digital versatile disks (DVDs), floppy disks, and Blu-ray disks, where disks typically reproduce data magnetically, and optical disks reproduce data optically using lasers. The above combinations are also included within the scope of non-transitory computer-readable and processor-readable media. In addition, the operations of the method or algorithm may reside on a non-transitory processor-readable storage medium and / or computer-readable storage medium as one or any combination or set of codes and / or instructions, which may be incorporated into a computer program product.

[0147] The above description of the disclosed embodiments is provided to enable those skilled in the art to implement or use the claims. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the claims. Therefore, the present disclosure is not intended to be limited to the embodiments shown herein, but should be given the broadest scope consistent with the following claims and the principles and novel features disclosed herein.

Claims

1. A method for supporting an immersive three-dimensional group conversation performed by a processor of a wireless device, comprising: receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph indicates control of at least an own graphics output node by the wireless device and control of corresponding one or more other graphics output nodes by each of one or more participant-computing devices; and The immersive three-dimensional group session is output for presentation on a display of the wireless device based at least in part on the one or more components of the own graphics output node and one or more other components of the one or more other graphics output nodes.

2. The method according to claim 1, wherein: The scene graph for the immersive 3D group session references an external media stream for each of the one or more other graphics output nodes.

3. The method according to claim 2, further comprising: A session description protocol (SDP) for the immersive 3D group session is received, wherein the SDP declares an external media stream for each of the one or more other graphics output nodes.

4. The method according to claim 3, wherein: The SDP indicates an address of a data channel over which the scene graph is to be shared; and Receiving the scene graph includes downloading the scene graph via the data channel.

5. The method according to claim 1, further comprising: One or more components of the owning graphics output node are controlled relative to the three-dimensional space of the immersive three-dimensional group session.

6. The method according to claim 5, further comprising: sending one or more components of the owning graphical output node in a first media stream to other participant computing devices of the one or more participant computing devices; as well as Components for receiving other graphical output nodes in a media stream from each other participant-computing device of the plurality of participant-computing devices.

7. The method according to claim 5, wherein: Controlling one or more components of the own graphics output node relative to the three-dimensional space of the immersive three-dimensional group session includes controlling one or more components of the own graphics output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session.

8. The method according to claim 7, wherein: Controlling one or more components of the own graphics output node based at least in part on a determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session includes controlling one or more components of the own graphics output node based at least in part on a determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session and a determined orientation of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session.

9. The method according to claim 1, wherein: The immersive 3D group session is a WebRTC session.

10. A wireless device comprising: A processor configured with processor-executable instructions to: receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph indicates control of at least an own graphics output node by the wireless device and control of corresponding one or more other graphics output nodes by each of one or more participant-computing devices; and The immersive three-dimensional group session is output for presentation on a display of the wireless device based at least in part on the one or more components of the own graphics output node and one or more other components of the one or more other graphics output nodes.

11. The wireless device of claim 10, wherein: The scene graph for the immersive 3D group session references an external media stream for each of the one or more other graphics output nodes.

12. The wireless device of claim 11, wherein: The processor is also configured with processor-executable instructions to: A session description protocol (SDP) for the immersive 3D group session is received, wherein the SDP declares an external media stream for each of the one or more other graphics output nodes.

13. The wireless device of claim 12, wherein: The SDP indicates an address of a data channel over which the scene graph is to be shared; and Receiving the scene graph includes downloading the scene graph via the data channel.

14. The wireless device of claim 10, wherein: The processor is also configured with processor-executable instructions to: One or more components of the owning graphics output node are controlled relative to the three-dimensional space of the immersive three-dimensional group session.

15. The wireless device of claim 14, wherein: The processor is also configured with processor-executable instructions to: sending one or more components of the owning graphical output node in a first media stream to other participant computing devices of the one or more participant computing devices; as well as Components for receiving other graphical output nodes in a media stream from each other participant-computing device of the plurality of participant-computing devices.

16. The wireless device of claim 14, wherein: The processor is also configured with processor-executable instructions to control one or more components of the own graphics output node relative to the three-dimensional space of the immersive three-dimensional group session by controlling one or more components of the own graphics output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session.

17. The wireless device of claim 16, wherein: The processor is also configured with processor-executable instructions to control one or more components of the own graphics output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session and the determined orientation of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session.

18. The wireless device of claim 10, wherein: The immersive 3D group session is a WebRTC session.

19. A non-transitory processor-readable medium having stored thereon processor-executable instructions configured to cause a processor of a wireless device to perform operations comprising: receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph indicates control of at least an own graphics output node by the wireless device and control of corresponding one or more other graphics output nodes by each of one or more participant-computing devices; and The immersive three-dimensional group session is output for presentation on a display of the wireless device based at least in part on the one or more components of the own graphics output node and one or more other components of the one or more other graphics output nodes.

20. The non-transitory processor-readable medium of claim 19, wherein: A scene graph of an immersive 3D group session scene graph for the immersive 3D group session references an external media stream of each of the one or more other graphics output nodes.

21. The non-transitory processor-readable medium of claim 20, wherein: The stored processor-executable instructions are configured to cause a processor of the wireless device to perform operations, the operations further comprising: A session description protocol (SDP) for the immersive 3D group session is received, wherein the SDP declares an external media stream for each of the one or more other graphics output nodes.

22. The non-transitory processor-readable medium of claim 21 , wherein: The SDP indicates an address of a data channel over which the scene graph is to be shared; and Receiving the scene graph includes downloading the scene graph via the data channel.

23. The non-transitory processor-readable medium of claim 19, wherein: The stored processor-executable instructions are configured to cause a processor of the wireless device to perform operations, the operations further comprising: One or more components of the owning graphics output node are controlled relative to the three-dimensional space of the immersive three-dimensional group session.

24. The non-transitory processor-readable medium of claim 23, wherein: The stored processor-executable instructions are configured to cause a processor of the wireless device to perform operations, the operations further comprising: sending one or more components of the owning graphical output node in a first media stream to other participant computing devices of the one or more participant computing devices; and Components for receiving other graphical output nodes in a media stream from each other participant-computing device of the plurality of participant-computing devices.

25. The non-transitory processor-readable medium of claim 23, wherein: The stored processor-executable instructions are configured to cause the processor of the wireless device to perform operations so that one or more components of the own graphics output node are controlled relative to the three-dimensional space of the immersive three-dimensional group session, including: controlling one or more components of the own graphics output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session.

26. The non-transitory processor-readable medium of claim 25, wherein: The stored processor-executable instructions are configured to cause the processor of the wireless device to perform operations so that one or more components of the autonomous graphics output node are controlled at least in part based on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session, including: controlling one or more components of the autonomous graphics output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session and the determined orientation of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session.

27. The non-transitory processor-readable medium of claim 19, the immersive three-dimensional group session is a Web Real-Time Communications (WebRTC) session.

28. A wireless device comprising: means for receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph indicates control of at least an own graphics output node by the wireless device and control of corresponding one or more other graphics output nodes by each of one or more participant computing devices; and Means for outputting the immersive three-dimensional group session for presentation on a display of the wireless device based at least in part on one or more components of the own graphics output node and one or more other components of the one or more other graphics output nodes.

29. The wireless device of claim 28, wherein: The scene graph for the immersive 3D group session references an external media stream for each of the one or more other graphics output nodes.

30. The wireless device of claim 29, further comprising: Means for receiving a Session Description Protocol (SDP) for the immersive 3D group session, wherein the SDP declares an external media stream for each of the one or more other graphics output nodes.

Images