Operating system level management of group communication sessions

Abstract

A device for providing operating system managed group communication sessions can include a memory and at least one processor. The at least one processor can be configured to receive, by an operating system level process executing on the device and from an application process executing on the device, a request to initiate a group session between a user associated with the device and a further user. The at least one processor can be further configured to identify, by the operating system level process, a further device associated with the further user. The at least one processor can be further configured to initiate, by the operating system level process, the group session with the user via the further device. The at least one processor can be further configured to manage, by the operating system level process, the group session.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 083,010, filed September 24, 2020, the disclosure of which is hereby incorporated by reference in its entirety. TECHNICAL FIELD

[0003] This description relates generally to communication management, including operating system level management of group communication sessions. BACKGROUND

[0004] Augmented reality technology aims to bridge the gap between virtual and physical environments by providing an augmented physical environment that is augmented with electronic information. Thus, the electronic information appears to be part of the physical environment as perceived by a user when the user views the physical environment via the augmented reality technology. BRIEF DESCRIPTION OF DRAWINGS

[0005] The features of the subject technology are set forth in the appended claims. However, for purpose of explanation, several implementations of the subject technology are set forth in the following figures.

[0006] Figure 1 An example system architecture including various electronic devices in which the subject system can be implemented is shown in accordance with one or more implementations.

[0007] Figure 2 A block diagram showing example features of an electronic device in accordance with one or more implementations is shown.

[0008] Figure 3 A flow diagram showing an example process for operating system level management of group communication sessions in accordance with one or more implementations is shown.

[0009] Figure 4 An example process flow for providing a delegation token to a group for shared access to content protected by a digital rights management (DRM) system in accordance with one or more implementations is shown.

[0010] Figure 5 An example process flow for sharing group access to content protected by a DRM system in accordance with one or more implementations is shown.

[0011] Figure 6 An example process flow for removing a user from a group that has shared access to content protected by a DRM system in accordance with one or more implementations is shown.

[0012] Figure 7 An electronic system with which one or more implementations of the subject technology can be implemented is shown. DETAILED DESCRIPTION

[0013] The detailed description is set forth with specific reference to the figures. Discussions of specific embodiments are intended to provide a description of various configurations of the subject technology and are not intended to represent the only configurations in which the subject technology can be practiced. The accompanying drawings are incorporated in and constitute a part of the detailed description. The detailed description includes specific details to provide a thorough understanding of the subject technology. However, the subject technology is not limited to the specific details set forth in this document. One or more other embodiments of the subject technology are possible using one or more of the other

[0014] A physical environment refers to a physical world that people are able to sense and / or interact with without the aid of electronic devices. A physical environment can include physical features such as physical surfaces or physical objects. For example, a physical environment corresponds to a physical park that includes physical trees, physical buildings, and physical people. People are able to directly sense and / or interact with a physical environment such as through sight, touch, hearing, taste, and smell. In contrast, an extended reality (XR) environment refers to a fully or partially simulated environment that people sense and / or interact with via electronic devices. For example, an XR environment can include augmented reality (AR) content, mixed reality (MR) content, virtual reality (VR) content, etc. In the case of an XR system, a subset of people’s physical motions, or representations thereof, are tracked, and in response, one or more characteristics of one or more virtual objects simulated in the XR system are adjusted in a manner consistent with at least one physical law. For example, an XR system can detect head movements and, in response, adjust graphical content and an acoustic field presented to the people in a manner similar to how such views and sounds would change in a physical environment. As another example, an XR system can detect movements of an electronic device (e.g., a mobile phone, a tablet, a laptop, etc.) presenting an XR environment and, in response, adjust graphical content and an acoustic field presented to the people in a manner similar to how such views and sounds would change in a physical environment. In some cases (e.g., for accessibility reasons), an XR system can adjust characteristics of graphical content in an XR environment in response to representations of physical motions (e.g., voice commands).

[0015] There are many different types of electronic systems that enable people to sense and / or interact with various XR environments. Examples include head-mounted systems, projection-based systems, heads-up displays (HUDs), vehicle windshields with integrated display capability, windows with integrated display capability, displays formed as lenses designed to be placed on a person's eyes (e.g., similar to contact lenses), headphones / earphones, speaker arrays, input systems (e.g., wearable or handheld controllers with or without haptic feedback), smartphones, tablets, and desktop / laptop computers. A head-mounted system can have an integrated opaque display and one or more speakers. Alternatively, a head-mounted system can be configured to accept an external opaque display (e.g., a smartphone). A head-mounted system can incorporate one or more imaging sensors to capture images or video of the physical environment, and / or one or more microphones to capture audio of the physical environment. Rather than an opaque display, a head-mounted system can have a transparent or translucent display. The transparent or translucent display can have a medium through which light representative of an image is directed to a person's eyes. The display can utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser scanning light source, or any combination of these technologies. The medium can be an optical waveguide, hologram medium, optical combiner, optical reflector, or any combination thereof. In some implementations, a transparent or translucent display can be configured to selectively become opaque. Projection-based systems can employ retinal projection technology that projects graphical images onto a person's retinas. Projection systems can also be configured to project virtual objects into the physical environment, for example, as a hologram or on a physical surface.

[0016] Implementations of the subject technology described herein provide operating system-level management of group communication sessions between user devices. A group communication session (also referred to herein as a group session) allows multiple users to be present (e.g., simultaneously present) in a common XR environment via their respective devices. The users can be physically co-located in the same physical location and / or one or more of the users can be located in physically remote locations. By providing operating system-level management of group communication sessions, the subject system allows a single application to access operating system-level group communication functionality through application programming interface (API) calls to the operating system-level group session framework, rather than each application having to individually develop its own group session functionality to access the operating system-level group communication functionality. Thus, the subject system allows any single application to provide users with access to a group session for accessing / interacting with the application's content without the application having to develop its own group session functionality / framework and without the application having to consider whether the application was designed for multi-user access.

[0017] By moving group session management to the operating system level, the subject systems can also provide tighter control around user authentication, digital rights management, and user access rights / permissions during a group communication session. For example, the operating system level framework can authenticate each user participating in a group session as well as the user's avatar for the group session, such as to confirm that the avatar's appearance generally matches the user's physical appearance. In this way, a user participating in a group session with another user can be confident that the other user is who they claim to be (via user authentication) and that the other user is the person the user intended to participate in the group session with (via avatar authentication).

[0018] The operating system level group session framework can also enforce DRM controls around content being viewed and / or shared in a group session, such as by preventing copying / redistribution of the content, controlling the number of concurrent users that have access to the content, and the like. In this way, content providers can ensure that their content rights are maintained when the content is shared among users in a group session. Similarly, the operating system level group session framework can provide a user, e.g., a session initiating user, with discrete control over user access rights / permissions during a group session. In this way, the operating system level group session framework allows the session initiating user to decide which other users can interact with, modify, remove, and / or view any particular application content during the session, as well as which other users can record and / or later annotate any portion of the group session.

[0019] Figure 1 An example system architecture 100 including various electronic devices in which the subject systems can be implemented is shown in accordance with one or more implementations. However, not all of the depicted components can be used in all implementations, and one or more implementations can include additional or different components than those shown in the figure. Variations in the arrangement and type of the components can be made without departing from the spirit or ambit of the claims as set forth herein. Additional components, different components, or fewer components can be provided.

[0020] The system architecture 100 includes electronic device 105, handheld electronic device 104, electronic device 110, electronic device 115, and server 120. For explanatory purposes, the system architecture 100 is shown in Figure 1 including electronic device 105, handheld electronic device 104, electronic device 110, electronic device 115, and server 120; however, the system architecture 100 can include any number of electronic devices and any number of servers or data centers including multiple servers.

[0021] The electronic device 105 can be implemented, for example, as a tablet device, a smartphone, and / or a head-mounted portable system (e.g., worn by the user 101). The electronic device 105 includes a display system capable of presenting a visualization of an extended reality environment to a user. The electronic device 105 can be powered with a battery and / or another power source. In one example, the display system of the electronic device 105 provides a stereoscopic presentation of an extended reality environment to a user, enabling a three-dimensional visual display of a particular scene rendering. In one or more implementations, instead of or in addition to utilizing the electronic device 105 to access an extended reality environment, a user can use a handheld electronic device 104, such as a tablet, a watch, a mobile device, etc.

[0022] The electronic device 105 can include one or more cameras, such as camera 150 (e.g., a visible light camera, an infrared camera, etc.). In addition, the electronic device 105 can include various sensors 152, including but not limited to cameras, image sensors, touch sensors, microphones, inertial measurement units (IMUs), heart rate sensors, temperature sensors, depth sensors (e.g., lidar sensors, radar sensors, sonar sensors, time-of-flight sensors, etc.), GPS sensors, Wi-Fi sensors, near-field communication sensors, radio frequency sensors, etc. In addition, the electronic device 105 can include hardware elements that can receive user input, such as hardware buttons or switches. User input detected by such sensors and / or hardware elements correspond to various input modalities, for example, for initiating a group communication session (group session) within an application. For example, such input modalities can include, but are not limited to, facial tracking, eye tracking (e.g., gaze direction), hand tracking, posture tracking, biometric readings (e.g., heart rate, pulse, pupil dilation, respiration, temperature, electroencephalogram, olfaction), recognizing speech or audio (e.g., specific hot words), and activating buttons or switches, etc.

[0023] In one or more implementations, the electronic device 105 can be communicatively coupled to a base device, such as electronic device 110 and / or electronic device 115. Generally, such base devices can include more computational resources and / or available power compared to the electronic device 105. In one example, the electronic device 105 can operate in various modes. For example, the electronic device 105 can operate in an independent mode independent of any base device. When the electronic device 105 operates in the independent mode, the number of input modalities can be constrained by power and / or processing limitations of the electronic device 105, such as available battery power of the device. In response to the power limitations, the electronic device 105 can deactivate certain sensors within the device itself to conserve battery power and / or free up processing resources.

[0024] The electronic device 105 can also operate in a wireless tethered mode (e.g., connected with a base device via a wireless connection) to work in conjunction with a given base device. The electronic device 105 can also operate in a connected mode in which the electronic device 105 is physically connected to a base device (e.g., via a cable or some other physical connector) and can utilize power resources provided by the base device (e.g., in the case that the base device charges the electronic device 105 while physically connected).

[0025] When the electronic device 105 operates in the wireless tethered mode or the connected mode, processing user inputs and / or rendering at least a portion of an extended reality environment can be offloaded to the base device, thereby reducing the processing burden on the electronic device 105. For example, in one implementation, the electronic device 105 works in conjunction with the electronic device 110 or the electronic device 115 to generate an extended reality environment that includes physical and / or virtual objects that enable different forms of interaction (e.g., visual, auditory, and / or physical or haptic interaction) between a user and the generated extended reality environment in a real-time manner. In an example, the electronic device 105 provides a rendering of a scene corresponding to the extended reality environment that can be perceived by a user and interacted with in a real-time manner, such as a host environment for a group session with additional users. Additionally, as part of presenting the rendered scene, the electronic device 105 can provide sound and / or haptic or tactile feedback to the user. The content of the given rendered scene can depend on available processing power, network availability and capacity, available battery power, and current system workload.

[0026] The network 106 can communicatively (directly or indirectly) couple, for example, the electronic device 105, the electronic device 110, and / or the electronic device 115 with each other and / or the server 120. In one or more implementations, the network 106 can be an interconnected network of devices that can include or be communicatively coupled to the Internet.

[0027] The electronic device 110 can include a touchscreen and can be, for example, a smartphone that includes a touchscreen, a portable computing device such as a laptop computer that includes a touchscreen, a companion device (e.g., a digital camera, a headset) that includes a touchscreen, a tablet device that includes a touchscreen, a wearable device (such as a watch, a wristband, etc.) that includes a touchscreen, any other appropriate device that includes, for example, a touchscreen, or any electronic device that has a touchpad. In one or more implementations, the electronic device 110 can not include a touchscreen, but can support touchscreen-like gestures, such as in an extended reality environment. In one or more implementations, the electronic device 110 can include a touchpad. In Figure 1In particular implementations, electronic device 110 is depicted as a mobile smartphone device having a touchscreen, by way of example. In one or more implementations, electronic device 110, handheld electronic device 104, and / or electronic device 105 can be and / or can include all or a portion of the electronic devices described below with respect to the electronic system described below with respect to FIG. 1. Figure 7 In one or more implementations, electronic device 110 can be another device, such as an Internet Protocol (IP) camera, a tablet, or a companion device such as an electronic stylus, etc.

[0028] Electronic device 115 can be, for example, a desktop computer, a portable computing device such as a laptop computer, a smartphone, a companion device (e.g., a digital camera, headphones), a tablet device, a wearable device such as a watch, a wristband, etc. In particular implementations, electronic device 115 is depicted as a desktop computer, by way of example. Electronic device 115 can be and / or can include all or a portion of the electronic system described below with respect to FIG. 1. Figure 1 Figure 7 In particular implementations, electronic device 115 is depicted as a desktop computer, by way of example. Electronic device 115 can be and / or can include all or a portion of the electronic system described below with respect to FIG. 1.

[0029] Server 120 can form all or a portion of a computer network or server group 130, such as in a cloud computing or data center implementation. For example, server 120 stores data and software, and includes specific hardware (e.g., processors, graphics processors, and other specialized or custom processors) for rendering and generating content for extended reality environments such as graphics, images, video, audio, and multimedia files. In one implementation, server 120 can function as a cloud storage server that stores any of the aforementioned extended reality content generated by the above-mentioned devices and / or server 120.

[0030] Figure 2 A block diagram illustrating various components that can be included in electronic device 105 is shown in accordance with aspects of the present disclosure. As Figure 2 ​As shown, the electronic device 105 can include one or more cameras such as camera 150 that capture images and / or video of a physical environment surrounding the electronic device, one or more sensors 152 that obtain environmental information (e.g., depth information) associated with the physical environment surrounding the electronic device 105. The sensors 152 can include depth sensors (e.g., time-of-flight sensors, infrared sensors, radar, sonar, lasers, etc.), one or more microphones, and / or other types of sensors for sensing the physical environment. In some implementations, the electronic device 105 also includes a communication circuit 208 for communicating with the electronic device 110, the electronic device 115, the server 120, and / or other devices and / or systems. The communication circuit 208 can include radio frequency (RF) communication circuitry for detecting radio frequency identification (RFID) tags, Bluetooth Low Energy (BLE) communication circuitry, other near-field communication (NFC) circuitry, WiFi communication circuitry, cellular communication circuitry, and / or other wired and / or wireless communication circuitry.

[0031] As shown, the electronic device 105 includes processing circuitry 204 (e.g., one or more processors and / or integrated circuits) and a memory 206. The memory 206 can store (e.g., temporarily or permanently) content generated and / or otherwise obtained by the electronic device 105. In some operational scenarios, the memory 206 can temporarily store images of the physical environment captured by the camera 150, depth information corresponding to images generated using a depth sensor of the sensors 152, for example, a mesh corresponding to the physical environment, virtual objects such as virtual objects generated by the processing circuitry 204 to include virtual content, and / or virtual depth information for the virtual objects. The memory 206 can store (e.g., temporarily or permanently) intermediate images and / or information generated by the processing circuitry 204 for combining images of the physical environment and virtual objects and / or virtual images to form (e.g., synthesize) images for display by the display 200, such as by compositing one or more virtual objects onto a pass-through video stream obtained from one or more of the cameras 150.

[0032] Memory 206 can store instructions or code for execution by processing circuitry 204, such as, for example, operating system code corresponding to an operating system installed on electronic device 105 and application code corresponding to one or more applications installed on electronic device 105. The operating system code, when executed, can correspond to one or more operating system-level processes, such as a process that supports group session management. The operating system-level processes can have administrative privileges (e.g., unrestricted access) to logical resources (e.g., file system resources) and / or physical resources (e.g., hardware resources such as camera 150, sensors 152, etc.), while the application code, when executed, can correspond to one or more application processes that have restricted access or no access to one or more of the logical and / or physical resources.

[0033] However, the application code can access one or more function calls of the operating system-level code, such as through an application programming interface (API) for group session management functionality. For example, the application code can access function calls for requesting initiation of a group communication session (group session), requesting termination of a group communication session, requesting addition of a user to an existing group communication session or removal of a user from an existing group communication session, etc. In this way, one or more installed applications can access group session functionality through the operating system-level processes without needing to separately develop a group session framework. Exemplary processes for an application initiating a group session via an operating system-level framework are discussed further below with respect to Figure 3 Exemplary processes for an application initiating a group session via an operating system-level framework are discussed further below with respect to

[0034] Figure 3 A flow diagram of an exemplary process 300 of operating system-level group session management in accordance with one or more implementations is shown. For explanatory purposes, process 300 is primarily described herein with reference to electronic device 105 of Figure 1 and Figure 2 However, process 300 is not limited to electronic device 105 of Figure 1 and Figure 2 and one or more other components of other suitable devices, including electronic device 110, electronic device 115, and / or server 120. Further for explanatory purposes, some of the blocks of process 300 are described herein as occurring in serial or in linear fashion. However, multiple blocks of process 300 can occur in parallel. Further, the blocks of process 300 need not be performed in the order shown and / or one or more of the blocks of process 300 need not be performed, and / or can be replaced by other operations.

[0035] The process 300 can be initiated when an operating system-level process executing on the electronic device 105 receives a request from an application process executing on the electronic device 105 to initiate a group session with another user (302). In one or more implementations, the electronic device 105 can be an electronic device that supports displaying XR content in an XR environment, such as a smartphone, a tablet device, or a head-mounted system worn by a user. For example, an application developer developing an application for such an electronic device 105 can have the option of opting into operating system-level management of access to group sessions. If the application developer opts into a particular application, users of the application can have access to one or more group session management functions from within the application, such as initiating a group session, terminating a group session, adding a user to a group session, and the like.

[0036] For example, a user of the application can be able to select a user interface element to initiate a group session with one or more other users (e.g., the user’s contacts), the user can be able to use a voice command to initiate a group session, or the user can be able to use generally any available input modality to initiate a group session from within the application. The application process can receive the request from the user input, and can then communicate the request to the operating system-level group session management process. The request communicated to the operating system-level process can include, for example, one or more user identifiers corresponding to one or more users with which the group session is being initiated.

[0037] The operating system-level process can identify one or more other electronic devices associated with the received user identifiers, such as the electronic device 104 (304). For example, the operating system-level process can transmit a request to a server, such as the server 120, to obtain one or more device identifiers corresponding to the one or more received user identifiers.

[0038] The operating system-level process can then initiate a group session with the other user via the other electronic device 104 of the other user (306). For example, the operating system-level process can authenticate the user and / or the other user and a representative of the user (as discussed further below), and if the other electronic device 104 supports displaying XR content in an XR environment, the operating system-level process can communicate with a peer operating system-level process of the electronic device 104 to initiate a group session within, for example, a host environment as specified by the initiating user of the electronic device 105 (as discussed further below).

[0039] In one or more implementations, the operating system-level process can authenticate the user of the electronic device 105, such as based on biometric information (iris scan, facial scan, fingerprint scan, biorhythm scan, etc.), and authenticate the representative that is being used by the user for the group session. For example, a user initiating a group session can be able to select (e.g., via an application process) an option to require that the user use an authenticated representative (e.g., a representative that has an appearance that conforms to the appearance of the corresponding user, rather than a non-conforming representative such as a clown) for the group session. Such an option can be desirable, for example, in a business setting or otherwise when a user wants to be able to visually verify the identity of the user with whom they are interacting. In such a case, the operating system-level process can obtain images, depth information, facial information from one or more cameras 150, and / or other information that can be used to authenticate that the visual appearance of the representative used by the user of the electronic device 105 generally conforms to the physical appearance of the user of the electronic device 105. For example, the operating system-level process can compare the representative and the user facial feature locations, sizes, etc., hair style / length / color, skin tone, eye color, or generally any attribute and / or characteristic that can be used to visually authenticate that the appearance of the representative conforms to the physical appearance of the user.

[0040] In one or more implementations, the operating system-level process can automatically generate a representative of the user when a representative is required to be authenticated. That is, instead of comparing the physical attributes of the user to the visual attributes of a representative created by the user, the operating system-level process can automatically scan the facial attributes and / or physical appearance of the user, and can generate a representative that visually conforms to the physical appearance of the user.

[0041] The operating system-level process of the electronic device 105 can then exchange signed certificates (e.g., using a hardware-bound private key) with the peer operating system-level process of the electronic device 104 that authenticate that the users of the electronic devices 104, 105 correspond to the user identifiers selected for the group session, and that the appearance of the representative selected by the user for the group session generally conforms to the physical appearance of the user.

[0042] In one or more implementations, authenticating the users of the electronic devices 104, 105 can be performed periodically throughout the group session, such as to confirm that the same users are still participating in the group session with the respective electronic devices 104, 105. Alternatively or additionally, if one or more of the electronic devices 104, 105 is a device that includes a sensor that can detect when a user has set down or removed the electronic device, such as a smartphone, tablet device, or head-mounted system, the user authentication can be maintained until the release or removal of the electronic device has been detected (e.g., by the hand of the user).

[0043] After authenticating the user and / or the user’s avatars, the operating system level process can determine an appropriate host environment for initiating the group session within. The host environment can be based on, for example, the user’s physical environment of the electronic device 105, the physical environment of the additional user of the electronic device 104, a mix of physical environments, or a virtual environment (e.g., a replication of the virtual environment and / or the additional physical environment). For example, the user of the electronic device 105 initiating the group session can be authorized to select the appropriate environment for hosting the group session.

[0044] In one or more implementations, the mix of each user’s physical environment can include a portion of each user’s physical environment with a mix / merge of environments / geometry in between. In one or more implementations, a host environment (e.g., a virtual environment, a replication of the physical environment, etc.) can be created and retained or saved, such that the user can subsequently have additional group sessions in the same host environment.

[0045] In the case where the group session is being hosted in an environment based on the user’s physical environment of the electronic device 105, the operating system level process of the electronic device 105 can coordinate display of pass-through video streams corresponding to the physical environment, for example, from one or more of the cameras 150, in conjunction with computer-generated avatars and / or application content corresponding to the additional user.

[0046] The operating system level process of the electronic device 105 can also provide information to the peer operating system level process executing on the electronic device 104 for generating a reconstruction of the physical environment of the user of the electronic device 105. The information can include, for example, previously generated and stored meshes and textures of objects in the physical environment, pass-through video streams (and any corresponding depth information) of the portion of the physical environment being viewed by the user of the electronic device 105, and previously stored video and / or images / textures / meshes of the portion of the physical environment not being viewed by the user of the electronic device 105 (e.g., behind the user that would be in the field of view of the user of the electronic device 104).

[0047] In one or more implementations, if the additional electronic device 104 does not support display of XR content in an XR environment (e.g., if the electronic device 105 is an XR device and the electronic device 104 is a mobile device or tablet that has not been updated to include XR functionality), the operating system level process can initiate the group session with an existing communication application on the electronic device 104, such as an existing video conferencing application. For example, the operating system level process can convert any XR content to video streams and can provide the video streams to the electronic device 104. Similarly, the operating system level process can convert received video streams (e.g., and any received corresponding depth information) to XR content that can be displayed to the user of the electronic device 105, and / or the operating system level process can display the received video streams in their original two-dimensional form.

[0048] After establishing the host environment for the group session, the operating system level process can receive application content from the application process (308). For example, the application process can transmit the application content (such as XR content or generally any content) and / or a declarative description of the application content to the operating system level process via inter-process communication (IPC).

[0049] The operating system level process can then manage the group session based at least in part on the received application content (310). For example, the operating system level process can determine an appropriate way to display the application content in the host environment of the group session (such as based on the host environment, the number of users in the group session, other application content being displayed in the group session, etc.). In this way, a developer of the application corresponding to the application process does not need to develop any separate frameworks to support group sessions, to support multiple concurrent users, and / or to support appropriate physical properties, geometry, shading, etc. for displaying the application content within an XR environment. For example, the operating system level process can coordinate displaying at least a portion of the application content with displaying at least a portion of a pass-through video feed, and can handle corresponding display properties of the application content, e.g., physical properties, geometry, shading, etc.

[0050] In one or more implementations, because the application process provides the application content to the operating system level process for display, the application process can be isolated from interaction / communication with the host environment of the group session and users within the group session (e.g., in addition to direct interaction with the application content). In this way, the operating system level process can prevent the application process from accessing pass-through video streams and / or information for reconstructing physical environments of users, as well as interactions between users during the group session, which protects the privacy of each of these users in the group session.

[0051] The operating system level process can further coordinate and / or synchronize with peer operating system level processes executing on additional electronic devices 104. For example, in addition to coordinating / synchronizing user interactions with displayed content, the operating system level process can also coordinate display of content, such as the application content, and the aforementioned host environment. For example, the operating system level process can manage conflict resolution with respect to user interactions with the application content, such as by prioritizing interactions by the user that initiated the group session.

[0052] In one or more implementations, if the application corresponding to the application content is not installed on the electronic device 104 of the additional user, the operating system-level process of the electronic device 105 can coordinate with the peer operating system-level process of the electronic device 104 to cause the electronic device 104 to download and install at least a portion of the application (e.g., a portion of the application corresponding to the application content and / or a viewer / lightweight version of the application corresponding to the application content).

[0053] In one or more implementations, the group session can begin while the additional electronic device 104 is in the process of downloading and installing the application. In this case, the operating system-level process of the electronic device 105 can serialize the application content (e.g., in a non-interactive version of the application content) into a format (e.g., a video stream or a universal / standard XR format) that can be displayed by the peer operating system-level process of the additional electronic device 104 without the application being installed. In one or more implementations, if the additional electronic device 104 is unable to download and / or install the application (e.g., because the additional user does not have a license for the application, because the additional electronic device 104 does not have space to install additional applications, or for any reason), the serialized version of the application content can be used for the entire group session.

[0054] After the additional electronic device 104 has downloaded and installed the application, the peer operating system-level process on the additional electronic device 104 can notify the operating system-level process on the electronic device 105 that the application has been installed. The operating system-level process on the electronic device 105 can then seamlessly transition from providing the serialized version of the application content (e.g., a non-interactive version and / or a two-dimensional version) to providing the application content (e.g., an interactive version and / or a three-dimensional version). In one or more implementations, one or more visual cues can be provided, such as by changing the color, hue, brightness, or generally any visible characteristic of the application content, to indicate to the user and / or the additional user that the application content has transitioned to an interactive version.

[0055] In one or more implementations, application content can correspond to and / or identify digital rights management (DRM) protected content. For example, users in a group session can wish to view a particular movie or other video content together in the group session. In this case, the operating system level process of electronic device 105 can provide a network identifier to the peer operating system level process of electronic device 104 for obtaining the DRM content, such as from a content server. The operating system level process of electronic device 105 can also facilitate providing the peer operating system level process of electronic device 104 with appropriate keys, such as a content key, for accessing (e.g., decrypting) the content. The operating system level process of electronic device 105 can then synchronize playback of the content (separately by each electronic device 104, 105) such as by exchanging timestamp and / or other metadata information with the peer operating system level process on electronic device 104. The following is discussed with respect to FIGS. 6A-6C. Figures 4 to 6 An example process flow for managing a group session with DRM content is further discussed.

[0056] In one or more implementations, a user in a group session can request, for example from within a further application process, that content from the further application process be added to the group session. The request can be passed from the further application process to the operating system level process. The operating system level process can then receive the other application content from the further application and can coordinate display of the other application content within the host environment of the group session.

[0057] In one or more embodiments, a user initiating a group session can be provided with permission controls to discretely control viewing, modification, removal, interaction, and / or other rights / permissions with respect to other users in the group session and with respect to any content (or a portion thereof) being displayed (or presented) in the group session. For example, a user can set viewing restrictions with respect to a particular portion of a particular content segment, such as a document (or a portion thereof), for one user, but not for other users in the group session.

[0058] In one or more implementations, a user that initiates a group session can be able to record the group session for subsequent review and / or interaction. The user can also be able to provide recording permissions (e.g., subject to any other review / interaction limitations set for other users) to one or more other users in the group session. The electronic device 105 can store a recording of the group session, such as locally and / or securely at the server 120, and the user and / or the additional users can replay the recording at a later time. The recording can include depth information, mesh information, other geometric shapes, etc., such that the replay of the recording can be experienced in the same host environment as the original group session. During the replay of the recording, the user can be able to annotate a portion of the recording, such as a particular time, location, and / or content within the recording. The annotation can be, for example, an audio recording, a note, a video recording, etc. The annotation can be stored with the recording, and can be presented during subsequent replays of the recording.

[0059] Although the process 300 has been primarily described with respect to two users within a group session, a group session can include any number of users. For example, during a group session, one of the users (in an authorized situation) such as the user of the electronic device 105 can request (e.g., via an application process) to add an additional user to the group session. The application process can pass the request to the operating system level process, which can coordinate with a peer operating system level process on an additional electronic device of the additional user. The operating system level process of the electronic device 105 can also coordinate with a peer operating system level process on the electronic device 104 to add the additional user to the group.

[0060] In one or more implementations, the operating system level process on each of the respective electronic devices 104, 105 can manage the group session in a self-organizing manner, such that when a user requests to leave the group session, the electronic devices of the remaining users can continue to participate in the group session. However, in one or more implementations, the operating system level process of the electronic device 105 of the user that initiated the group session can handle the primary management functions for managing the group session (e.g., synchronizing content, user interactions, adding / removing users, conflict resolution, etc.). In this case, when a request to leave the group session is received from the user of the electronic device 105, the operating system level process of the electronic device 105 can transfer the primary management functions (e.g., with permission of the user) to the electronic devices of the users that are still in the group session. In one or more implementations, the user of the electronic device 105 can designate a particular user that stays in the group session to assume the primary management functions. In one or more implementations, the electronic device that assumes the primary management functions can be determined automatically by the operating system level process based on one or more of processing resources, power constraints, bandwidth constraints, etc.

[0061] Figure 4An example process flow 400 of providing a delegation token to a group for shared access to content protected by a digital rights management (DRM) system is shown in accordance with one or more implementations. For purposes of explanation, the process flow 400 is described herein primarily with reference to the electronic device 105 of Figure 1 and Figure 2 The process flow 400 is not limited to the electronic device 105 of Figure 1 and Figure 2 and one or more blocks (or operations) of the process flow 400 can be performed by one or more other components of other suitable devices, including the electronic device 110, the electronic device 115, and / or the server 120. Further for purposes of explanation, some blocks of the process flow 400 are described herein as occurring in sequence or in a linear fashion. However, multiple blocks of the process flow 400 can occur in parallel. Moreover, the blocks of the process flow 400 need not be performed in the order shown, and / or one or more blocks of the process flow 400 need not be performed and / or can be replaced by other operations.

[0062] The process flow 400 includes a license server 402, a verification server 404, a content owner device 406, and a viewing group device 408. The license server 402 and / or the verification server 404 can be and / or can include all or a portion of the server 120. In one or more implementations, the license server 402 and the verification server 404 can be the same server and / or can be different processes on the same server. The content owner device 406 can be an electronic device of a user in a group session, such as the electronic device 105, that owns DRM-protected content, e.g., a movie, that they want to share / view with other users in the group session. The viewing group device 408 can be an electronic device of other users in the group session, such as the electronic devices 104, 110, and / or 115. Although the viewing group devices 408 are discussed collectively, it should be understood that the described operations are performed by one or more of, or each of, these devices in the viewing group.

[0063] When a user of the content owner device 406 requests to share DRM content with the viewing group devices 408, the process flow 400 begins. In one or more implementations, the content owner device 406 and one or more of the viewing group devices 408 can already be in a group session, or can initiate a group session with the request to share DRM content from the content owner device 406. When the request to share content is received, the content owner device 406 provides a network identifier (e.g., a movie link) and a content identifier (e.g., a movie identifier or MovieID) to the viewing group devices 408 for fetching the content, and the content owner device 406 generates a secure playback context (SPC) for the content based on the content identifier, such as based on the movie identifier (410). The SPC can include, for example, a hardware unique (HU) identifier of the content owner device 406. In one or more implementations, the SPC can be a cryptographic key request. One or more of the viewing group devices 408 receive the network identifier for fetching (and / or streaming) the content, and generate an SPC for the content based on the content identifier, such as the movie identifier (412). Each SPC can include, for example, a hardware unique identifier of each respective device in the viewing group devices 408.

[0064] One or more of the viewing group devices 408 provide the generated SPCs to the content owner device 406, and the content owner device 406 provides the SPCs and corresponding user identifiers and an authorization token for accessing the content to the verification server 404 (414). The verification server 404 verifies that the user of the content owner device 406 owns the content, for example, based on the authorization token, creates a viewing group including the user identifier of the user and a generated group identifier (GroupId), and provides a list of SPCs to the license server 402 for further verification (416). The license server 402 resolves hardware unique identifiers from the list of SPCs, creates a delegation token, and provides the list of hardware unique identifiers and the delegation token to the verification server 404 (418). The verification server 404 updates the group list with the list of hardware unique identifiers, and sends the delegation token to the content owner device 406 (420). The content owner device 406 provides the delegation token to the devices of the viewing group devices 408 (422). The viewing group devices 408 receive the delegation token (424), which can then be used to access a content key, as discussed further below with respect to FIG. 5. Figure 5 Further discussion.

[0065] In one or more implementations, the delegation token can include one or more data fields, such as a version, a random number, a content identifier (e.g., a movie identifier), a group identifier, a hardware unique identifier of the owner device, a list of hardware unique identifiers of the viewing group devices, and a creation date. The random number can be, for example, a random 64-bit number that can change each time the token is updated. The list of hardware unique identifiers can be modified and recreated when the token expires. The validity window of the delegation token can be a certain time, after which the rental request will not be redeemed. In one or more implementations, the validity window can be a fixed value for a given asset type, such as a large value that allows the playback of a video content to be completed. The delegation token can be encrypted and / or processed, for example, using a hash-based message authentication code (HMAC).

[0066] Figure 5 An example process flow 500 of a group accessing content protected by a DRM system is shown in accordance with one or more implementations. For explanatory purposes, the process flow 500 is primarily described herein with reference to the electronic device 105 of Figure 1 and Figure 2 However, the process flow 500 is not limited to the electronic device 105 of Figure 1 and Figure 2 and one or more blocks (or operations) of the process flow 500 can be performed by one or more other components of other suitable devices, including the electronic device 110, the electronic device 115, and / or the server 120. Further for explanatory purposes, some of the blocks of the process flow 500 are described herein as occurring in sequence or in a linear fashion. However, multiple blocks of the process flow 500 can occur in parallel. Moreover, the blocks of the process flow 500 need not be performed in the order shown, and / or one or more blocks of the process flow 500 need not be performed and / or can be replaced by other operations.

[0067] Once the viewing group device 408 has obtained the delegation token (as described above with respect to the process flow 400), the viewing group device 408 can send a request to the server 120 to redeem the delegation token (block 504). The request can include the delegation token and the viewing group device 408’s hardware unique identifier. Figure 4The one or more viewing group devices in viewing group devices 408 can generate an SPC for the content identifier (e.g., a movie identifier) and provide the SPC and a delegation token to validation server 404 (502) upon viewing the content. Validation server 404 receives the SPC and the delegation token and passes the SPC and the delegation token, along with the content key, to license server 402 for validation (504). License server 402 validates the delegation token, generates a content key context (CKC) that can be responsive to the encrypted key, and provides the group identifier, the list of hardware unique identifiers, and the CKC to validation server 404 (506). Validation server updates the lease information (e.g., expiration time) for the list of hardware unique identifiers and provides the CKC to viewing group devices 408 (508). Viewing group devices 408 receive the CKC and await initiation of playback by content owner device 406 (510).

[0068] Content owner device 406 obtains the CKC for the content identifier (501), transmits an indication of a playback start time to viewing group devices 408, and initiates playback at the playback start time (512). Viewing group devices 408 receive the indication and initiate playback at the indicated time (e.g., by accessing the content from a content server via the received network identifier and decrypting the content using the content key corresponding to the CKC) (514). In one or more implementations, playback can be controlled by an operating system-level process on viewing group devices 408. In this way, during playback, the operating system-level process on viewing group devices 408 can prevent copying, recording, and / or redistribution of the DRM-protected content being viewed.

[0069] In one or more embodiments, if a recording consists of a group communication session during which DRM-protected content is being viewed, the DRM-protected content can be obfuscated from a user viewing the recording when the user has no rights to the DRM-protected content.

[0070] If the lease (e.g., expiration time) for the CKC expires before playback is complete, viewing group devices 408 transmit a request to resume the lease to validation server 404 (516). The request can include the SPC and the delegation token. Validation server 404 receives the request and passes the SPC, the delegation token, and the content key to license server 402 for validation (518). License server 402 validates the delegation token, generates a content key context that can be responsive to the encrypted key, and provides the group identifier, the list of hardware unique identifiers, and the CKC to validation server 404 (520). Validation server 404 updates the group table with a playback start marker and provides the CKC to viewing group devices 408 (522). Viewing group devices 408 receive the CKC and continue accessing the content (524).

[0071] Figure 6 An example process flow 600 is shown that removes a user from a group that has accessed DRM system protected content, in accordance with one or more implementations. For explanatory purposes, the process flow 600 is primarily described herein with reference to the electronic device 105 of Figure 1 and Figure 2 However, the process flow 600 is not limited to the electronic device 105 of Figure 1 and Figure 2 and one or more blocks (or operations) of the process 600 can be performed by one or more other components of other suitable devices, including the electronic device 110, the electronic device 115, and / or the server 120. Further for explanatory purposes, some of the blocks of the process flow 600 are described herein as occurring in sequence or in a linear fashion. However, multiple blocks of the process flow 600 can occur in parallel. Moreover, the blocks of the process flow 600 need not be performed in the order shown, and / or one or more blocks of the process flow 600 need not be performed and / or can be replaced by other operations.

[0072] The process flow 600 begins when the user of the content owner device 406 requests to remove another user from shared viewing. For example, the request can be automatically generated when the other user leaves the group session, or the request can be generated when the user of the content owner device 406 wants to manually remove the other user for any other reason. The content owner device 406 transmits a remove user request to the verification server 404, the remove user request including a user identifier (UserId) of the user being removed, a delegation token, and an authorization token of the user of the content owner device 406 (602).

[0073] The verification server 404 receives the request, verifies the user of the content owner device 406 based at least in part on the authorization token, and transmits the delegation token to the license server 402 for verification (604). The license server 402 receives the delegation token, verifies the delegation token, and then transmits a list of hardware unique identifiers and a group identifier to the verification server 404 (606). The verification server 404 removes the hardware unique identifier of the electronic device corresponding to the user being removed from the list of hardware unique identifiers of the group, and transmits a confirmation message to the content owner device 406 (608). The content owner device 406 receives the confirmation message confirming that the other user has been removed (610).

[0074] In one or more implementations, the content owner device 406 can leave the group session and / or stop viewing the content, and other viewing group devices 408 can continue viewing the content. In one or more implementations, additional user devices can be added to the viewing group devices 408 at any time (e.g., during playback) using the process flows described above.

[0075] The present disclosure recognizes that the use of such personal information data, in the present technology, can be used to the benefit of users. For example, personal information data can be used to operate system-level group session management.

[0076] The present disclosure contemplates that the entities responsible for the collection, analysis, disclosure, transfer, storage, or other use of such personal information data will comply with well-established privacy policies and / or privacy practices. In particular, such entities should implement and consistently apply privacy policies that are designed to meet or exceed industry or governmental requirements for maintaining the privacy of user data. Such policies should be readily available to users and should be at least partially determined or influenced by the users themselves. Users should have the ability to opt in or opt out of collection, use, or disclosure of their personal information data. In addition, such entities should take any needed steps for safeguarding and securing access to such personal information data and ensuring that others with access to the personal information data adhere to their privacy policies and procedures. Further, such entities can subject themselves to evaluation by third parties to certify their adherence to widely accepted privacy policies and practices. In addition, users should be able to access, modify, or delete personal information data associated with them on such entities' systems. Moreover, policies and practices should be adapted to the particular types of personal information data being collected and / or accessed and the ways in which such personal information data is collected and / or accessed by the relevant entity.

[0077] Regardless of the above, the present disclosure also contemplates embodiments in which users selectively block the use of, or access to, personal information data. That is, the present disclosure contemplates that hardware and / or software elements can be provided to prevent or block access to such personal information data. For example, in the case of system-level group session management, the subject technology can be configured to allow users to select to "opt in" or "opt out" of permitting the collection and / or sharing of personal information data about them. In addition to providing the "opt in" and "opt out" options, the present disclosure contemplates providing notifications relating to the access or use of personal information data. For instance, a user can be notified upon download of an application that their personal information data will be accessed and then reminded again just before the personal information data is accessed by the application.

[0078] Moreover, it is the intent of the present disclosure that any personalized information data utilized by implementations of the present disclosure should be managed and handled in a way that minimizes risks of unintentional or unauthorized access or use. Risk can be minimized by limiting the collection of data and deleting data once it is no longer needed. In addition, and when applicable, data de-identification can be employed to protect user privacy. De-identification can be facilitated through practices such as removing identifiers, controlling the amount or

[0079] Accordingly, while the present disclosure broadly covers technologies using personal information data, the present disclosure also contemplates implementations that do not involve such personal information data. That is, various implementations of the technology covered by the present disclosure are compatible with data that does not include personal information data.

[0080] Figure 7 An electronic system 700 with which one or more specific implementations of the subject technology can be implemented is shown. The electronic system 700 can be, or can be a part of, an electronic device 105, a handheld electronic device 104, an electronic device 110, an electronic device 115, and / or a server 120 as shown. The electronic system 700 can include various types of computer readable media and interfaces for various other types of computer readable media. The electronic system 700 includes a bus 708, one or more processing unit(s) 712, a system memory 704 (and / or cache), a ROM 710, the permanent storage device 702, an input device interface 714, an output device interface 706, and one or more network interfaces 716, or a subset or superset thereof. Figure 1

[0081] The bus 708 generally represents what can be a complex series of buses and bridges that allow for communication among various internal devices of the electronic system 700. In one or more specific implementations, the bus 708 communicatively connects the one or more processing unit(s) 712 with the ROM 710, the system memory 704, and the permanent storage device 702. The one or more processing unit(s) 712 fetches instructions from these various memory units and processes them to execute the processes of the present subject disclosure. In different implementations, the one or more processing unit(s) 712 can be a single processor or multiple processors.

[0082] ​ROM 710 stores static data and instructions required by one or more processing units 712 and other modules of electronic system 700. On the other hand, persistent storage device 702 can be a read-write memory device. Persistent storage device 702 can be a non-volatile memory cell that stores instructions and data even when electronic system 700 is powered off. In one or more embodiments, mass storage devices (such as magnetic disks or optical disks and their corresponding disk drives) can be used as persistent storage device 702.

[0083] In one or more embodiments, a removable storage device (such as a floppy disk, flash drive, and its corresponding disk drive) may be used as persistent storage device 702. Like persistent storage device 702, system memory 704 may be a read-write memory device. However, unlike persistent storage device 702, system memory 704 may be volatile read-write memory, such as random access memory. System memory 704 may store any instructions and data that one or more processing units 712 may need during operation. In one or more embodiments, the processes disclosed in this subject matter are stored in system memory 704, persistent storage device 702, and / or ROM 710 (each implemented as a non-transitory computer-readable medium). One or more processing units 712 retrieve instructions to be executed and data to be processed from these various memory units to perform the processes of one or more embodiments.

[0084] Bus 708 is also connected to input device interface 714 and output device interface 706. Input device interface 714 enables a user to transmit information and select commands to electronic system 700. Input devices that can be used with input device interface 714 may include, for example, an alphanumeric keypad and pointing devices (also known as "cursor control devices"). Output device interface 706 may, for example, enable the display of images generated by electronic system 700. Output devices that can be used with output device interface 706 may include, for example, printers and display devices such as liquid crystal displays (LCDs), light-emitting diode (LED) displays, organic light-emitting diode (OLED) displays, flexible displays, flat panel displays, solid-state displays, projectors, or any other device for outputting information. One or more embodiments may include devices that act as both input and output devices, such as touchscreens. In these embodiments, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, voice, or tactile input.

[0085] Finally, as Figure 7As shown, bus 708 also couples electronic system 700 to one or more networks and / or to one or more network nodes, such as a telecommunications network, local area network ("LAN"), wide area network ("WAN"), the Internet, etc. Figure 1 Electronic device 110 is shown. In this manner, electronic system 700 can be part of a computer network, such as a LAN, a wide area network ("WAN"), or an intranet, or can be part of a network of networks, such as the Internet. Any or all components of electronic system 700 can be used in conjunction with the subject disclosure.

[0086] The functions described above can be implemented in computer software, firmware or hardware. The techniques can be implemented using one or more computer program products. Programmable processors and computers can include or be

[0087] Some implementations include electronic components, such as microprocessors, storage and memory that store computer program instructions in a machine -readable or computer-readable medium (also referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer- readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a variety of recordable / rewritable DVD discs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and / or solid-state hard drives, only and recordable Blu-ray discs, ultra-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable media can store the computer program, which is executable by at least one processing unit and includes sets of instructions for performing various operations. Examples of computer programs, or computer code, include machine code, such as produced by a compiler, and files including higher-level code that can be executed by a computer, an electronic component, or a microprocessor using an interpreter.

[0088] While the above discussion primarily refers to microprocessor or multi-core processors that execute software, some implementations are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some implementations, such integrated circuits execute instructions that are stored on the circuit itself.

[0089] As used in the description of the specification and any claims of the present application, the terms “computer”, “server”, “processor”, and “memory” all refer to electronic or other technological devices. These terms exclude humans or groups of humans. For purposes of the specification, the term display or being displayed means display on an electronic device. As used in the description of the specification and any claims of the present application, the terms “computer readable medium” and “computer readable media” are entirely restricted to tangible, physical objects that store information in a form readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral signals.

[0090] To provide for interaction with a user, implementations of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device of the user; for example, by sending web pages to a web browser on a user’s client device in response to requests received from the web browser.

[0091] Implementations of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), an inter-network (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0092] The computing system can include clients and servers. A client and server are generally remote from each other and can interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data (e.g., an HTML page) to a client device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the client device). Data generated at the client device (e.g., a result of the user interaction) can be received from the client device at the server.

[0093] According to aspects of the disclosure, a method is provided that includes receiving, by an operating system level process executing on a device and from an application process executing on the device, a request to initiate a group session between a user associated with the device and a further user; identifying, by the operating system level process, a further device associated with the further user; initiating, by the operating system level process, the group session between the user and the further user via the further device; receiving, by the operating system level process and from the application process, application content; and managing, by the operating system level process, the group session based at least in part on the received application content.

[0094] According to aspects of the disclosure, a device is provided that includes a memory; and at least one processor configured to: receive, by an operating system level process executing on a device and from an application process executing on the device, a request to initiate a group session between a user associated with the device and a further user; identify, by the operating system level process, a further device associated with the further user; initiate, by the operating system level process, the group session with the user via the further device; and manage, by the operating system level process, the group session.

[0095] According to aspects of the disclosure, a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: receiving, by an operating system level process executing on a device and from an application process executing on the device, a request to initiate a group session between a user associated with the device and a further user; identifying, by the operating system level process, a further device associated with the further user; initiating, by the operating system level process, the group session between the user and the further user via the further device; and managing, by the operating system level process, the group session.

[0096] Particular implementations within the scope of the present disclosure can be partially or fully implemented by one or more tangible computer-readable storage media (or multiple tangible computer-readable storage media of the same or multiple types) having instructions (or one or more sets of instructions) stored

[0097] A computer-readable storage medium can be any storage medium, tangible or non-transitory, that can be read by a general or special purpose computing device, including any processing electronics and / or processing circuitry capable of executing instructions. For example, without limitation, a computer-readable medium can include any volatile, non-volatile, or transitory storage medium, including any semiconductor-based memory, such as RAM, DRAM, SRAM, T-RAM, Z-RAM, and TTRAM. A computer-readable medium can also include any nonvolatile memory, such as ROM, PROM, EPROM, EEPROM, NVRAM, flash memory, nvSRAM, FeRAM, FeTRAM, MRAM, PRAM, CBRAM, SONOS, RRAM, NRAM, racetrack memory, FJG, and Millipede memory.

[0098] Furthermore, a computer-readable storage medium can include any non-semiconductor memory, such as optical disk storage, magnetic disk storage, magnetic tape, other magnetic storage devices, or any other medium capable of storing one or more instructions. In one or more particular implementations, a tangible computer-readable storage medium can be directly coupled to a computing device, while in other implementations, a tangible computer-readable storage medium can be indirectly coupled to a computing device, such as via one or more wired connections, one or more wireless connections, or any combination thereof.

[0099] Instructions can be directly executable or can be used to develop executable instructions. For example, instructions can be implemented as executable or non-executable machine code, or as high-level language instructions that can be compiled to produce executable or non-executable machine code. Furthermore, instructions can also be implemented as data, or can include data. Computer-executable instructions can also be organized in any format, including routines, subroutines, programs, data structures, objects, modules, applications, applets, functions, etc. As recognized by those of skill in the art, details including, but not limited to, the number, structure, sequence, and organization of instructions can vary significantly without altering the underlying logic, function, processing, and output.

[0100] While the above discussion primarily refers to microprocessor or multi-core processors that execute software, some or all of the implementations can be performed by one or more integrated circuits, such as ASICs or FPGAs. In one or more particular implementations, such integrated circuits execute instructions stored on the integrated circuits themselves.

[0101] Those skilled in the art will recognize that the various exemplary blocks, modules, elements, components, methods, and algorithms described herein can be implemented as electronic hardware, computer software, or a combination of both. To illustrate this interchangeability between hardware and software, the various exemplary blocks, modules, elements, components, methods, and algorithms have been generally described above in terms of functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the described functionality in different ways for each specific application. Various components and blocks can be arranged differently (e.g., in different orders or divided in different ways) without departing from the scope of the subject matter.

[0102] It should be understood that the specific order or hierarchy of the boxes in the process disclosed in this invention is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of the boxes in the process may be rearranged or all shown boxes may be executed. Any box within these boxes may be executed simultaneously. In one or more embodiments, multitasking and parallel processing may be advantageous. Furthermore, the division of various system components in the above embodiments should not be construed as requiring such division in all embodiments, and it should be understood that program components and systems may generally be integrated together in a single software product or packaged into multiple software products.

[0103] As used in this specification and any claim of this patent application, the terms "base station," "receiver," "computer," "server," "processor," and "memory" refer to electronic devices or other technical devices. These terms exclude persons or groups of persons. For the purposes of this specification, the terms "display" or "being displayed" mean displaying on an electronic device.

[0104] As used herein, the phrase "at least one of" following a series of items separated by the terms "and" or "or" modifies the list as a whole, not each member of the list (i.e., each item). The phrase "at least one of" does not require selection of at least one of each of the listed items; rather, it allows for the inclusion of at least one of any one item and / or at least one of any combination of items and / or at least one of each item. For example, the phrases "at least one of A, B, and C" or "at least one of A, B, or C" respectively refer to only A, only B, or only C; any combination of A, B, and C; and / or at least one of each of A, B, and C.

[0105] The predicates “configured to,” “operable to,” and “programmed to” do not imply any specific tangible or intangible modification to a particular subject but are intended to be used interchangeably. In one or more embodiments, a processor configured to monitor and control operations or components may also mean that the processor is programmed to monitor and control operations or that the processor is operable to monitor and control operations. Similarly, a processor configured to execute code may be interpreted as a processor programmed to execute code or operable to execute code.

[0106] Phrases such as aspect, that aspect, on the other hand, some aspects, one or more aspects, implementation, that implementation, another implementation, some implementations, one or more implementations, implementation scheme, that implementation scheme, another implementation scheme, some implementation schemes, one or more implementation schemes, configuration, that configuration, other configuration, some configurations, one or more configurations, subject matter technology, disclosure, this disclosure, other variations thereof, etc., are for convenience only and do not imply that disclosures involving one or more such phrases are essential to the subject matter technology, nor do they imply that such disclosures apply to all configurations of the subject matter technology. Disclosures involving one or more such phrases may apply to all configurations or one or more configurations. Disclosures involving one or more such phrases may provide one or more examples. Phrases such as aspect or some aspects may refer to one or more aspects, and this applies similarly to the other foregoing phrases.

[0107] The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment described herein as “exemplary” or “example” is not necessarily to be construed as preferred or superior to other specific embodiments. Furthermore, the terms “comprising,” “having,” etc., as used in the specification or claims, are intended to be inclusive, similar to how “comprising” is interpreted when used as a transitional word in the claims.

[0108] All structural and functional equivalents of elements throughout the various aspects described herein that are known or later become apparent to those skilled in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be made public, regardless of whether such disclosure is expressly stated in the claims. No claim element should be interpreted in accordance with 35 U.S.SC §112(f) unless the element is expressly stated using the phrase “means for…” or, in the case of a method claim, using the phrase “step for…”.

[0109] The preceding description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language claims, wherein reference to an element in the singular is not intended to mean "one and only one" unless specifically so stated, and wherein "some" indicates "one or more." Unless otherwise specified, the term "some" means one or more. The word "or" is used in the inclusive sense (i.e., "and / or"), unless otherwise indicated. The terms "a," "an," and "the" are used in the sense that they can mean one or more than one (i.e., "one or more"), unless otherwise indicated. The terms "plurality" and "a plurality" mean two or more than two (i.e., two or more than two) unless otherwise indicated. The terms "another," "additional" and "another" mean one or more than one (i.e., one or more than one), unless otherwise indicated. The terms "comprises," "comprising," "includes," "including," "has," "having," "contains," "containing," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, has, contains, or contains elements or steps not recited in the claims are still within the scope of such claims.

Claims

1. A method for managing group sessions, comprising: The system receives requests to initiate group sessions between users associated with the device and other users through an operating system-level process running on the device and from an application process running on the device. The operating system-level process identifies another device associated with the other user; Initiating the group session between the user and the other user via the operating system-level process through the other device, wherein initiating the group session between the user and the other user via the operating system-level process through the other device includes: Determine the host environment in which the group session is displayed on the device and the other device; and The group session is initiated within the host environment by displaying the host environment on the device. Receive application content through the operating system-level process and from the application process; and The group session is managed, at least in part, through the operating system-level process by displaying the received application content within the host environment.

2. The method of claim 1, wherein the host environment includes at least one of the following: the user's physical environment, the other user's additional physical environment, a virtual environment, or a hybrid environment including the user's physical environment and the other user's additional physical environment.

3. The method of claim 2, wherein the host environment includes the user's physical environment, and initiating the group session within the host environment includes: Receive the video stream of the user's physical environment; as well as Information is provided to the other device for generating a reconstruction of the user's physical environment, the information being generated at least in part based on the video stream.

4. The method of claim 3, wherein managing the group session via the operating system-level process, at least in part by displaying the received application content within the host environment, further comprises: The application process displays at least a portion of the video stream of the user's physical environment and at least a portion of the received application content.

5. The method of claim 3, wherein the information for generating the reconstruction of the user's physical environment includes information corresponding to portions of the physical environment not included in the video stream.

6. The method according to claim 3, further comprising: Prevent the application process from accessing the video stream or the information used to generate the reconstruction of the user's physical environment.

7. The method according to claim 1, further comprising: The user who authenticates the device is a representative of the user; as well as The user's representative is provided to the other device for display in the group session.

8. The method of claim 7, wherein the representative authenticating the user of the device further comprises: Verify that one or more virtual attributes represented by the user conform to one or more physical attributes of the user.

9. The method according to claim 1, further comprising: Store the records of the group sessions; After the group session has ended, the record of the group session is replayed for the user in response to a request for the record of the group session; During the replay of the record of the group session, user input for commenting on portions of the record of the group session is received; as well as Store the annotation associated with the portion of the record of the group session.

10. The method of claim 9, further comprising: After the annotation is stored, the record of the group session is replayed to the user in response to a further request for the record of the group session, wherein the annotation is displayed in conjunction with the portion of the record that is displayed.

11. The method of claim 1, wherein the device comprises a head-mounted system, and the additional device comprises a handheld mobile device or a tablet device.

12. The method of claim 1, wherein when the request to initiate the group session is received, the application corresponding to the application process is not installed on the other device, and initiating the group session between the user and the other user via the other device through the operating system-level process further comprises: The operating system-level process causes the other device to download and install at least a portion of the application.

13. The method of claim 12, wherein the group session is initiated before at least said portion of the application has been downloaded and installed on the other device, and the group session is managed at least in part based on the received application content via the operating system-level process, comprising: Provide a non-interactive version of the received application content to the other device.

14. The method of claim 13, further comprising: After the other device has downloaded and installed at least a portion of the application, the process switches from providing the other device with a non-interactive version of the received application content to providing the other device with an interactive version of the received application content.

15. The method of claim 14, wherein the non-interactive version of the received application content comprises two-dimensional content, and the interactive version of the received application content comprises three-dimensional content.

16. The method according to claim 1, further comprising: Receive content from other applications through the operating system-level process and from other application processes; as well as The group session is managed at least in part by the operating system-level process based on application content received from the application process and additional application content received from the other application process.

17. The method according to claim 1, further comprising: Receive additional requests to add additional users to the group session through the operating system-level process executing on the device and from the application process executing on the device; as well as The additional user is added to the group session via the additional user's additional device through the operating system-level process.

18. The method of claim 1, wherein managing the group session via the operating system-level process, at least in part, by displaying received application content within the host environment, comprises: Receive a request from the user to leave the group session; as well as The group session is transferred to at least one of the other operating system-level processes of the other device or the additional operating system-level processes of the additional device via the operating system-level process.

19. The method of claim 1, wherein the application content corresponds to content protected by digital rights management, and managing the group session via the operating system-level process, at least in part, by displaying the received application content within the host environment, comprises: Through the operating system-level process, and by providing the other device with a network identifier for retrieving the content from the content server; as well as The operating system-level process facilitates the provision of a key for accessing the acquired content to the other device.

20. The method of claim 19, wherein managing the group session via the operating system-level process, at least in part, by displaying received application content within the host environment, further comprises: The playback of the content on the device and the other device is synchronized through the operating system-level process.

21. The method of claim 1, wherein managing the group session via the operating system-level process, at least in part by displaying received application content within the host environment, further comprises: The operating system-level process controls at least one of the other user's interactions, modifications, or copies relative to the group session.

22. An electronic device, comprising: Memory; as well as At least one processor, said at least one processor being configured to: The system receives requests to initiate group sessions between users associated with the device and other users through an operating system-level process running on the device and from an application process running on the device. The operating system-level process identifies another device associated with the other user; Initiating the group session with the user via the other device through the operating system-level process, wherein initiating the group session with the user via the other device through the operating system-level process includes: Determine the host environment in which the group session is displayed on the device and the other device; as well as By displaying the host environment on the device, and initiating the group session within the host environment; and The group session is managed at least in part by the operating system-level process through displaying application content received from the application process within the host environment.

23. A non-transitory machine-readable medium comprising code, said code, when executed by one or more processors, causing said one or more processors to perform operations including: The system receives requests to initiate group sessions between users associated with the device and other users through an operating system-level process running on the device and from an application process running on the device. The operating system-level process identifies another device associated with the other user; Initiating the group session between the user and the other user via the operating system-level process through the other device, wherein initiating the group session between the user and the other user via the operating system-level process through the other device includes: Determine the host environment in which the group session is displayed on the device and the other device; as well as By displaying the host environment on the device, and initiating the group session within the host environment; and The group session is managed at least in part by the operating system-level process through displaying application content received from the application process within the host environment.

Images