Suspended mode features for virtual reality systems.

Abstract

Generally, this disclosure describes a suspend mode feature for a virtual reality system, and more particularly, describes system configurations and techniques for generating a suspend mode environment during the execution of a virtual reality application and presenting and controlling user interface (UI) elements within the suspend mode environment. The suspend mode feature allows a user interacting with one or more other users (or one or more avatars corresponding to one or more other users) within a virtual reality (AR) environment to suspend the AR environment from the user's perspective. While in the suspend mode environment, the user can observe and select other users from the AR environment. For example, from the suspend mode environment, the user can select another user to be a friend, message the user, mute the user, block the user, or report the user.

Description

[Technical field]

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 905,098, filed Sep. 24, 2019, the entire contents of which are incorporated herein by reference.

[0002] The present disclosure relates generally to artificial reality systems, such as virtual reality systems, mixed reality systems and / or augmented reality systems. [Background technology]

[0003] Artificial reality systems are becoming increasingly ubiquitous for applications in many areas, such as computer gaming, health and safety, industry, and education. As some examples, artificial reality systems are integrated into mobile devices, gaming consoles, personal computers, movie theaters, and theme parks. In general, artificial reality is a form of reality that is conditioned in some manner prior to presentation to a user, which may include, for example, virtual reality, augmented reality, mixed reality, hybrid reality, or any combination and / or derivative thereof.

[0004] A typical artificial reality system includes one or more devices for rendering and displaying content to a user. As an example, a synthetic reality system may incorporate a head mounted display (HMD) worn by a user and configured to output synthetic reality content to the user. The synthetic reality content may include entirely generated content or generated content combined with captured content (e.g., real-world video and / or images). In operation, a user typically interacts with the synthetic reality system to select content, launch applications, or otherwise configure the system. Summary of the Invention

[0005] In general, the present disclosure describes a suspended mode feature for a synthetic reality system. More specifically, the present disclosure describes system configurations and techniques for generating a suspended mode environment during execution of a synthetic reality application and presenting and controlling user interface (UI) elements within the suspended mode environment. The suspended mode feature allows a user interacting with one or more other users (or one or more avatars corresponding to one or more other users) within a synthetic reality (AR) environment to suspend the AR environment from the user's perspective. While in the suspended mode environment, the user is able to observe and select other users from the AR environment. For example, the user may select another user to be friends with, message the user, mute the user, block the user, or report the user from the suspended mode environment.

[0006] According to a first aspect of the present invention, there is provided a synthetic reality system comprising: a head mounted display (HMD) configured to output synthetic reality content to a user of the HMD, the synthetic reality content including one or more avatars corresponding to one or more other users participating in the synthetic reality environment; and a interruption mode engine configured to enable a interruption mode environment for a user of the HMD in response to user input, the interruption mode engine at least one of: replacing the synthetic reality environment with the interruption mode environment in the HMD, muting the one or more avatars and the synthetic reality environment in the HMD, or rendering one or more avatars at a static position within the interruption mode environment.

[0007] The artificial reality system may further include an imaging device configured to capture image data representing the physical environment, a gesture detector configured to identify from the image data a user interface (UI) activation gesture comprising moving an inside wrist area of ​​a hand of a user of the HMD in a field of view of the imaging device, a UI engine configured to generate a interruption mode UI element in response to the UI activation gesture, and / or a rendering engine configured to render the interruption mode UI element near or at least partially on the user's hand.

[0008] The user input may correspond to a pause mode selection gesture including a placement of one or more fingers of a virtual hand of the user's avatar over the pause mode UI element. The gesture detector may be further configured to identify the pause mode selection gesture from the image data.

[0009] The interrupted mode engine may be further configured to, in response to the interrupted mode environment being enabled, move one or more avatars away from the user's avatar when the one or more avatars are within a personal space threshold from the user's avatar.

[0010] The suspended mode engine may be further configured to replace a virtual background and virtual objects in the synthetic reality environment with a solid-color background in the suspended mode environment.

[0011] The suspended mode engine may be further configured to play background music in the suspended mode environment.

[0012] The HMD may include a first HMD, and a user of the first HMD may include the first user. The system may further include a second HMD of a second user corresponding to one of the one or more other avatars. The second HMD may be configured to output artificial reality content, including the artificial reality environment, to the second user. The avatar of the first user may stop participating in the artificial reality environment after the suspended mode is enabled for the first user in the first HMD. The second user may continue to participate in the artificial reality environment in the second HMD after the suspended mode is enabled for the first user in the first HMD.

[0013] The second HMD may render virtual graphics corresponding to the first user of the first HMD after the suspend mode is enabled for the first user in the first HMD.

[0014] A second user of the second HMD may not be able to interact with the avatar of the first user of the first HMD within the synthetic reality environment after the suspended mode is enabled for the first user in the first HMD.

[0015] The user input may include a first user input. The suspended mode environment may include a first suspended mode environment. The suspended mode engine may be further configured to enable a second suspended mode environment, different from the first suspended mode environment, for a second user of the second HMD in response to a second user input from a second user. The second user of the second HMD may not be able to interact with an avatar of the first user within the first suspended mode environment or the second suspended mode environment.

[0016] The interruption mode engine may be further configured to render a user interface (UI) element for each of the one or more avatars in the interruption mode environment, each UI element being selectable via a selection gesture performed by a user, and / or in response to detecting a selection gesture of a first UI element corresponding to a first avatar of the one or more avatars, to do at least one of: render a first selectable virtual element configured, when selected, to mute the first avatar in the HMD when the user returns to the synthetic reality environment; render a second selectable virtual element configured, when selected, to block the first avatar from interacting with the user's avatar when the user returns to the synthetic reality environment; and render a third selectable virtual element configured, when selected, to report the first avatar to a review entity associated with the AR experience.

[0017] The first avatar may be highlighted when a first UI element that corresponds to the first avatar is selected.

[0018] The first avatar, if blocked by the user, may not be able to interact with the user's avatar in the synthetic reality environment or any other synthetic reality environment, regardless of whether a suspended mode environment has been enabled for the user.

[0019] According to a second aspect of the present invention, a method is provided that includes outputting synthetic reality content in a head mounted display (HMD) of a synthetic reality system to a user of the HMD, the synthetic reality content including one or more avatars corresponding to one or more other users participating in the synthetic reality environment, and in response to user input, enabling a suspended mode environment for the user of the HMD by at least one of: replacing the synthetic reality environment with the suspended mode environment in the HMD, muting the one or more avatars and the synthetic reality environment in the HMD, or rendering one or more avatars at a static position within the suspended mode environment.

[0020] The method may further include acquiring, by the artificial reality system, image data via an imaging device, identifying from the image data a user interface (UI) activation gesture comprising moving an inner wrist region of a hand of a user of the HMD in a field of view of the imaging device, and in response to the UI activation gesture, generating a break mode UI element and / or rendering the break mode UI element near or at least partially on the user's hand.

[0021] The user input may correspond to a break mode selection gesture including a placement of one or more fingers of a virtual hand of the user's avatar over the break mode UI element. The method may further include identifying the break mode selection gesture from the image data.

[0022] The suspended mode engine may be further configured to replace a virtual background and virtual objects in the synthetic reality environment with a monochromatic background in the suspended mode environment.

[0023] Enabling the interrupted mode environment may further include, in response to the interrupted mode environment being enabled, moving one or more avatars away from the user's avatar when the one or more avatars are within a personal space threshold from the user's avatar.

[0024] A user's avatar may cease participating in the synthetic reality environment after the suspended mode is enabled for the user in the HMD. One or more avatars corresponding to one or more other users may continue to participate in the synthetic reality environment after the suspended mode is enabled for the user in the HMD.

[0025] According to a third aspect of the present invention, there is provided a computer-readable storage medium comprising instructions that configure one or more programmable processors to output synthetic reality content at a head mounted display (HMD) of a synthetic reality system to a user of the HMD, the synthetic reality content including one or more avatars corresponding to one or more other users participating in the synthetic reality environment, and in response to user input, enable a suspended mode environment by at least one of: replacing the synthetic reality environment with the suspended mode environment at the HMD, muting the one or more avatars and the synthetic reality environment at the HMD, or rendering one or more avatars at a static position within the suspended mode environment.

[0026] The present technique may provide one or more technical improvements that provide at least one practical application. For example, the present technique allows a user to quickly enable a suspend mode in an HMD while continuing to receive data corresponding to the synthetic reality environment and allowing other users to continue participating in the synthetic reality environment in other HMDs. By continuing to receive data corresponding to the synthetic reality environment while other users continue participating in the synthetic reality environment in other HMDs, the HMD will not have a backlog of data updates to process when returning to the synthetic reality environment from the suspend mode environment, which allows the user to return to participating in the synthetic reality environment more quickly by reducing data processing delays for a more pleasurable user experience.

[0027] The details of one or more examples of the techniques of this disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques will become apparent from the description and drawings, and from the claims. [Brief description of the drawings]

[0028] [Figure 1A] FIG. 1 illustrates an example virtual reality system having a suspended mode feature in accordance with the techniques of this disclosure. [Figure 1B] FIG. 1 illustrates another example virtual reality system having a suspended mode feature in accordance with the techniques of this disclosure. [Figure 2A] FIG. 1 illustrates an example HMD that operates in accordance with the techniques of this disclosure. [Figure 2B] FIG. 2 illustrates another exemplary HMD in accordance with the techniques described in this disclosure. [Diagram 3] 1 is a block diagram illustrating an example implementation of a console, a virtual reality application server, and an HMD of a virtual reality system that generates a suspended mode environment while a virtual reality application is running, in accordance with the techniques of this disclosure. [Figure 4]1 is a block diagram illustrating an example HMD and a virtual reality application server of a virtual reality system for generating a suspended mode environment while a virtual reality application is running in accordance with techniques of this disclosure. [Diagram 5] FIG. 1 is a block diagram illustrating a more detailed example implementation of a distributed architecture for a multi-device virtual reality system, where one or more devices are implemented using one or more SoC integrated circuits in each device. [Figure 6] 1 is a flowchart illustrating example operations for enabling a suspended mode environment for a virtual reality system according to aspects of the present disclosure. [Figure 7] 1 is a flowchart illustrating example operations while in a suspended mode environment for a virtual reality system in accordance with aspects of the present disclosure. [Figure 8] 1 illustrates a user interaction with an exemplary gesture-driven UI for entering a suspended mode environment while a virtual reality application is running, in accordance with the techniques of this disclosure. [Figure 9] FIG. 2 illustrates an example virtual reality environment for a virtual reality application in accordance with the techniques of this disclosure. [Figure 10] FIG. 2 illustrates an example of a suspended mode environment in accordance with techniques of this disclosure. [Figure 11] FIG. 2 illustrates another example of a suspended mode environment in accordance with the techniques of this disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] Like reference numbers refer to like elements throughout the figures and description.

[0030] 1A is a diagram illustrating an example virtual reality system 10 having a suspended mode feature in accordance with the techniques of this disclosure. In some examples, the suspended mode feature may be referred to as a safe mode or safe zone feature. In the example of FIG. 1A, the virtual reality (AR) system 10 includes a head mounted display (HMD) 112, a console 106, a virtual reality application server 107, and in some examples, one or more external sensors 90.

[0031] As shown, the HMD 112 is generally worn by the user 110 and comprises an electronic display and optical assembly for presenting the artificial reality content 122 to the user 110. Additionally, the HMD 112 may include one or more sensors (e.g., accelerometers) for tracking movement of the HMD and may include one or more imaging devices 138, e.g., cameras, line scanners, etc., for capturing image data of the surrounding physical environment. Although shown as a head-mounted display, the AR system 10 may alternatively or additionally include glasses or other display devices for presenting the artificial reality content 122 to the user 110.

[0032] In this example, the console 106 is shown as a single computing device, such as a gaming console, a workstation, a desktop computer, or a laptop. In other examples, the console 106 may be distributed across multiple computing devices, such as a distributed computing network, a data center, or a cloud computing system. The console 106, the HMD 112, and the sensor 90 may be communicatively coupled via a network 104, as shown in this example, which may be a wired or wireless network, such as Wi-Fi, a mesh network, or a short-range wireless communication medium. Although the HMD 112 is shown in this example as being in communication with the console 106, e.g., tethered to the console 106 or in wireless communication with the console 106, in some implementations, the HMD 112 operates as a standalone mobile virtual reality system.

[0033] In this example, the HMD 112 is shown in communication with a virtual reality application server 107 over the network 104, which performs back-end network operations for virtual reality applications executing on the console 106 and / or the HMD 112. In some examples, the virtual reality application server 107 may correspond to a server or another computing resource in any form. For example, the virtual reality application server 107 may be a physical computing device or may be a component of a cloud computing system, a server farm, and / or a server cluster (or portions thereof). Thus, the virtual reality application server 107 may represent one or more physical computing devices, virtual computing devices, virtual machines, containers, and / or other virtualized computing devices that provide services to client devices and other devices or systems that execute a particular virtual reality application. For example, the virtual reality application server 107 may manage relationships between users (e.g., friendships), maintain profiles for users, coordinate games with one or more users (e.g., place one or more users in a game, keep scores), etc.

[0034] In the example of FIG. 1A, the user 110 views a synthetic reality content 122 constructed and rendered by a synthetic reality application executing on the console 106 and / or HMD 112. In some examples, the synthetic reality system 10 uses information captured from a real-world 3D physical environment to render the synthetic reality content 122 for display to the user 110. As an example, the synthetic reality content 122 may be a home gaming application, in which the user 110 is rendered as an avatar including a virtual hand 136 and a virtual arm 137 in the synthetic reality environment 124. The synthetic reality content 122 may also include one or more other users rendered as avatars 120 and one or more virtual objects 128A, 128B in the synthetic reality environment 124. In some examples, the synthetic reality content 122 may include a mixture of real-world imagery and virtual objects, e.g., mixed reality and / or augmented reality. In other examples, the artificial reality content 122 may be, for example, a social networking application, a video conferencing application, a navigation application, an educational application, a training or simulation application, or other types of applications that implement artificial reality. In some examples, the artificial reality content 122 may be rendered from a first person perspective for an avatar associated with a user of the HMD 112.

[0035] During operation, the artificial reality application constructs artificial reality content 122 for display to the user 110 by tracking and calculating pose information for a frame of reference, typically the viewing perspective of the HMD 112. Using the HMD 112 as a frame of reference and based on a current field of view 130 determined by the current estimated pose of the HMD 112, the artificial reality application renders 3D artificial reality content, which in some examples may be at least partially overlaid on the real-world 3D physical environment of the user 110. During this process, the artificial reality application uses sensed data received from the HMD 112, such as movement information and user commands, and in some examples data from any external sensors 90, such as an external camera, to capture 3D information in the real-world physical environment, such as movements by and / or feature tracking information about the user 110. Based on the sensed data, the artificial reality application determines a current pose for the frame of reference of the HMD 112 and renders the artificial reality content 122 according to the current pose.

[0036] The virtual reality system 10 may trigger the generation and rendering of virtual content items based on the user's 110 current field of view 130 or other conditions, as may be determined by real-time eye tracking of the user 110. More specifically, the imaging device 138 of the HMD 112 captures image data representing objects in the real-world physical environment that are within the field of view 130 of the imaging device 138. The field of view 130 generally corresponds to the viewing perspective of the HMD 112. In some examples, the virtual reality application presents the virtual reality content 122, including mixed reality and / or augmented reality. In some examples, the virtual reality application may render images of real-world objects, such as parts of the user's 110 hand 132 and / or arm 134, that are within the field of view 130 along with the virtual objects, such as within the virtual reality content 122. In other examples, the virtual reality application may render a virtual representation of the portion of the user's 110's hand 132 and / or arm 134 that is within the field of view 130 (e.g., render a real-world object as a virtual object, such as a virtual hand 136) within the virtual reality content 122. In either example, the user 110 may be able to observe the portion of the user's hand 132, arm 134, and / or any other real-world object that is within the field of view 130 within the virtual reality content 122. In other examples, the virtual reality application may not render a representation of the user's hand 132 or arm 134.

[0037] In some examples, based on the sensed data, the virtual reality system 10 detects gestures performed by the user 110 and, in response to detecting one or more particular gestures, generates one or more user interface elements that may be overlaid on the underlying virtual reality content 122 presented to the user 110, such as the user interface (UI) menu 140 in FIG. 1A. More specifically, the virtual reality system 10 performs object recognition within image data captured by the imaging device 138 of the HMD 112 to identify the hand 132, optionally including identifying individual fingers or thumbs, and / or all or parts of the arm 134 of the user 110. The virtual reality system 10 tracks the position, orientation, and configuration of the hand 132 (optionally including specific digits of the hand) and / or parts of the arm 134 over a sliding window of time. The virtual reality system 10 analyzes any tracked movement, configuration, position, and / or orientation of the hand 132 and / or arm 134 portions to identify one or more gestures performed by a particular object, e.g., the hand 132 (including particular fingers of the hand) and / or arm 134 portions of the user 110. To detect the gesture(s), the virtual reality application may compare the movement, configuration, position, and / or orientation of the hand 132 and / or arm 134 portions to gesture definitions stored in a gesture library of the virtual reality system 10, where each gesture in the gesture library may each be mapped to one or more actions. In some examples, detecting the movement may include tracking the position of one or more of the fingers (individual fingers and thumb) of the hand 132, including whether one or more fingers are positioned in an area corresponding to a virtual object (e.g., a UI element).In other examples, detecting movement may include tracking the orientation of the hand 132 (e.g., fingers pointing towards or away from the HMD 112) and / or the orientation of the arm 134 (i.e., arm normal pointing towards the HMD 112) relative to the current pose of the HMD 112, including whether an inner wrist region of the hand 132 is pointing towards the HMD 112. The position and orientation of the hand 132 (or portion thereof) may alternatively be referred to as the pose of the hand 132 (or portion thereof).

[0038] According to the techniques of this disclosure, the virtual reality system 10 may interrupt the AR environment from the perspective of the user 110 and generate a interrupt mode environment in response to the user 110 enabling a interrupt mode feature in the HMD 112 during execution of a virtual reality application. For example, the interrupt mode feature may replace the virtual reality environment 124 with the interrupt mode environment in the virtual reality content 122 in response to user input selecting a interrupt mode UI element 142, as described in further detail below, and render one or more avatars from the virtual reality environment 124 in a static position in the HMD 112. The user 110 may then select any avatar 120 present in the interrupt mode environment and perform additional actions on the user corresponding to the selected avatar from the interrupt mode environment, including viewing the user's profile, friending the user, messaging the user, muting the user, blocking the user, or reporting the user. In some examples, the artificial reality application server 107 implements the artificial reality application backend network operations to friend, message, block, or report the user from the suspended mode environment. For example, the artificial reality application may formulate instructions implemented by the artificial reality application server 107 to get a profile, friend, message, mute, block, or report the user based on user input on a user interface presented to the user 110 in the HMD 112. In some examples, the suspended mode feature may replace the user's 110 avatar with a virtual graphic in the synthetic reality environment 124 while the avatar 120 continues to participate in the synthetic reality environment 124. In this way, the avatar 120 may not interact with the avatar of the HMD 112 (and vice versa) while the suspended mode feature is enabled in the HMD 112. In some examples, the suspended mode environment may be referred to as a safe mode or safe zone environment.

[0039] In some examples, the virtual reality system 10 may detect one or more UI activation gestures intended to trigger a desired response by the virtual reality application, such as triggering the display of a UI element used to select the suspended mode environment and / or triggering the execution of the suspended mode environment. For example, the user 110 may move the inner wrist region of the hand 132 into the current field of view 130 to trigger the display of a UI menu 140 (e.g., a virtual icon) near or at least partially on the virtual hand 136 and / or virtual arm 137. This UI activation gesture may be similar to a user looking at a watch or any other wearable device. Although FIG. 1A shows the UI menu 140 displayed on the inner surface (e.g., wrist region) of the virtual arm 137, it should be understood that the UI menu 140 may be displayed near or partially on any surface(s) of the virtual hand 136 and / or virtual arm 137.

[0040] As shown in FIG. 1A , UI menu 140 may include a break mode UI element 142 that, when selected, enables a break mode feature and a personal UI element 144 that, when selected, causes a personal menu item to be displayed.

[0041] In some examples, the personal menu items displayed in response to user input selecting a personal UI element 144 may include a settings UI element that, when selected, allows the user to change the synthetic reality application settings or preferences, a people UI element that, when selected, allows the user to view friends and / or people in the current synthetic reality application or environment, a places UI element that, when selected, allows the user to view information about the current synthetic reality environment 124 or previously visited places / environments, or a search UI element that, when selected, allows the user to search for people, places, synthetic reality experiences (e.g., games, applications), etc. In some examples, the synthetic reality application server 107 performs backend network operations in response to user input in the personal menu items to change the synthetic reality application settings or preferences, obtain information about the user in the synthetic reality environment, obtain information about the synthetic reality environment or other environments, or perform a search based on instructions formulated by the synthetic reality application.

[0042] In some examples, the user 110 may select the break mode UI element 142 or the personal UI element 144 with a selection gesture that includes the placement of one or more fingers on or near the corresponding UI element (e.g., on a physical surface on which the break mode UI element 142 or the personal UI element 144 is rendered or near a physical location corresponding to the UI element). In the example shown in FIG. 1A, the UI menu 140 is presented on an inner surface of the virtual arm 137, and the user 110 may touch a portion of the arm 134 that corresponds to the virtual location of the break mode UI element 142 or the personal UI element 144, which provides natural haptic feedback to the user 110 when selecting either UI element. That is, physical contact between one or more fingers of the user's hand and the user's arm 136 may provide the user with a simulation of the sensations felt by the user when directly interacting with a physical user input object, such as a button on a physical keyboard or other physical input device. This may simplify and improve the accuracy of gesture detection. By defining a UI activation gesture based on the user's hand movements, the user can quickly and intuitively activate the UI menu 140 to enable the suspension mode feature. In this manner, the user 110 can pause the synthetic reality environment 124 without having to remove the HMD 112 and while the HMD 112 continues to receive data corresponding to the synthetic reality environment 124 as other users continue to participate in the synthetic reality environment 124. By continuing to receive data corresponding to the synthetic reality environment 124 while other users continue to participate in the synthetic reality environment 124, the HMD 112 will not have a backlog of data updates to process when the user 110 returns to the synthetic reality environment 124 from the suspension mode environment, which allows the user 110 to return to participating in the synthetic reality environment 124 more quickly by reducing data processing delays for a more pleasing user experience.

[0043] 1B illustrates another example virtual reality system 20 with a suspended mode feature for concurrently executing virtual reality applications in accordance with the techniques of this disclosure. In the example of FIG. 1B, virtual reality system 20 includes external cameras 102A and 102B (collectively, “external cameras 102”), HMDs 112A-112C (collectively, “HMDs 112”), controllers 114A and 114B (collectively, “controllers 114”), console 106, virtual reality application server 107, and sensor 90.

[0044] As shown in FIG. 1B, the artificial reality system 20 represents a multi-user environment in which multiple artificial reality applications executing on the console 106 and / or HMD 112 are running concurrently and displayed on a common rendered scene that is presented to each of the users 110A-110C (collectively, “users 110”) based on the current viewing perspective of the corresponding frame of reference for each user. That is, in this example, each of the multiple artificial reality applications builds the artificial reality environment 124 by tracking and calculating pose information for the frame of reference for each of the HMDs 112. The artificial reality system 20 uses data received from the camera 102, the HMD 112, and the controller 114 to capture 3D information in the real-world environment, such as movements by the user 110 and / or tracking information about the user 110 and the object 108 for use in calculating updated pose information for the corresponding frame of reference of the HMD 112. In some examples, each user 110 may be in the same or different real-world environments (e.g., in the same or different physical locations). Either way, multiple virtual reality applications may render the virtual reality content 122 having virtual objects 128A-128C (collectively, "virtual objects 128") as spatially overlaid on real-world objects 108A-108C (collectively, "real-world objects 108") on the same scene in the virtual reality environment 124 based on a current viewing perspective determined for the HMD 112C. Additionally, from the perspective of the HMD 112C, the virtual reality system 20 renders avatars 120A, 120B within the virtual reality environment 124 based on estimated positions for the users 110A, 110B, respectively. The virtual reality content 122 shown in FIG. 1B may be an example of a virtual reality environment prior to enabling a suspended mode feature in accordance with the techniques of this disclosure.

[0045] Each of the HMDs 112 operates concurrently within the virtual reality system 20. In the example of FIG. 1B, each of the users 110 may be a "participant" (or "player") in multiple virtual reality applications, and any of the users 110 may be a "spectator" or "observer" in multiple virtual reality applications. The HMD 112C may operate substantially similarly to the HMD 112 of FIG. 1A by tracking the hand 132 and / or arm 134 of the user 110C. The HMD 112A may also operate substantially similarly to the HMD 112 of FIG. 1A by tracking the movement of the hands 132A, 132B of the user 110A and receive user input. The HMD 112B may receive user input from a controller 114 held by the user 110B. The controller 114 may be in communication with the HMD 112B using near-field communication of short-range wireless communication such as Bluetooth, using a wired communication link, or using another type of communication link.

[0046] 1B, in addition to or as an alternative to image data captured via camera 138 of HMD 112C, input data from external camera 102 may be used to track and detect specific movements, configurations, positions, and / or orientations of the hand and arm of user 110, such as hand 132 of user 110C, including movements of individual and / or combinations of digits (fingers, thumbs) of the hand. Imaging devices 102 and 138 may capture images in the visible light spectrum, infrared spectrum, or other spectrum. Image processing described herein to identify objects, object poses, and gestures may include, for example, processing infrared images, visible light spectrum images, etc.

[0047] In a manner similar to the example described above with respect to FIG. 1A, the console 106 and / or HMD 112C of the virtual reality system 20 may detect one or more UI activation gestures intended to trigger a desired response by the virtual reality application, such as triggering the display of a UI element used to select a suspension mode environment and / or triggering execution of the suspension mode environment. For example, the user 110 may move the inner wrist region (or any other region) of the hand 132 into the current field of view 130 to trigger the display of a UI menu 140 including a suspension mode UI element 142 and a personal UI element 144, at least partially on or near the virtual wrist of the virtual hand 136. The user 110C may input a user input selecting the suspension mode UI element 142 to enable the suspension mode feature. For example, user 110C may select the pause mode UI element 142 with a touch gesture that includes placement of one or more fingers of the user's hand in an area corresponding to where the pause mode UI element 142 is rendered, such that one or more virtual fingers of an avatar touch the pause mode UI element 142 in the artificial reality environment 124.

[0048] When activated, the suspension mode feature will cause the HMD 112C to freeze the avatars 120A, 120B (e.g., render the avatars 120A, 120B in a static position), mute all audio coming from the artificial reality environment 124 (including from the avatars 120A, 120B), and / or replace the artificial reality environment 124 around the avatars 120A, 120B with a suspension mode environment. That is, the HMD 112C will replace everything in the artificial content 122 except the avatars 120A, 120B with a suspension mode environment that includes a monochromatic background. For example, the HMD 112C will remove (e.g., cease displaying) all virtual objects 128 other than the avatars 120A, 120B while in the suspension mode environment. In some examples, the HMD 112C will remove weapons or handheld objects from the avatars 120A, 120B. In some examples, the HMD 112C will disable the microphone so that the avatars 120A, 120B do not hear the user 110. In some examples, the user 110 may not be able to interact with the synthetic reality environment 124 when the suspended mode feature is enabled. In this manner, the user 110C may pause the synthetic reality environment 124 without having to remove the HMD 112C and while continuing to receive data corresponding to the synthetic reality environment 124 as the users 110A, 110B continue to participate in the synthetic reality environment 124. By continuing to receive data corresponding to the synthetic reality environment 124 while the users 110A, 110B continue to participate in the synthetic reality environment 124 in the HMDs 112A, 112B, respectively, the HMD 112C will not have a backlog of data updates to process when returning to the synthetic reality environment 124 from the suspended mode environment, which allows the user to return to participate in the synthetic reality environment 124 more quickly by reducing data processing delays for a more pleasing user experience.

[0049] In some examples, the interrupt mode feature will move one or more of the frozen avatars away from the user 110C in the interrupt mode environment displayed in the HMD 112C when one or more avatars are within a personal space threshold (e.g., within 1 or 2 meters) from the user when the interrupt mode feature is enabled. For example, the HMD 112C will move avatar 120A back a certain distance (e.g., 2 or 3 meters) if avatar 120 was within a personal space threshold from the user 110C when the interrupt mode feature was activated in the HMD 112C. In some examples, the user 110C may be able to rotate the field of view 130 (e.g., 360 degrees in all directions) while in the interrupt mode environment, but the avatar corresponding to the user 110C will not be able to move from its current location in the interrupt mode environment. In some examples, the avatar corresponding to the user 110C may move around within a certain radius (e.g., 3 meters) in the interrupt mode environment. In some examples, only avatars of one or more other users that were in a 360-degree line of sight from the user 110C when the suspended mode feature was enabled will be rendered while in the suspended mode environment. For example, avatars for other users that were far away, at a higher altitude (e.g., on a virtual mountaintop or any other virtual object), or at a lower altitude (e.g., on the bottom of a virtual ditch) while in the synthetic reality environment 124 when the suspended mode feature was enabled, but were still visible from a 360-degree line of sight from the user 110C, will be displayed in the suspended mode environment at the HMD 112C. Conversely, an avatar for another user that was behind a virtual wall or otherwise occluded by a virtual object when the suspended mode feature was enabled will not be rendered in the suspended mode environment. For example, the HMD 112C will not display an avatar of a user that is hidden behind a virtual object 128A in the suspended mode environment at the HMD 112C, even after removing the virtual object 128A.In some examples, the HMD 112C will play ambient sounds (e.g., background sounds or music) while displaying the interrupted mode environment to the user 110C.

[0050] In some examples, users 110A and / or 110B may also enable a interruption mode feature in HMDs 112A and / or 112B, respectively. For example, user 110A may enable a interruption mode feature in HMD 112A via user input during a period of time during which the interruption mode feature is also enabled in HMD 112C. During that period of time, HMD 112C may display a first interruption mode environment and HMD 112A may display a second interruption mode environment that is different from the first interruption mode environment. For example, user 11C may not be able to interact with avatar 120A in the first interruption mode environment and user 110A may not be able to interact with an avatar corresponding to user 110C in the second interruption mode environment. In some examples, avatar 120A may not be displayed in the first interruption mode environment by HMD 112C when the interruption mode feature is enabled in HMD 112A, and vice versa. It should be understood that while the first interruption mode environment and the second interruption mode environment may differ, both the first interruption mode environment and the second interruption mode environment may include the same background and / or ambient sounds (e.g., the two environments may look and feel the same).

[0051] 2A is a diagram illustrating an example HMD 112 that operates according to the techniques of this disclosure. The HMD 112 of FIG. 2A may be an example of any of the HMDs 112 of FIG. 1A and FIG. 1B. The HMD 112 may be part of a virtual reality system, such as the virtual reality systems 10 and 20 of FIG. 1A and FIG. 1B, or may operate as a standalone mobile virtual reality system configured to implement the techniques described herein.

[0052] In this example, the HMD 112 includes a front rigid body and a band for fixing the HMD 112 to the user. Additionally, the HMD 112 includes an interior-facing electronic display 203 configured to present artificial reality content to the user. The electronic display 203 can be any suitable display technology, such as a liquid crystal display (LCD), a quantum dot display, a dot matrix display, a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a cathode ray tube (CRT) display, e-ink, or a monochrome, color, or any other type of display capable of generating visual output. In some examples, the electronic display is a stereoscopic display for providing separate images to each eye of the user. In some examples, the known orientation and position of the display 203 relative to the front rigid body of the HMD 112 is used as a reference system, also referred to as a local origin, when tracking the position and orientation of the HMD 112 to render the artificial reality content according to the current viewing perspective of the HMD 112 and the user. In other examples, the HMD may take the form of other wearable head-mounted displays, such as glasses or goggles.

[0053] 2A, in this example, the HMD 112 further includes one or more motion sensors 206, such as one or more accelerometers (also referred to as inertial measurement units or "IMUs") that output data indicative of the current acceleration of the HMD 112, a GPS sensor that outputs data indicative of the location of the HMD 112, radar or sonar that outputs data indicative of the distance of the HMD 112 from various objects, or other sensors that provide an indication of the location or orientation of the HMD 112 or other objects in the physical environment. Additionally, the HMD 112 may include integrated imaging devices 138A and 138B (collectively, "imaging devices 138"), such as video cameras, laser scanners, Doppler radar scanners, depth scanners, etc., configured to output image data representative of the physical environment. More specifically, the imaging devices 138 capture image data representative of objects in the physical environment (including the hand 132) that are within the fields of view 130A, 130B of the imaging devices 138, which generally correspond to the viewing perspective of the HMD 112. The HMD 112 includes an internal control unit 210, which may include an internal power supply and one or more printed circuit boards having one or more processors, memory, and hardware to provide an operating environment for processing sensed data and executing programmable operations for presenting artificial reality content on the display 203.

[0054] FIG. 2B is a diagram illustrating another example HMD 112 according to the techniques described in this disclosure. As shown in FIG. 2B, the HMD 112 may take the form of glasses. The HMD 112 of FIG. 2A may be an example of any of the HMDs 112 of FIG. 1A and FIG. 1B. The HMD 112 may be part of a virtual reality system, such as the virtual reality systems 10, 20 of FIG. 1A, FIG. 1B, or may operate as a standalone mobile virtual reality system configured to implement the techniques described herein.

[0055] In this example, the HMD 112 is a pair of glasses with a front frame including a bridge that allows the HMD 112 to rest on the user's nose and temples (or "arms") that extend over the user's ears to secure the HMD 112 to the user. Additionally, the HMD 112 of FIG. 2B includes inward-facing electronic displays 203A and 203B (collectively, "electronic displays 203") configured to present artificial reality content to the user. The electronic displays 203 can be any suitable display technology, such as a liquid crystal display (LCD), a quantum dot display, a dot matrix display, a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a cathode ray tube (CRT) display, e-ink, or any other type of display capable of generating a visual output, such as monochrome, color, or any other type of display. In the example shown in FIG. 2B, the electronic displays 203 form a stereoscopic display to provide separate images to each eye of the user. In some examples, the known orientation and position of the display 203 relative to the front frame of the HMD 112 is used as a reference system, also referred to as a local origin, when tracking the position and orientation of the HMD 112 to render artificial reality content according to the HMD 112 and the user's current viewing perspective.

[0056] 2B, in this example, the HMD 112 further includes one or more motion sensors 206, such as one or more accelerometers (also referred to as inertial measurement units or "IMUs") that output data indicative of the current acceleration of the HMD 112, a GPS sensor that outputs data indicative of the location of the HMD 112, radar or sonar that outputs data indicative of the distance of the HMD 112 from various objects, or other sensors that provide an indication of the location or orientation of the HMD 112 or other objects in the physical environment. Additionally, the HMD 112 may include integrated imaging devices 138A and 138B (collectively, "imaging devices 138"), such as video cameras, laser scanners, Doppler radar scanners, depth scanners, etc., configured to output image data representative of the physical environment. The HMD 112 includes an internal control unit 210, which may include an internal power supply and one or more printed circuit boards having one or more processors, memory, and hardware to process sensed data and provide an operating environment for executing programmable operations for presenting artificial reality content on the display 203.

[0057] 3 is a block diagram illustrating an example implementation of a console 106, a synthetic reality application server 107, and an HMD 112 of a synthetic reality system that generates a suspended mode environment during execution of a synthetic reality application in accordance with techniques of this disclosure. In the example of FIG. 3, the console 106 performs pose tracking, gesture detection, and generation and rendering for a synthetic reality application that runs and outputs content for display on the electronic display 203 of the HMD 112. In some examples, the pose tracking, gesture detection, and / or generation and rendering for the synthetic reality application may be implemented in the HMD 112, as described below with reference to FIG. 4.

[0058] In this example, the HMD 112 includes one or more processors 302 and memory 304 that, in some examples, provide a computer platform for running an operating system 305, which may be, for example, an embedded real-time multitasking operating system, or other type of operating system. The operating system 305, in turn, provides a multitasking operating environment for running one or more software components 307, including an application engine 340. As described with respect to the examples of FIGS. 2A and 2B, the processor 302 is coupled to the electronic display 203, the motion sensor 206, and the imaging device 138. The processor 302 is further coupled to one or more I / O interfaces 319, which provide an I / O interface for communicating with other devices, such as display devices, imaging devices, other HMDs, etc. Additionally, the one or more I / O interfaces 319 may include one or more wired or wireless network interface controllers (NICs) for communicating with a network, such as the network 104. In some examples, the processor 302 and memory 304 may be separate, discrete components. In other examples, memory 304 may be on-chip memory co-located with processor 302 within a single integrated circuit.

[0059] In general, the console 106 is a computing device that processes image and tracking information received from the camera 102 (FIG. 1B) and / or the HMD 112 to perform gesture detection and user interface generation for the HMD 112. In some examples, the console 106 is a single computing device, such as a server, a workstation, a desktop computer, a laptop, or a gaming system. In some examples, at least a portion of the console 106, such as the processor 312 and / or the memory 314, may be distributed across a network, such as a cloud computing system, across a data center, or the Internet, another public or private communications network, e.g., broadband, cellular, Wi-Fi, and / or other type of communications network for transmitting data between computing systems, servers, and computing devices (including HMDs).

[0060] In the example of FIG. 3, the console 106 includes one or more processors 312 and memory 314 that, in some examples, provide a computer platform for executing an operating system 316, which may be, for example, an embedded real-time multitasking operating system, or other type of operating system. The operating system 316 then provides a multitasking operating environment for executing one or more software components 317. The processor 312 is coupled to one or more I / O interfaces 315, which provide one or more I / O interfaces for communicating with devices, such as keyboards, game controllers, display devices, imaging devices, HMDs, input devices, etc. Additionally, the one or more I / O interfaces 315 may include one or more wired or wireless network interface controllers (NICs) for communicating with a network, such as the network 104. Each of the processors 302, 312 may include any one or more of a multi-core processor, controller, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA), or equivalent discrete or integrated logic circuitry. The memory 304, 314 may include any type of memory for storing data and executable software instructions, such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electronically erasable programmable read-only memory (EEPROM), and flash memory.

[0061] The software applications 317 of the console 106 include an application engine 320, a rendering engine 322, a gesture detector 324, a pose tracker 326, a user interface engine 328, a suspended mode engine 332, and a synthetic reality application engine 334. Generally, the application engine 320 includes functionality for providing and presenting synthetic reality applications, e.g., social networking applications, teleconferencing applications, gaming applications, navigation applications, educational applications, training or simulation applications, and the like. The application engine 320 may include, for example, one or more software packages, software libraries, hardware drivers, and / or application program interfaces (APIs) for implementing synthetic reality applications on the console 106. In response to control by the application engine 320, the rendering engine 322 generates 3D synthetic reality content for display to a user by an application engine 340 of the HMD 112.

[0062] The application engine 320 and the rendering engine 322 construct artificial content for display to the user 110 according to the current pose information for the frame of reference determined by the pose tracker 326, typically the viewing perspective of the HMD 112. Based on the current viewing perspective, the rendering engine 322 constructs 3D artificial reality content that, in some cases, may be overlaid, at least in part, on the real-world 3D environment of the user 110. During this process, the pose tracker 326 operates on sensed data received from the HMD 112, such as movement information and user commands, and in some examples, data from any external sensors 90, such as an external camera (FIGS. 1A, 1B), to capture 3D information in the real-world environment, such as movements by and / or feature tracking information about the user 110. Based on the sensed data, the pose tracker 326 determines a current pose with respect to a frame of reference of the HMD 112 and constructs artificial reality content according to the current pose for communication to the HMD 112 via one or more I / O interfaces 315 for display to the user 110. In some examples, the application engine 340 enables a user of the HMD 112 to capture image data of the artificial content to be displayed on the electronic device 203.

[0063] Based on sensed data from either the imaging device 138 or 102, or other sensor device, the gesture detector 324 analyzes tracked movements, configurations, positions, and / or orientations of the HMD 112 and / or the user's physical objects (e.g., hands, arms, wrists, fingers, palms, thumbs) to identify one or more gestures performed by the user 110. More specifically, the gesture detector 324 analyzes recognized objects in image data captured by the imaging device 138 and / or the sensor 90 and external camera 102 of the HMD 112 to identify the hands and / or arms of the user 110, and tracks the movements of the hands and / or arms relative to the HMD 112 to identify gestures performed by the user 110. The gesture detector 324 may track movements, including changes in the positions and orientations of the arms, hands, and / or fingers, based on the captured image data, and compare the object's movement vectors to one or more entries in the gesture library 330 to detect a gesture or combination of gestures performed by the user 110.

[0064] Some entries in the gesture library 330 may each define a gesture as a series or pattern of movements, such as the relative path or spatial translation and rotation of a user's hand, a particular finger, thumb, wrist, and / or arm. Some entries in the gesture library 330 may each define a gesture as a configuration, position, and / or orientation of a user's hand and / or arm (or parts thereof) at a particular time or over a period of time. Other examples of gesture types are possible. Additionally, each entry in the gesture library 330 may specify, for a defined gesture or series of gestures, conditions under which the gesture or series of gestures is required to trigger an action, such as a spatial relationship to the current field of view of the HMD 112, a spatial relationship to a particular area currently being observed by the user, as may be determined by the individual's real-time eye tracking, the type of artificial content being displayed, the type of application being executed, etc.

[0065] Each of the entries in the gesture library 330 may further specify a desired response or action to be performed by the software application 317 for each defined gesture or gesture combination / series of gestures. For example, according to the techniques of this disclosure, several special gestures may be predefined, and thus, in response to detecting one of the predefined gestures, the application engine 340 may generate a UI menu 140 including a suspension mode UI element 142 for display to the user 110 as an overlay on the virtual content of the artificial reality application's artificial reality environment 124 (e.g., as shown in FIG. 1A, FIG. 1B). These predefined gestures for generating the UI menu 140 may include moving the inner wrist region (or any region) of the user's hand into the current field of view of the HMD 112 (e.g., similar to a user looking at a watch or other wearable device). Other gestures may include a selection gesture that includes the placement of one or more fingers of the user's hand near a physical location corresponding to a UI element such that one or more fingers of the virtual hand of the user's avatar in the HMD 112 contacts the UI element in the artificial reality environment.

[0066] In response to the gesture detector 324 detecting a selection gesture on the interrupt mode UI element 142, the interrupt mode engine 332 may enable a interrupt mode feature in the HMD 112 during execution of the synthetic reality application. For example, the interrupt mode engine 332 will replace the synthetic reality environment 124 with the interrupt mode environment, mute all audio coming from the synthetic reality environment 124, and / or render any one or more avatars from the synthetic reality environment in a static pose (e.g., position and orientation) in the HMD 112. In some examples, the interrupt mode environment includes a monochromatic background. In some examples, the interrupt mode engine 332 will move one or more of the frozen avatars away from the user in the interrupt mode environment displayed on the electronic display 203 of the HMD 112 when the interrupt mode feature is enabled and the one or more avatars are within a personal space threshold (e.g., within one or two meters) from the user. In some examples, the interrupted mode engine 332 will play ambient sounds (e.g., background sounds or music) while displaying the interrupted mode environment on the electronic display 203 of the HMD 112. In some examples, the interrupted mode engine 332 will disable the ability of the user of the HMD 112 to capture image data of the artificial content displayed on the electronic device 203. For example, the user of the HMD 112 will not be able to capture image data of a rendered avatar in the interrupted mode environment. In some examples, the interrupted mode engine 332 will disable the microphone of the HMD 112 or cease transmitting audio from the HMD 112 to other users so that other users in the synthetic reality environment cannot hear the user of the HMD 112.

[0067] In some examples, the interrupted mode engine 332 allows a user of the HMD 112 to select a rendered avatar corresponding to another user from the interrupted mode environment and to perform one or more actions corresponding to that avatar / user. For example, the user of the HMD 112 may select an avatar and view the corresponding user's profile, or friend, message, block, or report the user. In some examples, the interrupted mode engine 332 may formulate instructions to be implemented by the virtual reality application server 107 to get the profile, friend, message, mute, block, or report the user based on user input on a user interface presented to the user 110 in the HMD 112.

[0068] In this example, the HMD 112 and the console 106 are shown in communication (e.g., via network 104) with a virtual reality application server 107, which performs back-end network operations for virtual reality applications executing on the console 106 and / or HMD 112. As shown in Figure 3, the virtual reality application server 107 may include a virtual reality application engine 334 and I / O interface(s) 336. The virtual reality application server may include additional hardware and software components, such as one or more processors, memory, operating systems, etc. These other components are not shown for simplicity.

[0069] The synthetic reality application engine 334 may manage relationships (e.g., friendships) between users, maintain profiles for users, coordinate games with one or more users (e.g., placing one or more users in a game, keeping scores), or perform any other network-end operations for running synthetic reality applications on the console 106 and / or HMD 112. For example, the synthetic reality application engine 334 may include functionality for retrieving a profile of a particular user or for friending, messaging, blocking, or reporting a user in response to user input received from the HMD 112 and / or console 106. In some examples, the synthetic reality application engine 334 may include one or more software packages, software libraries, hardware drivers, and / or application program interfaces (APIs) for implementing profile, friending, messaging, blocking, and / or reporting operations at the network level.

[0070] The virtual reality application engine 334 may access one or more databases (not shown) containing user profiles via the I / O interface(s) 336. These user profiles may each include one or more pictures and / or information about a particular user. For example, if a user of the HMD 112 blocks another user, the other user will not be able to see or otherwise interact with the user of the HMD 112 in the virtual reality environment (or in any other virtual reality environment or application) (or vice versa), search for the user of the HMD 112 on the network, message the user of the HMD 112 via the virtual reality application server 107, or become friends with the user of the HMD 112. In some examples, the virtual reality application engine 334 may prevent the two users from being later placed in the same virtual reality environment or application (e.g., in the same game). In some examples, the user of the HMD 112 may indicate whether to immediately block another user upon completion of a current synthetic reality application (e.g., completion of a game) or when the user exits the synthetic reality environment. In some examples, the user of the HMD 112 may indicate to mute another user in the synthetic reality environment and block the other user when the user of the HMD 112 exits the synthetic reality environment (e.g., mute another user during a current game and block the other user when the game ends). In some examples, the blocked user will not be able to see, hear, or otherwise interact with the user and / or the user's avatar in the synthetic reality environment. If the user of the HMD 112 mutes another user, the user of the HMD 112 will not be able to hear audio coming from the other user when the user of the HMD 112 returns to the synthetic reality environment. For example, the interrupt mode engine 332 will mute audio coming from the other user in the HMD 112.If a user of the HMD 112 reports another user, the synthetic reality application engine 334 will flag the user for review by a review entity associated with the synthetic reality environment or application. In some examples, the synthetic reality application engine 334 may record interactions between the user of the HMD 112 and another user within the synthetic reality environment for review.

[0071] In some examples, the console 106 and / or HMD 112 continue to receive data corresponding to the synthetic reality environment (e.g., game data, position data) via the I / O interfaces 315 and / or 319 (e.g., from other HMDs, consoles, and / or synthetic reality application server 107) while the suspended mode feature is enabled in the HMD 112. For example, the console 106 and / or HMD 112 may continue to receive position information for one or more avatars that continue to participate in the synthetic reality environment even if the positions of the one or more avatars in the suspended mode environment are not updated. In this manner, a user of the HMD 112 may pause the synthetic reality environment while other users continue to participate in the synthetic reality environment. By continuing to receive data corresponding to the synthetic reality environment, the HMD 112 and / or console 106 will not have a backlog of data updates to process when the user of the HMD 112 returns to the synthetic reality environment from the suspended mode environment, which allows the user of the HMD 112 to more quickly resume participating in the synthetic reality environment 124 by reducing data processing delays for a more pleasing user experience.

[0072] FIG. 4 is a block diagram illustrating an example HMD 112 and a virtual reality application server of a virtual reality system that generates a suspended mode environment while a virtual reality application is running in accordance with the techniques of this disclosure.

[0073] 3, the HMD 112 includes one or more processors 302 and memory 304 that, in some examples, provide a computer platform for running an operating system 305, which may be, for example, an embedded real-time multitasking operating system, or other type of operating system. The operating system 305, in turn, provides a multitasking operating environment for executing one or more software components 417. Additionally, the processor(s) 302 are coupled to the electronic display 203, the motion sensor 206, the I / O interface 319, and the imaging device 138.

[0074] 4, the software components 417 include an application engine 440, a rendering engine 422, a gesture detector 424, a pose tracker 426, an interface engine 428, and a break mode engine 432. In various examples, the software components 417 operate similarly to their counterpart components (e.g., the application engine 340, the rendering engine 322, the gesture detector 324, the pose tracker 326, the user interface engine 328, and the break mode engine 332) of the console 106 of FIG. 3. In some examples, the rendering engine 422 builds 3D artificial reality content that can be at least partially overlaid on the real-world physical environment of the user 110.

[0075] 3, the application engine 440 and the rendering engine 422 construct artificial content for display to the user 110 according to the current pose information with respect to the frame of reference determined by the pose tracker 426, typically the viewing perspective of the HMD 112. Based on the sensed data, the pose tracker 426 determines the current pose with respect to the frame of reference of the HMD 112 and constructs the artificial reality content for display to the user 110 according to the current pose.

[0076] Further, based on the sensed data, gesture detector 424 analyzes the tracked movement, configuration, position, and / or orientation of the user's objects (e.g., hand, arm, wrist, fingers, palm, thumb) to identify one or more gestures performed by user 110. Gesture detector 424 may track movements, including changes in position and orientation, of the hand, fingers, and / or arm based on the captured image data and compare the object's motion vectors to one or more entries in gesture library 430 to detect a gesture or combination of gestures performed by user 110. Gesture library 430 is similar to gesture library 330 of FIG. 3.

[0077] Based on the gesture detector 424 detecting the UI activation gesture, the application engine 440 may generate a UI menu 140 including a suspension mode UI element 142 for display to the user 110 as an overlay on the virtual content of the synthetic reality environment 124 of the synthetic reality application (e.g., as described above with reference to FIGS. 1A, 1B, 3). The gesture detector may then detect a selection gesture on the suspension mode UI element 142, and the suspension mode engine 432 may, in response, replace the synthetic reality environment of the synthetic reality application with the suspension mode environment, mute all audio coming from the synthetic reality environment, and / or render any avatar or avatars from the synthetic reality environment in a static position, as described above with reference to FIG. The interruption mode engine 432 may further enable a user of the HMD 112 to select a frozen avatar corresponding to another user from the interruption mode environment and perform one or more actions in the HMD 112, including viewing the corresponding user's profile or friending, messaging, blocking, or reporting the user, as described above with reference to FIG. 3. In some examples, the virtual reality application server 107 implements virtual reality application backend network functionality for obtaining the user's profile or friending, messaging, blocking, or reporting the user, as described above with reference to FIG. 3. In some examples, the interruption mode engine 432 implements functionality for muting another user in the HMD 112, as described above with reference to FIG. 3.

[0078] FIG. 5 is a block diagram illustrating a more detailed example implementation of a distributed architecture for a multi-device artificial reality system, where one or more devices are implemented using one or more SoC integrated circuits in each device. In some examples, the artificial reality system includes a peripheral device 502 that operates with the HMD 112. In this example, the peripheral device 502 is a physical real-world device having a surface on which the artificial reality system overlays virtual content. The peripheral device 502 may include one or more presence-sensitive surfaces for detecting user input by detecting the presence of one or more objects (e.g., fingers, stylus) touching or hovering over a location of the presence-sensitive surface. In some examples, the peripheral device 502 may include an output display, which may be a presence-sensitive display. In some examples, the peripheral device 502 may be a smartphone, tablet computer, personal digital assistant (PDA), or other handheld device. In some examples, the peripheral device 502 may be a smart watch, a smart ring, or other wearable device. The peripheral device 502 may also be part of a kiosk or other fixed or mobile system. The peripheral device 502 may or may not include a display device for outputting content to a screen.

[0079] In general, the SoC shown in Figure 5 represents a collection of specialized integrated circuits arranged in a distributed architecture, where each SoC integrated circuit includes various specialized functional blocks configured to provide an operating environment for an artificial reality application. Figure 5 is just one example arrangement of SoC integrated circuits. A distributed architecture for a multi-device artificial reality system may include any collection and / or arrangement of SoC integrated circuits.

[0080] In this example, the SoC 530A of the HMD 112 comprises functional blocks including tracking 570, encryption / decryption 580, coprocessor 582, security processor 583, and interface 584. Tracking 570 provides functional blocks for eye tracking 572 ("eye 572"), hand tracking 574 ("hand 574"), depth tracking 576 ("depth 576"), and / or simultaneous localization and mapping (SLAM) 578 ("SLAM 578"). For example, the HMD 112 may receive input from one or more accelerometers (also referred to as inertial measurement units or "IMUs") that output data indicative of the current acceleration of the HMD 112, a GPS sensor that outputs data indicative of the location of the HMD 112, radar or sonar that outputs data indicative of the distance of the HMD 112 from various objects, or other sensors that provide an indication of the location or orientation of the HMD 112 or other objects in the physical environment. The HMD 112 may also receive image data from one or more imaging devices 588A-588N (collectively, “imaging devices 588”). The imaging devices may include video cameras, laser scanners, Doppler radar scanners, depth scanners, etc. configured to output image data representative of the physical environment. More specifically, the imaging devices capture image data representative of objects in the physical environment (including the peripheral device 502 and / or hands) that are within a field of view of the imaging devices, which generally corresponds to a viewing perspective of the HMD 112. Based on the sensed data and / or image data, the tracking 570, for example, determines a current pose with respect to a frame of reference of the HMD 112 and renders the artificial reality content according to the current pose.

[0081] Encryption / Decryption 580 is a functional block for encrypting outgoing data communicated to the peripheral device 502 or security server and decrypting incoming data communicated from the peripheral device 502 or security server. Encryption / Decryption 580 may support symmetric key cryptography to encrypt / decrypt data with a session key (e.g., a private symmetric key).

[0082] The co-application processor 582 includes various processors, such as a video processing unit, a graphics processing unit, a digital signal processor, an encoder and / or a decoder, and / or others. According to the techniques described in this disclosure, all or part of the back-end shell may be of hardware, software, or a combination of hardware and software. For example, a back-end shell of a concurrent application engine may run on the co-application processor 582. Multiple virtual reality applications may run concurrently on the co-application processor 582.

[0083] The security processor 583 provides secure device attestation and mutual authentication of the HMD 112 when pairing with a device used in cooperation with the AR environment, such as the peripheral device 606. The security processor 583 may authenticate the SoCs 530A to 530C of the HMD 112.

[0084] The interface 584 is a functional block that includes one or more interfaces for connecting to the functional blocks of the SoC 530A. As an example, the interface 584 may include a Peripheral Component Interconnect Express (PCIe) slot. The SoC 530A may connect to the SoCs 530B, 530C using the interface 584. The SoC 530A may connect to a communication device (e.g., a wireless transmitter) using the interface 584 to communicate with other devices, e.g., the peripheral device 136.

[0085] SoCs 530B and 530C each represent a display controller for outputting synthetic reality content on a respective display, e.g., displays 586A, 586B (collectively, “displays 586”). In this example, SoC 530B may include a display controller for display 558A for outputting synthetic reality content for the user's left eye 587A. For example, SoC 530B may include a decryption block 592A, a decoder block 594A, a display controller 596A, and / or a pixel driver 598A for outputting synthetic reality content on display 586A. Similarly, SoC 530C may include a display controller for display 558B for outputting synthetic reality content for the user's right eye 587B. For example, SoC 530C may include a decryption block 592B, a decoder block 594B, a display controller 596B, and / or a pixel driver 598B for generating and outputting synthetic reality content on display 586B. The display 558 may include a light emitting diode (LED) display, an organic LED (OLED), a quantum dot LED (QLED), an electronic paper (E-ink) display, a liquid crystal display (LCD), or other types of displays for displaying AR content.

[0086] Peripheral device 502 includes SoCs 510A and 510B configured to support artificial reality applications. In this example, SoC 510A includes functional blocks including tracking 540, encryption / decryption 550, display processor 552, interface 554, and security processor 556. Tracking 540 is a functional block that provides eye tracking 542 ("eye 542"), hand tracking 544 ("hand 544"), depth tracking 546 ("depth 546"), and / or simultaneous localization and mapping (SLAM) 548 ("SLAM 548"). For example, the peripheral device 502 may receive inputs from one or more accelerometers (also referred to as inertial measurement units or "IMUs") that output data indicative of the peripheral device 502's current acceleration, a GPS sensor that outputs data indicative of the peripheral device 502's location, a radar or sonar that outputs data indicative of the peripheral device 502's distance from various objects, or other sensors that provide an indication of the location or orientation of the peripheral device 502 or other objects in the physical environment. The peripheral device 502 may also receive image data from one or more imaging devices, such as video cameras, laser scanners, Doppler radar scanners, depth scanners, etc., configured to output image data representative of the physical environment, in some examples. Based on the sensed data and / or image data, the tracking block 540 may, for example, determine a current pose of the peripheral device 502 with respect to a frame of reference and render artificial reality content in the HMD 112 according to the current pose.

[0087] The encryption / decryption 550 encrypts outgoing data communicated to the HMD 112 or security server and decrypts incoming data communicated from the HMD 112 or security server. The encryption / decryption 550 may support symmetric key cryptography to encrypt / decrypt data using a session key (e.g., a secret symmetric key).

[0088] The display processor 552 includes one or more processors, such as a video processing unit, a graphics processing unit, an encoder and / or a decoder, and / or others, for rendering synthetic reality content to the HMD 112.

[0089] The interface 554 includes one or more interfaces for connecting to the functional blocks of the SoC 510A. As an example, the interface 584 may include a Peripheral Component Interconnect Express (PCIe) slot. The SoC 510A may connect to the SoC 510B using the interface 584. The SoC 510A may connect to one or more communication devices (e.g., wireless transmitters) using the interface 584 to communicate with other devices, e.g., the HMD 112.

[0090] 5, the security processor 556 provides secure device attestation and mutual authentication of the peripheral device 502 when pairing with a device used in cooperation within the AR environment, such as the HMD 112. The security processor 556 may authenticate the SoCs 510A, 510B of the peripheral device 502.

[0091] The SoC 510B includes a co-application processor 560 and an application processor 562. In this example, the co-application processor 560 includes various processors, such as a visual processing unit (VPU), a graphics processing unit (GPU), and / or a central processing unit (CPU). The application processor 562 may include processing units for executing one or more artificial reality applications, for example, for generating and rendering a virtual user interface on a surface of the peripheral device 502 and / or for detecting gestures performed by a user with respect to the peripheral device 502.

[0092] 6 is a flowchart 600 illustrating example operations for enabling a suspended mode environment for a virtual reality system (e.g., virtual reality systems 10, 20 of FIGS. 1A, 1B) according to an embodiment of the present disclosure. For purposes of example and explanation, the method of FIG. 6 is described with respect to the HMD 112. However, it should be understood that the console 106 may perform one or more of the operations of the method.

[0093] First, the HMD 112 outputs artificial reality content to the user (602). For example, the HMD 112 may output a synthetic reality environment for a synthetic reality application, such as a social networking application, a teleconferencing application, a gaming application, a navigation application, an educational application, a training or simulation application, etc. The synthetic reality content may include one or more avatars corresponding to one or more other users of other HMDs participating in the synthetic reality environment. Next, the HMD 112 may detect a UI activation gesture including the user moving the inner wrist region (or any region) of the user's hand into the field of view of the HMD 112 (e.g., as shown in Figures 1A, 1B, 7), and the HMD 112 will output a UI menu on or near any surface of the virtual hand(s) corresponding, in part, to the user's hand (604). The UI menu may include a break mode UI element and / or a personal UI element, as described above with reference to Figures 1A, 1B.

[0094] The HMD 112 may then detect a break mode selection gesture for a break mode UI element in the UI menu (606). For example, the break mode selection gesture may include placement of one or more fingers (physical or virtual) of the user on or near the corresponding UI element (e.g., on a physical surface on which the break mode UI element is rendered or near a physical location corresponding to where the break mode UI element is rendered). In response to detecting the break mode selection gesture, the HMD 112 enables the break mode, including freezing one or more other avatars (e.g., rendering one or more avatars in a static position) and outputting the break mode environment in the HMD 112 (608). In some examples, one or more users may continue to participate in the synthetic reality environment while the break mode is enabled in the HMD 112. In some examples, the HMD 112 may continue to receive data corresponding to the synthetic reality environment, including game status (e.g., score, remaining life) information and / or location information for one or more avatars in the synthetic environment, but the user of the HMD 112 may not participate in the synthetic reality environment from the suspended mode environment. The HMD 112 may process this data in the background (e.g., without displaying updates in the suspended mode environment).

[0095] The HMD 112 will then determine whether the user wishes to exit the suspension mode (610). For example, the HMD 112 may determine whether the user selected a UI element within the suspension mode environment. In response to determining that the user does not wish to exit the suspension mode (the no branch of 610), the HMD 112 will remain in the suspension mode (e.g., returning to 608). In response to determining that the user wishes to exit the suspension mode (the yes branch of 610), the HMD 112 will output the synthetic reality content, including the synthetic reality environment, to the user of the HMD 112 (e.g., returning to 602). Because the HMD 112 has continued to receive and process data corresponding to the synthetic reality environment, the HMD 112 may return the user from the suspension mode environment to the synthetic reality environment more quickly because the HMD 112 will not have a backlog of data to process. In this manner, the transition between the suspension mode environment and the synthetic reality environment may be sharp (e.g., instantaneous or otherwise rapid). In some examples, the HMD 112 may render a fading effect when transitioning between the suspended mode environment and the synthetic reality environment.

[0096] 7 is a flowchart 700 illustrating example operations within a suspended mode environment for a virtual reality system (e.g., virtual reality systems 10, 20 of FIGS. 1A, 1B) according to aspects of the disclosure. For purposes of example and explanation, the method of FIG. 7 is described with respect to an HMD 112. However, it should be understood that the console 106 may perform one or more of the operations of the method. For example, the flowchart 700 is an example of functionality (e.g., step 608 of the flowchart 600 of FIG. 6) performed by the suspended mode engine 332 or 432 to enable a suspended mode feature in the HMD 112.

[0097] The HMD 112 replaces the synthetic reality environment of the synthetic reality application with the suspended mode environment (702). For example, the HMD 112 may replace the synthetic reality content rendered in the HMD 112 with a monochromatic background. This may include ceasing to display structures, ground / floor, objects (including handheld objects such as weapons), etc. that were in the synthetic reality environment. For example, if an avatar was standing on a table when the suspended mode was enabled, the table will not be displayed in the suspended mode environment, and the avatar may appear to be floating in the suspended mode environment. In some examples, the background of the suspended mode environment may be changed by a user of the HMD 112 (e.g., to other color(s), to other images, or to include one or more patters).

[0098] The HMD 112 will also mute audio from the synthetic reality environment at the HMD 112 (704). For example, the HMD 112 will mute music, voice or other noises from one or more avatars, background noise, etc. In some examples, the HMD 112 may play ambient sounds (e.g., background sounds, music, or white noise) corresponding to the suspended mode environment. For example, the HMD 112 may play sounds from nature (e.g., waves lapping on a beach), relaxing music, or other sounds. In some examples, the ambient sounds played in the suspended mode environment may be changed by the user of the HMD 112 (e.g., to other music or sounds). In some examples, the HMD 112 disables its microphone or ceases transmitting audio to other users or devices (e.g., other HMDs, the console 106) so that other users in the synthetic reality environment cannot hear the user of the HMD 112.

[0099] The HMD 112 will render one or more avatars corresponding to the other users in a static pose (e.g., position and orientation) within the suspended mode environment (706). The static pose may correspond to a pose that the one or more avatars were in within the artificial reality environment when the suspended mode was enabled in the HMD 112. In some examples, the HMD 112 may determine whether one or more of the rendered avatars are within a personal space threshold (e.g., within 1 or 2 meters) from a user of the HMD 112 (708). In response to a determination that one or more rendered avatars are within the personal space threshold (yes branch of 708), the HMD 112 moves one or more rendered avatars back a certain distance (e.g., 2 or 3 meters). For example, the HMD 112 may render an animation that slides one or more rendered avatars back within the personal space threshold. In some examples, the HMD 112 may simply render one or more rendered avatars in a static position at or above the personal space threshold. Either way, flowchart 700 continues at 712 .

[0100] In response to determining that one or more rendered avatars are not within the personal space threshold (the no branch of 708), the HMD 112 performs one or more actions corresponding to one or more rendered avatars in the suspended mode environment based on the user input (712). For example, the user may select another user's avatar (e.g., as described above with reference to FIGS. 1A, 3) to friend, message, mute, block, or report the user from the suspended mode environment. In some examples, the virtual reality application server 107 performs back-end network operations to friend, message, block, or report the user from the suspended mode environment (e.g., as described above with reference to FIGS. 1A, 3). For example, if a user of an HMD 112 blocks another user at the network level, the other user will not be able to see or otherwise interact with the user of the HMD 112 in the synthetic reality environment (or in any other synthetic reality environment or application) (or vice versa), search for the user of the HMD 112 on the network, message the user of the HMD 112, or become friends with the user of the HMD 112. If the user of the HMD 112 and the other user were already friends, blocking the other user may unfriend the two users. In some examples, the console 106 may prevent the two users from being later placed in the same synthetic reality environment or synthetic reality application (e.g., in the same game). In some examples, the user of the HMD 112 may indicate whether to block the other user immediately (so that the blocking will be effective when the user returns to the synthetic reality environment from the suspended mode environment) after completion of the current synthetic reality application (e.g., completion of a game) or when the user exits the synthetic reality environment.In some examples, a blocked user will not be able to see, hear, or otherwise interact with the user and / or the user's avatar in the synthetic reality environment. If a user of the HMD 112 mutes another user, the user of the HMD 112 will not be able to hear audio coming from the other user when the user of the HMD 112 returns to the synthetic reality environment. In some examples, the HMD 112 will continue to receive audio data from the other user, but will mute the audio.

[0101] FIG. 8 illustrates a user interaction with an exemplary gesture-driven UI for entering a suspension mode environment during execution of a virtual reality application, according to the techniques of this disclosure. In particular, FIG. 8 illustrates virtual hands 136A, 136B (collectively, “virtual hands 136”) and a UI menu 140 including a suspension mode UI element 142 and a personal UI element 144 near the virtual hand 136A. In some examples, the HMD 112 may have been rendered in response to the HMD 112 detecting that an inner wrist region of the user's hand corresponding to the virtual hand 136 has come into the field of view of the HMD 112 (e.g., after detecting a UI activation gesture as described above with reference to FIG. 1A). FIG. 8 further illustrates the user performing a suspension mode selection gesture including placement of a virtual finger 802 on the suspension mode UI element 142, which corresponds to the user placing the finger in a physical area corresponding to where the suspension mode UI element was rendered. In response to the HMD 112 detecting (e.g., via captured image data of a user's hand) a selection gesture for the break mode UI element 142, the HMD 112 enables the break mode feature. By defining UI activation and selection gestures based on the user's hand movements, a user may quickly and intuitively enable the break mode feature.

[0102] FIG. 9 illustrates an example synthetic reality environment 924 of a synthetic reality application presented to a user at an HMD 112 in accordance with the techniques of this disclosure. In the example illustrated in FIG. 9, the synthetic reality environment 924 includes avatars 920A, 920B, 920C, 920D, 920E, and 920F (collectively, “avatars 920”), virtual objects 928A and 928B (collectively, “virtual objects 928”), and a virtual handheld object 914. In particular, the avatars 920 may correspond to other users of the HMD, and the synthetic reality environment 924 may correspond to a gaming application. In some examples, one or more of the avatars 920 may wear costumes, masks, or accessories. For example, the avatar 920E is wearing the mask 902 in FIG. 9. In some examples, the outfit or costume worn by each avatar may be personalized by the respective user. In some examples, the avatars 920 may include graphic elements with information about the corresponding user. For example, Figure 9 shows a graphical element with the corresponding user's first name for avatar 920. In other examples, these graphics may additionally or alternatively include game scores, remaining life, or any other information regarding the corresponding user. In some examples, this graphical element may not be included in the synthetic reality environment 924. The synthetic reality environment 924 shown in Figure 9 may be an example of a synthetic reality environment prior to enabling a suspended mode feature in accordance with the techniques of this disclosure.

[0103] Although FIG. 9 shows six avatars corresponding to six different users, other avatars may be present in the artificial reality environment 924 but may not be visible to the user of the HMD 112 because they are not in the field of view of the HMD 112, are behind a virtual object (e.g., behind a virtual wall 928A), or are too far away (e.g., located beyond a visibility threshold). For example, the user of the HMD 112 may see the additional avatars by moving his field of view (e.g., turning around or looking behind the virtual wall 928A). FIG. 9 also shows a virtual graphic 904 corresponding to an unavailable user (i.e., Jacob). The virtual graphic 904 may include a profile picture of the user and a message indicating that the user is away or otherwise unavailable. In some examples, the user corresponding to the virtual graphic 904 may not be available because the user has paused his HMD, removed his HMD, enabled a suspend mode feature in his HMD, etc. In either case, the virtual graphic 904 may or may not include an indication of why the user is unavailable. In some examples, the virtual graphic 904 may be a "coin" or token that corresponds to the user.

[0104] Figure 10 is a diagram illustrating an example suspended mode environment 1024 in accordance with the techniques of this disclosure. In the example shown in Figure 10, the suspended mode environment 1024 corresponds to the synthetic reality environment 924 in Figure 9. That is, Figure 10 is an example suspended mode environment 1024 displayed in the HMD 112 after a user enables a suspended mode feature while participating in the synthetic reality environment 924 of Figure 9.

[0105] As shown in Figure 10, the suspended mode environment 1024 includes avatars 1020A, 1020B, 1020C, 1020D, 1020E, and 1020F (collectively, "avatars 1020"), which correspond to avatars 920A, 920B, 920C, 920D, 920E, and 920F, respectively, of Figure 9. The HMD 112 presents the avatars 1020 in a static pose (e.g., position and orientation) corresponding to the pose the avatars 1020 were in when a user of the HMD 112 enabled the suspended mode feature. Although the avatars 1020 are presented in a static pose in the suspended mode environment 1024 in the HMD 112, the avatars 920 may continue to participate in the synthetic environment 924 of Figure 9. The HMD 112 may continue to receive data regarding the synthetic reality environment 924, including pose information for each of the avatars 920, while the suspended mode feature is enabled. That is, the synthetic reality environment 924 is not paused or otherwise stopped for the avatar 920 when the user of the HMD 112 enters the suspension mode environment 1024. In particular, the HMD 112 will not display the virtual background of the synthetic environment 924 (including the virtual object 928) in the suspension mode environment 1024, but instead present a different background (e.g., a monochromatic background). For example, the suspension mode environment 1024 will not include the virtual object 928 or the virtual handheld object 914 of FIG. 9. In particular, the HMD 112 will not display in the suspension mode environment 1024 an avatar that was behind (e.g., occluded by) the virtual object 928 in the synthetic reality environment 924. In some examples, the HMD 112 continues to present the costume, mask, or accessories of the avatar 920. For example, the HMD 112 will present the avatar 1020E with the mask 1002, which corresponds to the avatar 920E and mask 902 in FIG. In some examples, avatars 1020 may include graphical elements with the first names of the corresponding users for some of avatars 920. In other examples, these graphics may additionally or alternatively include game scores, remaining lives, or any other information regarding the corresponding users.In some examples, this graphical element may not be included in the artificial reality environment 924, but may be included in the suspended mode environment 1024. In some examples, this graphical element may not be included in the suspended mode environment 1024.

[0106] The HMD 112 may further display a number of virtual UI elements corresponding to one or more avatars that were in the virtual reality environment 924 when the user enabled the interrupt mode feature. For example, the interrupt mode environment 1024 includes UI elements 1050A, 1050B, 1050C, 1050D, and 1050 (collectively, "UI elements 1050" or "information panels 1050"). Each of the UI elements 1050 may include a picture, a name, or any other information corresponding to a respective user of the avatar, as shown in FIG. 10. In some examples, the UI elements 1050 may include one or more UI elements for avatars that are not currently visible in the interrupt mode environment 1024. For example, the avatar corresponding to UI element 1050A in FIG. 10 is not visible. The avatar may not be visible because the avatar was not in the line of sight of the HMD 112, was behind a virtual object or another avatar, or was "away" when the user of the HMD 112 enabled the suspension mode feature. The HMD 112 may also present UI element 1054 with instructions, additional information, and / or virtual UI elements for exiting the suspension mode environment 1024 and returning to the artificial reality environment 924. In the example shown in FIG. 10, the UI element 1054 indicates that the user of the HMD 112 may tap a user profile (e.g., any of the UI elements 1050) for additional options. In this example, the user has selected avatar 1020A by tapping UI element 1050C. In some examples, the user of the HMD 112 may select avatar 1020 by looking at or pointing at the avatar. In some examples, a user of the HMD 112 may select an avatar 1020 by voice selection, including, for example, verbally describing the physical appearance of the avatar (e.g., describing the avatar's size, clothing, accessories, or any other distinctive characteristic) or saying the avatar's name. In this way, the user does not have to scroll through various UI elements 1050 to find a particular avatar / user.In some examples, the HMD 112 may visually mute the avatars 1050 until they are selected. For example, the avatars 1020B-1020F may be presented in grayscale or otherwise presented differently than in the artificial reality environment 924 (represented by dotted lines in FIG. 10). The HMD 112 may also highlight and / or present the selected avatar 1020A in true color (e.g., as in the artificial reality environment 924). For example, the HMD 112 may present a spotlight or base 1056 under the avatar 1020A and a line 1058 from the user of the HMD 112 to the base 1056. The base 1056 and the line 1058 may move between the avatars 1020 when the user selects a different avatar 1020.

[0107] In response to selecting the user 1020A, the HMD 112 renders a interruption mode UI 1052 that includes selectable virtual elements that enable the user to engage with the user through the interruption mode UI 1052. For example, the user may friend the user 1020A, message the user 1020A, mute the user 1020A, block the user 1020A, and / or report the user 1020A by selecting a corresponding virtual element in the interruption mode UI 1052. In some examples, the interruption mode UI 1052 may include virtual UI elements for exiting the interruption mode environment 1024 and for returning to the synthetic reality environment 924.

[0108] In some examples, blocking is performed at a network or server level (e.g., by the synthetic reality application engine 334 described with reference to FIG. 3) and muting is performed at an HMD level (e.g., by the suspend mode engine 432 described with reference to FIG. 4). For example, if a user blocks user 1020A on the network, user 1020A will not be able to see or otherwise interact with the user in synthetic reality environment 924 (or in any other synthetic reality environment or application), search for the user on the network, message the user, or befriend the user after the user exits the suspend mode environment. In some examples, the user may indicate whether user 902 should be blocked immediately after completion of the current synthetic reality application (e.g., completion of a game) or when the user exits synthetic reality environment 924. In some examples, a user of the HMD 112 may indicate that they would like to mute another user within the synthetic reality environment and block that other user when the user of the HMD 112 exits the synthetic reality environment (e.g., mute another user during a current game and block the other user when the game ends). In some examples, the blocked user will not be able to see, hear, or otherwise interact with the user and / or the user's avatar. For example, referring again to FIG. 9, if the user of the HMD 112 blocks avatar 920A and avatar 920B later speaks to the user when the user of the HMD 112 returns to the synthetic reality environment 924, avatar 920B will appear to not be speaking to anyone from the perspective of the blocked user corresponding to avatar 920A.

[0109] If a user mutes another user, the user will not be able to hear audio coming from the other user while in the synthetic reality environment 924 because the HMD 112 will mute the audio coming from the other user. If a user of the HMD 112 reports another user, the console 106 (or any other device from the synthetic reality system) will flag the user for review by a review entity associated with the synthetic reality environment or application. In some examples, the synthetic reality system may record interactions between the user of the HMD 112 and another user within the synthetic reality environment 924 for review.

[0110] The avatar 920 will continue to participate in the synthetic reality environment 924 while the suspended mode feature is enabled in the HMD 112, but the avatar corresponding to the user of the HMD 112 will be disabled in the synthetic reality environment 924, and the avatar 920 of FIG. 9 will not be able to observe or interact with the avatar corresponding to the user of the HMD 112. Instead, the user corresponding to the avatar 920 may see a graphic similar to the indication 904 of FIG. 9 indicating that the user is unavailable, without indicating that the user of the HMD 112 has enabled suspended mode. In some examples, the user of the HMD 112 may not be able to participate in the synthetic reality environment 924. In some examples, the user of the HMD 112 may not be able to photograph or record the avatar 1020 in the suspended mode environment 1024. Because a user's avatar may represent an extension of the user (e.g., the user's agent or persona in a synthetic reality environment), the suspended mode feature can help ensure that other users do not photograph, record, or interact with the user's avatar, and vice versa, while the suspended mode feature is activated, without the user's knowledge or consent.

[0111] While the user is in the suspended mode environment 1024, network communication continues between the user's HMDs in the synthetic reality environment (including the user's HMD in that mode). For example, the HMD 112 will continue to receive updates from the other HMDs regarding the synthetic reality application, the synthetic reality environment, and / or other avatars. In this manner, the user may pause the synthetic reality application and immediately re-enter the synthetic reality environment 1024 at a later time in the synthetic reality application's current state. Because the synthetic reality application continued while the user was in the suspended mode environment 1024, any of the avatars 920 may be located in a different location and / or may have exited the synthetic reality environment 924 when the user exits the suspended mode environment 1024 and re-enters the synthetic reality environment 924. Because the HMD 112 will have already processed updates to process while suspended mode was enabled, the HMD 112 will be able to immediately display the updated pose of the avatar 920 because the HMD 112 will not have a backlog of those updates.

[0112] It should be understood that the suspended mode feature is different from the paused mode. For example, in a suspended mode environment, a user of the HMD 112 may not be able to access or update synthetic reality application settings or preferences. As another example, an avatar for a user of the HMD 112 may be replaced with a virtual graphic (e.g., virtual graphic 904 of FIG. 9 ) in the synthetic reality environment 924 while the avatar 920 continues to participate in the synthetic reality environment 924. In this manner, the avatar 920 may not interact with the avatar of the HMD 112 (and vice versa) while the suspended mode feature is enabled in the HMD 112.

[0113] FIG. 11 is a diagram illustrating another example of a suspended mode environment 1124 according to the techniques of this disclosure. In the example shown in FIG. 11, the suspended mode environment 1124 is shown from a different perspective than the suspended mode environment 1024 of FIG. 10. In particular, the suspended mode environment 1124 is shown from a third-person perspective in FIG. 11. The suspended mode environment 1124 includes an avatar 1110 corresponding to a user of the HMD 112 and avatars 1120A, 1120B, and 1120C (collectively, "avatars 1120") corresponding to other users in the synthetic reality environment. As in FIG. 10, the HMD 112 presents the avatar 1020 in a static pose in the suspended mode environment 1124. As shown in FIG. 11, the avatar 1110 may be presented on a platform or base 1160.

[0114] In some examples, the HMD 112 will move one or more avatars 1120 away from the avatar 1110 when the one or more avatars 1120 are within a personal space threshold (e.g., within 1 or 2 meters) from the avatar 1110, as described above with reference to FIG. 7. In some examples, the avatar 1110 may be able to rotate its field of view while in the suspended mode environment, but will not be able to move from its current location in the suspended mode environment 1124. For example, the avatar 1110 may turn 360 degrees and look up and down, but will not be able to walk to another location in the suspended mode environment 1124. In some examples, the avatar 1110 may move around within a radius platform 1160 in the suspended mode environment. In some examples, only avatars that were in a 360-degree line of sight from the avatar 1110 when the user of the HMD 112 enabled the suspended mode feature will be rendered in the suspended mode environment. For example, an avatar 1120 that was at a higher level / altitude (e.g., on top of a virtual mountaintop or any other virtual object) or at a lower level / altitude (e.g., on the bottom of a virtual ditch) but within a 360-degree line of sight from the avatar 1110 (even though the user may not be looking at that avatar) when the user enabled the suspension mode feature will be displayed when that avatar is within the field of view of the HMD in the suspension mode environment 1124. An avatar for another user that was behind a virtual wall or otherwise occluded by a virtual object may not be rendered in the suspension mode environment 1124. In some examples, the HMD 112 may continue to render the clothing and accessories of the avatar 1120 in the suspension mode environment 1124, including clothing, headgear, shoes, costume elements, or any other elements that make up the avatar's persona. For example, the HMD 112 may continue to render virtual headphones 1162 on the avatar 1120B. In this manner, the user may more easily identify each of the avatars 1120 within the suspended mode environment 1124.

[0115] While in the suspension mode environment 1124, the avatar 1110 may select any of the avatars 1120 by performing a selection gesture on any of the UI elements 1150A, 1150B, or 1150C (collectively, "UI elements 1150") to perform additional actions (e.g., as described with reference to the UI elements 1050 in FIG. 10). For example, when the avatar 1110 selects one of the UI elements 1150 (e.g., through a touch gesture), the HMD 112 renders a virtual suspension mode UI 1152. When an avatar is selected, the HMD 112 highlights the selected avatar and / or de-emphasizes the other avatars (e.g., grays out the other avatars). For example, the avatar 1110 may select the avatar 1120A, and the HMD 112 may render a spotlight or base 1156 under the avatar 1120A and present a line 1158 from the platform 1160 to the base 1056. The base 1056 and the line 1058 may move between the avatars 1120 when the user selects a different avatar 1020. Additionally or alternatively, the HMD 112 de-emphasizes the non-selected avatars 1120B and 1120C, for example, by graying them out or fading them out (represented as dotted lines in FIG. 11 ).

[0116] The techniques described in this disclosure may be implemented, at least in part, in hardware, software, firmware, or any combination thereof. For example, various aspects of the described techniques may be implemented in one or more processors, including one or more microprocessors, DSPs, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or any other equivalent integrated or discrete logic circuits, as well as any combination of such components. The term "processor" or "processing circuitry" may generally refer to any of the above logic circuits, alone or in combination with other logic circuits, or any other equivalent circuitry. A control unit comprising hardware may also implement one or more of the techniques of this disclosure.

[0117] Such hardware, software, and firmware may be implemented within the same device or within separate devices to support various operations and functions described in this disclosure. Furthermore, any of the described units, modules, or components may be implemented together or separately as distinct but interoperable logical devices. The illustration of different features as modules or units is intended to highlight different functional aspects and does not necessarily imply that such modules or units must be realized by separate hardware or software components. Rather, the functionality associated with one or more modules or units may be performed by separate hardware or software components or incorporated within common or separate hardware or software components.

[0118] The techniques described in this disclosure may also be embodied or encoded in a computer-readable medium, such as a computer-readable storage medium containing instructions. The instructions embedded or encoded in the computer-readable storage medium may, for example, cause a programmable processor, or other processor, to perform a method when the instructions are executed. The computer-readable storage medium may include a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electronically erasable programmable read-only memory (EEPROM), a flash memory, a hard disk, a CD-ROM, a floppy disk, a cassette, a magnetic medium, an optical medium, or other computer-readable medium.

[0119] As described in various examples herein, the techniques of the present disclosure may include or be implemented with a synthetic reality system. As described, synthetic reality is a form of reality that is adjusted in some manner before presentation to a user, which may include, for example, virtual reality (VR), augmented reality (AR), mixed reality (MR), hybrid reality, or any combination and / or derivative thereof. Synthetic reality content may include fully generated content or generated content combined with captured content (e.g., real-world photos). Synthetic reality content may include video, audio, haptic feedback, or any combination thereof, any of which may be presented in a single channel (such as stereo video that creates a three-dimensional effect for the viewer) or multiple channels. Furthermore, in some embodiments, the synthetic reality may be associated with applications, products, accessories, services, or any combination thereof, for example, used to create content in the synthetic reality and / or used in the synthetic reality (e.g., performing activities in the synthetic reality). A virtual reality system that provides virtual reality content may be implemented on a variety of platforms, including a head mounted display (HMD) connected to a host computer system, a standalone HMD, a mobile device or computing system, or any other hardware platform capable of providing virtual reality content to one or more observers.

[0120] In particular embodiments, one or more objects (e.g., content or other types of objects) of a computing system may be associated with one or more privacy settings. The one or more objects may be stored in or possibly associated with any suitable computing system or application, such as, for example, a social networking system, a client system, a third party system, a social networking application, a messaging application, a photo sharing application, or any other suitable computing system or application. Although the examples described herein are in the context of an online social network, these privacy settings may apply to any other suitable computing system. The privacy settings (or "access settings") for an object may be stored in any suitable manner, such as, for example, in association with the object, in an index on an authorization server, in another suitable manner, or any suitable combination thereof. The privacy settings for an object may specify how the object (or particular information associated with the object) may be accessed, stored, or possibly used (e.g., viewed, shared, modified, copied, executed, surfaced, or identified) within the online social network. When the privacy settings for an object allow a particular user or other entity to access the object, the object may be described as being "visible" with respect to that user or other entity. By way of example and not by way of limitation, a user of an online social network may specify privacy settings for a user profile page that identify a set of users who may access work history information on the user profile page, thus excluding other users from accessing that information.

[0121] In certain embodiments, a privacy setting for an object may specify a "blocked list" of users or other entities that should not be allowed to access certain information associated with the object. In certain embodiments, the blocked list may include third party entities. The blocked list may specify one or more users or entities for which the object is not visible. By way of example and not by way of limitation, a user may specify a set of users that should not have access to a photo album associated with the user, thus excluding those users from accessing the photo album (and possibly allowing some users not in the user's designated set to access the photo album). In certain embodiments, privacy settings may be associated with a particular social graph element. The privacy settings of a social graph element, such as a node or edge, may specify how the social graph element, information associated with the social graph element, or objects associated with the social graph element may be accessed using an online social network. By way of example and not by way of limitation, a particular concept node corresponding to a particular photo may have a privacy setting that specifies that the photo may only be accessed by the user tagged in the photo and friends of the user tagged in the photo. In particular embodiments, privacy settings may allow a user to opt in or out of having their content, information, or actions stored / logged by the social networking system or shared with other systems (e.g., third party systems). Although this disclosure describes the use of particular privacy settings in particular manners, this disclosure contemplates the use of any suitable privacy settings in any suitable manner.

[0122] In certain embodiments, the privacy settings may be based on one or more nodes or edges of the social graph. The privacy settings may be specified for one or more edges or edge types of the social graph, or for one or more nodes or node types of the social graph. A privacy setting applied to a particular edge connecting two nodes may control whether a relationship between two entities corresponding to the nodes is visible to other users of the online social network. Similarly, a privacy setting applied to a particular node may control whether a user or concept corresponding to the node is visible to other users of the online social network. By way of example and not by way of limitation, a first user may share an object to a social networking system. The object may be associated with a concept node connected by an edge to the first user's user node. The first user may specify a privacy setting that applies to a particular edge connecting to the object's concept node, or may specify a privacy setting that applies to all edges connecting to the concept node. By way of another example and not by way of limitation, a first user may share a set of objects (e.g., a set of images) of a particular object type. The first user may specify privacy settings for all objects associated with the first user of a particular object type as having a particular privacy setting (e.g., specifying that all images posted by the first user are visible only to the first user's friends and / or users tagged in the images).

[0123] In certain embodiments, the social networking system may present a "privacy wizard" to the first user (e.g., within a web page, a module, one or more dialog boxes, or any other suitable interface) to assist the first user in specifying one or more privacy settings. The privacy wizard may display instructions, preferred privacy-related information, current privacy settings, one or more input fields for accepting one or more inputs from the first user specifying changes or confirmations of privacy settings, or any suitable combination thereof. In certain embodiments, the social networking system may provide the first user with "dashboard" functionality that may display the first user's current privacy settings to the first user. The dashboard functionality may be displayed to the first user at any suitable time (e.g., following input from the first user that summons the dashboard functionality, following the occurrence of a particular event or trigger action). The dashboard functionality may enable the first user to modify one or more of the first user's current privacy settings at any time and in any suitable manner (e.g., redirecting the first user to the privacy wizard).

[0124] The privacy settings associated with an object may specify any suitable granularity of granted access or denial of access. By way of example and not by way of limitation, access or denial of access may be specified for particular users (e.g., only me, my roommates, my boss), users within a particular degree of separation (e.g., friends, friends of friends), user groups (e.g., gaming clubs, my family), user networks (e.g., employees of a particular employer, students or alumni of a particular university), all users ("public"), no users ("private"), users of a third party system, particular applications (e.g., third party applications, external websites), other suitable entities, or any suitable combination thereof. Although this disclosure describes a particular granularity of granted access or denial of access, this disclosure contemplates any suitable granularity of granted access or denial of access.

[0125] In certain embodiments, one or more servers may be authorization / privacy servers for enforcing privacy settings. In response to a request from a user (or other entity) for a particular object stored in the data store, the social networking system may send a request for the object to the data store. The request may identify the user associated with the request, and the object may only be sent to the user (or the user's client system) if the authorization server determines that the user is authorized to access the object based on the privacy settings associated with the object. If the requesting user is not authorized to access the object, the authorization server may prevent the requested object from being retrieved from the data store or may prevent the requested object from being sent to the user. In a search query context, an object may be provided as a search result only if the querying user is authorized to access the object, e.g., only if the privacy settings for the object allow the object to be surfaced to, discovered by, or possibly visible to the querying user. In certain embodiments, the object may represent content that is visible to the user through the user's news feed. By way of example and not by way of limitation, one or more objects may be visible on the user's "trending" page. In certain embodiments, an object may correspond to a particular user. An object may be content associated with a particular user, or may be a particular user's account or information stored in a social networking system or other computing system. By way of example and not by way of limitation, a first user may view one or more second users of an online social network through a "People You May Know" feature of the online social network or by viewing the first user's list of friends.By way of example and not by way of limitation, a first user may specify that the first user does not want to see an object associated with a particular second user in the first user's news feed or friend list. If the privacy settings for the object do not allow the object to be surfaced to, discovered by, or visible to the user, the object may be filtered out from search results. Although this disclosure describes enforcing privacy settings in a particular manner, this disclosure contemplates enforcing privacy settings in any suitable manner.

[0126] In certain embodiments, different objects of the same type associated with a user may have different privacy settings. Different types of objects associated with a user may have different types of privacy settings. By way of example and not by way of limitation, a first user may specify that the first user's status updates are public, but images shared by the first user are visible only to the first user's friends on the online social network. By way of another example and not by way of limitation, a user may specify different privacy settings for different types of entities, such as individual users, friends of friends, followers, user groups, or corporate entities. By way of another example and not by way of limitation, a first user may specify groups of users that may view videos posted by the first user, while not making those videos visible to the first user's employer. In certain embodiments, different privacy settings may be provided for different user groups or demographics. By way of example and not by way of limitation, a first user may specify that other users who attend the same university as the first user may view the first user's pictures, but other users who are family members of the first user may not view those same pictures.

[0127] In certain embodiments, a social networking system may provide one or more default privacy settings for each object of a particular object type. The privacy settings for an object set to a default value may be changed by a user associated with the object. By way of example and not by way of limitation, all images posted by a first user may have a default privacy setting that is visible only to the first user's friends, and for a particular image, the first user may change the privacy setting so that the image is visible to friends and friends of friends.

[0128] In certain embodiments, privacy settings may allow a first user to specify (e.g., by opting out, by not opting in) whether the social networking system may receive, collect, log, or store certain objects or information associated with the first user for any purpose. In certain embodiments, privacy settings may allow a first user to specify whether a particular application or process may access, store, or use certain objects or information associated with the first user. Privacy settings may allow a first user to opt in or out of an object or information being accessed, stored, or used by a specific application or process. The social networking system may access such information to provide a particular function or service to the first user, and the social networking system may not have access to the information for any other purpose. Before accessing, storing, or using such objects or information, the social networking system may prompt the user to provide privacy settings that specify which applications or processes, if any, may access, store, or use the object or information before allowing any such action. By way of example and not by way of limitation, a first user may send a message to a second user via an application related to an online social network (e.g., a messaging app) and may specify privacy settings that such messages should not be stored by the social networking system.

[0129] In certain embodiments, a user may specify whether a particular type of object or information associated with a first user may be accessed, stored, or used by the social networking system. By way of example and not by way of limitation, a first user may specify that images sent by the first user through the social networking system may not be stored by the social networking system. By way of another example and not by way of limitation, a first user may specify that messages sent from the first user to a particular second user may not be stored by the social networking system. By way of yet another example and not by way of limitation, a first user may specify that all objects sent through a particular application may be saved by the social networking system.

[0130] In certain embodiments, privacy settings may allow a first user to specify whether a particular object or information associated with the first user may be accessed from a particular client system or third party system. Privacy settings may allow a first user to opt in or out of objects or information being accessed from a particular device (e.g., a phone book on the user's smartphone), from a particular application (e.g., a messaging app), or from a particular system (e.g., an email server). The social networking system may provide default privacy settings for each device, system, or application, and / or the first user may be prompted to specify particular privacy settings for each context. By way of example and not by way of limitation, a first user may utilize a location services feature of the social networking system to provide recommendations for restaurants or other places in close proximity to the user. The first user's default privacy settings may specify that the social networking system may use location information provided from the first user's client device to provide location-based services, but may not store the first user's location information or provide it to any third party systems. The first user may then update their privacy settings to allow the location information to be used by a third-party image sharing application to geotag the photo.

Claims

1. a head mounted display (HMD) configured to output artificial reality content to a user of the HMD, the artificial reality content including one or more avatars corresponding to one or more other users participating in the artificial reality environment; a interruption mode engine configured to, in response to a user input, enable, for the user of the HMD, a interruption mode environment in which an avatar corresponding to the user ceases participating in the synthetic reality environment, and in each HMD of the one or more other users, one or more avatars corresponding to the one or more other users continue participating in the synthetic reality environment, the interruption mode engine comprising at least Rendering the one or more avatars at a static position within the suspended mode environment at the HMD of the user while continuing to have one or more avatars corresponding to the one or more other users participate in the synthetic reality environment at each HMD of the one or more other users. , interrupt mode engine and An artificial reality system comprising:

2. an imaging device configured to capture image data representative of a physical environment; a gesture detector configured to identify from the image data a user interface (UI) activation gesture comprising moving an inner wrist region of the user's hand of the HMD in a field of view of the imaging device; a UI engine configured to generate a break-mode UI element in response to the UI activation gesture; a rendering engine configured to render the interruption mode UI elements near or at least partially on the user's hand; Further equipped with 2. The artificial reality system of claim 1, wherein the user input corresponds to a pause mode selection gesture comprising placement of one or more fingers of a virtual hand of the user's avatar over the pause mode UI element, and the gesture detector is further configured to identify the pause mode selection gesture from the image data.

3. 3. The virtual reality system of claim 1, wherein the suspended mode engine is further configured to, in response to the suspended mode environment being enabled, move the one or more avatars away from the user's avatar when the one or more avatars are within a personal space threshold from the user's avatar.

4. the suspended mode engine is further configured to replace a virtual background and virtual objects in the synthetic reality environment with a monochromatic background in the suspended mode environment; Preferably, the suspended mode engine is further configured to play background music in the suspended mode environment. A virtual reality system according to any one of claims 1 to 3.

5. the HMD comprises a first HMD, the user of the first HMD comprises a first user, the system further comprises a second HMD of a second user corresponding to one of the one or more other avatars, the second HMD configured to output synthetic reality content comprising the synthetic reality environment to the second user; an avatar of the first user ceasing to participate in the synthetic reality environment after the interruption mode is enabled for the first user in the first HMD; the second user continues to participate in the synthetic reality environment at the second HMD after the interruption mode is enabled for the first user at the first HMD. A virtual reality system according to any one of claims 1 to 4.

6. 6. The artificial reality system of claim 5, wherein the second HMD renders virtual graphics corresponding to the first user of the first HMD after the interruption mode is enabled for the first user in the first HMD.

7. 7. The artificial reality system of claim 5 or 6, wherein the second user of the second HMD cannot interact with the avatar of the first user of the first HMD in the artificial reality environment after the interruption mode is enabled for the first user in the first HMD.

8. the user input comprises a first user input, and the suspension mode environment comprises a first suspension mode environment; the interruption mode engine is further configured to enable, in response to a second user input from the second user, a second interruption mode environment for the second user of the second HMD, the second interruption mode environment being different from the first interruption mode environment; the second user of the second HMD is unable to interact with the avatar of the first user within the first interrupted mode environment or the second interrupted mode environment; A virtual reality system according to any one of claims 5 to 7.

9. The interrupt mode engine includes: Rendering a user interface (UI) element for each of the one or more avatars within the suspended mode environment, each UI element being selectable via a selection gesture performed by the user; in response to detecting a selection gesture of a first UI element corresponding to a first avatar of the one or more avatars; rendering a first selectable virtual element configured, when selected, to mute the first avatar in the HMD when the user returns to the synthetic reality environment; rendering a second selectable virtual element configured, when selected, to block the first avatar from interacting with the user's avatar when the user returns to the virtual reality environment; rendering a third selectable virtual element configured, when selected, to report the first avatar to a review entity associated with the AR experience; and performing at least one of the following:

9. The virtual reality system of claim 1 , further configured to:

10. the first avatar is highlighted when the first UI element corresponding to the first avatar is selected; Preferably, the first avatar, if blocked by the user, will not be able to interact with the user's avatar in the synthetic reality environment or any other synthetic reality environment, regardless of whether the suspended mode environment has been enabled for the user.

10. The artificial reality system of claim 9.

11. outputting, in a head mounted display (HMD) of a virtual reality system, virtual reality content to a user of the HMD, the virtual reality content including one or more avatars corresponding to one or more other users participating in the virtual reality environment; a suspended mode environment for the user of the HMD in response to a user input, in which an avatar corresponding to the user ceases participating in the synthetic reality environment, while at each HMD of the one or more other users, one or more avatars corresponding to the one or more other users continue participating in the synthetic reality environment; Rendering the one or more avatars at a static position within the suspended mode environment at the HMD of the user while continuing to have one or more avatars corresponding to the one or more other users participate in the synthetic reality environment at each HMD of the one or more other users. It can be enabled by A method comprising:

12. acquiring image data via an imaging device by the virtual reality system; identifying from the image data a user interface (UI) activation gesture comprising moving an inner wrist region of the user's hand of the HMD in a field of view of the imaging device; generating a break-mode UI element in response to the UI activation gesture; rendering the pause mode UI element near or at least partially on the user's hand; Further comprising: Preferably, the user input corresponds to a pause mode selection gesture comprising the placement of one or more fingers of a virtual hand of the user's avatar on the pause mode UI element; The method of claim 11 , the method further comprising identifying the pause mode selection gesture from the image data.

13. The suspended mode engine further configured to replace a virtual background and virtual objects in the artificial reality environment with a monochromatic background in the suspended mode environment; Preferably, enabling the interrupted mode environment further comprises, in response to the interrupted mode environment being enabled, moving the one or more avatars away from the user's avatar when the one or more avatars are within a personal space threshold from the user's avatar.

13. The method according to claim 11 or 12.

14. 14. The method of claim 11, wherein the user's avatar stops participating in the synthetic reality environment after the interruption mode is enabled for the user on the HMD, and the one or more avatars corresponding to the one or more other users continue to participate in the synthetic reality environment after the interruption mode is enabled for the user on the HMD.

15. A computer-readable storage medium comprising instructions, the instructions comprising: outputting, in a head mounted display (HMD) of a virtual reality system, virtual reality content to a user of the HMD, the virtual reality content including one or more avatars corresponding to one or more other users participating in the virtual reality environment; a suspended mode environment in which, in response to a user input, an avatar corresponding to the user stops participating in the synthetic reality environment, while at each HMD of the one or more other users, one or more avatars corresponding to the one or more other users continue to participate in the synthetic reality environment; Rendering the one or more avatars at a static position within the suspended mode environment at the HMD of the user while continuing to have one or more avatars corresponding to the one or more other users participate in the synthetic reality environment at each HMD of the one or more other users. It can be enabled by 16. A computer-readable storage medium for configuring one or more programmable processors to perform the steps of:

Images