Immersive system with mixed reality mode and virtual reality mode

Abstract

An immersive system with a mixed reality mode and a virtual reality mode is disclosed. Aspects of the present disclosure relate to managing mixed reality and virtual reality modes of operation using an artificial reality system. The artificial reality system may be operable in a virtual reality mode in which the environment displayed to the user is computer-generated or a mixed reality mode in which the environment displayed to the user represents the user's real-world environment but includes one or more computer-generated virtual objects. Various embodiments of a mode manager may manage these operating modes of an artificial reality system, such as managing an operating mode that an XR system initiates entry upon initialization, managing transitions between these operating modes, and managing the display of virtual objects that persist in both operating modes.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 585,348, filed on September 26, 2023, entitled “Immersive System with Mixed Reality and Virtual Reality Modes,” and U.S. Non-Provisional Application No. 18 / 808,681, filed on August 19, 2024, both of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates to managing mixed reality operating modes and virtual reality operating modes using an artificial reality system. Background Art

[0004] Artificial reality (extra reality, XR) devices are becoming more and more common. As artificial reality devices become more popular, the applications implemented on such devices are becoming more complex. Augmented reality (AR) applications or mixed reality (MR) applications can provide interactive three-dimensional (3D) experiences that combine real-world environments with virtual objects. On the other hand, virtual reality (VR) applications can provide a completely independent 3D computer environment. Users can get AR, MR, and VR experiences through a head-mounted display (HMD) (e.g., glasses or a head-mounted viewer). Summary of the invention

[0005] Various aspects of the present disclosure relate to using an artificial reality system to manage mixed reality and virtual reality modes of operation. The artificial reality system may be capable of operating in a virtual reality mode, in which the environment displayed to the user is completely computer-generated, or in a mixed reality mode, in which the environment displayed to the user represents the user's real-world environment but includes one or more computer-generated virtual objects. Multiple embodiments of a mode manager can manage these modes of operation of the artificial reality system, such as managing the operating mode that the XR system boots into upon initialization, managing transitions between these operating modes, and managing the display of virtual objects that persist in both operating modes. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1is a block diagram showing an overview of a number of devices on which some embodiments of the present technology may operate.

[0007] Figure 2A is a line diagram illustrating a virtual reality head mounted viewer that may be used in some embodiments of the present technology.

[0008] Figure 2B is a line diagram illustrating a mixed reality head mounted viewer that may be used in some embodiments of the present technology.

[0009] Figure 2C is a line diagram illustrating a plurality of controllers that, in some implementations, a user may hold with one or both hands to interact with an artificial reality environment.

[0010] Figure 3 is a block diagram illustrating an overview of an environment in which some embodiments of the present technology may operate.

[0011] Figure 4 is a block diagram illustrating various components that may be used in a system employing the disclosed technology in some implementations.

[0012] Figure 5 is a conceptual diagram illustrating an example usage space of an artificial reality system.

[0013] FIG. 6A to FIG. 6K is a conceptual diagram showing example transitions between different artificial reality modes.

[0014] Figure 7 is a conceptual diagram illustrating an example mixed reality environment with virtual objects.

[0015] Figure 8 is a conceptual diagram illustrating an example virtual reality environment with virtual objects.

[0016] Fig. 9 is a flow chart showing a method for starting an artificial reality system in a mixed reality mode of operation or a virtual reality mode of operation in some embodiments of the present technology.

[0017] Fig.10 is a flow chart illustrating a method for enabling virtual objects to persist when transitioning between virtual reality mode and mixed reality mode in some embodiments of the present technology.

[0018] The technology described herein may be better understood by referring to the following detailed description in conjunction with the accompanying drawings, in which like reference numbers refer to the same elements or elements with similar functions. DETAILED DESCRIPTION

[0019] Various aspects of the present disclosure relate to using an artificial reality system to manage mixed reality and virtual reality modes of operation. The artificial reality system may be capable of operating in a virtual reality mode, in which the environment displayed to the user is completely computer-generated, or in a mixed reality mode, in which the environment displayed to the user represents the user's real-world environment but includes one or more computer-generated virtual objects. Multiple embodiments of a mode manager can manage these modes of operation of the artificial reality system, such as managing the operating mode that the XR system boots into upon initialization, managing transitions between these operating modes, and managing the display of virtual objects that persist in both operating modes.

[0020] In some embodiments, the mode manager automatically boots into an operating mode. For example, the mode manager can detect the mode in which the artificial reality system was last operating (e.g., virtual reality mode or mixed reality mode) and boot into the same mode once the artificial reality system is initialized. When booting into virtual reality mode, the mode manager can check to ensure that the user is in a use space before immersing the user in the virtual reality environment. The use space (e.g., a "guardian") can be a space in the user's real world environment that is defined so that the user can safely use the artificial reality system. In some embodiments, a previous virtual reality environment can be restored when booting into virtual reality mode and / or a default virtual reality environment can be displayed. When booting into mixed reality mode, the mode manager can restore the state of the mixed reality environment (e.g., virtual object state, virtual tablet state, etc.).

[0021] The mode manager can manage the transition from mixed reality mode to virtual reality mode or from virtual reality mode to mixed reality mode. For example, transitioning into virtual reality mode may include checking that the user is within the boundaries of the defined usage space. Transitioning into mixed reality mode may cause a virtual object, such as a completely virtual object or a virtual object indicator, to be displayed. The transition may be caused by a user action (e.g., a user tapping the artificial reality system HMD), a user movement (e.g., moving out of the defined usage space), and / or a software trigger. The mode manager may also cause one or more virtual objects to persist between modes. In some embodiments, the mode manager may change the display of these virtual objects depending on the operating mode. Changing the display of these virtual objects may include: changing the scale of the virtual object; changing the skin of the virtual object, and / or changing the distance at which the virtual object appears to be displayed from the user's perspective.

[0022] Embodiments of the disclosed technology may include an artificial reality system, or be implemented in conjunction with an artificial reality system. Artificial reality or hyperreality (XR) is a form of reality that has been adjusted in some way before being presented to a user, and artificial reality or hyperreality may include, for example, virtual reality (VR), mixed reality (MR) (which may include augmented reality (AR)), hybrid reality, or some combination and / or derivative thereof. Artificial reality content may include fully generated content or generated content combined with collected content (e.g., real-world photos). Artificial reality content may include video, audio, tactile feedback, or some combination thereof, any of which may be presented in a single channel or multiple channels (e.g., stereoscopic video that gives the viewer a three-dimensional effect). In addition, in some embodiments, artificial reality may be associated with an application, product, accessory, service, or some combination thereof, which is used, for example, to create content in artificial reality, and / or to be used in artificial reality (e.g., to perform activities in artificial reality). Artificial reality systems that provide artificial reality content can be implemented on a variety of platforms, including a head-mounted display (HMD) connected to a host computer system, a stand-alone HMD, a mobile device or computing system, a "cave" environment or other projection system, or any other hardware platform capable of providing artificial reality content to one or more viewers.

[0023] As used herein, "virtual reality" or "VR" refers to an immersive experience in which a user's visual input is controlled by a computing system. "Augmented reality" or "AR", "mixed reality" or "MR" refers to systems in which A) a user views images of the real world after they have passed through a computing system (e.g., a tablet with a camera on the back may capture multiple images of the real world, which may then be displayed on a screen on the side of the tablet opposite the camera, and the tablet may process and adjust or "augment" the images as they pass through the system, such as by adding virtual objects); or B) the light entering the user's eyes is generated in part by the computing system and in part from light reflected from objects in the real world (e.g., a head-mounted viewer may be formed as a pair of glasses with a see-through display that allows light from the real world to pass through a waveguide that simultaneously emits light from a projector in the MR head-mounted viewer, thereby allowing the MR head-mounted viewer to present virtual objects mixed with real objects that the user can see). As used herein, “artificial reality,” “hyperreality,” or “XR” refers to any of the following: VR, AR, MR, or any combination or hybrid thereof.

[0024] Several embodiments are discussed in more detail below with reference to the accompanying drawings. Figure 1 is a block diagram showing an overview of multiple devices on which some embodiments of the disclosed technology may run. These devices may include hardware components of a computing system 100 that use an artificial reality system to manage mixed reality operating modes and virtual reality operating modes. In various embodiments, the computing system 100 may include a single computing device 103 or multiple computing devices (e.g., computing device 101, computing device 102, and computing device 103) that communicate over a wired channel or a wireless channel to distribute processing and share input data. In some embodiments, the computing system 100 may include a standalone head-mounted viewer that is capable of providing a computer-created or enhanced experience to a user without the need for external processing or external sensors. In other embodiments, the computing system 100 may include multiple computing devices, such as a head-mounted viewer and a core processing component (e.g., a console, a mobile device, or a server system), where some processing operations are performed on the head-mounted viewer and other processing operations are transferred to the core processing component. The following is combined with Figure 2A and Figure 2B An example head mounted view device is described. In some implementations, the location data and environmental data may be collected solely by sensors included in the head mounted view device, while in other implementations, one or more of the plurality of non-head mounted view computing devices may include sensor components that may track environmental data or location data.

[0025] The computing system 100 may include one or more processors 110 (e.g., a central processing unit (CPU), a graphical processing unit (GPU), a holographic processing unit (HPU), etc.). The processor 110 may be a single processing unit or multiple processing units, which are located in one device or distributed in multiple devices (e.g., distributed in two or more computing devices among the computing devices 101 to 103).

[0026] The computing system 100 may include one or more input devices 120 that provide input to the processor 110, thereby informing the processor of actions. These actions may be communicated by a hardware controller that interprets the signals received from the input devices and transmits information to the processor 110 using a communication protocol. For example, each input device 120 may include a mouse, a keyboard, a touch screen, a touchpad, a wearable input device (e.g., a tactile glove, a bracelet, a ring, an earring, a necklace, a watch, etc.), a camera (or other light-based input device, such as an infrared sensor), a microphone, or other user input device.

[0027] The processor 110 can be coupled to other hardware devices, for example, by using an internal bus or an external bus (e.g., a peripheral component interconnect standard (PCI) bus, a small computer system interface (SCSI) bus) or a wireless connection. The processor 110 can communicate with a hardware controller of a device (e.g., a display 130). The display 130 can be used to display text and graphics. In some embodiments, the display 130 includes an input device, such as when the input device is a touch screen or is equipped with an eye movement direction monitoring system, the input device is part of the display. In some embodiments, the display is separate from the input device. Examples of display devices are: liquid crystal display (LCD) display screen; light emitting diode (LED) display screen; projection, holographic or augmented reality display (e.g., a head-up display device or a head-mounted device), etc. Other input / output (I / O) devices 140 may also be coupled to the processor, such as a network chip or card, a video chip or card, an audio chip or card, a universal serial bus (USB), FireWire or other external devices, a camera, a printer, a speaker, a compact disc read-only memory (CD-ROM) drive, a digital video disc (DVD) drive, a disk drive, etc.

[0028] In some embodiments, the computing system 100 can use input from the I / O device 140 (e.g., a camera, a depth sensor, an inertial motion unit (IMU) sensor, a global positioning system (GPS) unit, a laser radar (LiDAR) or other time-of-flight sensor, etc.) to identify and map the user's physical environment while tracking the user's position in the environment. A simultaneous localization and mapping (SLAM) system can generate a map (e.g., a topology, a grid, etc.) of an area (which can be a room, a building, an outdoor space, etc.), and / or obtain a map previously generated by the computing system 100 or another computing system that has mapped the area. The SLAM system can track users in the area based on factors such as GPS data, match identified objects and structures with mapped objects and structures, monitor acceleration and other position changes, etc.

[0029] The computing system 100 may include a communication device that can communicate wirelessly or based on wired communication with other local computing devices or network nodes. The communication device can communicate with another device or server through a network, for example, using the Transmission Control Protocol / Internet Protocol (TCP / IP). The computing system 100 can use the communication device to distribute operations on multiple network devices.

[0030] Processor 110 may access memory 150, which may be included on one of the computing devices of computing system 100, or may be distributed across the computing devices of computing system 100 or other multiple external devices. Memory includes one or more hardware devices for volatile storage or non-volatile storage, and may include both read-only memory and writable memory. For example, memory may include one or more of the following: random access memory (RAM), various cache memories, CPU registers, read-only memory (ROM), and writable non-volatile memory, such as flash memory, hard drive, floppy disk, compact disk (CD), DVD, magnetic storage device, tape drive, etc. Memory is not a propagating signal that is separated from the underlying hardware; therefore, memory is non-transitory. Memory 150 may include program memory 160, which stores programs and software, such as operating system 162, artificial reality mode manager 164, and other applications 166. The memory 150 may also include a data storage 170, which may include, for example, sensor data, virtual object data (e.g., structure data, state data, etc.), application data, configuration data, settings, user options or preferences, etc., which may be provided to the program memory 160 or any element of the computing system 100.

[0031] Some embodiments may operate with many other computing system environments or configurations. Examples of computing systems, environments, and / or configurations suitable for use with the present technology include, but are not limited to, XR head-mounted viewers, personal computers, server computers, handheld or laptop devices, cellular phones, wearable electronic devices, game consoles, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, networked personal computers (PCs), minicomputers, mainframe computers, or distributed computing environments that include any of the above systems or devices, etc.

[0032] Figure 2A2 is a line diagram of a virtual reality head mounted display (HMD) 200 according to some embodiments. The virtual reality HMD 200 includes a front rigid body 205 and a strap 210. The front rigid body 205 includes: one or more electronic display elements of an electronic display 245; an inertial motion unit (IMU) 215; one or more position sensors 220; a plurality of locators 225; and one or more computing units 230. The position sensors 220, IMU 215, and computing units 230 may be located inside the virtual reality HMD 200 and may not be visible to the user. In various embodiments, the IMU 215, the position sensors 220, and the locators 225 may track the movement and position of the virtual reality HMD 200 in the real world and in an artificial reality environment with three degrees of freedom (3DoF) or six degrees of freedom (6DoF). For example, the locators 225 may emit infrared beams that produce light spots on real objects around the virtual reality HMD 200. As another example, the IMU 215 may include, for example, one or more accelerometers, one or more gyroscopes, one or more magnetometers, one or more other non-camera-based position, force or orientation sensors, or a combination thereof. One or more cameras (not shown) integrated with the virtual reality HMD 200 may detect light points. The computing unit 230 in the virtual reality HMD 200 may use the detected light points to infer the position and movement of the virtual reality HMD 200 and recognize the shapes and positions of real objects around the virtual reality HMD 200.

[0033] The electronic display 245 can be integrated with the front rigid body 205 and can provide image light to the user according to the instructions of the computing unit 230. In various embodiments, the electronic display 245 can be a single electronic display or multiple electronic displays (for example, one display for each user's eye). Examples of the electronic display 245 include: a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, an active-matrix organic light-emitting diode (AMOLED) display, a display including one or more quantum dot light-emitting diode (QOLED) sub-pixels, a projector unit (for example, micro-LED, laser (LASER)), some other display, or some combination thereof.

[0034] In some embodiments, the virtual reality HMD 200 can be coupled to a core processing component such as a personal computer (PC) (not shown) and / or one or more external sensors (not shown). The external sensors can monitor the virtual reality HMD 200 (e.g., through light emitted from the virtual reality HMD 200), and the PC can use the HMD in combination with outputs from the IMU 215 and the position sensor 220 to determine the position and movement of the virtual reality HMD 200.

[0035] Figure 2B 2 is a line diagram of a mixed reality HMD system 250, which includes a mixed reality HMD 252 and a core processing component 254. The mixed reality HMD 252 and the core processing component 254 can communicate through a wireless connection (e.g., a 60 GHz link) as indicated by link 256. In other embodiments, the mixed reality HMD system 250 includes only a head-mounted viewer without an external computing device, or includes other wired or wireless connections between the mixed reality HMD 252 and the core processing component 254. The mixed reality HMD 252 includes a transparent display 258 and a frame 260. The frame 260 can accommodate various electronic components (not shown), such as a light projector (e.g., LASER, LED, etc.), a camera, an eye tracking sensor, a micro-electromechanical system (MEMS) component, a network component, etc.

[0036] The projector can be coupled to the transparent display 258, for example, via a plurality of optical elements, to display media to the user. These optical elements can include one or more waveguide components, one or more reflectors, one or more lenses, one or more mirrors, one or more collimators, one or more gratings, etc., for guiding the light from the projector to the user's eyes. The image data can be transmitted to the mixed reality HMD 252 via the link 256 from the core processing component 254. The controller in the mixed reality HMD 252 can convert the image data into a plurality of light pulses from the projector, which can be transmitted to the user's eyes as output light via the optical element. The output light can be mixed with the light passing through the transparent display 258, so that the output light can present the following virtual objects: these virtual objects look like they exist in the real world.

[0037] Similar to the virtual reality HMD 200, the mixed reality HMD system 250 may also include motion and position tracking units, cameras, light sources, etc., which allow the mixed reality HMD system 250 to track itself, for example, with 3DoF or 6DoF, track multiple parts of the user (e.g., hands, feet, head, or other body parts), draw virtual objects to appear stationary when the mixed reality HMD 252 moves, and make virtual objects react to gestures and other real-world objects.

[0038] Figure 2C A plurality of controllers 270 (including controllers 276A and 276B) are shown, which in some embodiments can be held by a user with one or both hands to interact with an artificial reality environment presented by the virtual reality HMD 200 and / or the mixed reality HMD system 250. The controllers 270 can communicate with the HMD directly or via an external device (e.g., the core processing component 254). Each controller can have its own IMU unit, position sensor, and / or can emit additional light points. The virtual reality HMD 200 or the mixed reality HMD system 250, external sensors, or sensors in the controllers can track these controller light points to determine the position and / or orientation of the controllers (e.g., tracking the controllers with 3DoF or 6DoF). The computing unit 230 or the core processing component 254 in the virtual reality HMD 200 can use this tracking to monitor the user's hand position and movement in combination with the IMU output and the position output. The controllers may also include various buttons (eg, buttons 272A-272F) and / or levers (eg, levers 274A and 274B) that a user may actuate to provide input and interact with objects.

[0039] In various embodiments, the virtual reality HMD 200 or mixed reality HMD system 250 may also include a number of additional subsystems for monitoring user interactions and indications of intent, such as an eye tracking unit, an audio system, various network components, etc. For example, in some embodiments, one or more cameras included in the virtual reality HMD 200 or mixed reality HMD system 250 instead of or in addition to the controller, or one or more cameras from a plurality of external cameras, may monitor the position and posture of the user's hands to determine gestures and other hand and body movements. As another example, one or more light sources may illuminate one or both eyes of the user, and the virtual reality HMD 200 or mixed reality HMD system 250 may use eye-facing cameras to capture reflections of such light to determine eye position (e.g., based on a set of reflected light around the user's cornea), model the user's eyes, and determine gaze direction.

[0040] Figure 3 3 is a block diagram illustrating an overview of an environment 300 in which some embodiments of the disclosed technology may operate. The environment 300 may include one or more client computing devices 305A to 305D, examples of which may include the computing system 100. In some embodiments, some of the plurality of client computing devices (e.g., client computing device 305B) may be a virtual reality HMD 200 or a mixed reality HMD system 250. The client computing device 305 may operate in a networked environment using a logical connection to one or more remote computers (e.g., a server computing device) through a network 330.

[0041] In some implementations, server computing device 310 may be an edge server that receives multiple client requests and coordinates fulfillment of these requests through other servers (e.g., server computing devices 320A to 320C). Server computing devices 310 and 320 may include a computing system, such as computing system 100. Although each server computing device 310 and 320 is logically shown as a single server, multiple server computing devices may each be a distributed computing environment that includes multiple computing devices located at the same physical location or at geographically different physical locations.

[0042] Client computing device 305 and server computing devices 310 and 320 can each act as a server or client of one or more other server / client devices. Server computing device 310 can be connected to database 315. Server computing devices 320A to 320C can each be connected to corresponding databases 325A to 325C. As discussed above, each server computing device 310 or 320 can correspond to a group of servers, and each of these servers can share a database, or can have its own database. Although databases 315 and 325 are logically shown as a single unit, databases 315 and 325 can each be a distributed computing environment containing multiple computing devices, which can be located in their corresponding servers, or can be located at the same physical location or at geographically different physical locations.

[0043] Network 330 may be a local area network (LAN), a wide area network (WAN), a mesh network, a hybrid network, or other wired or wireless network. Network 330 may be the Internet or some other public or private network. Client computing device 305 may be connected to network 330 via a network interface, such as by wired or wireless communications. Although the connections between server computing device 310 and multiple server computing devices 320 are shown as separate connections, these connections may be any type of local area network, wide area network, wired network, or wireless network including network 330 or a separate public or private network.

[0044] Figure 4 4 is a block diagram showing a plurality of components that can be used in a system that employs the disclosed technology in some embodiments. The plurality of components 400 can be included in a device of the computing system 100, or can be distributed on a plurality of devices in the devices of the computing system 100. The plurality of components 400 include hardware 410, middleware 420, and a plurality of dedicated components 430. As discussed above, the system implementing the disclosed technology can use various hardware, including a processing unit 412, a working memory 414, input and output devices 416 (e.g., a camera, a display, an IMU unit, a network connection, etc.), and a storage memory 418. In various embodiments, the storage memory 418 can be one or more of the following: a local device, an interface to a remote storage device, or a combination thereof. For example, the storage memory 418 can be one or more hard drives or flash drives accessible via a system bus, or can be a cloud storage provider (e.g., located in a database 315 or 325) or other network storage device accessible via one or more communication networks. In various implementations, the plurality of components 400 may be implemented in a client computing device (eg, client computing device 305 ), or implemented on a server computing device (eg, server computing devices 310 or 320 ).

[0045] Middleware 420 may include components that transfer resources between hardware 410 and multiple dedicated components 430. For example, middleware 420 may include an operating system, services, drivers, a basic input output system (BIOS), a controller circuit, or other hardware or software systems.

[0046] The plurality of specialized components 430 may include software or hardware configured to perform operations for managing mixed reality and virtual reality modes of operation using an artificial reality system. The plurality of specialized components 430 may include a virtual object manager 434; a VR controller 436; an MR controller 438; a transformation manager 440; and components and APIs (e.g., interface 432) that may be used to provide a user interface, transfer data, and control the specialized components. In some implementations, the plurality of components 400 may be located in a computing system distributed across multiple computing devices, or may be an interface to a server-based application that executes one or more of the plurality of specialized components 430. Although the plurality of specialized components 430 are depicted as separate components, the specialized components may be logical or other non-physical distinctions of functionality, and / or may be sub-modules or code blocks of one or more applications.

[0047] The virtual object manager 434 can manage the virtual objects of the XR system. For example, the virtual object manager 434 can cause virtual objects to be displayed in VR mode and / or MR mode. In some embodiments, the virtual object manager 434 can manage the display of virtual objects so that fully virtual objects (e.g., virtual objects that can respond to user interactions) and / or virtual object indicators (e.g., virtual objects with limited user interaction capabilities and / or including a lower resolution display) are displayed. For example, based on the operating mode and the executed application, the virtual object manager 434 can cause a given virtual object to be displayed as a fully virtual object or a virtual object indicator. In some embodiments, the virtual object manager 434 can cause one or more virtual objects to persist in both VR mode and MR mode. The virtual object manager 434 can also change the display of these persistent virtual objects based on the operating mode. Reference Fig. 9 The frame 910 and Fig.10 Blocks 1004 and 1010 of describe further details regarding the virtual object manager 434 .

[0048] The VR controller 436 can cause the XR system to operate in VR mode. In some embodiments, the VR controller 436 can cause the XR system to start in VR mode upon initialization. The VR controller can detect whether the user is within the boundaries of the usage space, and when the user is not within the usage space or moves out of the usage space, suspend the VR environment displayed to the user in VR mode. The VR controller 436 can interact with the virtual object manager 434 to display virtual objects in the VR environment. Fig. 9 Boxes 914, 916, 918, 920 and 922 and Fig.10 Blocks 1002 , 1004 , 1008 , and 1010 describe further details regarding VR controller 436 .

[0049] MR controller 438 can cause the XR system to operate in MR mode. In some embodiments, MR controller 438 can cause the XR system to boot into MR mode upon initialization. For example, MR controller 438 can repopulate virtual objects based on their last state and / or restore the state of a virtual tablet displayed in the MR environment. MR controller 438 can interact with virtual object manager 434 to display virtual objects in the MR environment. Fig. 9 Boxes 906, 908, 910 and 912 and Fig.10 Blocks 1002 , 1004 , 1008 , and 1010 of EMBODIMENT describe further details regarding the MR controller 438 .

[0050] The transition manager 440 can detect transition conditions and manage transitions between MR mode and VR mode. For example, a user action (e.g., tapping the XR system HMD, other suitable user actions) may include a detected transition condition. In another example, a user moving out of a usage space during VR mode may include a detected transition condition. In another example, a software trigger (e.g., virtual object user interaction, user selection of a mode through the XR system local software, etc.) may include a detected transition condition. The transition manager 440 may communicate with the VR controller 436 and the MR controller 438 to implement transitions between VR mode and MR mode. Additional details on tapping the HMD to trigger an action are provided in U.S. patent application number 18 / 068,048, entitled “Triggering Actions Based on Detected Motions on an Artificial Reality Device,” filed on December 19, 2022, which is incorporated herein by reference in its entirety. References Fig.10 Block 1006 of describes further details regarding the conversion manager 440 .

[0051] Some operating modes of the XR system (e.g., VR operating mode) include mechanisms for ensuring user safety. Figure 55 is a conceptual diagram illustrating an example use space of an artificial reality system. Schematic 500 is a real-world environment illustrating a use space 502. Use space 502 (e.g., a protected area) can be an XR system use space defined in a user's real-world environment. The XR system can detect that the user is within the boundaries of the use space 502 and immerse the user in a VR environment based on the detection. The use space 502 can have boundaries that can trigger warnings or other system actions if the user crosses the boundaries in the XR experience (e.g., when the XR system is operating in VR mode). In some embodiments, the XR system can automatically create the boundaries of the use space by identifying the maximum boundaries of the real-world environment around the XR system using a depth sensor and / or an imaging device. More details are described in U.S. patent application number 63 / 585,360, filed on September 26, 2023, entitled "AUTOMATIC GUARDIAN FOR ANARTIFICIAL REALITY ENVIRONMENT", which is incorporated herein by reference in its entirety. In other examples, a user may define the boundaries of a use space by executing a workflow utilizing the XR system (e.g., traversing the space while wearing the HMD and / or moving an XR system controller).

[0052] In some embodiments, the usage space may include dynamic boundaries, and a trigger condition that causes an operating mode transition may be detected relative to such dynamic boundaries. More details about the XR system usage space including dynamic boundaries are described in U.S. Provisional Application No. 63 / 585,450, filed on September 26, 2023, entitled “Selective Guardians for an Application Executing in an Artificial Reality Environment,” which is incorporated herein by reference in its entirety.

[0053] Various embodiments manage transitions between one or more VR modes and one or more MR modes of an XR system. FIG. 6A to FIG. 6K is a conceptual diagram showing example transitions between different artificial reality modes. Fig. 6ASchematic diagram 600A shows a workflow for transitioning from VR mode to MR mode through user action. In the example shown, the user is initially immersed in a VR environment through a VR home application (e.g., a native VR application of the XR system). The user can take an action that triggers a transition from VR mode to MR mode, such as double-clicking the HMD of the XR system or by performing any other suitable action. The XR system can, in response to the user action, switch from the VR home application (or any other suitable VR application) to MR mode, in which one or more fully virtual objects can be displayed to the user in the MR environment. In some embodiments, a fully virtual object can be an object that includes a structure (e.g., a three-dimensional structure) and can receive interaction from a user (e.g., sensed motion or other user input that interacts with the virtual object in the context of the XR environment). For example, a fully virtual object can perform a function in response to such a user interaction.

[0054] Figure 6B Schematic diagram 600B shows a workflow for transitioning from VR mode to MR mode through user movement. In the example shown, the user is initially immersed in a VR environment through a VR home application (e.g., a native VR application of the XR system). The user may initially be within the boundaries of the use space and then move out of the boundaries. In response to the user movement, the XR system may transition from the VR home application (or any other suitable VR application) to MR mode, in which one or more fully virtual objects may be displayed to the user in the MR environment so that the user can interact with these fully virtual objects. In some embodiments, when the user interacts with the fully virtual objects, the display of the boundaries of the use space that the user has moved out of may be removed from the MR environment.

[0055] Figure 6C Schematic diagram of 600C and Fig.6DSchematic diagram 600D of shows a workflow for transitioning from VR mode to MR mode through user action when the user is immersed through a VR application. In the example shown, the user is initially immersed in a VR environment through a VR application (e.g., a third-party VR application such as a VR game or social experience). The user can take an action that triggers the transition from VR mode to MR mode, such as performing a given number of taps on the HMD of the XR system, or performing any other suitable action. The XR system can respond to the user action by pausing the VR application and triggering a transparent display of the user's real world environment. During the suspension of the VR application and the transparent display, one or more virtual object indicators can be displayed. The virtual object indicator can be a virtual object with limited user interaction capabilities and / or including a lower resolution display. The user can interact with one of the multiple virtual object indicators displayed, but because the display is not a completely virtual object, the interaction can trigger a modality that requires the user to select an operating mode. For example, the user can switch to MR mode and exit the VR application, or continue the VR application and leave MR mode. When the user chooses to exit the VR application, the virtual object indicator can be converted to a completely virtual object in the MR environment.

[0056] Fig. 6E Schematic diagram of 600E and Fig. 6F Schematic diagram 600F shows a workflow for transitioning from VR mode to MR mode through user movement when the user is immersed through a VR application. In the example shown, the user is initially immersed in a VR environment through a VR application (e.g., a third-party VR application such as a VR game or social experience). The user may initially be within the boundary of the use space and then move out of the boundary. The XR system may pause the VR application and trigger a transparent display of the user's real world environment in response to the user's movement. During the suspension of the VR application and the transparent display, one or more virtual object indicators may be displayed. The user may interact with one of the multiple virtual object indicators displayed, but because the display is not a completely virtual object, the interaction may trigger a modality requiring the user to select an operating mode. For example, the user may switch to MR mode and exit the VR application, or continue the VR application and leave MR mode. When the user chooses to exit the VR application, the virtual object indicator may be converted to a completely virtual object in the MR environment.

[0057] Figure 6GSchematic diagram 600G shows a workflow for transitioning from MR mode to VR mode through user action. In the example shown, an MR environment including the user's real-world environment and virtual objects (e.g., fully virtual objects) is initially displayed to the user. The user can take an action that triggers a transition from MR mode to VR mode, such as tapping the HMD of the XR system or by performing any other suitable action. The XR system can switch from the MR environment to VR mode in response to the user action. In some embodiments, the XR system can scan the user's real-world environment to detect whether the user is within the boundaries of a predefined usage space. When the XR system detects that the user is in such a usage space, the user can be immersed in the VR environment. In some embodiments, the VR environment can be a VR home application (e.g., a native VR application of the XR system) or any other suitable VR application.

[0058] Figure 6H Schematic diagram of 600H and Fig.6I Schematic diagram 600I of shows a workflow for transitioning from MR mode to VR mode through user movement. In the example shown, an MR environment including the user's real-world environment and virtual objects (e.g., fully virtual objects) is initially displayed to the user. The user can take an action that triggers the transition from MR mode to VR mode, such as tapping the HMD of the XR system or by performing any other suitable action. The XR system can switch from the MR environment to the VR mode in response to the user action. In some embodiments, the XR system can scan the user's real-world environment to detect whether the user is within the boundaries of a predefined use space. When the XR system detects that the user is not in such a use space, the user can be prompted to return to the predefined use space, create a new use space, or continue in MR mode (rather than switching to VR mode). In response to detecting that the user has entered such a use space, the user can be immersed in a VR environment. In some embodiments, the VR environment can be a VR home application (e.g., a local VR application of the XR system) or any other suitable VR application.

[0059] Figure 6J Schematic diagram of 600J and Figure 6KSchematic diagram 600K shows a workflow for transitioning from MR mode to an immersive VR application in VR mode through user movement. In the example shown, an MR environment including the user's real-world environment and virtual objects (e.g., fully virtual objects) is initially displayed to the user. A software trigger (e.g., a virtual object interaction that triggers the execution of a VR application and a user's selection of a VR shortcut for a VR application, etc.) can cause a transition from MR mode to VR mode. The XR system can transition from the MR environment to VR mode in response to the software trigger. In some embodiments, the XR system can scan the user's real-world environment to detect whether the user is within the boundaries of a predefined usage space. When the XR system detects that the user is not in such a usage space, the user can be prompted to return to the predefined usage space, create a new usage space, or continue in MR mode (rather than switching to VR mode). In response to detecting that the user has entered such a usage space, the user can be immersed in a VR environment. In some embodiments, a VR application (e.g., a third-party VR application such as a VR game or social experience) can immerse the user in a VR environment.

[0060] In some implementations, virtual objects that persist when transitioning XR modes of operation (e.g., from MR to VR, from VR to MR, etc.) may be changed. Figure 7 is a conceptual diagram illustrating an example mixed reality environment with virtual objects, Figure 8 Schematic diagrams 700 and 800 illustrate an example virtual reality environment including a real world environment and virtual objects 702 and 704. Schematic diagram 800 illustrates a virtual reality environment including a computer generated environment and virtual objects 802 and 804.

[0061] Embodiments may transition from the MR environment of schematic diagram 700 to the VR environment of schematic diagram 800 such that virtual objects 702 and 704 persist as virtual objects 802 and 804. In the example shown, when transitioning from MR to VR, the scale and distance at which virtual object 704 is displayed appear to be changed from the user's perspective. Virtual object 804 is smaller in size and appears to be displayed farther from the user in the VR environment. In addition, in the example shown, when transitioning from MR to VR, the skin of virtual object 702 is changed, as depicted by virtual object 802 in the VR environment. A VR environment may have an almost unlimited size, and some real-world environments (e.g., rooms in a building) in a VR environment that correspond to an MR environment may be limited. Therefore, in a VR environment, a virtual object may move farther away from the user, but in an MR environment, such a distance may be impractical. In addition, a VR application may control the visual display of the VR environment, so the VR application may re-skin the virtual object (e.g., change the visual appearance) to maintain visual aesthetics.

[0062] Embodiments may also transition from the VR environment of schematic diagram 800 to the MR environment of schematic diagram 700, so that virtual objects 802 and 804 persist as virtual objects 702 and 704. In the example shown, when transitioning from VR to MR, the scale and distance of the display of virtual object 804 appear to be changed from the user's perspective. Virtual object 704 is larger in size and appears to be displayed closer to the user in the MR environment. In addition, in the example shown, when transitioning from VR to MR, the skin of virtual object 802 is changed, as depicted by virtual object 702 in the MR environment. VR environments can have almost unlimited sizes, and some real-world environments (e.g., rooms in a building) in VR environments corresponding to MR environments may be limited. Therefore, in an MR environment, virtual objects may move closer to the user due to the limitations of the size of the environment. In addition, VR applications can control the visual display of the VR environment, so VR applications can re-skin virtual objects to maintain visual aesthetics. When transitioning to MR, virtual objects can be changed to restore their original skin or to a skin different from the skin applied by the VR application.

[0063] Those skilled in the art will appreciate that the above Figures 1 to 8 The components shown in the flowcharts and in each of the flowcharts discussed below can be changed in a variety of ways. For example, the order of the logic can be rearranged, multiple sub-steps can be performed in parallel, the logic shown can be omitted, other logic can be included, etc. In some embodiments, one or more of the above components can perform one or more of the processes described below.

[0064] In some embodiments, the virtual tablet may include display objects of one or more applications executed at the XR system. The virtual tablet may provide a user with practical and user-friendly tools for interacting with applications while immersed in an XR environment (e.g., an MR environment). In some embodiments, the virtual tablet represents a virtual computer in the XR environment, where the user can interact with one or more applications.

[0065] In some embodiments, the state of a virtual tablet may be stored at a point in time, and the virtual tablet may be restored to the previously stored state. For example, the state of the virtual tablet may be stored when the XR system enters a sleep state or a shutdown state (or when transitioning from an MR environment). In some embodiments, the stored state may be used to restore the virtual tablet so that: the virtual tablet is located at the same location in the XR environment; and the content / one or more applications (including the state of one or more applications) displayed by the virtual tablet is restored. The virtual tablet may be viewed as a display object and / or overlay relative to the XR environment. More details about providing an overlay relative to the XR environment, storing the state of the overlay, and restoring the state of the overlay are described in U.S. application No. 18 / 068,071, entitled “Overlay on an Artificial Reality Environment,” filed on December 19, 2022, which is incorporated herein by reference in its entirety.

[0066] Fig. 9 is a flow chart illustrating a process 900 for starting an artificial reality system in a mixed reality operating mode or a virtual reality operating mode in some embodiments of the present technology. The process 900 can be triggered in response to starting the XR system.

[0067] At block 902, process 900 may start the XR system. For example, the system may be started by a power button, pressing a button on a handheld controller, etc. The XR system may be started from an off state or a sleep state to an active state. At block 904, process 900 may detect a last operating mode of the XR system. For example, a flag or state may be stored that indicates the last operating mode (e.g., MR or VR) of the XR system when entering the off state or the sleep state. When the last operating mode is MR, process 900 may proceed to block 906. When the last operating mode is VR, process 900 may proceed to block 914.

[0068] At box 906, process 900 may begin starting the XR system in MR mode. At box 908, process 900 may display the real-world environment to the user through the display of the XR system. For example, the MR mode may display an MR environment including a real-world environment and computer-generated virtual objects. At box 910, process 900 may repopulate the virtual objects based on the stored states of the virtual objects. For example, when the XR system enters a closed state or a sleep state in MR mode, the XR system may store the states of the virtual objects displayed in the MR environment. In some embodiments, the virtual objects may be repopulated and restored to these stored states, such as being displayed at the same location, displayed in the same posture or display state, and displayed with the same content.

[0069] At block 912, process 900 may restore the virtual tablet based on the stored state of the virtual tablet. For example, the virtual tablet may include display objects of one or more applications executed at the XR system. The virtual tablet may provide the user with practical and user-friendly tools for interacting with these applications in the MR environment.

[0070] When the XR system enters a sleep state or a shutdown state, the state of the virtual tablet can be stored. In some embodiments, the stored state can be used to restore the virtual tablet so that: the virtual tablet is located at the same location in the MR environment; and the content / one or more applications (including the state of one or more applications) displayed by the virtual tablet is restored.

[0071] At block 914, process 900 may begin booting in VR mode. At block 916, process 900 may scan the user's environment, such as the real-world environment around the user / XR system. At block 918, process 900 may determine whether the user is within the boundaries of the predefined use space. When the user is within the boundaries of the predefined use space, process 900 may proceed to block 922. When the user is not within the boundaries of the predefined use space, process 900 may proceed to block 920.

[0072] At block 920, process 900 may prompt the user to enter the boundaries of a predefined use space or create a new use space. For example, once the user is within the boundaries of the use space, process 900 may immerse the user in a VR environment. At block 922, process 900 may immerse the user in a VR environment.

[0073] Fig.10 1000 is a flowchart illustrating a process for making virtual objects persist when switching between virtual reality mode and mixed reality mode in some embodiments of the present technology. Process 1000 can be triggered in response to operating the XR system in VR mode or MR mode.

[0074] At block 1002, process 1000 can operate an XR system in a first XR mode. For example, the first XR mode can be either a VR mode or an MR mode.

[0075] At block 1004, process 1000 may display a virtual object in a first XR environment. For example, when the first XR mode is a VR mode, the first XR environment may be a VR environment with virtual objects. In another example, when the first XR mode is an MR mode, the first XR environment may be an MR environment with virtual objects.

[0076] At block 1006, process 600 may detect a transition condition for transitioning from the first XR mode to the second XR mode. Example transition conditions may include: a user action (e.g., tapping the HMD), a user movement that triggers a transition (e.g., the user moves out of the boundary of the use space), a software trigger (e.g., the user interacts with a virtual object that triggers a transition, the user selects an XR mode from the local XR system software, etc.), or any other suitable transition condition. When a transition condition is detected, process 1000 may proceed to block 1008. When a transition condition is not detected, process 1000 may loop back to block 1004, where the virtual object may continue to be displayed in the first XR environment until a transition condition is detected.

[0077] At box 1008, process 1000 can transition to a second XR mode. For example, the XR system can transition from a VR mode to an MR mode, or from an MR mode to a VR mode. At box 1010, process 1000 can change virtual objects that persist between the first XR environment and the second XR environment. For example, one or more displayed virtual objects that exist in the first XR mode can persist in the second XR mode. In some embodiments, changing the display of one or more virtual objects includes one or more of the following: changing the scale of the one or more virtual objects from the user's perspective; changing the skin of the one or more virtual objects (e.g., the visual appearance of wrapping around a structure); changing the distance at which the one or more virtual objects appear to be displayed from the user's perspective; or any combination of the above.

[0078] For example, when the XR system switches from VR mode to MR mode, the change may include: determining that the distance of one or more virtual objects from the user's perspective in the VR environment displayed during the VR mode is greater than a threshold; and in response to the determination, changing the distance at which the one or more virtual objects appear to be displayed from the user's perspective in the MR environment displayed during the MR mode. In addition, in response to the determination, the scale at which the one or more virtual objects appear to be displayed from the user's perspective in the MR environment may be changed. For example, the one or more virtual objects may appear to be displayed at a first scale from the user's perspective in the VR environment, the one or more virtual objects may appear to be displayed at a second scale from the user's perspective in the MR environment, and the adjustment may enlarge the one or more virtual objects in the MR environment from the user's perspective.

[0079] In some embodiments, the threshold distance that triggers a change in the distance and / or scale of a virtual object can be based on the size of the MR environment. For example, the MR environment can include virtual content (e.g., virtual objects) and real-world components (e.g., the real-world environment of the XR system). The XR system can determine the size of the real-world environment (e.g., the depth of the room in which the XR system is located) based on processing an image of the real-world environment through one or more computer vision models. The threshold distance can be based on a size metric (e.g., depth) of the real-world environment. In this example, a virtual object can be displayed in a VR environment at a distance that is incompatible with the real-world environment of the MR environment. Determining the threshold distance based on the size metric of the real-world environment can support changing the display of the virtual object so that the changed display is compatible with the real-world environment that constitutes part of the MR environment. In some embodiments, the threshold distance can be set to a size metric, can include a certain proportion of a size metric, or can be determined relative to a size metric in any other suitable manner.

[0080] In another example, when the XR system transitions from MR mode to VR mode, the change may include changing the skin of one or more virtual objects. For example, the one or more virtual objects may include a three-dimensional structure, and changing the skin of the one or more virtual objects may include changing one or more textures applied to the three-dimensional structure when the one or more virtual objects are displayed. In some embodiments, the one or more virtual objects are displayed with a first skin in an MR environment displayed during MR mode, and the one or more virtual objects are displayed with a second skin in a VR environment displayed during VR mode.

[0081] In some embodiments, during MR mode, the display of the skin of one or more virtual objects is controlled by a first software, and during VR mode, the display of the skin of one or more virtual objects is controlled by a second software, the first software applies the first skin in the MR environment, and the second software applies the second skin in the VR environment. For example, the first software can be a system shell, and the second software can be a third-party VR application.

[0082] In some embodiments, changing virtual objects that persist between a first XR environment and a second XR environment may include making the virtual objects appear the same to the user, but modifying the manner in which the virtual objects are displayed to be presentable in the new type of environment. For example, when moving to MR mode, virtual objects presented in VR mode may be reconfigured to be displayed consistently relative to the position of real-world objects, where the position may be related to the position of the virtual objects as perceived by the user in the VR environment. As another example, when moving to VR mode, virtual objects presented in MR mode may be reconfigured to appear to the user to be consistently positioned between environments, i.e., the position of the virtual objects is related to the position of the virtual objects as perceived by the user in the MR environment. This may include changing the manner in which virtual objects are anchored in the new mode, while appearing to the user as if the virtual objects are simply moving between modes.

[0083] In addition, changing the virtual objects that persist between the first XR environment and the second XR environment may include modifying the virtual objects to be consistent with the theme or rules of the new XR environment. For example, when moving to a VR environment with a medieval theme, the lightsaber presented in the MR environment may be reconfigured to look like a broadsword. In addition, the display, context, or function of the converted virtual object may be modified by the rules of the application that controls the new XR environment. Continuing with the previous example, the lightsaber in MR mode may already have the ability to cut off any other virtual object, but the application that controls the medieval VR environment may have a rule that specifies the upper limit of the ability of the object to cut off other objects, and the broadsword version of the virtual object cutting function may be set to only meet the upper limit. In this example, the first XR application may cause the virtual object to be displayed in the MR environment, the second XR application may cause the virtual object to be displayed in the VR environment, and in the context of the MR environment, the interaction of the virtual object with external elements (e.g., other virtual objects, users, user avatars, etc.) may be controlled by the first XR application, and in the context of the VR environment, the interaction of the virtual object with external elements may be controlled by the second XR application.

[0084] References in this specification to "embodiments" (e.g., "some embodiments," "various embodiments," "one embodiment," "an embodiment," etc.) mean that the particular features, structures, or characteristics described in connection with the embodiment are included in at least one embodiment of the present disclosure. The appearances of these phrases in different places in the specification do not necessarily all refer to the same embodiment, nor do they necessarily refer to separate embodiments or alternative embodiments that are mutually exclusive of other embodiments. In addition, various features are described that may be exhibited by some embodiments but not by other embodiments. Similarly, various requirements are described that may be requirements of some embodiments but not others.

[0085] As used herein, being higher than a threshold value means that the value of a comparison item is higher than other values ​​specified, the comparison item is located in a certain specified number of items with a maximum value, or the comparison item has a value within a specified maximum percentage value. As used herein, being lower than a threshold value means that the value of a comparison item is lower than other values ​​specified, the comparison item is located in a certain specified number of items with a minimum value, or the comparison item has a value within a specified minimum percentage value. As used herein, being within a threshold value means that the value of a comparison item is between two other values ​​specified, the comparison item is located in a middle specified number of items, or the comparison item has a value within a middle specified percentage range. When not otherwise defined, relative terms such as high or unimportant can be understood as assigning a value and determining how the value is compared with the established threshold value. For example, the phrase "selecting a fast connection" can be understood as selecting a connection with a value corresponding to the connection speed assigned thereto that is higher than a threshold value.

[0086] As used herein, the word "or" refers to any possible arrangement of a group of items. For example, the phrase "A, B or C" refers to any one of A, B, C, or any combination thereof, such as any one of the following: A; B; C; A and B; A and C; B and C; A, B and C; or any multiple of any item, such as A and A; B, B and C; A, A, B, C and C; etc.

[0087] Although the subject matter has been described in language specific to structural features and / or method actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Specific embodiments and implementations have been described herein for illustrative purposes, but various modifications may be made without departing from the scope of these embodiments and implementations. The specific features and actions described above are disclosed as example forms of implementing the appended claims. Therefore, these embodiments and implementations are not limited except as limited by the appended claims.

[0088] Any patent, patent application and other references mentioned above are incorporated herein by reference. If necessary, various aspects may be modified to provide further embodiments using the systems, functions and concepts of the various references mentioned above. If the statements or themes in the documents incorporated by reference conflict with the statements or themes of the present application, the present application shall prevail.

Claims

1. A method for switching between a virtual reality mode and a mixed reality mode, the method comprising: detecting, at the XR system, a trigger condition for transitioning from the first artificial reality mode to the second artificial reality mode; changing one or more user displays to transition from the first artificial reality mode to the second artificial reality mode, wherein: One or more displayed virtual objects present in the first artificial reality mode continue to exist in the second artificial reality mode; The display of the one or more virtual objects changes between the first artificial reality mode and the second artificial reality mode; and Changing the display of the one or more virtual objects includes one or more of the following: changing the scale of the one or more virtual objects from the user's perspective; changing the skin of the one or more virtual objects; changing the distance at which the one or more virtual objects appear to be displayed from the user's perspective; or any combination of the above.

2. The method according to claim 1, wherein: The first artificial reality mode includes a virtual reality mode, and the second artificial reality mode includes a mixed reality mode.

3. The method according to claim 2, wherein: Changing the display of the one or more virtual objects further comprises: determining that the distance of the one or more virtual objects from the user's perspective in the virtual reality environment displayed during the virtual reality mode is greater than a threshold; and In response to the determination, changing the distance at which the one or more virtual objects appear to be displayed from the user's perspective in a mixed reality environment displayed during the mixed reality mode.

4. The method according to claim 3, further comprising: In response to the determination, the scale at which the one or more virtual objects appear to be displayed in the mixed reality environment from the perspective of the user is changed, wherein the one or more virtual objects appear to be displayed at a first scale from the perspective of the user in the virtual reality environment, the one or more virtual objects appear to be displayed at a second scale from the perspective of the user in the mixed reality environment, and the adjustment magnifies the one or more virtual objects in the mixed reality environment from the perspective of the user.

5. The method according to claim 1, wherein: The first artificial reality mode includes a mixed reality mode, and the second artificial reality mode includes a virtual reality mode.

6. The method according to claim 5, wherein: Changing the display of the one or more virtual objects further comprises: Changing the skin of the one or more virtual objects, wherein the one or more virtual objects include a three-dimensional structure, and changing the skin of the one or more virtual objects includes: changing one or more textures applied to the three-dimensional structure when displaying the one or more virtual objects.

7. The method according to claim 6, wherein: The one or more virtual objects are displayed with a first skin in a mixed reality environment displayed during the mixed reality mode, and the one or more virtual objects are displayed with a second skin in a virtual reality environment displayed during the virtual reality mode.

8. The method according to claim 6, wherein: During the mixed reality mode, the display of the skin of the one or more virtual objects is controlled by a first software, and during the virtual reality mode, the display of the skin of the one or more virtual objects is controlled by a second software, the first software applies a first skin in the mixed reality environment, and the second software applies a second skin in the virtual reality environment.

9. The method according to claim 8, wherein: The first software includes a system shell and the second software includes a third party virtual reality application.

10. An artificial reality (XR) system that switches between a virtual reality mode and a mixed reality mode, the XR system comprising: one or more processors; as well as one or more memories storing instructions that, when executed by the one or more processors, cause the XR system to: detecting a trigger condition for switching from a first artificial reality mode to a second artificial reality mode; changing one or more user displays to transition from the first artificial reality mode to the second artificial reality mode, wherein one or more displayed virtual objects present in the first artificial reality mode persist in the second artificial reality mode; The display of the one or more virtual objects changes between the first artificial reality mode and the second artificial reality mode; and Changing the display of the one or more virtual objects includes one or more of the following: changing the scale of the one or more virtual objects from the user's perspective; changing the skin of the one or more virtual objects; changing the distance at which the one or more virtual objects appear to be displayed from the user's perspective; or any combination of the above.

11. The XR system of claim 10, wherein: The first artificial reality mode includes a virtual reality mode, and the second artificial reality mode includes a mixed reality mode.

12. The XR system of claim 11, wherein: Changing the display of the one or more virtual objects further comprises: determining that the distance of the one or more virtual objects from the user's perspective in the virtual reality environment displayed during the virtual reality mode is greater than a threshold; and In response to the determination, changing the distance at which the one or more virtual objects appear to be displayed from the user's perspective in a mixed reality environment displayed during the mixed reality mode.

13. The XR system of claim 12, wherein: The instructions, when executed by the one or more processors, further cause the XR system to: In response to the determination, the scale at which the one or more virtual objects appear to be displayed in the mixed reality environment from the perspective of the user is changed, wherein the one or more virtual objects appear to be displayed at a first scale from the perspective of the user in the virtual reality environment, the one or more virtual objects appear to be displayed at a second scale from the perspective of the user in the mixed reality environment, and the adjustment magnifies the one or more virtual objects in the mixed reality environment from the perspective of the user.

14. The XR system of claim 10, wherein: The first artificial reality mode includes a mixed reality mode, and the second artificial reality mode includes a virtual reality mode.

15. The XR system of claim 14, wherein: Changing the display of the one or more virtual objects further comprises: Changing the skin of the one or more virtual objects, wherein the one or more virtual objects include a three-dimensional structure, and changing the skin of the one or more virtual objects includes: changing one or more textures applied to the three-dimensional structure when displaying the one or more virtual objects.

16. The XR system of claim 15, wherein: The one or more virtual objects are displayed with a first skin in a mixed reality environment displayed during the mixed reality mode, and the one or more virtual objects are displayed with a second skin in a virtual reality environment displayed during the virtual reality mode.

17. The XR system of claim 15, wherein: During the mixed reality mode, the display of the skin of the one or more virtual objects is controlled by a first software, and during the virtual reality mode, the display of the skin of the one or more virtual objects is controlled by a second software, the first software applies a first skin in the mixed reality environment, and the second software applies a second skin in the virtual reality environment.

18. The XR system of claim 17, wherein: The first software includes a system shell and the second software includes a third party virtual reality application.

19. A computer-readable storage medium storing instructions for switching between a virtual reality mode and a mixed reality mode, the instructions, when executed by a computing system, causing the computing system to: detecting a trigger condition for switching from a first artificial reality mode to a second artificial reality mode; changing one or more user displays to transition from the first artificial reality mode to the second artificial reality mode, wherein: One or more displayed virtual objects present in the first artificial reality mode continue to exist in the second artificial reality mode; The display of the one or more virtual objects changes between the first artificial reality mode and the second artificial reality mode; and Changing the display of the one or more virtual objects includes one or more of the following: changing the scale of the one or more virtual objects from the user's perspective; changing the skin of the one or more virtual objects; changing the distance at which the one or more virtual objects appear to be displayed from the user's perspective; or any combination of the above.

20. The computer-readable storage medium of claim 19, wherein: The first artificial reality mode includes a virtual reality mode, and the second artificial reality mode includes a mixed reality mode, and changing the display of the one or more virtual objects further includes: determining that the distance of the one or more virtual objects from the user's perspective in the virtual reality environment displayed during the virtual reality mode is greater than a threshold; and In response to the determination, changing the distance at which the one or more virtual objects appear to be displayed from the user's perspective in a mixed reality environment displayed during the mixed reality mode.

Images