Method for providing immersive experience in environment

Abstract

To provide methods for providing an immersive experience in an environment.SOLUTION: A method includes: displaying, when detecting a first input corresponding to a request to initiate individual spatial effect via an input device while displaying a three-dimensional environment, the three-dimensional environment with the individual spatial effect at a first immersion level; displaying, when detecting an individual input while displaying the three-dimensional environment, the three-dimensional environment with the individual spatial effect at a second immersion level which is higher than the first immersion level, in accordance with a determination that the individual input is the first input; and displaying the three-dimensional environment with the individual spatial effect at a third immersion level which is lower than the first immersion level, via a display generation component, in accordance with a determination that the individual input is a second input which is different from the first input.SELECTED DRAWING: Figure 10A

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Patent Application No. 63 / 174,272, filed April 13, 2021, U.S. Provisional Patent Application No. 63 / 261,554, filed September 23, 2021, U.S. Provisional Patent Application No. 63 / 264,831, filed December 2, 2021, and U.S. Provisional Patent Application No. 63 / 362,799, filed April 11, 2022, the contents of which are incorporated herein by reference in their entirety for all purposes.

[0002] The present invention generally relates to computer systems having a display generation component and one or more input devices that present a graphical user interface, including but not limited to electronic devices that provide an immersive experience within a three-dimensional environment. [Background technology]

[0003] The development of computer systems for augmented reality has progressed significantly in recent years. Exemplary augmented reality environments include at least some virtual elements that replace or augment the physical world. Input devices such as cameras, controllers, joysticks, touch-sensitive surfaces, and touchscreen displays for computer systems and other electronic computing devices are used to interact with the virtual / augmented reality environment. Exemplary virtual elements include virtual objects, including digital images, video, text, icons, and control elements such as buttons and other graphics.

[0004] However, methods and interfaces for interacting with environments (e.g., applications, augmented reality environments, mixed reality environments, and virtual reality environments) that include at least some virtual elements are cumbersome, inefficient, and limited. For example, systems that provide insufficient feedback for performing actions associated with virtual objects, systems that require a series of inputs to achieve a desired result in an augmented reality environment, and systems in which manipulating virtual objects is complex and error-prone create a significant cognitive burden for users and detract from the experience of the virtual / augmented reality environment. In addition, these methods are unnecessarily time-consuming, thereby wasting energy. This latter consideration is particularly important in battery-operated devices. Summary of the Invention

[0005] Therefore, there is a need for a computer system having improved methods and interfaces for providing users with computer-generated experiences that make interaction with the computer system more efficient and intuitive for the user. Such methods and interfaces can optionally complement or replace conventional methods of providing users with computer-generated reality experiences. Such methods and interfaces reduce the number, extent, and / or type of inputs from the user by helping the user understand the connection between the input provided and the device response to that input, thereby creating a more efficient human-machine interface.

[0006] The above-mentioned deficiencies and other problems associated with user interfaces for computer systems having a display generation component and one or more input devices are reduced or eliminated by the disclosed system. In some embodiments, the computer system is a desktop computer with an associated display. In some embodiments, the computer system is a portable device (e.g., a notebook computer, a tablet computer, or a handheld device). In some embodiments, the computer system is a personal electronic device (e.g., a wearable electronic device such as a wristwatch or a head-mounted device). In some embodiments, the computer system has a touchpad. In some embodiments, the computer system has one or more cameras. In some embodiments, the computer system has a touch-sensitive display (also known as a "touch screen" or "touchscreen display"). In some embodiments, the computer system has one or more eye-tracking components. In some embodiments, the computer system has one or more hand-tracking components. In some embodiments, the computer system has one or more output devices in addition to the display generation component, the output devices including one or more tactile output generators and one or more audio output devices. In some embodiments, the computer system has a graphical user interface (GUI), one or more processors, memory, and one or more modules, programs, or instruction sets stored in the memory for performing a plurality of functions. In some embodiments, a user interacts with the GUI through stylus and / or finger contacts and gestures on the touch-sensitive surface, the movement of the user's eyes and hands in space relative to the GUI or the user's body as captured by cameras and other movement sensors, and voice input as captured by one or more audio input devices.In some embodiments, the functions performed through the interactions optionally include image editing, drawing, presenting, word processing, spreadsheet creation, game playing, making phone calls, video conferencing, emailing, instant messaging, training support, digital photography, digital videography, web browsing, digital music playback, note taking, and / or digital video playback, and executable instructions to perform those functions are optionally contained on a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors.

[0007] There is a need for electronic devices with improved methods and interfaces for interacting with objects in a three-dimensional environment. Such methods and interfaces can complement or replace conventional methods for interacting with objects in a three-dimensional environment. Such methods and interfaces reduce the number, extent, and / or type of input from a user, creating a more efficient human-machine interface.

[0008] In some embodiments, the electronic device alters the immersive level of the virtual environment and / or spatial effects within the three-dimensional environment based on the geometry of the physical environment surrounding the device. In some embodiments, the electronic device modifies the virtual environment and / or spatial effects in response to detecting movement of the device. In some embodiments, the electronic device moves a user interface of an application into or out of the virtual environment. In some embodiments, the electronic device selectively alters the display of simulated environments and / or atmospheric effects within the three-dimensional environment based on movement of objects associated with the user's viewpoint. In some embodiments, the electronic device provides feedback to the user in response to the user moving virtual objects into and / or within the simulated environment, according to some embodiments.

[0009] It should be noted that the various embodiments described above can be combined with any other embodiment described herein. The features and advantages described herein are not exhaustive, and many additional features and advantages will become apparent to those skilled in the art, particularly in light of the drawings, specification, and claims. Furthermore, it should be noted that the language used in this specification has been selected solely for the purposes of readability and explanation, and not to define or limit the subject matter of the present invention.

[0010] For a better understanding of the various described embodiments, reference should be made to the following Detailed Description of the Invention in conjunction with the following drawings, in which like reference numerals refer to corresponding parts throughout: [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram illustrating a computer system operating environment for providing a CGR experience, according to some embodiments.

[0012] [Figure 2] FIG. 1 is a block diagram illustrating a controller of a computer system configured to manage and coordinate a user's CGR experience, according to some embodiments.

[0013] [Figure 3] FIG. 1 is a block diagram illustrating display generation components of a computer system configured to provide a user with visual components of a CGR experience, according to some embodiments.

[0014] [Figure 4] FIG. 1 is a block diagram illustrating a hand tracking unit of a computer system configured to capture a user's gesture input, according to some embodiments.

[0015] [Figure 5]FIG. 1 is a block diagram illustrating an eye-tracking unit of a computer system configured to capture a user's gaze input, according to some embodiments.

[0016] [Figure 6A] 1 is a flowchart illustrating a glint-assisted gaze tracking pipeline, according to some embodiments.

[0017] [Figure 6B] 1 illustrates an exemplary environment for an electronic device for providing a CGR experience, according to some embodiments.

[0018] [Figure 7A] 1 illustrates an example of displaying a virtual environment, according to some embodiments. [Figure 7B] 1 illustrates an example of displaying a virtual environment, according to some embodiments. [Figure 7C] 1 illustrates an example of displaying a virtual environment, according to some embodiments. [Figure 7D] 1 illustrates an example of displaying a virtual environment, according to some embodiments. [Figure 7E] 1 illustrates an example of displaying a virtual environment, according to some embodiments. [Figure 7F] 1 illustrates an example of displaying a virtual environment, according to some embodiments. [Figure 7G] 1 illustrates an example of displaying a virtual environment, according to some embodiments. [Figure 7H] 1 illustrates an example of displaying a virtual environment, according to some embodiments.

[0019] [Figure 8A] 1 is a flowchart illustrating a method for displaying a virtual environment, according to some embodiments. [Figure 8B] 1 is a flowchart illustrating a method for displaying a virtual environment, according to some embodiments. [Figure 8C] 1 is a flowchart illustrating a method for displaying a virtual environment, according to some embodiments. [Figure 8D] 1 is a flowchart illustrating a method for displaying a virtual environment, according to some embodiments. [Figure 8E] 1 is a flowchart illustrating a method for displaying a virtual environment, according to some embodiments. [Figure 8F] 1 is a flowchart illustrating a method for displaying a virtual environment, according to some embodiments. [Figure 8G] 1 is a flowchart illustrating a method for displaying a virtual environment, according to some embodiments. [Figure 8H] 1 is a flowchart illustrating a method for displaying a virtual environment, according to some embodiments.

[0020] [Figure 9A] 1 illustrates an example of changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 9B] 1 illustrates an example of changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 9C] 1 illustrates an example of changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 9D] 1 illustrates an example of changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 9E] 1 illustrates an example of changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 9F] 1 illustrates an example of changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 9G] 1 illustrates an example of changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 9H] 1 illustrates an example of changing the immersion level of a three-dimensional environment, according to some embodiments.

[0021] [Figure 10A] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10B]1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10C] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10D] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10E] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10F] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10G] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10H] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10I] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10J] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10K] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10L] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10M] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10N] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10O] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments. [Figure 10P] 1 is a flowchart illustrating a method for changing the immersion level of a three-dimensional environment, according to some embodiments.

[0022] [Figure 11A] 1 illustrates an example of moving objects into and out of a virtual environment, according to some embodiments. [Figure 11B] 1 illustrates an example of moving objects into and out of a virtual environment, according to some embodiments. [Figure 11C] 1 illustrates an example of moving objects into and out of a virtual environment, according to some embodiments. [Figure 11D] 1 illustrates an example of moving objects into and out of a virtual environment, according to some embodiments.

[0023] [Figure 12A] 1 is a flowchart illustrating a method for moving objects into and out of a virtual environment, according to some embodiments. [Figure 12B] 1 is a flowchart illustrating a method for moving objects into and out of a virtual environment, according to some embodiments. [Figure 12C] 1 is a flowchart illustrating a method for moving objects into and out of a virtual environment, according to some embodiments. [Figure 12D] 1 is a flowchart illustrating a method for moving objects into and out of a virtual environment, according to some embodiments. [Figure 12E] 1 is a flowchart illustrating a method for moving objects into and out of a virtual environment, according to some embodiments. [Figure 12F] 1 is a flowchart illustrating a method for moving objects into and out of a virtual environment, according to some embodiments. [Figure 12G] 1 is a flowchart illustrating a method for moving objects into and out of a virtual environment, according to some embodiments.

[0024] [Figure 13A]10 illustrates an example of selectively altering the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on the movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 13B] 10 illustrates an example of selectively altering the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on the movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 13C] 10 illustrates an example of selectively altering the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on the movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 13D] 10 illustrates an example of selectively altering the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on the movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 13E] 10 illustrates an example of selectively altering the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on the movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 13F] 10 illustrates an example of selectively altering the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on the movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 13G] 10 illustrates an example of selectively altering the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on the movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 13H] 10 illustrates an example of selectively altering the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on the movement of an object associated with a user's viewpoint, according to some embodiments.

[0025] [Figure 14A] 1 is a flowchart illustrating a method for selectively modifying the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 14B] 1 is a flowchart illustrating a method for selectively modifying the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 14C] 1 is a flowchart illustrating a method for selectively modifying the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 14D] 1 is a flowchart illustrating a method for selectively modifying the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 14E] 1 is a flowchart illustrating a method for selectively modifying the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 14F] 1 is a flowchart illustrating a method for selectively modifying the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 14G] 1 is a flowchart illustrating a method for selectively modifying the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 14H] 1 is a flowchart illustrating a method for selectively modifying the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 14I] 1 is a flowchart illustrating a method for selectively modifying the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 14J]1 is a flowchart illustrating a method for selectively modifying the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 14K] 1 is a flowchart illustrating a method for selectively modifying the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on movement of an object associated with a user's viewpoint, according to some embodiments. [Figure 14L] 1 is a flowchart illustrating a method for selectively modifying the display of simulated environmental and / or atmospheric effects within a three-dimensional environment based on movement of an object associated with a user's viewpoint, according to some embodiments.

[0026] [Figure 15A] 10A-10C illustrate examples of providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. [Figure 15B] 10A-10C illustrate examples of providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. [Figure 15C] 10A-10C illustrate examples of providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. [Figure 15D] 10A-10C illustrate examples of providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. [Figure 15E] 10A-10C illustrate examples of providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. [Figure 15F] 10A-10C illustrate examples of providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. [Figure 15G]10A-10C illustrate examples of providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments.

[0027] [Figure 16A] 1 is a flowchart illustrating a method for providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. [Figure 16B] 1 is a flowchart illustrating a method for providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. [Figure 16C] 1 is a flowchart illustrating a method for providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. [Figure 16D] 1 is a flowchart illustrating a method for providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. [Figure 16E] 1 is a flowchart illustrating a method for providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. [Figure 16F] 1 is a flowchart illustrating a method for providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. [Figure 16G] 1 is a flowchart illustrating a method for providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. [Figure 16H]1 is a flowchart illustrating a method for providing feedback to a user in response to the user moving a virtual object to and / or within a simulated environment, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0028] The present disclosure relates to a user interface that provides a computer-generated reality (CGR) experience to a user, according to some embodiments.

[0029] The systems, methods, and GUIs described herein provide improved ways for electronic devices to interact with and manipulate objects in a three-dimensional environment.

[0030] In some embodiments, the computer system displays the virtual environment in a three-dimensional environment. In some embodiments, the virtual environment is displayed via a far-field process or a near-field process based on the geometry (e.g., size and / or shape) of the three-dimensional environment (e.g., optionally simulating the real-world environment surrounding the device). In some embodiments, the far-field process includes introducing the virtual environment from a location farthest from the user's viewpoint and gradually expanding the virtual environment toward the user's viewpoint (e.g., using information about the location, position, distance, etc. of objects in the environment). In some embodiments, the near-field process includes introducing the virtual environment from a location farthest from the user's viewpoint and expanding outward from the initial location (e.g., expanding the size of the virtual environment for the display generation component) without considering the distance and / or position of objects in the environment.

[0031] In some embodiments, the computer system displays a virtual environment and / or atmospheric effects within the three-dimensional environment. In some embodiments, displaying the atmospheric effects includes displaying one or more lighting and / or particle effects in the three-dimensional environment. In some embodiments, in response to detecting a movement of the device, portions of the virtual environment are de-highlighted, optionally without reducing the atmospheric effects. In some embodiments, in response to detecting a rotation of the user's body (e.g., simultaneously with the rotation of the device), the virtual environment is moved to a new location within the three-dimensional environment, optionally aligned with the user's body.

[0032] In some embodiments, the computer system displays the virtual environment simultaneously with the user interface of the application. In some embodiments, the user interface of the application may be moved into the virtual environment and treated as a virtual object residing in the virtual environment. In some embodiments, the user interface is automatically resized when moved into the virtual environment based on the distance of the user interface when moved into the virtual environment. In some embodiments, while displaying both the virtual environment and the user interface, the user may request that the user interface be displayed as an immersive environment. In some embodiments, in response to a request to display the user interface as an immersive environment, the previously displayed virtual environment is replaced with the immersive environment of the user interface.

[0033] In some embodiments, the computer system displays the simulated environment and / or atmospheric effects within the three-dimensional environment. In some embodiments, in response to detecting movement of the user, parts of the user (e.g., head, eyes, face, and / or body), the computer system, and / or other components of the computer system, the simulated environment is retracted within the three-dimensional environment to reveal portions of the physical environment. In some embodiments, the atmospheric effects are not reduced. In some embodiments, further movement of the user, parts of the user (e.g., head, eyes, face, and / or body), the computer system, and / or other components of the computer system causes the simulated environment to no longer be displayed.

[0034] In some embodiments, the electronic device displays a virtual object within a three-dimensional environment, the three-dimensional environment including a mimicked environment having a first visual appearance, the virtual object being located outside the mimicked environment within the three-dimensional environment. In some embodiments, in response to receiving an input that moves the virtual object into or into the mimicked environment, the electronic device displays the mimicked environment having a second visual appearance that is different from the first visual appearance. In some embodiments, changing the visual appearance of the mimicked environment includes changing the location of the mimicked environment, changing an immersion level associated with the mimicked environment, changing the opacity of the mimicked environment, and / or changing a color of the mimicked environment.

[0035] The processes described below enhance the usability of the device and streamline the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating / interacting with the device) through various techniques, including providing improved visual feedback to the user, reducing the number of inputs required to perform an operation, providing additional control options without cluttering the user interface with additional controls that are displayed, performing an operation without requiring further user input when a set of conditions is met, improving privacy and / or security, and / or other techniques. These techniques also reduce power usage and improve the device's battery life by allowing the user to use the device more quickly and efficiently.

[0036] 1-6 provide a description of an exemplary computer system for providing a CGR experience to a user (as described below with respect to methods 800, 1000, 1200, 1400, and 1600). In some embodiments, the CGR experience is provided to a user via an operating environment 100 that includes a computer system 101, as shown in FIG. The computer system 101 includes a controller 110 (e.g., a processor of a portable electronic device or a remote server), a display generation component 120 (e.g., a head-mounted device (HMD), a display, a projector, a touchscreen, etc.), one or more input devices 125 (e.g., an eye-tracking device 130, a hand-tracking device 140, other input devices 150), one or more output devices 155 (e.g., a speaker 160, a tactile output generator 170, and other output devices 180), one or more sensors 190 (e.g., an image sensor, a light sensor, a depth sensor, a touch sensor, an orientation sensor, a proximity sensor, a temperature sensor, a location sensor, a motion sensor, a speed sensor, etc.), and optionally one or more peripheral devices 195 (e.g., a home appliance, a wearable device, etc.). In some embodiments, one or more of the input device 125, the output device 155, the sensor 190, and the peripheral device 195 are integrated with the display generation component 120 (e.g., within a head-mounted or handheld device).

[0037] When describing a CGR experience, various terms are used to individually refer to several related, but distinct, environments that a user senses and / or with which the user can interact (e.g., using inputs detected by computer system 101 that cause the computer system generating the CGR experience to generate audio, visual, and / or haptic feedback corresponding to various inputs provided to computer system 101 generating the CGR experience). The following is a subset of these terms:

[0038] Physical Environment: The physical environment refers to the physical world that people can sense and / or interact with without the aid of electronic systems. A physical environment, such as a physical park, includes physical objects such as physical trees, physical buildings, and physical people. People can directly sense and / or interact with the physical environment through their senses, such as sight, touch, hearing, taste, and smell.

[0039] Computer-Generated Reality (or Extended Reality (XR)): In contrast, a Computer-Generated Reality (CGR) environment refers to a wholly or partially mimicked environment that people sense and / or interact with via electronic systems. In a CGR, a subset of a person's body movements or representations thereof are tracked, and one or more properties of one or more virtual objects simulated within the CGR environment are adjusted accordingly to behave according to at least one law of physics. For example, a CGR system may detect a person's head rotation and adjust the graphical content and sound field presented to the person accordingly, in a manner similar to how such views and sounds change in a physical environment. In some circumstances (e.g., for accessibility reasons), adjustments to the property(ies) of a virtual object(s) in a CGR environment may be made in response to a representation of a body movement (e.g., a voice command). A person may sense and / or interact with a CGR object using any one of these senses, including sight, hearing, touch, taste, and smell. For example, a person may sense and / or interact with audio objects that create a 3D or spatially expansive audio environment that provides the perception of a point sound source in 3D space. In another example, audio objects may enable audio transparency that selectively incorporates ambient sounds from the physical environment, with or without computer-generated audio. In some CGR environments, a person may sense and / or interact with only audio objects.

[0040] Examples of CGR include virtual reality and mixed reality.

[0041] Virtual Reality: A virtual reality (VR) environment refers to an emulated environment designed to be based entirely on computer-generated sensory input for one or more senses. A VR environment includes multiple virtual objects that a person can sense and / or interact with. For example, computer-generated images of trees, buildings, and avatars representing people are examples of virtual objects. A person can sense and / or interact with virtual objects in the VR environment through a simulation of the person's presence in the computer-generated environment and / or through a simulation of a subset of the person's physical movement within the computer-generated environment.

[0042] Mixed Reality: A mixed reality (MR) environment refers to a mimetic environment designed to incorporate sensory input from or representations of a physical environment in addition to including computer-generated sensory input (e.g., virtual objects), as opposed to a VR environment designed to be based entirely on computer-generated sensory input. On the virtual continuum, a mixed reality environment is anywhere between, but not including, a fully physical environment at one end and a virtual reality environment at the other. In some MR environments, computer-generated sensory input may respond to changes in sensory input from the physical environment. Some electronic systems for presenting MR environments may also track location and / or orientation relative to the physical environment to allow virtual objects to interact with real objects (i.e., physical items from the physical environment or representations thereof). For example, the system may account for movement so that a virtual tree appears stationary relative to the physical ground.

[0043] Examples of mixed reality include augmented reality and augmented virtuality.

[0044] Augmented reality: An augmented reality (AR) environment refers to a simulated environment in which one or more virtual objects are superimposed on a physical environment or a representation thereof. For example, an electronic system for presenting an AR environment may have a transparent or translucent display through which a person can directly view the physical environment. The system may be configured to present virtual objects on the transparent or translucent display, whereby a person using the system perceives the virtual objects superimposed on the physical environment. Alternatively, the system may have an opaque display and one or more imaging sensors that capture images or videos of the physical environment, which are representations of the physical environment. The system composites the images or videos with virtual objects and presents the composite on the opaque display. The person uses the system to indirectly view the physical environment through the images or videos of the physical environment and perceive the virtual objects superimposed on the physical environment. As used herein, video of a physical environment shown on an opaque display is referred to as "pass-through video," meaning that the system captures images of the physical environment using one or more image sensors and uses those images in presenting the AR environment on the opaque display. Alternatively, the system may include a projection system that projects virtual objects, e.g., as holograms, into a physical environment or onto a physical surface, such that a person using the system perceives the virtual objects superimposed on the physical environment. Augmented reality environments also refer to mimic environments in which a representation of a physical environment is transformed by computer-generated sensory information. For example, when providing pass-through video, a system may distort one or more sensor images to impose a selected perspective (e.g., viewpoint) different from the perspective captured by the imaging sensor. As another example, a representation of a physical environment may be distorted by graphically modifying (e.g., enlarging) portions thereof, thereby rendering the modified portions a non-photorealistic, altered version of the originally captured image. As a further example, a representation of a physical environment may be distorted by graphically removing or obscuring portions thereof.

[0045] Augmented Virtual: An augmented virtual (AV) environment refers to a mimicking environment in which a virtual or computer-generated environment incorporates one or more sensory inputs from a physical environment. The sensory inputs may be representations of one or more characteristics of the physical environment. For example, an AV park may have virtual trees and virtual buildings, while people with faces are realistically recreated from images taken of physical people. As another example, virtual objects may adopt the shape or color of physical items imaged by one or more imaging sensors. As a further example, virtual objects may adopt shadows that match the position of the sun in the physical environment.

[0046] Perspective-Locked Virtual Object: A virtual object is perspective-locked when the computer system displays the virtual object in the same location and / or position within the user's perspective, even as the user's perspective shifts (e.g., changes). In embodiments in which the computer system is a head-mounted device, the user's perspective is locked to the forward-facing orientation of the user's head (e.g., the user's perspective is at least a portion of the user's field of view when the user is looking straight ahead). Thus, the user's perspective remains fixed even as the user's line of sight moves without moving the user's head. In embodiments in which the computer system has a display generation component (e.g., a display screen) that can be repositioned relative to the user's head, the user's perspective is the augmented reality view being presented to the user on the display generation component of the computer system. For example, a perspective-locked virtual object that is displayed in the upper left corner of the user's perspective when the user's perspective is in a first orientation (e.g., the user's head is facing north) continues to be displayed in the upper left corner of the user's perspective when the user's perspective changes to a second orientation (e.g., the user's head is facing west). In other words, the location and / or position at which a viewpoint-locked virtual object is displayed in a user's viewpoint is independent of the user's position and / or orientation in the physical environment. In embodiments in which the computer system is a head-mounted device, the user's viewpoint is locked to the orientation of the user's head, such that the virtual object is also referred to as a "head-locked virtual object."

[0047] Environment-Locked Virtual Object: A virtual object is environment-locked (or "world-locked") when a computer system displays the virtual object at a location and / or position within a user's viewpoint that is based on (e.g., selected with reference to and / or anchored to) locations and / or objects within a three-dimensional environment (e.g., a physical environment or a virtual environment). As the user's viewpoint shifts, the locations and / or objects within the environment relative to the user's viewpoint change, resulting in the environment-locked virtual object appearing at a different location and / or position within the user's viewpoint. For example, an environment-locked virtual object locked to a tree directly in front of the user will appear centered within the user's viewpoint. If the user's viewpoint shifts to the right (e.g., the user's head is turned to the right) and the tree becomes more left-leaning within the user's viewpoint (e.g., the position of the tree within the user's viewpoint shifts), the environment-locked virtual object locked to the tree will appear more left-leaning within the user's viewpoint. In other words, the location and / or position at which the environment-locked virtual object appears within the user's viewpoint depends on the position and / or orientation of the location and / or object in the environment to which the virtual object is locked. In some embodiments, the computer system uses a stationary reference frame (e.g., a coordinate system fixed to a fixed location and / or object in the physical environment) to determine a position at which to display an environment-locked virtual object in the user's viewpoint. The environment-locked virtual object can be locked to a stationary portion of the environment (e.g., a floor, wall, table, or other stationary object) or can be locked to a moving portion of the environment (e.g., a vehicle, an animal, a person, or a representation of a part of the user's body that moves independent of the user's viewpoint, such as the user's hand, wrist, arm, or leg), so that the virtual object moves as the viewpoint or part of the environment moves in order to maintain a fixed relationship between the virtual object and the part of the environment.

[0048] In some embodiments, an environment-locked or viewpoint-locked virtual object exhibits delayed-following behavior, which reduces or delays the movement of the environment-locked or viewpoint-locked virtual object relative to the movement of a reference point that the virtual object is following. In some embodiments, when exhibiting delayed-following behavior, the computer system intentionally delays the movement of the virtual object when it detects movement of the reference point that the virtual object is following (e.g., a part of the environment, the viewpoint, or a point fixed relative to the viewpoint, such as a point between 5 and 300 cm from the viewpoint). For example, when the reference point (e.g., a part of the environment or the viewpoint) moves at a first speed, the virtual object is moved by the device to remain locked to the reference point, but at a second speed that is slower than the first speed (e.g., until the reference point stops or slows down, at which point the virtual object begins to catch up with the reference point). In some embodiments, when the virtual object exhibits delayed-following behavior, the device ignores small amounts of movement of the reference point (e.g., ignores movement of the reference point that is less than a threshold amount, such as movement between 0 and 5 degrees or movement between 0 and 50 cm). For example, when the reference point (e.g., a portion of the environment or a viewpoint to which the virtual object is locked) moves by a first amount, the distance between the reference point and the virtual object increases (e.g., because the virtual object is displayed to maintain a fixed or substantially fixed position relative to a viewpoint or portion of the environment different from the reference point to which the virtual object is locked), and when the reference point (e.g., a portion of the environment or a viewpoint to which the virtual object is locked) moves by a second amount greater than the first amount, the distance between the reference point and the virtual object initially increases (e.g., because the virtual object is displayed to maintain a fixed or substantially fixed position relative to a viewpoint or portion of the environment different from the reference point to which the virtual object is locked), and then decreases as the amount of movement of the reference point increases beyond a threshold (e.g., a “delayed following” threshold) as the virtual object is moved by the computer system to maintain a fixed or substantially fixed position relative to the reference point.In some embodiments, a virtual object that maintains a substantially fixed position relative to a reference point includes a virtual object that is displayed within a threshold distance (e.g., 1, 2, 3, 5, 15, 20, or 50 cm) of the reference point in one or more dimensions (e.g., above / below, left / right, and / or in front / behind relative to the position of the reference point).

[0049] Hardware: There are many different types of electronic systems that enable a person to sense and / or interact with various CGR environments. Examples include head-mounted systems, projection-based systems, heads-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays formed as lenses designed to be placed over a person's eyes (e.g., similar to contact lenses), headphones / earphones, speaker arrays, input systems (e.g., wearable or handheld controllers with or without haptic feedback), smartphones, tablets, and desktop / laptop computers. A head-mounted system may have one or more speaker(s) and an integrated opaque display. Alternatively, a head-mounted system may be configured to accept an external opaque display (e.g., a smartphone). A head-mounted system may incorporate one or more imaging sensors for capturing images or video of the physical environment and / or one or more microphones for capturing audio of the physical environment. A head-mounted system may have a transparent or translucent display rather than an opaque display. The transparent or translucent display may have a medium through which light representing an image is directed to a person's eyes. The display may utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser-scanned light source, or any combination of these technologies. The medium may be a light guide, a holographic medium, an optical combiner, an optical reflector, or any combination thereof. In one embodiment, the transparent or translucent display may be configured to be selectively opaque. The projection-based system may employ retinal projection technology to project a graphical image onto a person's retina. The projection system may also be configured to project virtual objects into the physical environment, for example, as holograms or as physical surfaces.In some embodiments, controller 110 is configured to manage and coordinate the user's CGR experience. In some embodiments, controller 110 includes a suitable combination of software, firmware, and / or hardware. Controller 110 is described in more detail below with reference to FIG. 2. In some embodiments, controller 110 is a computing device that is local or remote to scene 105 (e.g., the physical environment). For example, controller 110 is a local server located within scene 105. In another example, controller 110 is a remote server (e.g., a cloud server, a central server, etc.) located outside scene 105. In some embodiments, controller 110 is communicatively coupled to display generation component 120 (e.g., an HMD, a display, a projector, a touchscreen, etc.) via one or more wired or wireless communication channels 144 (e.g., BLUETOOTH, IEEE 802.11x, IEEE 802.16x, IEEE 802.3x, etc.). In another example, the controller 110 is contained within the housing (e.g., physical housing) of, or shares the same physical housing or support structure as, one or more of the display generation component 120 (e.g., an HMD or a portable electronic device including a display and one or more processors), one or more of the input devices 125, one or more of the output devices 155, one or more of the sensors 190, and / or one or more of the peripheral devices 195.

[0050] In some embodiments, display generation component 120 is configured to provide a CGR experience (e.g., at least a visual component of the CGR experience) to a user. In some embodiments, display generation component 120 includes a suitable combination of software, firmware, and / or hardware. Display generation component 120 is described in more detail below with reference to FIG. 3. In some embodiments, functionality of controller 110 is provided by and / or combined with display generation component 120.

[0051] According to some embodiments, the display generation component 120 provides a CGR experience to the user while the user is virtually and / or physically present within the scene 105.

[0052] In some embodiments, the display generation component is worn on a part of the user's body (e.g., on their head, their hand, etc.). Thus, display generation component 120 includes one or more CGR displays provided for displaying CGR content. For example, in various embodiments, display generation component 120 surrounds the user's field of view. In some embodiments, display generation component 120 is a handheld device (e.g., a smartphone or tablet) configured to present CGR content, where the user holds the device with a display pointed toward the user's field of view and a camera pointed toward scene 105. In some embodiments, the handheld device is optionally located within a housing worn on the user's head. In some embodiments, the handheld device is optionally located on a support (e.g., a tripod) in front of the user. In some embodiments, display generation component 120 is a CGR chamber, housing, or room configured to present CGR content without the user wearing or holding display generation component 120. Many user interfaces described with reference to one type of hardware for displaying CGR content (e.g., a handheld device or a device on a tripod) may be implemented on another type of hardware for displaying CGR content (e.g., an HMD or other wearable computing device). For example, a user interface showing interactions with CGR content triggered based on interactions occurring in the space in front of a handheld or tripod-mounted device may be implemented similarly to an HMD in which the interactions occur in the space in front of the HMD and the CGR content responses are displayed via the HMD. Similarly, a user interface showing interactions with CGR content triggered based on movement of a handheld or tripod-mounted device relative to the physical environment (e.g., scene 105 or a part of the user's body (e.g., the user's eye(s), head, or hands)) may be implemented similarly to an HMD in which the movement is caused by movement of the HMD relative to the physical environment (e.g., scene 105 or a part of the user's body (e.g., the user's eye(s), head, or hands)).

[0053] While relevant features of operating environment 100 are shown in FIG. 1, those skilled in the art will understand from this disclosure that various other features have not been shown for the sake of brevity so as not to obscure more pertinent aspects of the exemplary embodiments disclosed herein.

[0054] 2 is a block diagram of an example controller 110, according to some embodiments. While certain features are shown, those skilled in the art will understand from this disclosure that various other features are not shown for the sake of brevity so as not to obscure more pertinent aspects of the embodiments disclosed herein. Thus, by way of non-limiting example, in some embodiments, the controller 110 includes one or more processing units 202 (e.g., a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a graphics processing unit (GPU), a central processing unit (CPU), a processing core, etc.), one or more input / output (I / O) devices 206, one or more communication interfaces 208 (e.g., Universal Serial Bus (USB), FIREWIRE, THUNDERBOLT, IEEE 802.3x, IEEE 802.11x, IEEE 802.16x, Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Global Positioning System (GPS), Infrared (IR), BLUETOOTH, ZIGBEE, or similar types of interfaces), one or more programming (e.g., I / O) interfaces 210, memory 220, and one or more communication buses 204 for interconnecting these and various other components.

[0055] In some embodiments, one or more communication buses 204 include circuitry that interconnects and controls communication between system components. In some embodiments, one or more I / O devices 206 include at least one of a keyboard, a mouse, a touchpad, a joystick, one or more microphones, one or more speakers, one or more image sensors, one or more displays, etc.

[0056] Memory 220 includes high-speed random-access memory, such as dynamic random-access memory (DRAM), static random-access memory (SRAM), double-data-rate random-access memory (DDRRAM), or other random-access solid-state memory devices. In some embodiments, memory 220 includes non-volatile memory, such as one or more magnetic storage devices, optical storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 220 optionally includes one or more storage devices located remotely from one or more processing units 202. Memory 220 includes a non-transitory computer-readable storage medium. In some embodiments, memory 220, or the non-transitory computer-readable storage medium of memory 220, stores the following programs, modules, and data structures, or a subset thereof, including optional operating system 230 and CGR experience module 240:

[0057] Operating system 230 includes instructions for handling various basic system services and performing hardware-dependent tasks. In some embodiments, CGR experience module 240 is configured to manage and coordinate one or more CGR experiences for one or more users (e.g., a single CGR experience for one or more users, or multiple CGR experiences for respective groups of one or more users). To that end, in various embodiments, CGR experience module 240 includes a data acquisition unit 242, a tracking unit 244, an adjustment unit 246, and a data transmission unit 248.

[0058] 1 , and optionally one or more of input devices 125, output devices 155, sensors 190, and / or peripheral devices 195. To that end, in various embodiments, data acquisition unit 242 includes instructions and / or logic therefor, as well as heuristics and metadata therefor.

[0059] In some embodiments, tracking unit 244 is configured to map scene 105 and track the position / location of at least display generation component 120 relative to scene 105 of FIG. 1 , and optionally relative to one or more of input device 125, output device 155, sensor 190, and / or peripheral device 195. To that end, in various embodiments, tracking unit 244 includes instructions and / or logic therefor, as well as heuristics and metadata therefor. In some embodiments, tracking unit 244 includes hand tracking unit 243 and / or eye tracking unit 245. In some embodiments, hand tracking unit 243 is configured to track the position / location of one or more parts of a user's hand and / or the movement of one or more parts of a user's hand relative to scene 105 of FIG. 1 , relative to display generation component 120, and / or relative to a coordinate system defined relative to the user's hand. Hand tracking unit 243 is described in more detail below with respect to FIG. 4. In some embodiments, eye tracking unit 245 is configured to track the position and movement of the user's gaze (or, more broadly, the user's eyes, face, or head) relative to scene 105 (e.g., relative to the physical environment and / or the user (e.g., the user's hands)) or relative to CGR content displayed via display generation component 120. Eye tracking unit 245 is described in more detail below with respect to FIG. 5.

[0060] In some embodiments, adjustment unit 246 is configured to manage and adjust the CGR experience presented to the user by display generation component 120 and, optionally, by one or more of output devices 155 and / or peripheral devices 195. To that end, in various embodiments, adjustment unit 246 includes instructions and / or logic therefor, as well as heuristics and metadata therefor.

[0061] In some embodiments, data transmission unit 248 is configured to transmit data (e.g., presentation data, location data, etc.) to at least display generation component 120, and optionally to one or more of input device 125, output device 155, sensor 190, and / or peripheral device 195. To that end, in various embodiments, data transmission unit 248 includes instructions and / or logic therefor, as well as heuristics and metadata therefor.

[0062] Although the data acquisition unit 242, the tracking unit 244 (e.g., including the eye tracking unit 243 and the hand tracking unit 244), the adjustment unit 246, and the data transmission unit 248 are shown as being present on a single device (e.g., the controller 110), it should be understood that in other embodiments, any combination of the data acquisition unit 242, the tracking unit 244 (e.g., including the eye tracking unit 243 and the hand tracking unit 244), the adjustment unit 246, and the data transmission unit 248 may be located within separate computing devices.

[0063] Furthermore, Figure 2 is intended more to illustrate the functionality of various features that may be present in particular embodiments, as opposed to a structural overview of the embodiments described herein. As will be recognized by those skilled in the art, items shown separately can be combined and some items can be separated. For example, some functional modules shown separately in Figure 2 can be implemented in a single module, and various functions of a single functional block can be implemented by one or more functional blocks in various embodiments. The actual number of modules, as well as the division of specific functions and how functions are allocated among them, will vary from implementation to implementation and, in some embodiments, will depend in part on the particular combination of hardware, software, and / or firmware selected for a particular implementation.

[0064] 3 is a block diagram of an example of a display generation component 120, according to some embodiments. While certain features are shown, those skilled in the art will understand from this disclosure that, for the sake of brevity, various other features are not shown so as not to obscure more pertinent aspects of the embodiments disclosed herein. To that end, by way of non-limiting example, in some embodiments, the HMD 120 includes one or more processing units 302 (e.g., microprocessors, ASICs, FPGAs, GPUs, CPUs, processing cores, etc.), one or more input / output (I / O) devices and sensors 306, one or more communication interfaces 308 (e.g., USB, FIREWIRE, THUNDERBOLT, IEEE 802.3x, IEEE 802.11x, IEEE 802.16x, GSM, CDMA, TDMA, GPS, infrared, BLUETOOTH, ZIGBEE, and / or similar types of interfaces), one or more programming (e.g., I / O) interfaces 310, one or more CGR displays 312, one or more optional inward-facing and / or outward-facing image sensors 314, memory 320, and one or more communication buses 304 for interconnecting these and various other components.

[0065] In some embodiments, the one or more communication buses 304 include circuitry that interconnects and controls communications between system components. In some embodiments, the one or more I / O devices and sensors 306 include at least one of an inertial measurement unit (IMU), an accelerometer, a gyroscope, a thermometer, one or more physiological sensors (e.g., a blood pressure monitor, a heart rate monitor, a blood oxygen sensor, a blood glucose sensor, etc.), one or more microphones, one or more speakers, a haptic engine, one or more depth sensors (e.g., structured light, time of flight, etc.), etc.

[0066] In some embodiments, one or more CGR displays 312 are configured to provide a CGR experience to a user. In some embodiments, one or more CGR displays 312 correspond to holographic, digital light processing (DLP), liquid crystal display (LCD), liquid crystal on silicon (LCoS), organic light-emitting field-effect transistor (OLET), organic light-emitting diode (OLED), surface-conduction electron-emissive element display (SED), field-emission display (FED), quantum dot light-emitting diode (QD-LED), MEMS, and / or similar display types. In some embodiments, one or more CGR displays 312 correspond to a waveguide display, such as a diffractive, reflective, polarized, holographic, etc. For example, the HMD 120 includes a single CGR display. In another example, the HMD 120 includes a CGR display for each eye of the user. In some embodiments, one or more CGR displays 312 are capable of presenting MR or VR content. In some embodiments, one or more CGR displays 312 are capable of presenting MR or VR content.

[0067] In some embodiments, the one or more image sensors 314 are configured to acquire image data corresponding to at least a portion of the user's face, including the user's eyes (and may be referred to as eye-tracking cameras). In some embodiments, the one or more image sensors 314 are configured to acquire image data corresponding to at least a portion of the user's hand(s) and optionally the user's arm(s) (and may be referred to as hand-tracking cameras). In some embodiments, the one or more image sensors 314 are configured to face forward to acquire image data corresponding to a scene viewed by the user when the HMD 120 is not present (and may be referred to as scene cameras). The one or more optional image sensors 314 may include one or more RGB cameras (e.g., with a complementary metal-oxide semiconductor (CMOS) image sensor or a charge-coupled device (CCD) image sensor), one or more infrared (IR) cameras, one or more event-based cameras, and / or the like.

[0068] Memory 320 includes high-speed random-access memory, such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices. In some embodiments, memory 320 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 320 optionally includes one or more storage devices located remotely from one or more processing units 302. Memory 320 includes a non-transitory computer-readable storage medium. In some embodiments, memory 320, or its non-transitory computer-readable storage medium, stores the following programs, modules, and data structures, or a subset thereof, including an optional operating system 330 and a CGR presentation module 340:

[0069] The operating system 330 includes instructions for handling various basic system services and for performing hardware-dependent tasks. In some embodiments, the CGR presentation module 340 is configured to present CGR content to a user via one or more CGR displays 312. To that end, in various embodiments, the CGR presentation module 340 includes a data acquisition unit 342, a CGR presentation unit 344, a CGR map generation unit 346, and a data transmission unit 348.

[0070] In some embodiments, the data acquisition unit 342 is configured to acquire data (e.g., presentation data, interaction data, sensor data, location data, etc.) from at least the controller 110 of Figure 1. To that end, in various embodiments, the data acquisition unit 342 includes instructions and / or logic therefor, as well as heuristics and metadata therefor.

[0071] In some embodiments, the CGR presentation unit 344 is configured to present CGR content via one or more CGR displays 312. To that end, in various embodiments, the CGR presentation unit 344 includes instructions and / or logic therefor, as well as heuristics and metadata therefor.

[0072] In some embodiments, the CGR map generation unit 346 is configured to generate a CGR map (e.g., a 3D map of a mixed reality scene or a map of a physical environment in which computer-generated objects can be placed) based on the media content data. To that end, in various embodiments, the CGR map generation unit 346 includes instructions and / or logic therefor, as well as heuristics and metadata therefor.

[0073] In some embodiments, data transmission unit 348 is configured to transmit data (e.g., presentation data, location data, etc.) to at least controller 110, and optionally to one or more of input device 125, output device 155, sensor 190, and / or peripheral device 195. To that end, in various embodiments, data transmission unit 348 includes instructions and / or logic therefor, as well as heuristics and metadata therefor.

[0074] Although the data acquisition unit 342, the CGR presentation unit 344, the CGR map generation unit 346, and the data transmission unit 348 are shown as residing on a single device (e.g., the display generation component 120 of FIG. 1), it should be understood that in other embodiments, any combination of the data acquisition unit 342, the CGR presentation unit 344, the CGR map generation unit 346, and the data transmission unit 348 may be located within separate computing devices.

[0075] Furthermore, Figure 3 is intended more to illustrate the functionality of various features that may be present in particular implementations, as opposed to a structural overview of the embodiments described herein. As will be recognized by those skilled in the art, items shown separately can be combined and some items can be separated. For example, some functional modules shown separately in Figure 3 can be implemented within a single module, and various functions of a single functional block can be performed by one or more functional blocks in various embodiments. The actual number of modules, as well as the division of specific functions and how functions are allocated among them, will vary from implementation to implementation and, in some embodiments, will depend in part on the particular combination of hardware, software, and / or firmware selected for a particular implementation.

[0076] 4 is a schematic diagram of an example embodiment of a hand tracking device 140. In some embodiments, the hand tracking device 140 (FIG. 1) is controlled by a hand tracking unit 243 (FIG. 2) to track the location / position of one or more parts of a user's hand and / or the movement of one or more parts of the user's hand relative to the scene 105 of FIG. 1 (e.g., relative to a portion of the physical environment surrounding the user, relative to the display generation component 120, or relative to a portion of the user (e.g., the user's face, eyes, or head), and / or relative to a coordinate system defined relative to the user's hand). In some embodiments, the hand tracking device 140 is part of the display generation component 120 (e.g., embedded in or attached to a head-mounted device). In some embodiments, the hand tracking device 140 is separate from the display generation component 120 (e.g., located in a separate housing or attached to a separate physical support structure).

[0077] In some embodiments, the hand tracking device 140 includes an image sensor 404 (e.g., one or more IR cameras, 3D cameras, depth cameras, and / or color cameras) that captures three-dimensional scene information including at least the hand 406 of a human user. The image sensor 404 captures hand images with sufficient resolution to allow for differentiation of the fingers and their respective positions. The image sensor 404 typically captures images of other parts of the user's body, or all of the body, and can have either zoom capabilities or a dedicated sensor with high magnification to capture hand images at a desired resolution. In some embodiments, the image sensor 404 also captures 2D color video images of the hand 406 and other elements of the scene. In some embodiments, the image sensor 404 is used in conjunction with or functions as an image sensor that captures the physical environment of the scene 105. In some embodiments, the image sensor 404 is positioned relative to the user or the user's environment such that the field of view of the image sensor, or a portion thereof, is used to define an interaction space in which hand movements captured by the image sensor are processed as inputs to the controller 110.

[0078] In some embodiments, image sensor 404 outputs a sequence of frames containing 3D map data (and possibly color image data) to controller 110, which extracts high-level information from the map data. This high-level information is provided, typically via an application program interface (API), to an application running on the controller, which drives display generation component 120 accordingly. For example, a user can interact with software running on controller 110 by moving their hand 408 and changing the posture of their hand.

[0079] In some embodiments, the image sensor 404 projects a spot pattern onto a scene including the hand 406 and captures an image of the projected pattern. In some embodiments, the controller 110 calculates the 3D coordinates of points in the scene (including points on the surface of the user's hand) by triangulation based on the lateral shift of the pattern's spots. This approach is advantageous in that it does not require the user to hold or wear any type of beacon, sensor, or other marker. This provides depth coordinates of points in the scene relative to a predetermined reference plane at a specific distance from the image sensor 404. In this disclosure, the image sensor 404 is assumed to define a set of orthogonal x, y, and z axes such that the depth coordinate of a point in the scene corresponds to the z-component measured by the image sensor. Alternatively, the hand tracking device 440 can use other 3D mapping methods, such as stereoscopic imaging or time-of-flight measurements, based on single or multiple cameras or other types of sensors.

[0080] In some embodiments, the hand tracking device 140 captures and processes a time sequence of depth maps containing the user's hand while the user moves the hand (e.g., the entire hand or one or more fingers). Software running on the image sensor 404 and / or a processor in the controller 110 processes the 3D map data to extract patch descriptors of the hand in these depth maps. The software matches these descriptors with patch descriptors stored in the database 408, based on a previous learning process, to estimate the pose of the hand in each frame. The pose typically includes the 3D locations of the user's wrist joints and fingertips.

[0081] The software can also analyze hand and / or finger trajectories across multiple frames in a sequence to identify gestures. The pose estimation functionality described herein may be interleaved with motion tracking functionality, whereby patch-based pose estimation is performed only once every two (or more) frames, while tracking is used to discover pose changes that occur across the remaining frames. The pose, motion, and gesture information is provided to an application program running on controller 110 via the API described above. This program can, for example, move and modify an image presented on display generation component 120 or perform other functions in response to the pose and / or gesture information.

[0082] In some embodiments, the gesture includes an air gesture, which is detected without (or independent of) the user touching an input element that is part of a device (e.g., computer system 101, one or more input devices 125, and / or hand tracking device 140) and is based on detected movement of a part of the user's body in the air (e.g., head, one or more arms, one or more hands, one or more fingers, and / or one or more legs), including movement of the user's body relative to an absolute reference (e.g., the angle of the user's arm relative to the ground or the distance of the user's hand relative to the ground), movement of the user's body relative to another part of the user's body (e.g., movement of the user's hand relative to the user's shoulder, movement of one of the user's hands relative to another of the user's hands, and / or movement of a user's finger relative to another finger or part of the user's hand), and / or absolute movement of the user's body part (e.g., a tap gesture involving movement of a hand in a predetermined posture by a predetermined amount and / or speed, or a shake gesture involving a predetermined speed or amount of rotation of the user's body part).

[0083] In some embodiments, input gestures used in various examples and embodiments described herein include air gestures performed by the movement of a user's finger(s) relative to other finger(s) or part(s) of the user's hand to interact with a CGR or XR environment (e.g., a virtual or mixed reality environment), according to some embodiments. In some embodiments, an air gesture is a gesture that is detected without the user touching an input element that is part of the device (or independent of an input element that is part of the device) and is based on detected movement of a part of the user's body, including movement of the user's body relative to an absolute reference (e.g., the angle of the user's arm relative to the ground, or the distance of the user's hand relative to the ground), movement of the user's body relative to another part of the user's body (e.g., movement of the user's hand relative to the user's shoulder, movement of the user's other hand relative to one of the user's hands, and / or movement of the user's fingers relative to another finger or part of the user's hand), and / or absolute movement of a part of the user's body (e.g., a tap gesture that includes movement of the hand in a predetermined pose by a predetermined amount and / or speed, or a shake gesture that includes rotation of a part of the user's body at a predetermined speed or amount).

[0084] In some embodiments where the input gesture is an air gesture (e.g., in the absence of physical contact with an input device that provides a computer system with information about which user interface element is the target of the user input, such as contact with a user interface element displayed on a touchscreen or contact with a mouse or trackpad to move a cursor to a user interface element), the gesture takes into account the user's attention (e.g., gaze) to determine the target of the user input (e.g., in the case of direct input, as described below). Thus, in implementations that include air gestures, the input gesture is detected attention (e.g., gaze) to a user interface element in combination with (e.g., simultaneous with) movement of the user's finger(s) and / or hand to perform pinch and / or tap input, as described in more detail below.

[0085] In some embodiments, an input gesture directed at a user interface object is performed directly or indirectly with reference to the user interface object. For example, user input is performed directly at a user interface object in response to performing an input gesture with the user's hand at a position corresponding to the user interface object's position in the three-dimensional environment (e.g., as determined based on the user's current viewpoint). In some embodiments, an input gesture is performed indirectly at a user interface object in response to detecting the user's attention (e.g., gaze) to the user interface object while performing the input gesture while the user's hand position is not at a position corresponding to the user interface object's position in the three-dimensional environment. For example, for a direct input gesture, a user can direct the user's input at a user interface object by initiating the gesture at or near a position corresponding to the user interface object's displayed position (e.g., within a distance of 0.5 cm, 1 cm, 5 cm, or 0-5 cm, measured from an outer edge of the option or a central portion of the option). For indirect input gestures, a user can direct their input to a user interface object by paying attention to the user interface object (e.g., by gazing at the user interface object), and while paying attention to the option, the user initiates an input gesture (e.g., at any position detectable by the computer system) (e.g., at a position that does not correspond to the displayed position of the user interface object).

[0086] In some embodiments, input gestures (e.g., air gestures) used in various examples and embodiments described herein include pinch inputs and tap inputs for interacting with a virtual or mixed reality environment, according to some embodiments. For example, pinch inputs and tap inputs, as described below, are performed as air gestures.

[0087] In some embodiments, the pinch input is part of an air gesture, including one or more of a pinch gesture, a long pinch gesture, a pinch-and-drag gesture, or a double pinch gesture. For example, a pinch gesture that is an air gesture includes moving two or more fingers of a hand to contact each other, i.e., optionally with a short break (e.g., within 0-1 second) after contact with each other. A long pinch gesture that is an air gesture includes moving two or more fingers of a hand to contact each other for at least a threshold amount of time (e.g., at least 1 second) before detecting a break in contact with each other. For example, a long pinch gesture includes a user holding a pinch gesture (e.g., when two or more fingers are in contact), and the long pinch gesture continues until a break in contact between the two or more fingers is detected. In some embodiments, a double pinch gesture that is an air gesture includes two (e.g., or more) pinch inputs (e.g., performed by the same hand) that are detected immediately in succession (e.g., within a predetermined period of time) after each other. For example, a user performs a first pinch input (e.g., a pinch input or a long pinch input), releases the first pinch input (e.g., breaking contact between two or more fingers), and performs a second pinch input within a predetermined period of time (e.g., within 1 second or 2 seconds) after releasing the first pinch input.

[0088] In some embodiments, a pinch-and-drag gesture that is an air gesture includes a pinch gesture (e.g., a pinch gesture or a long pinch gesture) performed in conjunction with (e.g., followed by) a drag input that changes the position of a user's hand from a first position (e.g., a start position of the drag) to a second position (e.g., an end position of the drag). In some embodiments, a user maintains the pinch gesture while performing the drag input and releases the pinch gesture (e.g., spreading two or more fingers apart) to end the drag gesture (e.g., at the second position). In some embodiments, the pinch input and the drag input are performed by the same hand (e.g., a user pinches two or more fingers together and moves the same hand to a second position in the air with a drag gesture). In some embodiments, the pinch input is performed by a user's first hand and the drag input is performed by the user's second hand (e.g., the user's second hand moves from a first position to a second position in the air while the user continues the pinch input with the user's first hand). In some embodiments, an input gesture that is an air gesture includes an input (e.g., a pinch input and / or a tap input) performed using both of a user's hands. For example, the input gesture includes two (e.g., or more) pinch inputs performed in conjunction with each other (e.g., simultaneously or within a predetermined period of time). For example, a first pinch gesture (e.g., a pinch input, a long pinch input, or a pinch and drag input) performed using a first hand of the user and a second pinch input performed using the other hand (e.g., a second of the user's hands) in conjunction with performing the pinch input using the first hand. In some embodiments, a movement between a user's hands (e.g., to increase and / or decrease the distance or relative orientation between the user's hands).

[0089] In some embodiments, a tap input (e.g., directed toward a user interface element) performed as an air gesture includes movement(s) of a user's finger(s) toward the user interface element, movement of a user's hand toward a user interface element, optionally with the user's finger(s) extended toward the user interface element, a downward movement of a user's finger (e.g., mimicking a mouse click action or a tap on a touchscreen), or other predefined movement of the user's hand. In some embodiments, a tap input performed as an air gesture is detected based on movement characteristics of the finger or hand performing the tap gesture, moving the finger or hand away from the user's viewpoint and / or toward the object that is the target of the tap input followed by an end of the movement. In some embodiments, an end of the movement is detected based on a change in movement characteristics of the finger or hand performing the tap gesture (e.g., an end of movement away from the user's viewpoint and / or toward the object that is the target of the tap input, a reversal of the direction of movement of the finger or hand, and / or a reversal of the direction of acceleration of the movement of the finger or hand).

[0090] In some embodiments, the user's attention is determined to be directed to a portion of the three-dimensional environment based on detecting a gaze directed to the portion of the three-dimensional environment (optionally, without requiring other conditions). In some embodiments, the device determines that the user's attention is directed to the portion of the three-dimensional environment based on detecting a gaze directed to the portion of the three-dimensional environment with one or more additional conditions, such as requiring the gaze to be directed to the portion of the three-dimensional environment for at least a threshold duration (e.g., dwell time) while the user's viewpoint is within a distance threshold from the portion of the three-dimensional environment, and / or requiring the gaze to be directed to the portion of the three-dimensional environment, and if one of the additional conditions is not met, the device determines that the user's attention is not directed to the portion of the three-dimensional environment to which the gaze is directed (e.g., until one or more additional conditions are met).

[0091] In some embodiments, detection of a ready configuration of a user or a portion of a user is detected by a computer system, and detection of a ready configuration of the hands is used by the computer system as an indication that the user is likely preparing to interact with the computer system using one or more air gesture inputs performed with the hands (e.g., pinch, tap, pinch and drag, double pinch, long pinch, or other air gestures described herein). For example, the ready state of a hand is determined based on whether the hand has a predetermined hand geometry (e.g., a pre-pinch geometry with the thumb and one or more fingers extended and spaced apart, ready to perform a pinch or grab gesture, or a pre-tap geometry with one or more fingers extended and the palm facing away from the user), whether the hand is in a predetermined position relative to the user's viewpoint (e.g., below the user's head, above the user's waist, extended at least 15 cm, 20 cm, 25 cm, 30 cm, or 50 cm from the body), and / or whether the hand has moved in a particular manner (e.g., above the user's waist, moved toward an area in front of the user below the user's head, or away from the user's body or legs). In some embodiments, the ready state is used to determine whether an interactive element of a user interface is responsive to attentional (e.g., gaze) input.

[0092] In some embodiments, the software may be downloaded to the controller 110 in electronic form, for example, over a network, or alternatively may be provided on a tangible, non-transitory medium, such as an optical, magnetic, or electronic memory medium. In some embodiments, the database 408 is similarly stored in memory associated with the controller 110. Alternatively, or additionally, some or all of the described functionality of the computer may be implemented in dedicated hardware, such as a custom or semi-custom integrated circuit or a programmable digital signal processor (DSP). While the controller 110 is shown in FIG. 4 as, by way of example, a separate unit from the image sensor 440, some or all of the processing functionality of the controller may be implemented by a suitable microprocessor and software, or by dedicated circuitry within the housing of the hand tracking device 402, or otherwise associated with the image sensor 404. In some embodiments, at least some of these processing functions may be performed by a suitable processor integrated with the display generation component 120 (e.g., in a television set, handheld device, or head-mounted device) or using any other suitable computerized device, such as a game console or media player. The sensing function of the image sensor 404 may likewise be integrated into a computer or other computerized device that is controlled by the sensor output.

[0093] FIG. 4 also includes a schematic diagram of a depth map 410 captured by the image sensor 404, according to some embodiments. The depth map includes a matrix of pixels having respective depth values, as described above. A pixel 412 corresponding to the hand 406 is segmented from the background and wrist in this map. The intensity of each pixel in the depth map 410 is inversely proportional to the depth value, i.e., the measured z-distance from the image sensor 404, with increasing gray levels as depth increases. The controller 110 processes these depth values ​​to identify and segment components of the image (i.e., groups of adjacent pixels) that have characteristics of a human hand. These characteristics can include, for example, the overall size, shape, and frame-to-frame motion of the depth map sequence.

[0094] 4 also schematically illustrates a hand skeleton 414 that the controller 110 ultimately extracts from the depth map 410 of the hand 406, according to some embodiments. In FIG. 4, the skeleton 414 is superimposed on a hand background 416 that was segmented from the original depth map. In some embodiments, key feature points on the hand (e.g., knuckles, fingertips, center of the palm, end of the hand where it connects to the wrist, etc.), and optionally the wrist or arm connected to the hand, are identified and positioned on the hand skeleton 414. In some embodiments, the location and movement of these key feature points over multiple image frames are used by the controller 110 to determine hand gestures performed by the hand or the current state of the hand, according to some embodiments.

[0095] FIG. 5 illustrates an exemplary embodiment of eye tracking device 130 (FIG. 1). In some embodiments, eye tracking device 130 is controlled by eye tracking unit 245 (FIG. 2) to track the position and movement of a user's gaze relative to scene 105 or relative to CGR content displayed via display generation component 120. In some embodiments, eye tracking device 130 is integrated with display generation component 120. For example, in some embodiments, if display generation component 120 is a head-mounted device, such as a headset, helmet, goggles, or glasses, or a handheld device disposed in a wearable frame, the head-mounted device includes both components for generating CGR content for viewing by the user and components for tracking the user's gaze relative to the CGR content. In some embodiments, eye tracking device 130 is separate from display generation component 120. For example, if the display generation component is a handheld device or a CGR chamber, eye tracking device 130 is optionally a device separate from the handheld device or the CGR chamber. In some embodiments, eye tracking device 130 is a head-mounted device or part of a head-mounted device. In some embodiments, head-mounted eye tracking device 130 is optionally used in conjunction with head-mounted or non-head-mounted display generation components. In some embodiments, eye tracking device 130 is not a head-mounted device, and is optionally used in conjunction with head-mounted display generation components. In some embodiments, eye tracking device 130 is not a head-mounted device, and is optionally part of non-head-mounted display generation components.

[0096] In some embodiments, the display generation component 120 uses a display mechanism (e.g., left and right near-eye display panels) that displays frames including left and right images in front of the user's eyes to provide the user with a 3D virtual view. For example, a head-mounted display generation component may include left and right optical lenses (referred to herein as eyepieces) positioned between the display and the user's eyes. In some embodiments, the display generation component may include or be coupled to one or more external video cameras that capture video of the user's environment for display. In some embodiments, the head-mounted display generation component may have a transparent or translucent display that allows the user to view the physical environment directly and display virtual objects on the transparent or translucent display. In some embodiments, the display generation component projects virtual objects into the physical environment. The virtual objects are projected, for example, onto a physical surface or as a hologram, allowing an individual using the system to observe the virtual objects superimposed on the physical environment. In such cases, separate display panels and image frames for the left and right eyes may not be required.

[0097] As shown in FIG. 5 , in some embodiments, the gaze tracking device 130 includes at least one eye tracking camera (e.g., an infrared (IR) or near-IR (NIR) camera) and an illumination source (e.g., an IR or NIR light source such as an array or ring of LEDs) that emits light (e.g., IR or NIR light) toward the user's eyes. The eye tracking camera may be aimed at the user's eyes to receive reflected IR or NIR light from the light source directly from the eyes, or alternatively, may be aimed at a “hot” mirror positioned between the user's eyes and a display panel that reflects the IR or NIR light from the eyes to the eye tracking camera while allowing visual light to pass through. The gaze tracking device 130 optionally captures images of the user's eyes (e.g., as a video stream captured at 60-120 frames per second (fps)), analyzes the images, generates gaze tracking information, and communicates the gaze tracking information to the controller 110. In some embodiments, the user's eyes are tracked separately by their respective eye tracking cameras and illumination sources. In some embodiments, only one eye of the user is tracked by a separate eye-tracking camera and lighting source.

[0098] In some embodiments, the eye tracking device 130 is calibrated using a device-specific calibration process to determine the eye tracking device's parameters for the particular operating environment 100, such as the 3D geometric relationships and parameters of the LEDs, camera, hot mirror (if present), eyepiece, and display screen. The device-specific calibration process may be performed at a factory or another facility before delivery of the AR / VR equipment to the end user. The device-specific calibration process may be an automatic or manual calibration process. The user-specific calibration process may include estimation of a particular user's eye parameters, such as pupil location, central visual location, optical axis, visual axis, eye spacing, etc. According to some embodiments, once the device-specific and user-specific parameters for the eye tracking device 130 have been determined, images captured by the eye tracking camera can be processed using glint-assisted methods to determine the user's current visual axis and viewpoint relative to the display.

[0099] As shown in FIG. 5, eye tracking device 130 (e.g., 130A or 130B) includes an eyepiece(s) 520 and a gaze tracking system including at least one eye tracking camera 540 (e.g., an infrared (IR) or near-IR (NIR) camera) positioned on the side of the user's face where eye tracking occurs and an illumination source 530 (e.g., an IR or NIR light source such as an array or ring of NIR light emitting diodes (LEDs)) that emits light (e.g., IR or NIR light) toward the user's eye(s) 592. The eye tracking camera 540 may be positioned between the user's eye(s) 592 and the display 510 (e.g., the left or right display panel of a head-mounted display, or the display of a handheld device, a projector, etc.) and may be directed at a mirror 550 that reflects IR or NIR light from the eye(s) 592 while transmitting visible light (e.g., as shown at the top of FIG. 5), or may be directed at the user's eye(s) 592 to receive reflected IR or NIR light from the eye(s) 592 (e.g., as shown at the bottom of FIG. 5).

[0100] In some embodiments, controller 110 renders AR or VR frames 562 (e.g., left and right frames for left and right display panels) and provides frames 562 to display 510. Controller 110 uses gaze tracking input 542 from eye tracking camera 540 for various purposes, such as in processing frames 562 for display. Controller 110 optionally estimates the user's viewpoint on display 510 based on gaze tracking input 542 obtained from eye tracking camera 540, using a glint-assisted method or other suitable method. The viewpoint estimated from gaze tracking input 542 is optionally used to determine the direction the user is currently looking.

[0101] Some possible use cases of the user's current gaze direction are described below, but are not intended to be limiting. As an exemplary use case, the controller 110 can render virtual content differently based on the determined user's gaze direction. For example, the controller 110 may generate virtual content with higher resolution in a central visual area determined from the user's current gaze direction than in a peripheral area. As another example, the controller may position or move virtual content within a view based at least in part on the user's current gaze direction. As another example, the controller may display particular virtual content within a view based at least in part on the user's current gaze direction. As another exemplary use case in an AR application, the controller 110 can orient an external camera to capture the physical environment of the CGR experience and focus in the determined direction. The external camera's autofocus mechanism can then focus on an object or surface within the environment the user is currently viewing on the display 510. As another exemplary use case, eyepiece 520 may be a focusable lens, and eye-tracking information is used by the controller to adjust the focus of eyepiece 520 so that the virtual object the user is currently looking at has the proper binocular coordination to match the convergence of the user's eyes 592. Controller 110 can utilize the eye-tracking information to orient and focus eyepiece 520 so that close objects the user is looking at appear at the correct distance.

[0102] In some embodiments, the eye tracking device is part of a head-mounted device that includes a display (e.g., display 510), two eyepieces (e.g., eyepiece(s) 520), an eye tracking camera (e.g., eye tracking camera(s) 540), and a light source (e.g., light source 530 (e.g., IR or NIR LED)) attached to the wearable housing. The light source emits light (e.g., IR or NIR light) toward the user's eye(s) 592. In some embodiments, the light sources may be arranged in a ring or circle around each lens, as shown in FIG. 5. In some embodiments, eight light sources 530 (e.g., LEDs) are arranged around each lens 520, as an example. However, more or fewer light sources 530 may be used, and other arrangements and locations of the light sources 530 may be employed.

[0103] In some embodiments, the display 510 emits light in the visible light range and not in the IR or NIR range, and therefore does not introduce noise into the gaze tracking system. Note that the location and angle of the eye tracking camera(s) 540 are given by way of example and are not intended to be limiting. In some embodiments, a single eye tracking camera 540 is located on each side of the user's face. In some embodiments, two or more NIR cameras 540 may be used on each side of the user's face. In some embodiments, a camera 540 with a wider field of view (FOV) and a camera 540 with a narrower FOV may be used on each side of the user's face. In some embodiments, a camera 540 operating at one wavelength (e.g., 850 nm) and a camera 540 operating at a different wavelength (e.g., 940 nm) may be used on each side of the user's face.

[0104] Embodiments of an eye tracking system such as that shown in FIG. 5 may be used, for example, in computer-generated reality, virtual reality, and / or mixed reality applications to provide a user with a computer-generated reality, virtual reality, augmented reality, and / or augmented virtual experience.

[0105] FIG. 6A illustrates a glint-assisted gaze tracking pipeline according to some embodiments. In some embodiments, the gaze tracking pipeline is implemented by a glint-assisted gaze tracking system (e.g., eye tracking device 130 as shown in FIGS. 1 and 5). The glint-assisted gaze tracking system can maintain a tracking state. Initially, the tracking state is off or "no." When in the tracking state, the glint-assisted gaze tracking system tracks the pupil contour and glint in the current frame using prior information from the previous frame when analyzing the current frame. When not in the tracking state, the glint-assisted gaze tracking system attempts to detect the pupil and glint in the current frame, and if successful, initializes the tracking state to "yes" and continues to the next frame in the tracking state.

[0106] As shown in FIG. 6A, an eye-tracking camera may capture left and right images of a user's left and right eyes. The captured images are then input into an eye-tracking pipeline for processing beginning at 610. As indicated by the arrow returning to element 600, the eye-tracking system may continue to capture images of the user's eyes at a rate of, for example, 60-120 frames per second. In some embodiments, each set of captured images may be input into the pipeline for processing. However, in some embodiments, or under some conditions, not all captured frames are processed by the pipeline.

[0107] At 610, if the tracking status is yes for the currently captured image, the method proceeds to element 640. If the tracking status is no at 610, the image is analyzed to detect the user's pupil and glint in the image, as shown at 620. If the pupil and glint are successfully detected at 630, the method proceeds to element 640. If not, the method returns to element 610 to process the next image of the user's eyes.

[0108] At 640, proceeding from element 410, the current frame is analyzed to track pupils and glints based in part on previous information from the previous frame. At 640, proceeding from element 630, a tracking state is initialized based on the detected pupils and glints in the current frame. The results of the processing at element 640 are checked to ensure that the tracking or detection results are reliable. For example, the results can be checked to determine whether a sufficient number of glints are successfully tracked or detected in the current frame to perform pupil and gaze estimation. At 650, if the results are not reliable, the tracking state is set to no and the method returns to element 610 to process the next image of the user's eyes. At 650, if the results are reliable, the method proceeds to element 670. At 670, the tracking state is set to yes (if not already yes) and the pupil and glint information is passed to element 680 to estimate the user's gaze point.

[0109] 6A is intended to serve as an example of eye-tracking technology that may be used in particular implementations. As will be recognized by those skilled in the art, other eye-tracking technologies, now existing or developed in the future, may be used in place of or in combination with the glint-assisted eye-tracking technology described herein in computer system 101 to provide a user with a CGR experience according to various embodiments.

[0110] FIG. 6B illustrates an exemplary environment for electronic device 101 for providing a CGR experience, according to some embodiments. In FIG. 6B, real-world environment 602 includes electronic device 101, user 608, and real-world objects (e.g., table 604). As shown in FIG. 6B, electronic device 101 is optionally tripod-mounted or otherwise secured to real-world environment 602 so that one or more hands of user 608 are free (e.g., user 608 is optionally not holding device 101 with one or more hands). As described above, device 101 optionally has one or more groups of sensors located on different sides of device 101. For example, device 101 optionally includes sensor group 612-1 and sensor group 612-2 located on the “rear” and “front” sides of device 101, respectively (e.g., capable of capturing information from each side of device 101). As used herein, the front side of the device 101 is the side that faces the user 608 and the back side of the device 101 is the side that faces away from the user 608 .

[0111] In some embodiments, sensor group 612-2 includes an eye tracking unit (e.g., eye tracking unit 245 described above with reference to FIG. 2) that includes one or more sensors for tracking the eyes and / or gaze of a user, and the eye tracking unit can "watch" user 608 and track the eye(s) of user 608 in the manner described above. In some embodiments, the eye tracking unit of device 101 can capture the movement, orientation, and / or gaze of the eyes of user 608 and process the movement, orientation, and / or gaze as input.

[0112] In some embodiments, sensor group 612-1 includes a hand tracking unit (e.g., hand tracking unit 243 described above with reference to FIG. 2) that can track one or more hands of user 608 held on the “back” side of device 101, as shown in FIG. 6B. In some embodiments, a hand tracking unit is optionally included in sensor group 612-2 so that user 608 can additionally or alternatively hold one or more hands on the “front” side of device 101 while device 101 tracks the position of the one or more hands. As described above, the hand tracking unit of device 101 can capture the movements, positions, and / or gestures of one or more hands of user 608 and process the movements, positions, and / or gestures as input.

[0113] In some embodiments, sensor group 612-1 optionally includes one or more sensors (e.g., image sensor 404 described above with reference to FIG. 4 ) configured to capture images of real-world environment 602, including table 604. As described above, device 101 can capture images of portions (e.g., part or all) of real-world environment 602 and present the captured portions of real-world environment 602 to the user via one or more display generation components of device 101 (e.g., a display of device 101 optionally located on a side of device 101 facing the user, opposite the side of device 101 facing the captured portions of real-world environment 602).

[0114] In some embodiments, the captured portion of the real-world environment 602 is used to provide the user with a CGR experience, e.g., a mixed reality environment in which one or more virtual objects are overlaid on a representation of the real-world environment 602.

[0115] Accordingly, the description herein describes several embodiments of three-dimensional environments (e.g., CGR environments) that include representations of real-world objects and representations of virtual objects. For example, the three-dimensional environment optionally includes a representation of a table present in a physical environment that is captured and displayed within the three-dimensional environment (e.g., actively via a camera and display of the electronic device, or passively via a transparent or translucent display of the electronic device). As mentioned above, the three-dimensional environment is optionally a mixed reality system based on a physical environment, where the three-dimensional environment is captured by one or more sensors of the device and displayed via a display generation component. As a mixed reality system, the device can optionally display portions and / or objects of the physical environment such that each of the portions and / or objects of the physical environment appears to exist within the three-dimensional environment displayed by the electronic device. Similarly, the device can optionally display virtual objects in the three-dimensional environment such that the virtual objects appear to exist within the real world (e.g., the physical environment) by placing the virtual objects at respective locations in the three-dimensional environment that have corresponding locations in the real world. For example, the device optionally displays a vase in a manner that makes it appear as if the real vase were placed on a table in the physical environment. In some embodiments, each location in the three-dimensional environment has a corresponding location in the physical environment. Thus, when a device is described as displaying a virtual object at a location distinct from a physical object (e.g., at or near the location of a user's hand, or on or near a physical table, etc.), the device displays the virtual object at a particular location in the three-dimensional environment in a manner that makes it appear as if the virtual object were at or near the physical object in the physical world (e.g., the virtual object is displayed at a location in the three-dimensional environment that corresponds to the location in the physical environment where the virtual object would be displayed if the virtual object were a real object at that particular location).

[0116] In some embodiments, real-world objects present in the physical environment that are displayed in the three-dimensional environment can interact with virtual objects that exist only in the three-dimensional environment. For example, the three-dimensional environment can include a table and a vase placed on the table, where the table is a view (or representation) of the physical table in the physical environment and the vase is a virtual object.

[0117] Similarly, a user can optionally use one or more hands to interact with virtual objects in the three-dimensional environment as if the virtual objects were real objects in the physical environment. For example, as described above, one or more sensors of the device optionally capture one or more of the user's hands and display a representation of the user's hands in the three-dimensional environment (e.g., in a manner similar to displaying real-world objects in the three-dimensional environment described above), or in some embodiments, due to the transparency / translucency of the user interface, or the projection of the user interface onto a transparent / translucent surface, or the portion of the display generation component displaying the projection of the user interface to the user's eyes or field of view of the user's eyes, the user's hands are visible through the display generation component by the ability to see the physical environment through the user interface. Thus, in some embodiments, the user's hands are displayed at discrete locations in the three-dimensional environment and are treated as if they were objects in the three-dimensional environment that can interact with virtual objects in the three-dimensional environment as if they were actual physical objects in the physical environment. In some embodiments, a user can move their hands to cause representations of their hands in the three-dimensional environment to move in coordination with the movement of the user's hands.

[0118] In some of the embodiments described below, the device can optionally determine an “effective” distance between a physical object in the physical world and a virtual object in the three-dimensional environment, for example, to determine whether a physical object is interacting with a virtual object (e.g., whether a hand is touching, grabbing, holding, etc., a virtual object, or whether it is within a threshold distance from the virtual object). For example, when determining whether and / or how a user is interacting with a virtual object, the device determines the distance between the user's hand and the virtual object. In some embodiments, the device determines the distance between the user's hand and the virtual object by determining the distance between the location of the hand in the three-dimensional environment and the location of a target virtual object in the three-dimensional environment. For example, one or more of the user's hands are placed at specific positions in the physical world, which the device optionally captures and displays at specific corresponding positions in the three-dimensional environment (e.g., positions in the three-dimensional environment at which the hands are displayed, if the hands are virtual rather than physical hands). The positions of the hands in the three-dimensional environment are optionally compared to the positions of the target virtual objects in the three-dimensional environment to determine the distance between the user's one or more hands and the virtual object. In some embodiments, the device optionally determines the distance between a physical object and a virtual object by comparing positions in the physical world (e.g., as opposed to comparing positions in a three-dimensional environment). For example, when determining the distance between one or more of a user's hands and a virtual object, the device optionally determines the corresponding location in the physical world of the virtual object (e.g., the position at which the virtual object is located in the physical world if the virtual object is a physical object rather than a virtual object), and then determines the distance between the corresponding physical position and the user's one or more hands.In some embodiments, the same techniques are optionally used to determine the distance between any physical object and any virtual object. Thus, as described herein, when determining whether a physical object is in contact with a virtual object or whether a physical object is within a threshold distance of a virtual object, the device optionally performs any of the techniques described above to map the location of the physical object to the three-dimensional environment and / or to map the location of the virtual object to the physical world.

[0119] In some embodiments, the same or similar techniques are used to determine where and what a user's gaze is directed at and / or where and what a physical stylus held by the user is directed at. For example, if a user's gaze is directed at a particular position in the physical environment, the device optionally determines a corresponding position in the three-dimensional environment, and if a virtual object is located at that corresponding virtual position, the device optionally determines that the user's gaze is directed at that virtual object. Similarly, the device can optionally determine where the physical stylus is pointing in the physical world based on the orientation of the physical stylus. In some embodiments, based on this determination, the device determines a corresponding virtual position in the three-dimensional environment that corresponds to the location in the physical world where the stylus is pointing, and optionally determines that the stylus is pointing to the corresponding virtual position in the three-dimensional environment.

[0120] Similarly, embodiments described herein may refer to the location of a user (e.g., a user of a device) and / or the location of a device within a three-dimensional environment. In some embodiments, a user of a device is holding, wearing, or otherwise located at or near the electronic device. Thus, in some embodiments, the location of the device is used as a proxy for the location of the user. In some embodiments, the location of the device and / or user within the physical environment corresponds to a distinct location within the three-dimensional environment. In some embodiments, the distinct location is a location from which a “camera” or “view” of the three-dimensional environment extends. For example, if a user stands at a location facing a distinct portion of the physical environment displayed by the display generation component, the location of the device is the location within the physical environment (and its corresponding location within the three-dimensional environment) at which the user would see objects within the physical environment in the same position, orientation, and / or size (e.g., absolutely and / or relative to each other) as the objects are displayed by the display generation component of the device. Similarly, if the virtual objects displayed in the three-dimensional environment were physical objects in the physical environment (e.g., the virtual objects are located in the same physical environment location and have the same physical environment size and orientation as in the three-dimensional environment), the location of the device and / or user is the position at which the user would see the virtual objects in the physical environment in the same position, orientation, and / or size (e.g., absolutely and / or relative to each other and to real-world objects) as displayed by the display generation component of the device.

[0121] In this disclosure, various input methods are described with respect to interaction with a computer system. Where one example is provided using one input device or input method and another example is provided using a different input device or input method, it should be understood that each example may be compatible with, and optionally utilize, the input device or input method described with respect to the other example. Similarly, various output methods are described with respect to interaction with a computer system. Where one example is provided using one output device or output method and another example is provided using a different output device or output method, it should be understood that each example may be compatible with, and optionally utilize, the output device or output method described with respect to the other example. Similarly, various methods are described with respect to interaction with a virtual environment or a mixed reality environment via a computer system. Where one example is provided using interaction with a virtual environment and another example is provided using a mixed reality environment, it should be understood that each example may be compatible with, and optionally utilize, the method described with respect to the other example. Thus, this disclosure discloses embodiments that are combinations of features of multiple examples, without exhaustively listing all features of the embodiments in the description of each exemplary embodiment.

[0122] Furthermore, for methods described herein in which one or more steps are conditioned on one or more conditions being satisfied, it should be understood that the described method can be repeated in multiple iterations, such that over the course of the iterations, all of the conditions on which the method steps are conditioned are satisfied in different iterations of the method. For example, if a method requires performing a first step if a condition is satisfied and a second step if the condition is not satisfied, one skilled in the art will understand that the steps recited in the claim are repeated in a particular order until the conditions are satisfied and then no longer satisfied. Thus, a method described with one or more steps that depend on one or more conditions being satisfied can be rewritten as a method that is repeated until each condition recited in the method is satisfied. However, this is not required for system or computer-readable medium claims in which the system or computer-readable medium includes instructions for performing a conditional action based on the satisfaction of the corresponding one or more conditions, and thus can determine whether a contingency is met without explicitly repeating the method steps until all conditions on which the method steps are conditioned are satisfied. Those skilled in the art will also understand that, as with methods having conditional steps, the system or computer-readable storage medium may repeat the steps of the method as many times as necessary to ensure that all of the conditional steps have been performed. User Interface and Related Processes

[0123] We now turn our attention to embodiments of user interfaces (“UIs”) and associated processes that may be executed in a computer system, such as a portable multifunction device or a head-mounted device, equipped with a display generation component, one or more input devices, and (optionally) one or more cameras.

[0124] 7A-7H show examples of displaying a virtual environment, according to some embodiments.

[0125] FIG. 7A illustrates electronic device 101 displaying a three-dimensional environment 704 on a user interface via a display generation component (e.g., display generation component 120 of FIG. 1 ). As described above with reference to FIGS. 1-6 , electronic device 101 optionally includes a display generation component (e.g., a touchscreen) and multiple image sensors (e.g., image sensor 314 of FIG. 3 ). The image sensors optionally include one or more of a visible light camera, an infrared camera, a depth sensor, or any other sensor that electronic device 101 can use to capture one or more images of a user or a portion of a user while the user interacts with electronic device 101. In some embodiments, the user interfaces illustrated below may also be implemented on a head-mounted display that includes a display generation component that displays the user interface to a user and sensors for detecting the physical environment and / or movement of the user's hands (e.g., external sensors facing outward from the user) and / or the user's line of sight (e.g., internal sensors facing inward toward the user's face). The figures herein illustrate a three-dimensional environment presented to a user by device 101 (e.g., and displayed by a display generation component of device 101) and an overhead view of the physical and / or three-dimensional environment associated with device 101 (e.g., overhead view 718 of FIG. 7A) to show the relative locations of objects in the real-world environment and the locations of virtual objects in the three-dimensional environment.

[0126] 7A , device 101 captures one or more images of real-world environment 702 (e.g., operating environment 100) around device 101, including one or more objects in real-world environment 702 around device 101. In some embodiments, device 101 displays a representation of the real-world environment in a three-dimensional environment 704. For example, three-dimensional environment 704 includes a back corner of a room, a representation of corner table 708a, a representation of desk 710a, a representation of picture frame 706 on the back wall of the room, a representation of coffee table 714a, and a representation of side table 712a. Thus, three-dimensional environment 704 optionally recreates portions of real-world environment 702 such that three-dimensional environment 704 appears to the user as if the user were physically located in real-world environment 702 (e.g., optionally from the perspective of the user's current location in real-world environment 702, facing in the direction the user is currently facing).

[0127] 7A, corner table 708a, desk 710b, side table 712b, and coffee table 714b are real objects in real-world environment 702 captured by one or more sensors of device 101, and representations of which are included in three-dimensional environment 704 (e.g., realistic representations, simplified representations, cartoons, caricatures, etc.). In some embodiments, real-world environment 702 also includes a picture frame (e.g., represented in three-dimensional environment 704 by representation 706) and a sofa 719.

[0128] 7A , a user 720 of device 101 is sitting on a couch 719 and is holding (e.g., or wearing, e.g., if device 101 is a head-mounted device) device 101 so that it is facing the other end of the room (e.g., so that one or more sensors are facing the other end of the room), thus capturing a corner table 708 a, a desk 710 b, a side table 712 b, and a coffee table 714 b and displaying representations of the objects in three-dimensional environment 704. In some embodiments, device 101 is capable of determining the geometry of at least a portion of real-world environment 702 (e.g., the portion of real-world environment 702 that device 101 is facing, all of the portion of real-world environment 702, etc.). In some embodiments, device 101 determines the geometry of real-world environment 702 using one or more sensors, such as a visible light sensor (e.g., a camera), a depth sensor (e.g., a time-of-flight sensor), or a combination of sensors (e.g., multiple sensors of the same type, sensors of different types, etc.). 7A, device 101 determines that there is a threshold amount of physical space in front of device 101 because user 720 is sitting on couch 719. For example, the distance from user 720 (e.g., and therefore from device 101) to the back wall (e.g., where desk 710b is located) is greater than the threshold distance (e.g., greater than 3 feet, 6 feet, 10 feet, 30 feet, etc.). Thus, because three-dimensional environment 704 is at least a partial reproduction of real-world environment 702 as described above, three-dimensional environment 704 optionally reflects the available space between user 720 and the back wall of real-world environment 702 (e.g., the distance from the viewpoint of three-dimensional environment 704 (e.g., the "camera" position from where the user views three-dimensional environment 704) to the back wall of three-dimensional environment 704 is the same or similar to the distance from user 720 to the back wall of real-world environment 702), and three-dimensional environment 704 has at least a threshold amount of space (e.g., because real-world environment 702 has at least a threshold amount of space in front of user 720).In some embodiments, as described in further detail below with respect to Figures 7C-7H, the three-dimensional environment 704 is eligible for far-field virtual environment transition animation because there is at least a threshold amount of depth within the three-dimensional environment 704 in front of the user's perspective view.

[0129] 7A , device 101 displays an immersion level indicator 716. In some embodiments, immersion level indicator 716 indicates the current immersion level (e.g., out of a maximum number of immersion levels) at which device 101 is displayed in three-dimensional environment 704. In some embodiments, the immersion level includes the amount by which a view of the physical environment (e.g., a view of objects in real-world environment 702) is obscured by the virtual environment (e.g., an emulated environment, optionally different from the user's surrounding real-world environment 702), or the amount by which objects in the physical environment are modified to achieve a particular spatial effect (e.g., as described in more detail below with respect to method 1000). For example, a maximum immersion level (e.g., full immersion) optionally refers to a state in which none of the physical environment is viewable in three-dimensional environment 704 via display generation component 120, and the entire three-dimensional environment 704 is encompassed by the virtual environment. In some embodiments, an intermediate immersion level (e.g., an immersion level below maximum and above no immersion) refers to a state in which portions of the real-world environment 702 are visible within the three-dimensional environment 704 via the display generation component 120, and portions of the real-world environment 702 that would otherwise be visible (e.g., not due to immersion) are replaced by the virtual environment. In some embodiments, the immersion level indicator 716 optionally includes multiple elements associated with multiple immersion levels. In some embodiments, the multiple immersion levels are categorized into minor and major immersion levels, and the immersion level indicator 716 optionally includes minor tick marks (e.g., indicated by a square) and major tick marks (e.g., indicated by a square displayed with a triangle). In some embodiments, as the immersion level increases, the device 101 presents more elements of the virtual environment.In some embodiments, a minor tick refers to an immersion state in which a minor element is introduced or the magnitude of a previously introduced element is increased (e.g., increasing the volume, increasing the size of the virtual environment, etc.), and a major tick refers to an immersion state in which a major element is introduced (e.g., a new visual element, a new audio element, etc.), optionally in addition to increasing the magnitude of a previously introduced element.

[0130] 7A , immersion level indicator 716 indicates that the current immersion level is no immersion (indicated by the fact that none of the squares and / or triangles are filled in), and therefore, no virtual environment is displayed within three-dimensional environment 704. In some embodiments, immersion level indicator 716 is displayed when the current immersion level exceeds the no immersion level, and / or for a threshold time (e.g., 5 seconds, 10 seconds, 30 seconds, 1 minute, etc.) after the immersion level changes, and / or for a threshold time (e.g., 5 seconds, 10 seconds, 30 seconds, 1 minute, etc.) after the immersion level drops to zero.

[0131] 7B illustrates an embodiment similar to FIG. 7A , except that a user 720 is seated at a desk 710b, as shown in an overhead view 718 of the real-world environment 702. In some embodiments, the device 101 is positioned at the desk 710b and captures a view of the real-world environment 702 from the desk 710b to the back wall, such that the three-dimensional environment 704 is a view of the limited space from the user 720 to the back wall, including a representation of the back wall near the user, a representation 706 of a picture frame on the back wall, and a portion of a representation 710a of the desk 710b. In FIG. 7B , the device 101 determines that the user 720 is seated near the back wall of the room and therefore does not have a threshold amount of physical space in front of the device 101 in the room. In some embodiments, the device 101 makes this determination based on one or more of the following: the distance from the device 101 to the back wall of the room, the area between the device 101 and the back wall of the room, or the volume between the device 101 and the back wall of the room. In some embodiments, this determination by the device 101 is additionally or alternatively based on an average of the distance / area / volume, or the distance / area / volume of the closest point, or the distance / area / volume of the farthest point within a predefined region relative to the user's viewpoint within the three-dimensional environment 704.

[0132] For example, the distance from user 720 to the back wall (e.g., and therefore device 101) is less than the threshold distance described above. Thus, because three-dimensional environment 704 is at least a partial reproduction of real-world environment 702 as described above, three-dimensional environment 704 optionally reflects the available space between user 720 and the back wall of real-world environment 702 (e.g., the distance from the viewpoint of three-dimensional environment 704 to the back wall of three-dimensional environment 704 is the same or similar to the distance from user 720 to the back wall of real-world environment 702), and three-dimensional environment 704 does not have at least a threshold amount of space (e.g., because real-world environment 702 does not have at least a threshold amount of space in front of user 720). In some embodiments, because there is not at least a threshold amount of depth within three-dimensional environment 704 in front of the user's perspective, three-dimensional environment 704 is not eligible for far-field virtual environment transition animation and will instead perform near-field virtual environment transition animation, as described in further detail below with respect to FIGS. 7C-7H .

[0133] In some embodiments, implementing far-field transition animations when the real-world environment 702 has at least a threshold amount of space, but implementing near-field transition animations when the real-world environment 702 does not have at least a threshold amount of space, provides an optimized experience that reduces the risk of sickness or dizziness. For example, if a user is sitting in a location with a lot of space in front of the user, converting the space into a virtual environment with a potentially infinite horizon poses a low risk of visual dissonance and sickness, but if a user is sitting in a location with little space in front of the user, converting the space into a virtual environment with a potentially infinite horizon poses a high risk of visual dissonance and sickness. Thus, implementing far-field animations when the risk of sickness is low provides a comfortable experience for the user, while implementing near-field animations when the risk of sickness is high allows the user to view the virtual environment but introduces the environment to the user slowly to reduce the likelihood of sickness.

[0134] 7C and 7D show snapshots of far-field and near-field transition animations, respectively, for displaying a virtual environment within three-dimensional environment 704. In some embodiments, the virtual environment is a simulated three-dimensional environment that can be displayed in three-dimensional environment 704, optionally instead of (e.g., full immersion) or optionally simultaneously with (e.g., partial immersion) a representation of the physical environment. Some examples of virtual environments include a lake environment, a mountain environment, a sunset scene, a sunrise scene, a night environment, a grassland environment, a concert scene, etc. In some embodiments, the virtual environment is based on a real physical location, such as a museum, an aquarium, etc. In some embodiments, the virtual environment is an artist-designed location. Thus, displaying a virtual environment within three-dimensional environment 704 provides a user with a virtual experience as if the user were physically located within the virtual environment.

[0135] In some embodiments, a virtual environment is displayed in three-dimensional environment 704 in response to user input selecting an affordance associated with the individual virtual environment. For example, three-dimensional environment 704 includes one or more affordances for one or more virtual environments, from which a user can select an individual environment for display in three-dimensional environment 704. In some embodiments, in response to selecting an affordance for the individual environment, the individual environment is displayed by device 101 at a predetermined level of immersion, such as a partial immersion level or a full immersion level, as described in further detail below with respect to method 1000. In some embodiments, a virtual environment is displayed within three-dimensional environment 704 in response to user input that increases the immersion level of three-dimensional environment 704. For example, device 101 optionally includes a mechanical dial that can be rotated in a particular direction to increase the immersion level and in another direction to decrease the immersion level. Embodiments herein describe processes for displaying a virtual environment from no virtual environment displayed to an intermediate or fully immersive level, and / or for changing the immersion level of a virtual environment from a non-immersive or partially immersive level (e.g., a level less than fully immersive) to a higher partial or fully immersive level.

[0136] 7C-7D show first snapshots of a process for displaying virtual environment 722 at a high partial immersion level (e.g., with a large amount of three-dimensional environment 704 replaced with virtual environment 722, as in FIGS. 7G-7H ), it is understood that FIGS. 7C-7D optionally show snapshots of a process for displaying virtual environment 722 at a low partial immersion level (e.g., with a small amount of three-dimensional environment 704 replaced with virtual environment 722) and / or a full immersion level (e.g., with all of three-dimensional environment 704 replaced with virtual environment 722). In some embodiments, as described above, the process for displaying virtual environment 722 is triggered (e.g., initiated) in response to a user input that causes virtual environment 722 to be displayed (e.g., virtual environment 722 was not displayed when the user input was received) or in response to a user input that changes the immersion level of virtual environment 722 (e.g., virtual environment 722 was displayed when the user input was received).

[0137] 7C shows a first snapshot of the far-field process for displaying a virtual environment 722 when the immersion level is 1 (e.g., just above no immersion), as indicated by the immersion level indicator 716 with the leftmost square filled and the other squares (and triangles) unfilled. As described above, in FIG. 7C , the process for displaying the virtual environment 722 includes a far-field transition, which involves replacing portions of the physical environment and / or three-dimensional environment 704 with portions of a virtual environment based on the distance of those portions of the physical environment and / or three-dimensional environment 704 from the user's viewpoint. For example, the farthest position of the three-dimensional environment 704 (e.g., the farthest position of the real-world environment 702 that is visible within the three-dimensional environment 704) is replaced with the virtual environment, extending from its farthest position toward the user's viewpoint (e.g., as shown in FIGS. 7E and 7G ). 7C , the left and right corners of three-dimensional environment 704 correspond to locations farthest from the viewpoint of user 720, and thus the left corner of three-dimensional environment 704 is replaced with virtual environment 722-1a, and the right corner of three-dimensional environment 704 is replaced with virtual environment 722-2a (e.g., as also shown by virtual environment 722-1b and virtual environment 722-2b in overhead view 718). In some embodiments, virtual environment 722-1a and virtual environment 722-2a are part of the same virtual environment, but correspond to the respective left and right sides of virtual environment 722 (e.g., different portions of virtual environment 722). For example, virtual environment 722 is at least partially overlaid on three-dimensional environment 704 (e.g., optionally, virtual environment 722 can cover all of three-dimensional environment 704), but when the immersion level is zero (e.g., no immersion), the entire virtual environment 722 is hidden from view, but as the immersion level increases, portions of virtual environment 722 are revealed (e.g., displacing and / or obscuring the view of respective portions of the physical environment). Thus, in FIG. 7C , the leftmost portion of virtual environment 722 is exposed so that the user can see the leftmost portion where the left rear corner was previously displayed, and the rightmost portion of virtual environment 722 is exposed so that the user can see the rightmost portion of virtual environment 722 where the right rear corner was previously displayed.

[0138] In some embodiments, virtual environment 722 optionally has dimensions greater than those of real-world environment 702 (e.g., potentially of infinite depth if virtual environment 722 is a landscape that extends to the horizon), and thus by replacing portions of three-dimensional environment 704 with virtual environment 722, the replaced portions appear as if they had become visual portals into virtual environment 722. For example, overhead view 718 indicates that the left and right rear corners of real-world environment 702 are no longer present (e.g., no longer displayed) within three-dimensional environment 704 (e.g., as indicated by dotted lines), and instead is a view into virtual environment 722, optionally extending in the distance according to the size, dimensions, and / or depth of virtual environment 722 (e.g., extending outward from the left and right rear corners at a vertical angle from the viewpoint of user 720). Thus, in FIG. 7C , three-dimensional environment 704 appears to the user as an environment that is part real-world environment 702 and part virtual environment 722, as if portions of the user's real-world environment 702 have been transformed into virtual environment 722 so that the user can enter virtual environment 722 through the left and right rear corners of real-world environment 702.

[0139] 7D shows a first snapshot of a near-field process for displaying a virtual environment 722 when the immersion level is 1 (e.g., just above no immersion), as indicated by the immersion level indicator 716 with the leftmost square filled and the other squares (and triangles) unfilled. As mentioned above, in FIG. 7D , the process for displaying the virtual environment 722 includes a near-field transition that includes selecting a center (or other fixed) location that optionally corresponds to a surface in the three-dimensional environment 704 that is farthest from the user (e.g., in the direction of the orientation of the user's viewpoint into the three-dimensional environment 704) and expanding radially outward from the center location (e.g., optionally increasing the radius of a portal into the virtual environment regardless of the location and / or distance of objects in the three-dimensional environment 704 relative to the user 720, as described in more detail below with respect to FIGS. 7F and 7H ). While FIG. 7D shows a virtual environment 722a having a circular shape, it should be understood that other shapes (e.g., elliptical, cylindrical, square, rectangular, etc.) are possible. 7D , the back wall of real-world environment 702 is the farthest surface within three-dimensional environment 704 in the direction of the orientation of the user's viewpoint into three-dimensional environment 704, and therefore, starting at a location in the center of the back wall, device 101 replaces a circular area in the center of the back wall with virtual environment 722 a (e.g., as also illustrated by virtual environment 722 b in overhead view 718). In some embodiments, the z-position (e.g., depth) of virtual environment 722 a (e.g., the boundary between virtual environment 722 a and the real-world environment portion of three-dimensional environment 704) is not necessarily at the back wall, but optionally at a location closer to user 720 than the back wall. For example, in FIG. 7D , overhead view 718 shows virtual environment 722 b starting at a depth location in the center of table 710 b (e.g., where the boundary between real-world environment 702 and virtual environment 722 b is). In FIG. 7D , the height of virtual environment 722 coincides with the center of the surface of the rear wall (e.g., the boundary of virtual environment 722 is directly in front of the rear wall), so that virtual environment 722 may appear to start from the surface of the rear wall.In FIG. 7D , overhead view 718 indicates that the center of the back wall of real-world environment 702 is no longer present (e.g., no longer displayed) within three-dimensional environment 704 (e.g., as indicated by a dotted line), and instead is a view into virtual environment 722, optionally extending in distance according to the size, dimensions, and / or depth of virtual environment 722 (e.g., extending outward from the back wall at a vertical angle from the viewpoint of user 720).

[0140] 7C , virtual environment 722 is optionally superimposed on three-dimensional environment 704 but is hidden (e.g., thus replacing or obscuring a respective portion of three-dimensional environment 704) until the immersion level causes portions of virtual environment 722 to be exposed to the user. Thus, in FIG. 7D , device 101 exposes a portion of virtual environment 722 superimposed on the center of the back wall (e.g., from a depth location at the center of table 710 b outward based on the dimensions of virtual environment 722). Thus, in FIG. 7D , three-dimensional environment 704 appears to the user as an environment that is part real-world environment 702 and part virtual environment 722, as if a portion of the user's real-world environment 702 had been transformed into virtual environment 722. Thus, even if both Figures 7C and 7D show the same level of immersion in the transition to display the same virtual environment (e.g., virtual environment 722), the portion of virtual environment 722 visible in Figure 7D during the first snapshot of the near-field transition is different from the portion of virtual environment 722 visible in Figure 7C during the first snapshot of the far-field transition.

[0141] 7E-7F show a second snapshot of the process of displaying virtual environment 722 at a high immersion level (e.g., a continuation of the process described in FIGS. 7C-7D, respectively, and terminating at the immersion level described in FIGS. 7G-7H, respectively). As indicated by immersion level indicator 716, the second snapshot corresponds to an immersion level of 3, thus further increasing the amount of three-dimensional environment 704 replaced by virtual environment 722a and thus increasing the size of virtual environment 722a (e.g., increasing the size of the visual “portal” to virtual environment 722a).

[0142] For example, in FIG. 7E (e.g., which is a snapshot of a far-field transition animation), the virtual environment 722a expands toward the user (e.g., replacing more of the real-world environment portion of the three-dimensional environment 704) in response to an increasing level of immersion (e.g., based on the distance of those portions of the physical environment and / or three-dimensional environment 704 from the user's viewpoint). In some embodiments, expanding the virtual environment 722a includes expanding and / or moving the boundary between the virtual environment 722a and the real-world environment portion of the three-dimensional environment 704 toward the user (e.g., as also shown in the overhead view 718). In some embodiments, the boundary of the virtual environment 722a remains equidistant from the user 720 and thus appears to move toward the user in a circular and / or spherical shape (e.g., when considered in three dimensions). In some embodiments, other shapes for the expansion of the boundary of the virtual environment 722a are possible. For example, virtual environment 722a expands as a plane toward the user (e.g., a planar boundary parallel to the user that moves toward the user as virtual environment 722a expands). In some embodiments, as virtual environment 722a expands, portions of the real-world environment portion of three-dimensional environment 704 are no longer displayed and are replaced by portions of virtual environment 722a that were previously not displayed. For example, in FIG. 7E , the back wall, corner table 708a, and a portion (e.g., more distant portions) of desk 710a are no longer visible because they are further from user 720 than the boundary of virtual environment 722a. In some embodiments, objects in the real-world environment portions of the three-dimensional environment that are further from the boundary of virtual environment 722a (e.g., further from the user's viewpoint) are no longer displayed, while objects in the real-world environment portions of the three-dimensional environment that are closer to the boundary of virtual environment 722a (e.g., closer to the user's viewpoint) continue to be displayed. In some embodiments, if the object is closer than the boundary of the virtual environment 722a but has a height that would otherwise obscure part of the virtual environment 722a (e.g., desk 710a in FIG. 7E), the object is optionally removed from the three-dimensional environment 704.For example, the right rear corner of side table 712a obscures a portion of virtual environment 722a in FIG. 7E and is optionally removed from three-dimensional environment 704 (e.g., no longer displayed). In some embodiments, objects that are closer than the boundary of virtual environment 722a but would otherwise obscure a portion of virtual environment 722a continue to be displayed (e.g., such as side table 712a in FIG. 7E). In some embodiments, if an object straddles the boundary of virtual environment 722a (e.g., portions of the object are closer than the boundary and portions are further away than the boundary, such as desk 710a in FIG. 7E), the object is optionally removed from display (e.g., removed from three-dimensional environment 704 so that it is no longer visible). In some embodiments, if an object straddles the boundary, portions of the object that are closer than the boundary continue to be displayed, but portions of the object that are further away from the boundary are no longer displayed (e.g., such as desk 710a in FIG. 7E). In some embodiments, when an object straddles a boundary, the entire object continues to be displayed (e.g., optionally visually de-highlighted, such as grayed out, darkened, partially transparent, displayed as an outline, etc., as if that portion of the object were part of the simulated environment) such that part of the object is in the real-world portion of three-dimensional environment 704 and part of the object is in the simulated environment portion of three-dimensional environment 704. Thus, in some embodiments, the entire desk 710a is no longer displayed (e.g., removed from view) because at least a portion of the desk 710a is further than the boundary of virtual environment 722a. Alternatively, in some embodiments, the entire desk 710a continues to be displayed even when a portion of the desk 710a is further than the boundary of virtual environment 722a (e.g., such that the desk 710a appears to be partially within the virtual environment 722a and partially within the real-world environment portion of three-dimensional environment 704).

[0143] In some embodiments, expanding virtual environment 722a includes making more of virtual environment 722a visible. For example, if the boundary of virtual environment 722a is 10 feet away from the user, the user can see a portion of virtual environment 722a 10 feet away, but if the boundary of virtual environment 722a is expanded to 5 feet away from the user, the user can see a portion of virtual environment 722a 5 feet away, including the portion of virtual environment 722a 10 feet away (e.g., the portion that was shown when the boundary was 10 feet away) and an additional 5 feet of space that was previously invisible. Thus, as virtual environment 722a expands, more of the virtual environment becomes visible to the user. For example, in FIG. 7E, the area between two corners of the room (e.g., the area between 722-1a and 722-2a in FIG. 7C that previously contained a representation of the physical environment) is now occupied by virtual environment 722a. In some embodiments, expanding virtual environment 722a does not include moving virtual environment 722a closer to the user. For example, an object in virtual environment 722a that was previously 20 feet away from the user optionally does not move closer to the user as the boundary of virtual environment 722a moves toward the user, but rather remains 20 feet away. Instead, an object in virtual environment 722a that was, for example, 10 feet away and previously not exposed to the user (e.g., because the boundary is further away than 10 feet) is now optionally exposed to the user (e.g., because the boundary is moving closer than 10 feet away). Thus, expanding virtual environment 722a appears as if a visual portal into virtual environment 722a expands and / or moves closer to the user, without features of virtual environment 722a (e.g., objects therein) moving closer to the user.

[0144] 7F (e.g., which is a snapshot of a near-field transition animation), virtual environment 722a is optionally expanding radially outward from a central location (e.g., the position shown above in FIG. 7D ) (e.g., replacing more of the real-world environment portion of three-dimensional environment 704), regardless of the positions and / or distances of objects within three-dimensional environment 704 relative to user 720. In some embodiments, expanding virtual environment 722a radially outward optionally includes widening the size / portal of virtual environment 722a without moving the boundaries of virtual environment 722a closer to the user. For example, in FIG. 7F , virtual environment 722b has a radial size of 30 degrees (e.g., the angle formed by the left and right boundaries of virtual environment 722b is 30 degrees, and optionally the angle formed by the bottom and top boundaries of virtual environment 722b is 30 degrees), which is an increase in radial size compared to FIG. 7D . In some embodiments, the boundaries of virtual environment 722a expand in a circular manner (or in a spherical manner when considered in three dimensions), curving around user 720, as shown by virtual environment 722b in overhead view 718. In some embodiments, the boundaries of virtual environment 722a expand outward, optionally remaining equidistant from user 720 along the expanded boundaries (e.g., as if the boundaries of virtual environment 722a were expanding along the surface of a circle / sphere with user 720 as its center point). Thus, as shown in FIG. 7F , virtual environment 722a is a circular visual portal that not only increases in radius (e.g., expanding left / right and up / down, parallel to the plane of the back wall), but also curves around the user (e.g., changing depth, away from the plane of the back wall). As described above, expanding virtual environment 722a makes more of virtual environment 722a visible, replacing portions of the real-world environment. In Figure 7F, a larger area of ​​the back wall is no longer visible, replacing portions of the virtual environment 722a that were previously not displayed (e.g., compared to Figure 7D), and a portion of the desk 710a is no longer visible.In some embodiments, the curvature of the virtual environment 722a around the user (e.g., in the z-dimension) for the near-field transition animation (e.g., Figures 7D, 7F, and 7H) is the same as or similar (e.g., same radius, same shape, etc.) to the curvature of the virtual environment 722a (e.g., in the z-dimension) for the far-field transition animation (e.g., Figures 7C, 7E, and 7G).

[0145] 7G-7H show a third snapshot of the process of displaying virtual environment 722 at a high immersion level (e.g., optionally a final state, a stable state, the state of three-dimensional environment 704 at the end of the process of displaying virtual environment 722, including the first and second snapshots described above with respect to FIGS. 7C-7D, respectively, and 7E-7F, respectively). As indicated by immersion level indicator 716, the third snapshot corresponds to an immersion level of 6, thus further increasing the amount of three-dimensional environment 704 replaced by virtual environment 722a, and therefore increasing the size of virtual environment 722a (e.g., increasing the size of the visual “portal” to virtual environment 722a).

[0146] As shown in FIG. 7G illustrating a far-field transition animation, virtual environment 722a has further expanded such that the boundaries of virtual environment 722a move closer to user 720 (e.g., optionally while remaining equidistant to user 720 along the boundaries of virtual environment 722a). For example, a portion of the three-dimensional environment between representation 714a of coffee table 714b and representation 710a of desk 710a that was not previously occupied by virtual environment 722a is now occupied by virtual environment 722a. As described above, expanding the boundaries of virtual environment 722a toward the user does not involve moving objects within virtual environment 722a, but rather involves exposing portions of virtual environment 722a that are located closer to the user's viewpoint but were previously hidden (e.g., unmasking portions of virtual environment 722a). In some embodiments, as virtual environment 722a encompasses more of three-dimensional environment 704, representations of more physical objects extend beyond the boundaries of virtual environment 722a and are no longer displayed. 7G, representation 710a of desk 710b and representation 712a of side table 712b are no longer displayed, but representation 714a of coffee table 714b is still displayed because it is not located further than the boundaries of virtual environment 722a. Thus, although the physical environment is no longer visible beyond the boundaries of virtual environment 722a (e.g., only virtual environment 722a is present), the physical environment closer to the user than the boundaries of virtual environment 722a remains visible.

[0147] In Figure 7H, which shows the near-field transition animation, virtual environment 722a has expanded further radially outward compared to virtual environment 722a in Figure 7F. For example, in Figure 7H, virtual environment 722b has a radial size of 180 degrees (e.g., the left and right boundaries of virtual environment 722a extend immediately to the left and right of user 720, and optionally the top and bottom boundaries of virtual environment 722a extend immediately above and below user 720), which is an increase in radial size compared to Figure 7F; thus, more of desk 710a that was not previously occupied by virtual environment 722a is now occupied by virtual environment 722a. As described above, virtual environment 722a optionally expands radially as if along the surface of a sphere surrounding user 720. Thus, as virtual environment 722a expands, it appears as an expanding circle (e.g., in the x and y dimensions), but also wraps around the user (e.g., in the z dimension), as shown in overhead view 718. In some embodiments, virtual environment 722a does not extend closer to user 720, and thus a portion of desk 710b is still visible at the bottom of three-dimensional environment 704. As mentioned above, the shape of the boundary of virtual environment 722a is optionally the same in FIG. 7H (e.g., near-field transition animation) as in FIG. 7G (e.g., far-field transition animation).

[0148] 7G and 7H show an animation for displaying virtual environment 722a and / or a final state of the display of virtual environment 722a at a relatively high immersion level (e.g., a maximum immersion level or an immersion level greater than an intermediate immersion level), where the amount of three-dimensional environment 704 consumed by virtual environment 722a is the same regardless of whether the animation was a far-field transition animation or a near-field transition animation. The size and shape of virtual environment 722a in FIG. 7G (e.g., the far-field transition animation) is, optionally, the same as the size and shape of virtual environment 722a in FIG. 7H (e.g., the near-field transition animation). For example, the boundary of virtual environment 722a is the same shape between FIG. 7G and FIG. 7H and, optionally, is the same distance from the user's viewpoint between FIG. 7G and FIG. 7H. In some embodiments, the user's view of the virtual environment 722a in Figures 7G and 7H is the same (e.g., individual objects in the virtual environment 722a are at the same distance and in the same location relative to the user in Figures 7G and 7H).

[0149] Thus, in some embodiments, at a certain immersion level(s) (e.g., a maximum immersion level, a high immersion level, etc.), the virtual environment appears the same to the user (e.g., same size, same shape, same amount of the three-dimensional environment occupied by the virtual environment, same portion of the virtual environment, etc.) regardless of the amount of space in front of the user in the user's physical environment. In some embodiments, at a particular immersion level (e.g., a medium immersion level, a low immersion level, etc.), the virtual environment appears different based on whether there is enough space in front of the user or whether there is not enough space in front of the user to utilize far-field transition animations. For example, in the embodiment illustrated in Figures 7G-7H, at immersion levels higher than 6, the size and / or shape of the entrance into the virtual environment 722a is the same or similar between the far-field transition and the near-field transition, while at immersion levels lower than 6 (e.g., as described with reference to Figures 7C-7F), the size and / or shape of the entrance into the virtual environment 722a is different between the far-field transition and the near-field transition.

[0150] 8A-8H are flowcharts illustrating a method 800 for displaying a virtual environment, according to some embodiments. In some embodiments, method 800 is implemented in a computer system (e.g., computer system 101 of FIG. 1 , such as a tablet, smartphone, wearable computer, or head-mounted device) that includes a display generation component (e.g., display generation component 120 of FIGS. 1 , 3, and 4 ) (e.g., a head-up display, a display, a touchscreen, a projector, etc.) and one or more cameras (e.g., a camera pointing downward in a user's hand (e.g., color sensors, infrared sensors, and other depth-sensing cameras) or a camera pointing forward from the user's head). In some embodiments, method 800 is governed by instructions stored on a non-transitory computer-readable storage medium and executed by one or more processors of the computer system, such as one or more processors 202 of computer system 101 (e.g., control unit 110 of FIG. 1A ). Some operations of method 800 are, optionally, combined, and / or the order of some operations is, optionally, changed.

[0151] In method 800, in some embodiments, an electronic device (e.g., computer system 101 of FIG. 1 ) in communication with a display generation component and one or more input devices (e.g., a mobile device (e.g., a tablet, smartphone, media player, or wearable device) or a computer) receives input 802 corresponding to a request to display a first mimicked environment, such as a selection of an affordance or a representation of a mimicked environment while displaying a three-dimensional environment 704 such as one of FIGS. 7A-7B , via one or more input devices while displaying a particular environment (e.g., a representation of a physical environment, a mimicked environment, computer-generated reality, extended reality, etc.) via the display generation component (e.g., detects movements of one or more of a user's hands, movements of one or more of a user's eyes, and / or any other input corresponding to the request to display the first mimicked environment via a hand tracking device, an eye tracking device, or any other input device).

[0152] In some embodiments, the display generation component is a display integrated with the electronic device (optionally a touchscreen display), an external display such as a monitor, projector, television, or a hardware component (optionally built-in or external) for projecting a user interface and making the user interface visible to one or more users, etc. In some embodiments, the one or more input devices include electronic devices or components capable of receiving user input (e.g., capturing user input, detecting user input, etc.) and transmitting information related to the user input to the electronic device. Examples of input devices include a touchscreen, a mouse (e.g., external), a trackpad (optionally built-in or external), a touchpad (optionally built-in or external), a remote control device (e.g., external), another mobile device (e.g., separate from the electronic device), a handheld device (e.g., external), a controller (e.g., external), a camera, a depth sensor and / or a motion sensor (e.g., hand tracking sensor, hand motion sensor), etc.

[0153] In some embodiments, the individual environment is an augmented reality or mixed reality environment, optionally including virtual objects and / or representations of real-world objects in the physical world around the electronic device. In some embodiments, the individual environment is based at least on the physical environment around the electronic device. For example, the electronic device can capture visual information about the environment around the user (e.g., objects in the environment, the size and shape of the environment, etc.) and display at least a portion of the physical environment around the user to the user, optionally making it appear as if the user is still located within the physical environment. In some embodiments, the individual environment (e.g., the physical environment around the electronic device, a portion of the physical environment around the electronic device, etc.) is actively displayed to the user via a display generation component. In some embodiments, the individual environment (e.g., the physical environment around the electronic device, a portion of the physical environment around the electronic device, etc.) is passively presented to the user via a partially transparent or translucent display through which the user can see at least a portion of the physical environment.

[0154] In some embodiments, the request to display the first mimicked environment includes manipulating a rotary element, such as a mechanical or virtual dial, of or in communication with the electronic device. In some embodiments, the request includes selecting a displayed affordance and / or manipulating a displayed control element to increase the immersion level of the device and / or the individual environment. In some embodiments, the request includes a predefined gesture recognized as a request to increase the immersion level of the device and / or the individual environment. In some embodiments, the request includes a voice input requesting an increase in the immersion level of the device and / or the individual environment and / or requesting the display of the first mimicked environment. In some embodiments, the first mimicked environment includes a scene that at least partially covers at least a portion of the individual environment so that the user appears to be located within the scene (e.g., optionally, no longer located within the first mimicked environment). In some embodiments, the first mimicked environment is an atmosphere transformation that modifies one or more visual characteristics of the individual environment so that the individual environment appears to be located at a different time, location, and / or conditions (e.g., morning lighting instead of afternoon lighting, sunny instead of cloudy, etc.). In some embodiments, the immersion level includes the degree to which content displayed by the electronic device (e.g., the virtual environment) obscures background content (e.g., content other than the virtual environment) around / behind the virtual environment, and optionally includes the number of items of background content displayed and the visual characteristics (e.g., color, contrast, opacity) at which the background content is displayed, and / or the angular range of content displayed via the display generation component (e.g., 60 degrees for content displayed at low immersion, 120 degrees for content displayed at medium immersion, 180 degrees for content displayed at high immersion), and / or the percentage of the field of view displayed via the display generation that is consumed by the virtual environment (e.g., 33% of the field of view consumed by the virtual environment at low immersion, 66% of the field of view consumed by the virtual environment at medium immersion, 100% of the field of view consumed by the virtual environment at high immersion). In some embodiments, the background content is included in the background against which the virtual environment is displayed.In some embodiments, background content includes user interfaces (e.g., device-generated user interfaces corresponding to applications), virtual objects not associated with or included in the virtual environment (e.g., device-generated files, representations of other users, etc.), and / or real objects (e.g., pass-through objects representing real objects in the physical environment around the user's viewpoint, displayed by the device as visible through the display generation components and / or as visible through transparent or translucent display generation components, because the electronic device does not obscure / obstruct their visibility through the display generation components). In some embodiments, at a first (e.g., low) immersion level, background, virtual, and / or real objects are displayed in an obscured manner. For example, a virtual environment at a low immersion level is optionally displayed simultaneously with background content, and the background content is optionally displayed at full brightness, color, and / or translucency. In some embodiments, at a second (e.g., high) immersion level, background, virtual, and / or real objects are displayed in an obscured manner (e.g., dimmed, blurred, removed from view, etc.). For example, a separate virtual environment having a high immersion level may be displayed (e.g., in full screen or fully immersive mode) without simultaneously displaying background content. As another example, a virtual environment displayed at an intermediate immersion level may be simultaneously displayed with dimmed, blurred, or otherwise de-highlighted background content. In some embodiments, the visual characteristics of the background objects may differ between the background objects. For example, at a particular immersion level, one or more first background objects may be visually less highlighted (e.g., dimmed, blurred, or displayed with increased transparency) than one or more second background objects, and one or more third background objects may cease to be displayed.

[0155] In some embodiments, in response to receiving an input corresponding to a request to display the first simulated environment, the electronic device transitions (804) from a display of the individual environment to a display of the first simulated environment (e.g., displaying the first simulated environment in a portion of the display area (e.g., a subset of the display area, all of the display area, etc.), such as transitioning from a display of three-dimensional environment 704 without the virtual environment in FIGS. 7A-7B to a display of virtual environment 722 in FIGS. 7G-7H, respectively). In some embodiments, the portion of the individual environment is no longer displayed and / or no longer visible to the user, and a portion of the first simulated environment is displayed and / or visible in its place. In some embodiments, the portion of the individual environment is overlaid by the first simulated environment. In some embodiments, the portion of the individual environment is replaced by the first simulated environment.

[0156] In some embodiments, the transitioning includes transitioning 806 from displaying the individual environment to displaying the first simulated environment using a first type of transition, such as the far-field transitions shown in FIGS. 7C, 7E, and 7G, in accordance with a determination that the physical environment around the user's viewpoint meets one or more first criteria (e.g., a transition from displaying the individual environment to displaying the first simulated environment is a first type of transition if the physical environment has a size, area, depth (e.g., in front of the electronic device) that exceeds a threshold, the shape of the physical environment conforms to a particular pattern, and / or a portion of the physical environment relative to the device has a size, area, depth (e.g., space in front of the device, space to the left of the device, space to the right of the device, etc.) that exceeds a threshold). and transitioning (808) from displaying the respective environment to displaying the first mimicked environment using a second type of transition different from the first type of transition, such as the near-field transitions shown in Figures 7D, 7F, and 7H, in accordance with a determination that the environment satisfies one or more second criteria different from the one or more first criteria (e.g., if the physical environment around the user's viewpoint has a size, area, or depth (e.g., in front of the electronic device) that is less than a threshold, the shape of the physical environment fits a different pattern, and / or if a portion of the physical environment relative to the device has a size, area, or depth (e.g., the space in front of the device, the space to the left of the device, the space to the right of the device, etc.) that is less than a threshold, the transition from displaying the respective environment to displaying the first mimicked environment is the second type of transition).

[0157] Thus, the one or more first criteria include criteria that are met if the physical environment is greater than a threshold size (e.g., 20 square feet, 50 square feet, 100 square feet, 300 square feet, etc.), area, and / or depth (e.g., the depth of the physical environment in front of the user and / or device has a depth greater than 2 feet, 5 feet, 8 feet, 10 feet, etc.). In some embodiments, the depth of the physical environment is determined based on the distance from the user and / or device to the nearest physical object (e.g., optionally the nearest physical object other than the floor in front of the user and / or device, or to the left, right, or behind the user and / or device, such as a coffee table or desk). In some embodiments, the depth of the physical environment is determined based on the distance from a vertical plane such as a wall (e.g., optionally the nearest vertical plane, or optionally the nearest vertical plane in front of the user or device). In some embodiments, the depth of the physical environment is determined based on the distance from the nearest boundary of the physical environment (or the boundary of the physical environment directly in front of the user or device). As described above, the personalized environment may be based at least in part on the physical environment. Thus, in some embodiments, characteristics of the physical environment inform the transition between the personalized environment and the first simulated environment, e.g., to reduce transition shock, nausea, and / or cognitive dissonance. For example, if the physical environment around and / or in front of the user / electronic device has a maximum depth of more than four feet (e.g., the user is sitting at least four feet away from the wall opposite the user), the transition to the first simulated environment includes gradually transitioning portions of the personalized environment based on their distance from the electronic device and / or user. For example, the area of ​​the personalized environment farthest from the user transitions first, then the next-closest area transitions second, and so on, optionally continuing until the user and / or electronic device is reached or a predetermined distance in front of the user and / or electronic device (e.g., six inches in front of the user, one foot in front of the user, three feet in front of the user, etc.).In some embodiments, different portions of the individual environments transition at different times (e.g., closer portions transition after more distant portions). In some embodiments, non-adjacent areas of the individual environments transition simultaneously (e.g., different portions of the individual environments are not adjacent but are the same distance away from the user and / or electronic device). In some embodiments, a first type of transition appears to start at the farthest location and extend toward the user (e.g., as a plane moving toward the user or in a circle toward the user, e.g., such that the boundary of the mimicked environment is, optionally, equidistant from the user and / or device throughout the transition). Thus, in some embodiments, the first type of transition depends on differences in depth of objects within the individual environments (e.g., the order of which transition is first, second, third, last, etc. is based on depth information). In some embodiments, transitioning includes dissolving a separate portion of the individual environment into a separate portion of the first mimicked environment. In some embodiments, other transition animations are possible. In some embodiments, the portion of the individual environment that transitions into a portion of the first mimicked environment is added to a previously transitioned portion of the first mimicked environment. For example, a first portion of the individual environment transitions to a first portion of a first simulated environment, and then a second portion of the individual environment transitions to a second portion of the first simulated environment, with the first and second portions of the first simulated environment together forming a continuous simulated environment.

[0158] In some embodiments, the user's viewpoint is a location within the physical environment (e.g., and / or its corresponding location within the three-dimensional environment, depending on the context of its use) that has a perspective (e.g., view) of the physical environment that is the same as (or similar to) the perspective of the three-dimensional environment. For example, the three-dimensional environment is constructed to appear to the user as if the user were physically located at a distinct location within the real-world environment (e.g., representations of real-world objects within the three-dimensional environment appear to be at the same relative locations and the same distances away from the user as if the user were actually viewing the real-world objects within the real-world environment). Accordingly, the one or more second criteria include criteria that are met if the physical environment has less than a threshold size (e.g., 20 square feet, 50 square feet, 100 square feet, 300 square feet, etc.), area, and / or depth (e.g., the space in front of the user and / or device has a depth of less than 2 feet, 5 feet, 8 feet, 10 feet, etc.). In some embodiments, the second type of transition involves initiating the transition from a distinct location within the distinct environment and expanding outward from the distinct location. For example, if the physical environment surrounding and / or in front of the user has a maximum depth of less than 4 feet, the distinct environment transitions to the first simulated environment starting from a portion of the distinct environment farthest from the user (e.g., the furthest point within the physical environment), with the transition expanding outward (e.g., radially) from the starting position (e.g., with the starting position as the center of the transition and more of the distinct environment transitioning to the first simulated environment). In some embodiments, the second type of transition appears to initiate from a particular location within the distinct environment and expand outward (e.g., left / right, up / down) from that location, optionally expanding around the user (e.g., as an expanding circle as opposed to a plane expanding toward the user). In some embodiments, the second type of transition is implemented regardless of differences in depth in the distinct environments.For example, the transition extends outward from the starting position, optionally equally in all directions, regardless of whether the respective portion of the environment is closer or farther away (e.g., the order in which objects are "wrapped" by the mimicked environment does not depend on the depth of the object within the environment). In some embodiments, the second type of transition includes initiating the transition from a location on a surface (e.g., the surface of a table, the surface of a wall, etc.) and moving along the surface (e.g., toward the user). In some embodiments, the second type of transition includes initiating the transition from a discrete location (e.g., the center of the field of view, etc.) and extending radially outward along the user's field of view (e.g., extending in all directions from the discrete location in the same manner). In some embodiments, transitioning outward from a central reference location provides an initial reference point for the first mimicked environment, which reduces nausea and / or cognitive dissonance due to potentially transitioning from an environment with a short depth (e.g., the user's physical environment) to an environment with a distant depth (e.g., the first mimicked environment). In some embodiments, the first and second transition methods have different transition speeds, different cross-fade animations, and / or different orders for transitioning portions of the environment. In some embodiments, the results of the first and second transition methods are the same. For example, the initial and final conditions are the same, but the transitions from the initial conditions to the final conditions are different.

[0159] The above-described method of transitioning from one environment to another (e.g., by transitioning from a first environment to a second environment in a manner based on characteristics of the physical environment surrounding the device) provides a fast and efficient way of transitioning to a different environment in a manner that takes into account the characteristics of the physical environment and / or starting environment, and optionally reduces the abruptness of the transition, which simplifies the interaction between a user and the electronic device, improves usability of the electronic device, makes the user-device interface more efficient (e.g., by allowing a user to interact with a new environment faster and reducing the time required to adapt to changes), reduces potential disorientation that may potentially lead to symptoms of dizziness or motion sickness, thereby further reducing power usage and improving battery life of the electronic device by allowing a user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0160] In some embodiments, the one or more first criteria include a criterion that is met when a portion of the physical environment surrounding the user's viewpoint in a particular direction relative to the reference orientation of the electronic device is greater than a threshold distance from the user's viewpoint of the electronic device, such as physical environment 702 having a distance greater than a threshold distance from user 720 to the far wall in FIG. 7A , thus qualifying three-dimensional environment 704 for far-field transition (e.g., the first criterion is met if a portion of the electronic device's physical environment in front of the electronic device (e.g., in a “forward” direction and / or toward the center of the field of view of the viewpoint of the three-dimensional environment displayed by the device) has greater than a threshold amount of space (e.g., greater than 1 foot, 5 feet, 10 feet, 50 feet, 100 feet, etc.)) (810).

[0161] For example, if the device is located in the center of a room and a wall in front of the device (e.g., a boundary of the physical environment in front of the device) is more than 10 feet away, the one or more first criteria are met (e.g., if the wall in front of the device is less than 10 feet away, the one or more first criteria are optionally not met). In some embodiments, the boundary of the physical environment is defined by a wall of the physical environment (e.g., a vertical barrier above a threshold height, such as 5 feet, 8 feet, 12 feet, or a vertical barrier that spans the vertical distance from floor to ceiling, etc.). In some embodiments, objects in the physical environment that are not recognized as a vertical wall or that are below the threshold height are not considered when determining whether the physical environment is greater than a threshold distance from the electronic device. For example, a sofa, table, desk, counter, etc. in front of the electronic device are optionally not considered when determining whether the physical environment is greater than a threshold distance from the electronic device. In some embodiments, sofas, tables, desks, counters, etc. are considered. In some embodiments, the size and / or shape of the physical environment (e.g., including the size, shape, and / or location of objects within the physical environment) is determined via one or more sensors of the electronic device, such as a visible light sensor (e.g., a camera), a depth sensor (e.g., a time-of-flight sensor), etc. (e.g., individually or in combination).

[0162] The above-described method of displaying an environment via either a first type of transition or a second type of transition (e.g., based on whether the physical environment around the device meets certain criteria) ensures a smooth transition that takes into account the user's environment and reduces potential disorientation that may potentially lead to symptoms of dizziness or motion sickness, which simplifies the interaction between the user and the electronic device, improves the usability of the electronic device, and makes the user-device interface more efficient (e.g., automatically, without the user having to perform additional input to select between different types of transitions), thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0163] In some embodiments, the one or more second criteria include a criterion that is met when a portion of the physical environment surrounding the user's viewpoint in a particular direction relative to the reference orientation of the electronic device is less than a threshold distance from the electronic device (812), e.g., physical environment 702 has less than a threshold distance from user 720 in FIG. 7B to the back wall, thus making three-dimensional environment 704 eligible for near-field transition, and optionally not eligible for far-field transition (e.g., the second criterion is met if a portion of the electronic device's physical environment in front of the electronic device (e.g., in the “forward” direction and / or toward the center of the field of view of the viewpoint of the three-dimensional environment displayed by the device) has less than a threshold amount of space (e.g., more than 1 foot, 5 feet, 10 feet, 50 feet, 100 feet, etc.)).

[0164] For example, if a wall in front of the electronic device is less than a threshold distance from the electronic device, the electronic device performs a second type of transition when displaying the individual environment. In some embodiments, the second type of transition is a different type of transition from when there is more than a threshold amount of space to reduce the risk that the transition is uncomfortable for the user and / or to reduce the risk of causing symptoms of motion sickness. For example, if a user is presented with an environment with a small amount of depth (e.g., the user's physical environment) and suddenly transitions to an environment with a lot of space (e.g., a simulated environment), this effect may cause visual dissonance. In some embodiments, the one or more second criteria include a criterion that is met when the electronic device is unable to determine depth information of the physical environment surrounding the electronic device (e.g., optionally to a threshold accuracy level). For example, the one or more second criteria are met when environmental conditions prevent the device's sensors from determining the size and shape of the user's room (e.g., atmospheric interference, electromagnetic interference, etc.).

[0165] The above-described method of displaying an environment via either a first type of transition or a second type of transition (e.g., based on whether the physical environment around the device meets certain criteria) ensures a smooth transition that takes into account the user's environment and reduces potential disorientation that may potentially lead to symptoms of dizziness or motion sickness, which simplifies the interaction between the user and the electronic device, improves the usability of the electronic device, and makes the user-device interface more efficient (e.g., automatically, without the user having to perform additional input to select between different types of transitions), thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0166] In some embodiments, the individual environment includes a representation of a portion of the physical environment surrounding the user's viewpoint (e.g., at least a portion of the environment displayed and / or presented to the user is based on (e.g., is a realistic representation of) the physical environment surrounding the device and / or user, such as via actual pass-through through a display generation component (e.g., a transparent or translucent display generation component) or digital pass-through through a display generation component), and transitioning from displaying the individual environment to displaying the first mimicked environment includes replacing the display of a representation of the portion of the physical environment with a representation of the portion of the first mimicked environment (e.g., replacing a portion of the representation of the physical environment with the mimicked environment such that the environment presented to the user is partially a real-world physical environment and partially a virtual mimicked environment) (814), such as replacing the representation of a corner of physical environment 702 in three-dimensional environment 704 with a respective portion of virtual environment 722 (e.g., 722-1a and 722-1b, respectively) in FIG. 7C .

[0167] In some embodiments, the amount of the mimicked environment that is displayed (e.g., the amount of the physical environment that is not displayed) is based on the immersion level of the device and / or the mimicked environment. For example, increasing the immersion level causes more of the mimicked environment to be displayed, replacing and / or obscuring more of the representation of the physical environment, and decreasing the immersion level causes less of the mimicked environment to be displayed, exposing portions of the physical environment that were not previously displayed and / or obscured.

[0168] The above-described method of displaying a mimicked environment (e.g., by replacing portions of a representation of the physical environment with portions of the mimicked environment) provides a fast and efficient way to transition from displaying only the physical environment to also displaying the mimicked environment, which improves the usability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that could potentially lead to symptoms of dizziness or motion sickness, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, and reducing errors during use.

[0169] In some embodiments, the input corresponding to the request to display the first mimicked environment includes a user input (e.g., detecting and / or receiving a user's manipulation of a mechanical input element in communication with an electronic device, such as a dial or button, or a control element, such as a virtual button or virtual dial, displayed within the respective environment) such as a selection of an affordance or representation of the mimicked environment while currently displaying a virtual environment, such as any of Figures 7C-7H (816). In some embodiments, the user input is performed by one or more hands of the user and detected by one or more sensors of the device, such as a camera, motion sensor, hand tracking sensor, etc.

[0170] In some embodiments, transitioning from displaying the individual environment to displaying the first simulated environment includes transitioning from displaying the individual environment to displaying a first portion of the first simulated environment, e.g., transitioning from a simulated environment having a first immersion level to a simulated environment having a second immersion level (e.g., transitioning from one of FIGS. 7C-7H to another of FIGS. 7C-7H), in accordance with a determination that the user input corresponds to a request to transition to the first simulated environment by a first amount (e.g., the user input selecting a higher or lower individual immersion level than the current immersion level) (e.g., displaying a respective amount of the first simulated environment based on user input) (820); and transitioning from displaying the individual environment to displaying a second portion of the first simulated environment that is greater than the first portion (824) in accordance with a determination that the user input corresponds to a request to transition to the first simulated environment by a second amount greater than the first amount, e.g., transitioning from a simulated environment having a first immersion level to a simulated environment having a second immersion level (e.g., transitioning from any of Figures 7C-7H to any other of Figures 7C-7H) (e.g., if the user input is a request to increase (or decrease) the immersion level by two steps, the amount of the first simulated environment displayed is increased (or decreased) by an amount corresponding to the two steps of increased (or decreased) immersion) (818).

[0171] In some embodiments, the user input is a relative input that increases or decreases the immersion level by a particular amount (e.g., a request to increase the immersion level by one step, two steps, etc.). In some embodiments, the user input is a selection of a particular immersion level (e.g., a request to increase the immersion level by 50%, 75%, etc.). For example, if the user input is a request to increase the immersion level by one step, the amount of the first simulated environment displayed increases by an amount corresponding to one step of increased immersion. In some embodiments, if the user input is a request to increase the immersion level by two steps, the amount of the first simulated environment displayed increases by an amount corresponding to two steps of increased immersion. In some embodiments, increasing or decreasing the amount of immersion comprises gradually increasing or decreasing, respectively, the amount the first simulated environment is displayed (e.g., as opposed to abruptly displaying more or less of the first simulated environment).

[0172] The above-described method of increasing or decreasing the amount of simulated environment displayed (e.g., by gradually increasing or decreasing based on the amount requested by user input) provides a fast and efficient way to display more or less simulated environment, which improves the usability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that could potentially lead to symptoms of dizziness or motion sickness, and thereby further reduces power usage, improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0173] In some embodiments, the user input includes rotating a rotatable input element in communication with the electronic device (e.g., a mechanical dial on the electronic device or on another device in communication with the electronic device (e.g., a remote control device, another electronic device, etc.)), where the user input corresponding to the request to transition to the first simulated environment by a first amount includes rotating the rotatable input element by a first discrete amount (e.g., one increment (and / or first rotation amount), two increments (and / or a second larger rotation amount), etc. (e.g., one detent, two detents, etc.)), and the user input corresponding to the request to transition to the first simulated environment by a second discrete amount includes rotating the rotatable input element by a second discrete amount greater than the first discrete amount, e.g., rotating a mechanical dial attached to or in communication with the electronic device 101 (e.g., rotating the mechanical dial by a larger amount) (826). In some embodiments, rotating in a first direction corresponds to a request to increase the immersion level, and rotating in a second, opposite direction corresponds to a request to decrease the immersion level.

[0174] The above-described method of increasing or decreasing the amount of simulated environment displayed (e.g., by rotating a rotatable element by a certain amount) provides a quick and efficient way to display more or less simulated environment (e.g., by providing the user with a process for increasing or decreasing the current level of immersion without the user having to select a specific amount of immersion), which improves the usability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that could potentially lead to symptoms of dizziness or motion sickness, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, and reducing errors during use.

[0175] In some embodiments, the display via the display generation component of the first mimicked environment after transitioning from displaying an individual environment to displaying the first mimicked environment using a first type of transition is the same as the display via the display generation component of the first mimicked environment after transitioning from displaying an individual environment to displaying the first mimicked environment using a second type of transition, e.g., the virtual environment 722a of FIG. 7H appears the same or similar to FIG. 7H after completing their respective transitions (828) (e.g., the starting and ending states of the first and second type of transitions are the same or similar).

[0176] For example, when transitioning from a first immersion level to a second immersion level, the system determines whether to implement a first type of transition or a second type of transition. In some embodiments, the first and second types of transitions include different types of animations and different portions of the display and / or environment transition at different times, but the end result after the transition is completed is the same regardless of which type of transition is used. In some embodiments, transitioning from the first immersion level to the second immersion level includes transitioning through each intermediate immersion level between the first and second immersion levels. In some embodiments, the highest immersion level (e.g., maximum immersion level) appears the same or similar regardless of whether the physical environment meets the first or second criteria (e.g., regardless of whether the first or second type of transition is used). In some embodiments, the intermediate immersion level (e.g., an immersion level less than the maximum immersion level and greater than no immersion) differs based on whether the physical environment meets the first criterion or the second criterion. For example, the size and / or shape of the first simulated environment within the three-dimensional environment optionally differs depending on whether the physical environment meets the first criterion or the second criterion, but the size and shape of the first simulated environment when at the maximum immersion level is the same regardless of whether the physical environment meets the first criterion or the second criterion.

[0177] The above-described method of displaying a mimicked environment (e.g., by performing different types of transitions based on the situation but reaching the same final state after the transitions are completed) provides a fast and efficient way of displaying a mimicked environment (e.g., by providing a consistent experience regardless of what type of transition is used to display the mimicked environment), which improves the usability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that can potentially lead to symptoms of dizziness or motion sickness, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0178] In some embodiments, the first type of transition proceeds in a manner based on the difference in distance between 1) a first portion of the physical environment surrounding the user's viewpoint and 2) a second portion of the physical environment surrounding the user's viewpoint, such as a far-field transition involving the virtual environment 722 moving to a distance closer to the user 720 using position and distance information of objects within the physical environment 702 of Figures 7C, 7E, and 7G (e.g., the first type of transition takes into account the position of objects in the user's physical environment when determining which portions of the respective environments will transition to the first, second, etc. (e.g., the order of the transition)).

[0179] For example, a first type of transition optionally first replaces a portion of the representation of the physical environment farthest from the user (or device) with a portion of the mimicked environment, then replaces a portion of the representation of the physical environment next farthest from the user (or device), and so on.

[0180] In some embodiments, the second type of transition proceeds 832 in a manner that is independent of (e.g., not based on) the difference between 1) the distance between the electronic device and a first portion of the physical environment surrounding the user's viewpoint and 2) the distance between the electronic device and a second portion of the physical environment surrounding the user's viewpoint, such as virtual environment 722 extending radially from the central position, regardless of the distance and / or location of objects within the physical environment of Figures 7D, 7F, and 7H (e.g., the second type of transition does not take into account the positions of objects within the room or the distance of objects within the room).

[0181] In some embodiments, the second type of transition selects a discrete location to begin the transition, replaces the representation of the physical environment at the starting position with a portion of the mimicked environment, and then extends the replacement process outward from the initial point, regardless of whether the object is in front of or behind other objects (e.g., the extension is optionally based on the display area rather than what is displayed). In some embodiments, the starting position is the location in the representation of the physical environment that is the farthest point from the user's viewpoint. For example, if the device is facing a flat wall, the second type of transition optionally begins from the center of the flat wall. In some embodiments, as the second type of transition progresses, the mimicked environment extends from the starting position. In some embodiments, the mimicked environment extends outward in a circular shape. In some embodiments, the extension of the mimicked environment has a shape similar to the surface of a sphere (or hemisphere) surrounding the user (e.g., the device and / or the user are at the center point of the sphere (or hemisphere)), so that the mimicked environment extends as if it were extending along the surface of a sphere (e.g., an imaginary sphere not displayed). In some embodiments, as the mimicked environment expands, the boundaries of the mimicked environment expand outward (e.g., in the x and y directions), but also forward (e.g., in the z direction), optionally remaining the same distance from the user (e.g., because the radius of the sphere is constant throughout the sphere).

[0182] The above-described methods of displaying mimicked environments (e.g., by transitioning in a manner that takes into account position and depth information of the environment surrounding the device or in a manner that does not take into account position and depth information of the environment surrounding the device) provide tailored transition styles based on the type of environment the device is located in, which improves the usability of the electronic device, makes the user-device interface more efficient, and reduces potential disorientation that can potentially lead to symptoms of dizziness or motion sickness (e.g., without the user having to determine the ideal type of transition and perform additional input to select each type of transition), thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0183] In some embodiments, the second type of transition includes a cross-dissolve between the individual environment and the first mimicked environment, where the cross-dissolve begins in a portion of the physical environment that is farthest from the electronic device (834), as in FIG. 7D (e.g., the second type of transition begins in a location that corresponds to the farthest location in the physical environment).

[0184] In some embodiments, from the starting location, the second type of transition expands the mimicked environment outward relative to the display area (e.g., the display screen). For example, from the starting location, the mimicked environment expands radially by 1 cm of the display area per second. In some embodiments, cross-dissolving between the individual environment and the first mimicked environment includes fading away the individual environment while simultaneously fading in the first mimicked environment. In some embodiments, the individual environment and the first mimicked environment are displayed simultaneously during a portion of the transition (e.g., when the individual environment is fading out and when the first mimicked environment is fading in). In some embodiments, cross-dissolving between the individual environment and the first mimicked environment includes successively replacing respective portions of the individual environment with respective portions of the first mimicked environment until all respective portions of the individual environment are completely replaced with the first mimicked environment.

[0185] The above-described method of displaying a simulated environment (e.g., by crossfading from an individual environment to a first simulated environment outward from a starting location) provides a smooth and efficient transition, which improves the usability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that could potentially lead to symptoms of dizziness or motion sickness, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, and reducing errors during use.

[0186] In some embodiments, transitioning from displaying a distinct environment to displaying the first mimicked environment using a first type of transition includes, during a first portion of the transition (838), replacing, via the display generation component, the display of a representation of the distinct portion of the physical environment with a display of at least a first portion of the first mimicked environment (840) in accordance with a determination that the distinct portion of the physical environment around the user's viewpoint is greater than a first distance from the user's viewpoint, e.g., determining that a portion of the three-dimensional environment 704 corresponds to a portion of the physical environment 702 that is farther away than the distinct distance associated with the immersion level of 3 in FIG. 7E , and thus replacing the determined portion with the virtual environment 722a (e.g., if the first portion of the representation of the physical environment corresponds to a location farther from the user (or device) than the second portion, the first portion of the representation of the physical environment is replaced with the second portion). and, in accordance with a determination that the individual portions of the physical environment surrounding the user's viewpoint are less than a first distance from the user's viewpoint, ceasing to replace, via the display generation component, the display of the representation of the individual portions of the physical environment with the display of at least a first portion of the first mimicked environment (842); e.g., determining that portions of the three-dimensional environment 704 correspond to portions of the physical environment 702 that are closer than the individual distance associated with the immersion level of 3 in FIG. 7E and therefore maintaining those portions as representations of the physical environment (e.g., if the first portion of the representation of the physical environment is not far from the second portion, not replacing (e.g., not transitioning) the first portion with a portion of the first mimicked environment during the first portion of the transition (e.g., until after the second portion has transitioned) (836).

[0187] In some embodiments, during a second portion of the transition (e.g., after a first portion of the transition) (844), pursuant to a determination that the respective portion of the physical environment surrounding the user's viewpoint is greater than a second distance from the user's viewpoint that is different from the first distance, the display generation component replaces (846) the display of the representation of the respective portion of the physical environment with a display of at least a second portion of the first mimicked environment, e.g., determines that a portion of the three-dimensional environment 704 corresponds to a portion of the physical environment 702 that is farther away than the respective distance associated with the immersion level of 6 in FIG. 7G , and therefore replaces the determined portion with the virtual environment 722a (e.g., during the second portion of the transition, a second portion of the representation of the physical environment (e.g., the next farthest portion after the first portion) that is closer than the first portion is replaced with the second portion (e.g., during the second portion of the transition, a second portion of the representation of the physical environment (e.g., the next farthest portion after the first portion) is replaced with the second portion (e.g., during the second portion of the transition, a second portion of the representation of the physical environment (e.g., the next farthest portion after the first portion) that is closer than the first portion ... 7H ) and replaces (e.g., transitions to) the previous portion of the first mimicked environment, and, pursuant to a determination that the individual portion of the physical environment surrounding the user's viewpoint is less than a second distance from the user's viewpoint, the display generation component ceases (848) replacing the display of the representation of the individual portion of the physical environment with the display of at least a second portion of the first mimicked environment, e.g., determines that the portion of the three-dimensional environment 704 corresponds to a portion of the physical environment 702 that is closer than the individual distance associated with the immersion level of 6 in FIG. 7H , and therefore maintains those portions as representations of the physical environment (e.g., if the second portion of the representation of the physical environment is not the next-furthest portion after the first portion, does not replace the second portion of the representation (e.g., until the second portion becomes the next-furthest portion to which it has not yet transitioned)).

[0188] In some embodiments, the boundary between the physical environment and the first mimicked environment has a shape defined by the surface of a sphere around the user's viewpoint (e.g., with the device and / or user as a center point). In some embodiments, as the transition progresses, the sphere moves closer to the electronic device and / or user and / or decreases in size. In some embodiments, the portion of the sphere's surface that intersects with the physical environment defines the boundary between the physical environment and the mimicked environment (e.g., the mimicked environment is where it intersects and the physical environment is where it intersects). In some embodiments, as the sphere moves closer to the electronic device and / or user and / or the size of the sphere decreases, more of the sphere's surface intersects with the physical environment, thus making the mimicked environment appear to encompass more of the three-dimensional environment and move closer to the user's viewpoint (e.g., starting from the furthest location in the physical environment).

[0189] The above-described method of transitioning from displaying an individual environment to displaying a mimicked environment (e.g., by transitioning a portion of the individual environment that is more distant before transitioning a portion of the individual environment that is closer) provides a smooth and efficient transition, which improves the usability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that could potentially lead to symptoms of dizziness or motion sickness, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, and reducing errors during use.

[0190] In some embodiments, the first type of transition includes, during a first portion of the transition (852), in accordance with a determination that a distinct portion of the physical environment surrounding the user's viewpoint is greater than a first distance from the user's viewpoint (854), and in accordance with a determination that a second distinct portion of the physical environment surrounding the user's viewpoint is less than the first distance from the user's viewpoint, displaying, via the display generation component (856), a representation of a second distinct portion of the physical environment between the first portion of the first mimicked environment and the user's viewpoint, e.g., determining that a portion of the three-dimensional environment 704 corresponds to a portion of the physical environment 702 that is closer than the distinct distance associated with the immersion level of 3 in FIG. 7E and maintaining those portions as a representation of the physical environment (e.g., during the first portion of the transition, the device's physical environment If an object (having a representation in the individual environment) in the virtual environment is closer than the mimicked environment, the representation of the object is displayed in front of the mimicked environment (e.g., an architectural object, furniture, etc.), and, in accordance with a determination that a second individual portion of the physical environment around the user's viewpoint is greater than the first distance from the user's viewpoint, the display generation component ceases to display the representation of the second individual portion of the physical environment (858), e.g., determines that portions of the three-dimensional environment 704 correspond to portions of the physical environment 702 that are farther away than the individual distance associated with the immersion level of 3 in FIG. 7E and replaces those portions with individual portions of the virtual environment 722 (e.g., the object farther away than the mimicked environment is no longer displayed) (850). For example, the object is overlaid, obscured, and / or replaced by the mimicked environment.

[0191] In some embodiments, a representation of an object is displayed overlaying a portion of the mimicked environment if the representation of the object is located along the line of sight of the portion of the mimicked environment. Thus, objects of the physical environment are optionally displayed overlaid on (and / or occluding) portions of the mimicked environment and / or obscured portions of the mimicked environment if the respective object is closer to the user / device than the mimicked environment.

[0192] The above-described method of displaying a separate environment simultaneously with a representation of the physical environment (e.g., by displaying a representation of the object and optionally obscuring parts of the separate environment if the separate object is located between the user / device and the mimicked environment) provides a smooth and efficient transition, which improves operability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that could potentially lead to symptoms of dizziness or motion sickness, thereby further reducing power usage and improving battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, and reducing errors during use.

[0193] In some embodiments, the individual environment includes a representation of a physical environment around the user's viewpoint, and the first mimicked environment includes a first mimicked light source (860) (e.g., the individual environment simultaneously includes a representation of a physical environment and a mimicked environment, and the mimicked environment includes at least one light source). For example, the mimicked environment is an outdoor environment with the sun as a light source, or the mimicked environment is a room with a lamp as a light source, or an application user interface (e.g., a movie screen) that emits mimicked light.

[0194] In some embodiments, while simultaneously displaying, via the display generation component, at least a portion of the representation of the physical environment surrounding the user's viewpoint and at least a portion of the first mimicked environment (e.g., including the first mimicked light source), the electronic device displays (862) a portion of the representation of the physical environment with lighting effects based on the first mimicked light source, such as when the virtual environment 722a projects lighting effects onto the real-world environment portion of the three-dimensional environment 704 of FIG. 7E (e.g., a light source in the mimicked environment projects lighting effects onto the representation of the physical environment). For example, light from the sun in the mimicked environment is optionally displayed as if it were exiting the mimicked environment, entering the physical environment, and illuminating a portion of the representation of the physical environment. In some embodiments, the mimicked light source optionally causes a mimicked shadow to be displayed in the representation of the physical environment.

[0195] The above-described method of displaying lighting effects from a mimicked environment within a representation of the physical environment (e.g., by displaying lighting effects in portions of the physical environment due to light sources within the mimicked environment) increases the immersive effect of the mimicked environment (e.g., by blending aspects of the mimicked environment with the physical environment), which improves the usability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that can potentially lead to symptoms of dizziness or sickness, thereby further reducing power usage and improving battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, and reducing errors during use.

[0196] In some embodiments, the lighting effects are based on the geometry of the physical environment around the user's viewpoint (864), such as the shape and position of the room and objects in physical environment 702 of FIG. 7E (e.g., lighting effects from mimicked lighting sources in the mimicked environment are displayed as if they interact with objects in the physical environment according to physics). For example, objects within the line of sight of the lighting source experience lighting effects, but objects obscured (e.g., by virtual or physical objects) optionally do not experience lighting effects (e.g., and optionally have shadows cast on them by objects within the line of sight of the lighting source). In some embodiments, some objects receive scattered, reflected, or refracted lighting effects.

[0197] The above-described methods of displaying lighting effects from a mimicked environment in a representation of a physical environment (e.g., by displaying lighting effects on portions of the physical environment based on the physical properties of light from a lighting source) increase the immersive effect of the mimicked environment (e.g., by realistically projecting light from the lighting source onto the physical environment) and provide information and / or indication of what the device knows about the physical environment, which improves operability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that could potentially lead to symptoms of dizziness or sickness, thereby further reducing power usage and improving battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0198] In some embodiments, the lighting effects are based on the architecture of the physical environment around the user's viewpoint (866), such as the shape and position of rooms and objects in physical environment 702 of Figure 7E (e.g., lighting effects from simulated lighting sources in the simulated environment are displayed as if they interact with the boundaries of the physical environment according to physics). For example, the lighting sources optionally project light onto walls and through windows.

[0199] The above-described methods of displaying lighting effects from a mimicked environment in a representation of a physical environment (e.g., by displaying lighting effects on portions of the physical environment based on the physical properties of light from a lighting source) increase the immersive effect of the mimicked environment (e.g., by realistically projecting light from the lighting source onto the physical environment) and provide information and / or indication of what the device knows about the physical environment, which improves operability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that could potentially lead to symptoms of dizziness or sickness, thereby further reducing power usage and improving battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0200] In some embodiments, the individual environment includes a representation of the physical environment around the user's viewpoint, and the first mimicked environment includes a first volumetric effect (868), such as the spatial effect of FIG. 9B (e.g., an effect that changes the visual characteristics of space in the environment that is not occupied by objects), for example, an effect that changes the look and feel of air in the environment. In some embodiments, the volumetric effect includes a lighting effect, a fog effect, a smoke effect, etc.

[0201] In some embodiments, while simultaneously displaying, via the display generation component, at least a portion of the representation of the physical environment around the user's viewpoint and at least a portion of the first mimicked environment (e.g., including the first volumetric effect), the electronic device displays (870) a second volumetric effect based on the first volumetric effect, such as displaying a spatial effect on a real-world portion of the three-dimensional environment 704 based on the virtual environment 722 of FIG. 7E within the volume of the portion of the representation of the physical environment (e.g., displaying a volumetric effect in the representation of the physical environment based on the volumetric effect in the mimicked environment). For example, if the mimicked environment includes a fog effect, the fog effect is optionally displayed in the representation of the physical environment such that fog from the mimicked environment appears to drift within the physical environment.

[0202] The above-described method of displaying volumetric effects from the mimicked environment in a representation of the physical environment (e.g., by displaying volumetric effects in portions of the physical environment based on volumetric effects shown in the mimicked environment) increases the immersive effect of the mimicked environment (e.g., by allowing effects from the mimicked environment to spill over into the physical environment), which improves operability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that could potentially lead to symptoms of dizziness or sickness, thereby further reducing power usage and improving battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, and reducing errors during use.

[0203] In some embodiments, the first volumetric effect and the second volumetric effect include lighting effects (874) (e.g., the volumetric effect includes varying lighting and / or introducing lighting effects). For example, a fog volumetric effect includes varying lighting of virtual particles displayed in the physical environment to mimic the scattering / diffraction effects of fog.

[0204] The above-described method of displaying lighting effects from a mimicked environment within a representation of a physical environment (e.g., by displaying lighting effects on portions of the physical environment based on lighting effects within the mimicked environment) increases the immersive effect of the mimicked environment (e.g., by realistically projecting light from a lighting source onto the physical environment), which improves the operability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that can potentially lead to symptoms of dizziness or sickness, thereby further reducing power usage and improving battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, and reducing errors during use.

[0205] In some embodiments, the first volumetric effect and the second volumetric effect include particle effects (e.g., displaying visual effects (e.g., abstract visual effects), such as fire, smoke, dust, etc., in the physical environment (874). In some embodiments, the particle effect includes one or more simulated particles in the atmosphere of the physical environment that emit one or more visual effects.

[0206] The above-described method of displaying particle effects from a mimicked environment within a representation of the physical environment (e.g., by displaying particle effects on portions of the physical environment based on particle effects in the mimicked environment) increases the immersive effect of the mimicked environment (e.g., by realistically displaying particle effects in the physical environment that are optional extensions of the particle effects displayed in the mimicked environment), which improves operability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that could potentially lead to symptoms of dizziness or sickness, thereby further reducing power usage and improving battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, and reducing errors during use.

[0207] In some embodiments, while simultaneously displaying at least a portion of the individual environment and at least a portion of the first mimicked environment via the display generation component (876), pursuant to a determination that a boundary between the portion of the individual environment and the portion of the first mimicked environment intersects a representation of an individual object within the individual environment, the electronic device removes (878) the representation of the individual object from the display via the display generation component (e.g., completely removing the display of the object if the boundary of the mimicked environment intersects a representation of a physical object in the representation of the physical environment (e.g., as opposed to the object being entirely in front of or entirely behind the mimicked environment)). In some embodiments, instead of removing the display of the object, the electronic device displays the portion of the object that is not behind the boundary of the mimicked environment (e.g., displaying only a portion of the object and hiding the portion of the object behind the mimicked environment).

[0208] The above-described method of simultaneously displaying a mimicked environment and a representation of the physical environment (e.g., by removing objects in the physical environment that would otherwise be partially within the physical environment and partially within the mimicked environment) increases the immersive effect of the mimicked environment (e.g., by avoiding scenarios where the blending between the mimicked environment and the physical environment is unnatural), improves operability of the electronic device, makes the user-device interface more efficient, reduces potential disorientation that could potentially lead to symptoms of dizziness or sickness, thereby further reducing power usage and improving battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, and reducing errors during use.

[0209] 9A-9H show examples of varying (or not varying) the immersion level of a three-dimensional environment, according to some embodiments.

[0210] FIG. 9A illustrates electronic device 101 displaying a three-dimensional environment 904 on a user interface via a display generation component (e.g., display generation component 120 of FIG. 1 ). As described above with reference to FIGS. 1-6 , electronic device 101 optionally includes a display generation component (e.g., a touchscreen) and multiple image sensors (e.g., image sensor 314 of FIG. 3 ). The image sensors optionally include one or more of a visible light camera, an infrared camera, a depth sensor, or any other sensor that electronic device 101 can use to capture one or more images of a user or a portion of a user while the user interacts with electronic device 101. In some embodiments, the user interfaces illustrated below may also be implemented on a head-mounted display that includes a display generation component that displays the user interface to a user and sensors for detecting the physical environment and / or movement of the user's hands (e.g., external sensors facing outward from the user) and / or the user's line of sight (e.g., internal sensors facing inward toward the user's face).

[0211] 9A , device 101 captures one or more images of real-world environment 902 (e.g., operating environment 100) around device 101, including one or more objects in real-world environment 902 around device 101. In some embodiments, device 101 displays a representation of the real-world environment in a three-dimensional environment 904. For example, three-dimensional environment 904 includes a portion of a representation of desk 910a, a representation of coffee table 914a, and a representation of side table 912a. As shown in overhead view 918 of real-world environment 902 in FIG. 9A , corner table 908b, desk 910b, side table 912b, and coffee table 914b are real objects in real-world environment 902 captured by one or more sensors of device 101. In FIG. 9A, the three-dimensional environment 904 is currently at an immersion level of 3 (e.g., as indicated by the immersion level indicator 916), so that the virtual environment 922a encompasses a portion of the three-dimensional environment 904 and obscures the view of the back portion of the corner table 908b and the desk 910b (e.g., in a manner similar to the three-dimensional environment 704 described above with reference to FIGS. 7A-7H, such as FIG. 7E).

[0212] In some embodiments, when three-dimensional environment 904 is in a particular immersion level (e.g., a non-immersive level), device 101 generates one or more audio effects associated with virtual environment 922a (e.g., as indicated by speaker 924). In some embodiments, the audio effects include ambient sound effects heard in virtual environment 922a. For example, a peaceful farm virtual environment may include sounds such as frogs croaking, birds chirping, running water, and the like, which are not optionally present in physical environment 902 (e.g., the surrounding physical environment does not include these sounds), but which are generated by device 101 as if they were heard while the user was within physical environment 902. In some embodiments, providing audio effects increases the immersive effect of virtual environment 922a, thus giving the user the experience of being within virtual environment 922a.

[0213] In some embodiments, three-dimensional environment 904 is displayed at an intermediate immersion level, as in FIG. 9A , in response to user input requesting the display of a virtual environment (e.g., virtual environment 922a, etc.). For example, device 101 displays one or more selectable representations of the virtual environment (e.g., a buffet of virtual environments) that can be selected to display the selected virtual environment in an immersive experience as if the user were physically located within the selected virtual environment. In some embodiments, in response to user input selecting an individual virtual environment, device 101 displays the selected virtual environment at a default immersion level that is below a maximum immersion level (e.g., a highest allowable immersion level, optionally limited by system or user-defined settings) and above a minimum immersion level (e.g., an immersion level just above no immersion). In some embodiments, displaying the selected virtual environment at a default intermediate immersion level gently orients the user into the immersive virtual environment and allows the user to increase or decrease the immersion level as desired. In some embodiments, a user can increase or decrease the immersion level by rotating a mechanical dial in one direction or another or by selecting an affordance to increase or decrease the immersion level. In some embodiments, a user can select a particular immersion level by interacting with an immersion level indicator 916. For example, in response to a user performing a selection input on an element of the immersion level indicator 916 that is at the highest immersion level (e.g., the right-most square), the three-dimensional environment 904 displays the virtual environment 922a at the highest immersion level, whereas in response to a selection input directed at the lowest immersion level element of the immersion level indicator 916 (e.g., the left-most square), the virtual environment 922a is displayed at the lowest immersion level. In some embodiments, transitioning from one immersion level to another is described above with reference to method 800.

[0214] FIG. 9B illustrates an embodiment in which an immersive spatial effect (e.g., as opposed to an immersive virtual environment) is displayed at three immersion levels. In some embodiments, the spatial effect refers to artificially (e.g., virtually) modifying the physical atmosphere or the visual characteristics of one or more objects in the three-dimensional environment 904, optionally without replacing or removing one or more objects in the three-dimensional environment 904. For example, the spatial effect may include displaying the three-dimensional environment 904 as if lit by a sunset, displaying the three-dimensional environment 904 as if it were foggy, displaying the three-dimensional environment 904 as if it were dimly lit, etc. As shown in the overhead view 918, displaying the spatial effect optionally does not include displaying a virtual environment in place of portions of the real-world environment portion of the three-dimensional environment (e.g., it will be understood that both an immersive spatial effect and an immersive virtual environment can be displayed simultaneously, at the same immersion level or at different immersion levels). In FIG. 9B, the spatial effect includes displaying virtual light entering the physical environment through a window 915. In some embodiments, the virtual light entering through window 915 is a lighting effect that differs from the amount of real-world light entering through window 915 in real-world environment 902 (e.g., the virtual light is virtual and exists only in three-dimensional environment 904). In some embodiments, the virtual light causes lighting effects to be displayed in three-dimensional environment 904 according to particle physics. For example, the virtual lighting optionally causes glare on picture frame 906 on the back wall of the room, casts shadow 926 on table 910a, and casts a shadow on coffee table 914a (e.g., one or more of which are optionally not present in physical environment 902 or are present in physical environment 902 with different characteristics, such as lower or higher brightness, smaller or larger size, etc.), as shown in FIG. 9B . In some embodiments, the virtual light optionally increases or decreases the ambient light in three-dimensional environment 904 compared to the amount of actual light present in physical environment 902.

[0215] 9C illustrates three-dimensional environment 904 in response to an input that increases the immersion level of virtual environment 922 from 3 (e.g., as in FIG. 9A ) to an immersion level of 6 (e.g., as indicated by immersion level indicator 916). In some embodiments, increasing the immersion level causes virtual environment 922a to expand and encompass more of three-dimensional environment 904, thus replacing the representation of real-world environment 902, as shown in FIG. 9C (e.g., in a manner similar to that described above with respect to method 800). In some embodiments, increasing the immersion level optionally includes increasing one or more audio effects generated by device 101 (e.g., as indicated by speaker 924 providing a louder volume than in FIG. 9A ). In some embodiments, increasing one or more audio effects includes increasing the volume of audio effects provided at lower immersion levels, providing one or more new audio effects, and / or increasing the complexity and / or depth of audio effects, etc. For example, at lower immersion levels, device 101 may provide a babbling brook sound effect (e.g., associated with virtual environment 922a), at higher immersion levels, device 101 may add a bird chirp sound effect (e.g., associated with virtual environment 922a) to the bubbling brook sound effect, at even higher immersion levels, device 101 may add a frog chirp sound effect (e.g., associated with virtual environment 922a) to the babbling brook sound effect and the bird chirp sound effect, etc. In some embodiments, device 101 may increase or not increase (e.g., maintain or decrease) the volume of the generated sound effect when adding the above-mentioned sound effects to the audio it generates.

[0216] FIG. 9D illustrates three-dimensional environment 904 in response to an input that increases the immersion level of an immersive spatial effect from 3 (e.g., as in FIG. 9B ) to an immersion level of 6 (e.g., as indicated by immersion level indicator 916). In some embodiments, increasing the immersion level increases the magnitude of the spatial effect. In some embodiments, increasing the magnitude of the spatial effect includes increasing the amount of modification and / or transformation performed on real-world environment 902 and / or three-dimensional environment 904. For example, in FIG. 9D , the virtual light entering through window 915 (e.g., which, as described above, is a virtual modification of real-world environment 902) is increased in magnitude (e.g., increased brightness). In some embodiments, increasing the virtual light entering through window 915 brings more ambient light into three-dimensional environment 904, increasing the lighting effect. For example, in Figure 9D, the glare on picture frame 906 is increased as a result of increasing the virtual light from window 915, and shadow 926 is darker (e.g., due to the increased contrast between the ambient lighting of three-dimensional environment 904 and the shadow cast by desk 910a). In some embodiments, increasing the immersion level of the immersive spatial effect includes increasing one or more audio effects associated with the spatial effect in one or more ways similar to increasing the audio effects described in Figure 9C with respect to virtual environment 922a.

[0217] 9E-9F illustrate embodiments in which the viewpoint and / or perspective of device 101 has changed, for example, as a result of device 101 being pointed in a different direction without a change in the orientation of the user's 920 body (e.g., torso). For example, in FIGS. 9E-9F, the user has moved their hands to point device 101 to the left (e.g., rather than forward as in FIGS. 9A-9D), but has not rotated their shoulders and / or torso to point to the left. As another example in the case of a head-mounted device 101, the user has turned their head to the left without rotating their shoulders and / or torso to the left in conjunction with the rotation of their head. As described above, rotating device 101 points device 101 in a different direction, causing one or more sensors of device 101 to capture a different portion of real-world environment 902. In some embodiments, in response to detecting that device 101 has turned to face a different portion of real-world environment 902, device 101 updates the three-dimensional environment to rotate according to the rotation of device 101. For example, in Figures 9E-9F, because device 101 is currently facing left, three-dimensional environment 904 displays a representation of real-world environment 902 to the left of user 920, and optionally does not display the portion of real-world environment 902 in front of user 920, as if the user had turned their head to the left to see the portion of real-world environment 902 to the user's left (e.g., the amount that is no longer displayed is optionally based on the field of view of three-dimensional environment 904 and the amount of rotation experienced by device 101).

[0218] 9E-9F, the immersion level is increased to a maximum immersion level. Increasing the immersion level to a maximum in FIG. 9E optionally includes increasing the amount of three-dimensional environment 904 occupied by virtual environment 922a to a maximum amount, which optionally includes replacing all of three-dimensional environment 904 with virtual environment 922a, optionally except for a predetermined radius around user 920 (e.g., a radius of, for safety purposes, e.g., 6 inches, 1 foot, 2 feet, 5 feet, etc.), as reflected in overhead view 918 of FIG. 9E, and optionally increasing one or more audio effects to a maximum amount, in one or more of the manners described with reference to FIG. 9C.

[0219] 9E , because device 101 is oriented to face a different direction, the view of three-dimensional environment 904 displayed by device 101 is rotated to show a new perspective of three-dimensional environment 904 as a result of the new orientation. For example, because device 101 is now facing left, the view of three-dimensional environment 904 displayed by device 101 includes a view of the left side of real-world environment 902, including a representation of side table 923a. In some embodiments, because user 920's body (e.g., torso) did not rotate to face left, virtual environment 922a does not rotate in response to the change in device 101 orientation, but remains oriented (e.g., remains centered) in the direction that user 920's body is facing (e.g., forward), as shown by overhead view 918 (e.g., virtual environment 922a remains in the same location within three-dimensional environment 904 that it was displayed in before the change in device 101 orientation). In some embodiments, by rotating device 101 to face left, as shown in Figure 9E, a boundary between virtual environment 922a and real-world environment 902 is optionally exposed on device 101. In some embodiments, in response to detecting that device 101 has rotated a threshold amount (e.g., rotated 30 degrees, 45 degrees, 60 degrees, 90 degrees, 120 degrees, etc.), one or more audio effects are reduced (or removed), as indicated by speaker indicator 924 in Figure 9E. In some embodiments, the one or more audio effects are reduced in proportion to the magnitude of the change in orientation of device 101 as device 101 rotates within real-world environment 902, and are eliminated entirely when device 101 has rotated a threshold amount. In some embodiments, reducing one or more audio effects includes reducing the volume of one or more audio effects, reducing the complexity of one or more audio effects, canceling one or more audio tracks, or completely canceling all audio effects associated with the virtual environment 922a (e.g., in one or more ways opposite to that described with reference to FIG. 9C).In some embodiments, not rotating or moving the virtual environment 922a within the three-dimensional environment 904 to be oriented (e.g., centered) with the view of the three-dimensional environment 904 displayed by the device 101 and / or reducing one or more audio effects reduces the amount of immersion experienced by the user, thus allowing the user to interact with their physical environment. For example, a user may turn their head to interact with objects or people in their environment (e.g., thus, if the device 101 is a head-mounted device, rotating the device to face a new direction), and by not rotating the virtual environment 922a to be oriented with the view of the three-dimensional environment 904 displayed by the device 101 and / or reducing one or more audio effects, the user can interact with the objects or people in their environment without providing explicit input to reduce the immersion level of the virtual environment 922a. In some embodiments, subsequently rotating the device 101 to an amount of rotation less than the threshold amount of rotation optionally causes one or more audio effects to be returned to their previous level (e.g., a level associated with the maximum immersion level) (e.g., gradually depending on the difference between the current orientation of the device 101 and the location of the virtual environment 922a within the three-dimensional environment 904).

[0220] In Figure 9F, device 101 has been reoriented as described with reference to Figure 9E. In Figure 9F, because device 101 has been oriented to face a different direction, the view of three-dimensional environment 904 displayed by device 101 is rotated to display a new perspective as a result of the new orientation. In Figure 9F (e.g., as described with reference to Figures 9B and 9D), device 101 is displaying an immersive spatial effect, and because device 101 is displaying an immersive spatial effect, the user cannot see the boundary between the representation of the real-world environment and the virtual environment. As shown in Figure 9F, side table 923a casts a virtual shadow 928 as a result of virtual light entering from window 915 (e.g., as described above with reference to Figures 9B and 9D). Additionally or alternatively, device 101 optionally does not reduce one or more audio effects. In some embodiments, the immersive spatial effect is applied to the user's entire physical environment, and thus rotating device 101 so that it is oriented toward a different portion of the real-world environment does not reduce the amount and / or magnitude of the spatial effect provided by device 101; thus, rotating device 101 allows the user to see the effect of the spatial effect on different areas of their physical environment. In some embodiments, providing the immersive spatial effect does not obscure the real-world environment (e.g., as in the case of displaying a virtual environment), and therefore maintaining the spatial effect does not prevent the user from interacting with objects or people in the user's environment. For example, objects or people in their environment are not removed from view as a result of displaying the immersive spatial effect.

[0221] 9G-9H illustrate embodiments in which the viewpoint and / or perspective of device 101 has changed compared to FIGS. 9C and 9D, for example, as a result of user 920's body being rotated (e.g., both device 101 and user 920's shoulders and / or torso being rotated within real-world environment 902). For example, in FIGS. 9G-9H, user 920 continues to hold device 101 facing outward while rotating their body to face left, thus causing device 101 to also face left, as shown in overhead view 918. In some embodiments, device 101 can determine the orientation of user 920's body (e.g., torso and / or shoulders) via one or more sensors, such as a visible light sensor (e.g., a camera) and / or a depth sensor, facing towards the user.

[0222] 9G , in response to detecting that device 101 and user 920's body have rotated to face left, device 101 moves (e.g., rotates) virtual environment 922a within three-dimensional environment 904 so that it is oriented (e.g., centered) with the orientation of device 101 and / or user 920's body. In some embodiments, because device 101 is oriented to face the same direction as the user's body, virtual environment 922a is centered in the view of three-dimensional environment 904 displayed by device 101 (e.g., excluding a "cut-out" area around user 920, as described above) so that the entire view of three-dimensional environment 904 remains occupied by virtual environment 922a. In some embodiments, device 101 additionally or alternatively maintains one or more audio effects associated with virtual environment 922a. Thus, as described above, if device 101 is rotated without the user's body rotating while the virtual environment is displayed at an individual immersion level, the virtual environment is maintained in its absolute position within three-dimensional environment 904 even though device 101 is no longer facing in a direction oriented toward (e.g., centered on) the virtual environment (e.g., even though the user can no longer see the virtual environment and / or device 101 is no longer displaying the virtual environment due to displaying a different part of three-dimensional environment 904), but if device 101 is rotated with the user's body rotating in the same or similar manner (e.g., by the same or similar amount, in the same or similar direction, etc.) with device 101 and / or the user's body, the virtual environment is displayed at a new location within three-dimensional environment 904 and rotated so that the virtual environment is oriented toward (e.g., centered on) the user (e.g., maintaining the same location relative to the user's body orientation and optionally rotating using an acceleration / deceleration mechanism to avoid jerky animations).It is understood that if the user's body rotates without device 101 rotating, even though the orientation of the view of three-dimensional environment 904 does not change (e.g., because device 101 is not facing the new direction), the virtual environment is optionally moved within three-dimensional environment 904 according to the rotation of the user's body, which may optionally cause the virtual environment to rotate out of view (e.g., based on how much the user's body has rotated and the field of view of three-dimensional environment 904). In some embodiments, if the user's body rotates without device 101 rotating, the virtual environment does not rotate according to the rotation of the user's body (e.g., the virtual environment optionally rotates only if both the user's body and the device rotate, although it is understood that they need not rotate simultaneously).

[0223] 9H, in response to detecting that device 101 and user 920's body have rotated to face left (e.g., in a manner similar to FIG. 9G), the view of three-dimensional environment 904 displayed by device 101 is rotated to display a new perspective view as a result of device 101's new orientation, and device 101 continues to display an immersive spatial effect in a manner similar to FIG. 9F. As shown in FIG. 9H, side table 923a casts a virtual shadow 928 as a result of virtual light entering from window 915 (e.g., as described above in FIGS. 9B, 9D, and 9F). Additionally or alternatively, device 101 optionally does not reduce one or more audio effects. 9F and 9H, if device 101 is providing an immersive spatial effect at a discrete immersion level, then as device 101 is rotated to face a new direction, the immersive spatial effect continues to be provided without increasing or decreasing the amount of immersion (e.g., without increasing or decreasing audio effects, without increasing or decreasing visual effects, etc.), regardless of whether the user's body rotates with device 101. For example, three-dimensional environment 904 in FIG. 9H is the same as and / or similar to three-dimensional environment 904 in FIG. 9F, and / or the audio effects provided in FIG. 9H are the same as and / or similar to the audio effects provided in FIG. 9F.

[0224] Thus, as described above, device 101 can provide an immersive environment (e.g., as described with respect to FIGS. 9A, 9C, 9E, and 9G) and / or immersive atmospheric effects (e.g., as described with respect to FIGS. 9B, 9D, 9F, and 9H). In some embodiments, device 101 can provide both an immersive environment and immersive atmospheric effects simultaneously (e.g., optionally affecting the visual characteristics of only the real-world environment portion of three-dimensional environment 904, or optionally affecting the visual characteristics of both the real-world environment portion and the simulated environment portion of three-dimensional environment 904). In such embodiments, the immersive environment exhibits the behavior described above with respect to FIGS. 9A, 9C, 9E, and 9G, and the atmospheric effects exhibit the behavior described above with respect to FIGS. 9B, 9D, 9F, and 9H.

[0225] 10A-10P are flowcharts illustrating a method 1000 for changing the immersion level of a three-dimensional environment, according to some embodiments. In some embodiments, method 1000 is performed on a computer system (e.g., computer system 101 of FIG. 1 , such as a tablet, smartphone, wearable computer, or head-mounted device) that includes a display generation component (e.g., display generation component 120 of FIGS. 1, 3, and 4) (e.g., a head-up display, a display, a touchscreen, a projector, etc.) and one or more cameras (e.g., a camera pointing downward in a user's hand (e.g., color sensors, infrared sensors, and other depth-sensing cameras) or a camera pointing forward from the user's head). In some embodiments, method 1000 is governed by instructions stored on a non-transitory computer-readable storage medium and executed by one or more processors of the computer system, such as one or more processors 202 of computer system 101 (e.g., control unit 110 of FIG. 1A ). Some operations of method 1000 are, optionally, combined, and / or the order of some operations is, optionally, changed.

[0226] In method 1000, in some embodiments, while displaying a three-dimensional environment without an individual spatial effect (e.g., a representation of a physical environment, an imitation environment, a computer-generated reality, an extended reality, etc.) via a display generation component, an electronic device (e.g., computer system 101 of FIG. 1 ) in communication with the display generation component and one or more input devices (e.g., a mobile device (e.g., a tablet, smartphone, media player, or wearable device) or a computer) detects (1002) a first input corresponding to a request to initiate an individual spatial effect, such as a selection of an affordance of the representation of the individual spatial effect, via the one or more input devices (e.g., detects a movement of one or more of the user's hands, a movement and / or change in the line of sight of one or more of the user's eyes, and / or any other input corresponding to a request to initiate a spatial effect, via a hand tracking device, an eye tracking device, or any other input device) while the three-dimensional environment 904 is not displaying the spatial effect, as in FIG. 7A or 7B .

[0227] In some embodiments, the display generation component is a display integrated with the electronic device (optionally a touchscreen display), an external display such as a monitor, projector, television, or a hardware component (optionally built-in or external) for projecting a user interface and making the user interface visible to one or more users, etc. In some embodiments, the one or more input devices include electronic devices or components capable of receiving user input (e.g., capturing user input, detecting user input, etc.) and transmitting information related to the user input to the electronic device. Examples of input devices include a touchscreen, a mouse (e.g., external), a trackpad (optionally built-in or external), a touchpad (optionally built-in or external), a remote control device (e.g., external), another mobile device (e.g., separate from the electronic device), a handheld device (e.g., external), a controller (e.g., external), a camera, a depth sensor and / or a motion sensor (e.g., hand tracking sensor, hand motion sensor), etc.

[0228] In some embodiments, the three-dimensional environment is an augmented reality or mixed reality environment, optionally including virtual objects and / or including representations of real-world objects in the physical world around the electronic device. In some embodiments, the individual environment is based at least on the physical environment around the electronic device. For example, the electronic device can capture visual information about the environment around the user (e.g., objects in the environment, the size and shape of the environment, etc.) and display at least a portion of the physical environment around the user to the user, optionally making the user appear to be still located within the physical environment. In some embodiments, the three-dimensional environment is actively displayed to the user via a display generation component. In some embodiments, the three-dimensional environment is passively presented to the user via a partially transparent or translucent display, through which the user can see at least a portion of the physical environment. In some embodiments, the spatial effect is an atmospheric transformation that modifies one or more visual characteristics of the three-dimensional environment so that the three-dimensional environment appears to be located at a different time, location, and / or conditions (e.g., morning lighting instead of afternoon lighting, sunny instead of cloudy, foggy, etc.). For example, spatial effects include changing the warmth of the environment (e.g., the warmth of a lighting source), changing the lighting of the environment (e.g., the location, type, and / or brightness), changing shadows in the environment (e.g., adding a lighting source, removing a lighting source, or mimicking a change in the position of a lighting source), etc. In some embodiments, spatial effects optionally do not add or remove objects from the three-dimensional environment. In some embodiments, the electronic device may contextually recognize windows and modify what is viewable through the windows according to the spatial effect. For example, if the spatial effect includes displaying the three-dimensional environment as if it were in a rainforest, the atmosphere of the three-dimensional environment may be modified to appear dimly lit, damp, etc., and the windows may be modified so that the rainforest scene may be viewable through the windows.In some embodiments, the spatial effect includes displaying a mimicked environment within or simultaneously with the three-dimensional environment (eg, in a manner similar to that described above with respect to method 800).

[0229] In some embodiments, the request includes a selection of an affordance to increase the immersion level of the device and / or the individual environment, and / or a manipulation of a (e.g., mechanical) control element (e.g., similar to those described with reference to method 800), and / or a selection of an option associated with the individual spatial effect (e.g., from a plurality of available spatial effects). In some embodiments, the request includes a predefined gesture recognized as a request to initiate the spatial effect. In some embodiments, the request includes a voice input requesting the display of the individual spatial effect.

[0230] In some embodiments, in response to detecting the first input, the electronic device, via the display generation component, displays (1004) a three-dimensional environment with a distinct spatial effect at a first immersion level, such as in FIG. 9B (e.g., displays a spatial effect within the three-dimensional environment at an initial immersion level).

[0231] In some embodiments, displaying the spatial effect includes modifying one or more visual characteristics of one or more elements within the three-dimensional environment. As described above, modifying one or more visual characteristics can include changing the lighting temperature, direction, etc. of one or more lighting sources, which optionally causes the three-dimensional environment and / or objects within the three-dimensional environment to have a particular atmospheric effect and / or be located at a particular time and / or place, etc. In some embodiments, when the spatial effect is first displayed, it is displayed at an initial immersion level. In some embodiments, the initial immersion level is an intermediate immersion level (e.g., not a maximum immersion level and a minimum immersion level) that can be increased and / or decreased. In some embodiments, the initial immersion level is a minimum immersion level. In some embodiments, the initial immersion level is a maximum immersion level. In some embodiments, a higher immersion level corresponds to an increased spatial effect, and a lower immersion level corresponds to a decreased spatial effect. For example, a high immersion level for a sunset spatial effect optionally includes modifying the three-dimensional environment to have a strong orange glow (e.g., sunset lighting), and a low immersion level optionally includes modifying the three-dimensional environment to have a softer orange glow. In some embodiments, displaying the spatial effect includes displaying a mimicked environment within the three-dimensional environment. In some embodiments, displaying the mimicked environment includes transitioning a portion of the three-dimensional environment into the mimicked environment in a manner similar to that described above with respect to method 800. In some embodiments, displaying the mimicked environment at a first immersion level includes displaying a subset of the mimicked environment (e.g., displaying a first portion of the mimicked environment, but optionally not displaying a second portion of the mimicked environment that is closer to the user).

[0232] In some embodiments, while displaying the three-dimensional environment with the individual spatial effects at a first immersion level, the electronic device detects (1006) an individual input via one or more input devices (e.g., via a hand tracking device, an eye tracking device, or any other input device, a movement of one or more of the user's hands, a movement and / or change in line of sight of one or more of the user's eyes, and / or any other input corresponding to a request to change the immersion level of the individual spatial effects).

[0233] In some embodiments, the user input comprises manipulation of a rotary element such as a mechanical or virtual dial to increase or decrease the immersion level (e.g., rotate clockwise to increase, rotate counterclockwise to decrease, or vice versa). In some embodiments, the user input comprises manipulation of a displayed control element (e.g., a slider, an immersion indicator, etc.) to increase or decrease the immersion level. In some embodiments, the user input comprises voice input requesting an increase or decrease in the immersion level.

[0234] In some embodiments, in response to detecting the discrete input (1008), in accordance with a determination that the discrete input is a first input, the electronic device displays (1010), via the display generation component, the three-dimensional environment with the discrete spatial effect at a second immersion level higher than the first immersion level, such as in FIG. 9D (e.g., increasing the immersion level from the initial immersion level to a second, higher immersion level in accordance with the discrete input), and in accordance with a determination that the discrete input is a second input different from the first input, the electronic device displays (1012), via the display generation component, the three-dimensional environment with the discrete spatial effect at a third immersion level lower than the first immersion level, such as if the immersion level of the three-dimensional environment 904 in FIG. 9D was 2 or 1 (e.g., decreasing the immersion level from the initial immersion level to a third, lower immersion level in accordance with the discrete input).

[0235] For example, if the discrete input is a rotation of a rotating element in a direction associated with an increased immersion level, the immersion level for the spatial effect is increased from its initial position to a higher level. In some embodiments, increasing the immersion level includes amplifying or increasing the effect of the spatial effect on the three-dimensional environment. For example, if one of the spatial effects on the three-dimensional environment (e.g., one or more spatial effects associated with the discrete spatial effect) includes increasing the brightness of the three-dimensional environment, increasing from the initial immersion level to a higher immersion level includes further increasing the brightness of the three-dimensional environment. In some embodiments, increasing the immersion level to a higher level includes any one of increasing the brightness or intensity of one or more visual effects, decreasing the brightness or intensity of one or more visual effects, displaying a visual effect that was not previously displayed, displaying fewer visual effects, modifying a visual effect, increasing or decreasing one or more audio effects, playing more audio effects that were not previously played, playing fewer audio effects, modifying an audio effect, etc. In some embodiments, displaying the simulated environment at the second immersion level includes increasing the size of the simulated environment and / or displaying more of the simulated environment (e.g., displaying a first portion of the simulated environment and, optionally, a second portion of the simulated environment closer to the user).

[0236] For example, if the discrete input is a rotation of the rotating element in a direction associated with decreasing the immersion level, the immersion level for the spatial effect is decreased from its initial position to a lower level. In some embodiments, decreasing the immersion level includes decreasing the impact of the spatial effect on the three-dimensional environment. For example, if one of the spatial effects on the three-dimensional environment (e.g., one or more spatial effects associated with the discrete spatial effect) includes increasing the brightness of the three-dimensional environment, decreasing from the initial immersion level to a lower immersion level includes decreasing the brightness of the three-dimensional environment (e.g., optionally to a level lower than the ambient brightness of the three-dimensional environment without the discrete spatial effect, or optionally to a level equal to or greater than the ambient brightness of the three-dimensional environment without the discrete spatial effect). In some embodiments, decreasing the immersion level to a higher level includes any one of increasing the brightness or intensity of one or more visual effects, decreasing the brightness or intensity of one or more visual effects, displaying a visual effect not previously displayed, displaying fewer visual effects, modifying a visual effect, increasing or decreasing one or more audio effects, playing more audio effects not previously played, playing fewer audio effects, modifying an audio effect, etc. In some embodiments, displaying the mimicked environment at a third immersion level includes decreasing the size of the mimicked environment and / or displaying a smaller portion of the mimicked environment (e.g., displaying less of the mimicked environment than the first portion).

[0237] The above-described method of initiating individual spatial effects (e.g., by applying the individual spatial effects to the three-dimensional environment at an intermediate immersion level that can be increased or decreased) introduces spatial effects at a level less than the full immersion level (e.g., maintaining a display of at least a portion of the three-dimensional environment and reducing the potentially jarring effect of displaying the spatial effects), which simplifies the interaction between the user and the electronic device, improves the usability of the electronic device, and makes the user-device interface more efficient (e.g., by displaying the spatial effects at an initial immersion level and allowing the user to increase or decrease the magnitude of the spatial effects as desired), thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0238] In some embodiments, the individual spatial effect includes displaying at least a portion of the virtual environment, displaying the three-dimensional environment without the individual spatial effect includes displaying a representation of a portion of the physical environment around the user's viewpoint, and displaying the three-dimensional environment with the individual spatial effect includes replacing the display of at least a portion of the representation of the portion of the physical environment with a display of a portion of the virtual environment (1014), such as in FIG. 9A (e.g., the spatial effect includes displaying an imitation environment such as that described above with respect to method 800).

[0239] In some embodiments, the mimicked environment is displayed in (e.g., encompasses) a first portion of the three-dimensional environment, and in some embodiments, a second portion of the three-dimensional environment includes a representation of a portion of the physical environment (e.g., the real-world environment surrounding the user and / or device).

[0240] The above-described method of initiating the display of the mimicked environment (e.g., by initially displaying the mimicked environment at an intermediate immersion level that can be increased or decreased), introducing the mimicked environment at a level less than the full immersion level (e.g., maintaining the display of at least a portion of the three-dimensional environment and reducing the potentially jarring effect of displaying spatial effects), simplifying the interaction between the user and the electronic device (e.g., by displaying the mimicked environment at an initial immersion level and allowing the user to increase or decrease the amount of mimicked environment as desired), improving the usability of the electronic device, and making the user-device interface more efficient, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0241] In some embodiments, displaying the three-dimensional environment with the individual spatial effect includes presenting 1016 audio corresponding to the virtual environment, such as in FIG. 9A , as indicated by speakers 924 (e.g., the individual spatial effect includes playing (e.g., via speakers, via earphones, via headphones, etc.) audio effects associated with the displayed virtual environment and / or spatial effect). In some embodiments, the audio effects include environmental noises or sounds, such as running water, wind, birdsong, etc., that correspond to the visual content of the virtual environment.

[0242] In some embodiments, while displaying the three-dimensional environment with the individual spatial effects and presenting audio corresponding to the virtual environment, the electronic device detects (1018) via one or more input devices a movement of the electronic device corresponding to changing the orientation of a viewpoint of a user of the electronic device from a first orientation to the three-dimensional environment, such as in Figures 9E-9H (e.g., detecting that the user and / or device has changed orientation within its physical environment, whereby, optionally, a representation of the physical environment changes orientation in accordance with the change in orientation of the user and / or device).

[0243] For example, if a user rotates the device 45 degrees to the left, the device shifts the display of the three-dimensional environment 45 degrees to the left, exposing the portion of the three-dimensional environment to the left of what was previously displayed, and ceasing to display the portion of the three-dimensional environment to the right of what was previously displayed. In this way, the environment is adjusted so that the user appears to be looking around the three-dimensional environment in the same way that the user looks around their physical environment. In some embodiments, as the user and / or the device change orientation, the user can see the boundary between the physical environment and the virtual environment. In some embodiments, the virtual environment expands in the direction of the device rotation (e.g., follows the user's rotation, optionally with a delay). In some embodiments, the virtual environment does not expand in the direction of the device rotation.

[0244] In some embodiments, in response to detecting 1020 the virtual environment of the electronic device, in accordance with a determination that movement of the electronic device satisfies one or more criteria, including a criterion that is met when the orientation of the user's viewpoint toward the three-dimensional environment changes from a first orientation by more than a threshold amount, the electronic device de-highlights 1022 at least one component of the audio corresponding to the movement, such as in FIG. 9E (e.g., reducing the audible effect provided to the user if the user and / or device changes orientation by more than a threshold amount (e.g., by more than 30 degrees, 45 degrees, 90 degrees, etc.) or by a predetermined amount (e.g., by more than 45 degrees but less than 90 degrees, more than 60 degrees but less than 120 degrees, etc.)).

[0245] In some embodiments, reducing an audible effect includes disabling all audible effects associated with the spatial effect. In some embodiments, reducing an audible effect includes reducing the volume of an audible effect associated with the spatial effect. In some embodiments, reducing an audible effect involves removing discrete components and / or tracks from the audible effect (e.g., removing a bird song component but optionally keeping a running water component at the same volume as before).

[0246] The above-described method of reducing the effect of the audio component (e.g., by detecting when the viewpoint has changed by a threshold amount and de-highlighting the audio) provides a fast and efficient way of allowing a user to re-engage with the user's physical environment (e.g., by automatically reducing the audio effects when the user turns their head, e.g., toward a sound source and / or away from the virtual environment, thus allowing the user to hear sounds from the user's physical environment without having to perform additional input to de-highlight the audio effects), which simplifies the interaction between the user and the electronic device, improves the usability of the electronic device, and makes the user-device interface more efficient, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0247] In some embodiments, dehighlighting at least the components of the audio corresponding to the virtual environment includes reducing the degree to which the audio is presented as originating from different directions within the three-dimensional environment (e.g., reducing the volume of at least one component of the audio associated with a spatial effect) (1024), as in FIG. 9E. In some embodiments, reducing the degree includes reducing the volume of all components of the audio associated with a spatial effect. In some embodiments, reducing the degree includes reducing the volume of a subset of the components of the audio while maintaining the volume of other components of the audio. In some embodiments, dehighlighting at least the components of the audio includes reducing the number of audio channels, for example, from five audio channels (e.g., surround sound) to stereo (e.g., two audio channels) and / or from stereo to mono (e.g., one audio channel).

[0248] The above-described method of reducing the effect of the audio component (e.g., by detecting when the viewpoint has changed by a threshold amount and de-highlighting the audio) provides a fast and efficient way of allowing a user to re-engage with the user's physical environment (e.g., by automatically reducing the audio effects when the user turns their head, e.g., toward a sound source and / or away from the virtual environment, thus allowing the user to hear sounds from the user's physical environment without having to perform additional input to de-highlight the audio effects), which simplifies the interaction between the user and the electronic device, improves the usability of the electronic device, and makes the user-device interface more efficient, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0249] In some embodiments, dehighlighting at least the component of the audio corresponding to the virtual environment includes reducing the amount of noise cancellation with which the audio corresponding to the virtual environment is presented (e.g., reducing the noise cancellation effect of the electronic device and / or headphones / speakers / earphones through which the audio is being presented) (1026).

[0250] In some embodiments, the electronic device is in communication with one or more speakers (or any other noise- or audio-generating components). In some embodiments, the one or more speakers have one or more noise-canceling features that at least partially reduce (e.g., cancel) sounds from the user's physical environment (e.g., ambient sounds). In some embodiments, reducing the noise-canceling effect includes reducing the noise-canceling magnitude (e.g., canceling 10% less ambient noise, canceling 20% ​​less ambient noise, canceling 50% less ambient noise, etc.). In some embodiments, reducing the noise-canceling effect includes passively allowing ambient noise to reach the user (e.g., disabling noise-canceling). In some embodiments, reducing the noise-canceling effect includes enabling an active transparency listening mode (e.g., actively providing ambient sounds to the user, e.g., to overcome the passive acoustic attenuation effect of headphones and / or earphones).

[0251] The above-described methods of reducing the immersive effects of a virtual environment (e.g., by reducing the device's noise canceling capabilities) provide a fast and efficient way to allow a user to re-engage with the user's physical environment (e.g., by automatically reducing audio effects and allowing ambient noise to pass when the user turns their head, for example, toward the sound source and / or away from the virtual environment, thereby allowing the user to hear sounds from the user's physical environment without having to perform additional input to de-highlight audio effects), which simplifies the interaction between the user and the electronic device, improves the usability of the electronic device, and makes the user-device interface more efficient, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0252] In some embodiments, dehighlighting at least the components of the audio corresponding to the virtual environment includes reducing the number of audio components in which the audio corresponding to the virtual environment is presented (e.g., reducing or ceasing at least some of the audio components of the audio associated with the virtual environment) (1028), such as in FIG. 9E. For example, reducing the volume of one or more tracks of audio (e.g., to zero) while maintaining the volume of other tracks of audio.

[0253] The above-described method of reducing the immersive effect of a virtual environment (e.g., by reducing the number of audio components provided) provides a fast and efficient way to allow a user to re-engage with their physical environment (e.g., by automatically reducing the amount of audio effects and allowing ambient noise to pass through when the user turns their head, for example, towards the sound source and / or away from the virtual environment, without the user having to perform additional input to de-highlight audio effects), which simplifies the interaction between the user and the electronic device, improves the usability of the electronic device, and makes the user-device interface more efficient, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0254] In some embodiments, while displaying the three-dimensional environment with individual spatial effects at the first immersion level, the electronic device detects (1030) via one or more input devices a user input corresponding to a request to display the three-dimensional environment with individual spatial effects at a maximum immersion level (e.g., by interacting with a displayed immersion affordance or by receiving a user input interacting with a mechanical dial), such as detecting an input that increases the immersion level to 9, such as in FIGS. 9E-9F . In some embodiments, the user input includes a request to increase immersion to the maximum immersion level, such as by selecting an affordance associated with the highest immersion level and / or performing a rotational input on a mechanical dial associated with requesting the highest immersion level.

[0255] In some embodiments, in response to detecting a user input (1032) and determining that a user-defined setting independent of the user input has a first value (e.g., a system setting that sets a maximum immersion level is user adjustable), the electronic device, via the display generation component, displays (1034) the three-dimensional environment with the individual spatial effects at a first individual immersion level (e.g., if the user-defined setting for the maximum immersion level is a first value (e.g., 180 degrees), increasing the immersion level to the first value based on the user-defined setting). In accordance with a determination that the user-defined setting has a second value different from the first value, the electronic device, via the display generation component, displays (1036) the three-dimensional environment with the individual spatial effect at a second individual immersion level greater than the first individual immersion level, for example, if the maximum immersion level is capped by the user-defined setting in FIGS. 9E-9F (e.g., if the user-defined setting for the maximum immersion level is a second value (e.g., 360 degrees), increase the immersion level to the second value based on the user-defined setting).

[0256] For example, the maximum immersion level can be 45 degrees of immersion (e.g., 45 degrees of the three-dimensional environment relative to the user will be occupied by virtual objects / environment, and the remainder will be occupied by the physical environment), 90 degrees of immersion, 180 degrees of immersion, 360 degrees of immersion, etc. In some embodiments, the maximum immersion level can be set to be equal to the field of view (e.g., the maximum angle of immersion is equal to the angle of the field of view of the three-dimensional environment, such as at or about 180 degrees).

[0257] The above-described method of providing a user-defined setting for the maximum immersion level provides a quick and efficient way for a user to limit the maximum immersion level (e.g., by limiting immersion to a user-defined maximum immersion level without the user having to perform additional user input to decrease or increase the immersion level each time the immersion level is increased to the maximum), which simplifies the interaction between the user and the electronic device, improves the usability of the electronic device, and makes the user-device interface more efficient, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0258] In some embodiments, the three-dimensional environment is associated with an application (1038), such as in FIG. 11D (e.g., the three-dimensional environment is a user interface of the application). In some embodiments, the application can cause a virtual environment to be displayed. For example, a full-screen mode or presentation mode of the application optionally causes the virtual environment to be displayed, as described with reference to method 1200.

[0259] In some embodiments, while displaying the three-dimensional environment with individual spatial effects at the first immersion level, the electronic device detects (1040) user input via one or more input devices corresponding to a request to display the three-dimensional environment with individual spatial effects at a maximum immersion level (e.g., receiving user input interacting with a displayed immersion affordance or interacting with a mechanical dial). In some embodiments, the user input includes a request to increase immersion to the maximum immersion level, for example, selecting an affordance associated with the highest immersion level and / or performing a rotational input on a mechanical dial associated with requesting the highest immersion level.

[0260] In some embodiments, in response to detecting user input (1042), in accordance with a determination that the application satisfies one or more first criteria, the electronic device displays, via the display generation component, the three-dimensional environment with the individual spatial effects at a first individual immersion level (1044) (e.g., if the application has a maximum immersion level set at the first individual level, then display the individual spatial effects at the first individual immersion level), and in accordance with a determination that the application satisfies one or more second criteria different from the first criteria, the electronic device displays, via the display generation component, the three-dimensional environment with the individual spatial effects at a second individual immersion level higher than the first individual immersion level (1046), such as in FIG. 11D (e.g., if the maximum immersion level set by the application is the second individual level, then display the individual spatial effects at the second individual immersion level).

[0261] In some embodiments, an application-defined maximum immersion level overrides a system-set maximum immersion level (e.g., defined by a user, as described above). For example, if the maximum immersion level for an application is greater than the system's maximum immersion level, the maximum immersion level for the application controls. In some embodiments, the maximum immersion level is limited by the system's immersion level, even if the application's maximum immersion level is greater. In some embodiments, if the application-defined maximum immersion level is less than the system's immersion level, spatial effects are displayed at the application-defined maximum level in response to a request to display spatial effects at the maximum immersion level. Thus, in some embodiments, the immersion level is the lesser of the application-set maximum level or the system-level setting maximum level.

[0262] The above-described method of determining the maximum immersion level based on the level set by an application (e.g., by defining the maximum immersion level based on the level set by an active application) simplifies the interaction between the user and the electronic device, improves the usability of the electronic device, and makes the user-device interface more efficient, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0263] In some embodiments, the discrete spatial effect includes an atmospheric effect (e.g., a visual modification of at least a portion of the three-dimensional environment (e.g., not associated with an object within the three-dimensional environment)), displaying the three-dimensional environment without the discrete spatial effect includes displaying a representation of a portion of the physical environment surrounding the user's viewpoint without the atmospheric effect, and displaying the three-dimensional environment with the discrete spatial effect includes displaying a representation of a portion of the physical environment surrounding the user's viewpoint with the atmospheric effect (1048), such as in FIG. 9E (e.g., the atmospheric effect is displayed within the portion of the three-dimensional environment that includes the spatial effect, but not within the portion of the three-dimensional environment that does not include the spatial effect).

[0264] For example, displaying ambient lighting effects (e.g., sunrise, sunset, etc.), fog effects, mist effects, smoke / particle effects, etc. In some embodiments, atmospheric effects are those in which the air or empty space of a three-dimensional environment appears to be filled with physical effects.

[0265] For example, if a level is immersive and 30% of the "rooms" exhibit spatial effects and 70% of the "rooms" do not exhibit spatial effects (e.g., are representations of rooms that are not optionally modified by any spatial effects), then the 30% of the rooms with spatial effects will be displayed with a foggy atmosphere and the 70% of the rooms without spatial effects will not include a foggy atmosphere. In some embodiments, exhibiting spatial effects includes modifying portions of the physical environment to involve spatial effects (e.g., modifying the lighting of objects in the physical environment, modifying visual characteristics of objects in the physical environment, etc.).

[0266] The above-described method of displaying spatial effects (e.g., by displaying atmospheric effects in portions of the environment where the spatial effects are displayed, but not in portions of the environment that do not include the spatial effects) provides a fast and efficient way of displaying spatial effects in a subset of the environment, which simplifies the interaction between the user and the electronic device, improves the usability of the electronic device, and makes the user-device interface more efficient, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0267] In some embodiments, displaying the three-dimensional environment with the individual spatial effect includes presenting 1050 audio corresponding to the atmospheric effect (e.g., the individual spatial effect includes playing an audible effect associated with the displayed virtual environment and / or spatial effect), as in Figure 9B. In some embodiments, the audible effect includes noises or sounds associated with the atmospheric effect, such as running water, wind, birdsong, the crackling of burning wood, etc.

[0268] In some embodiments, while displaying the three-dimensional environment with the individual spatial effects and presenting audio corresponding to the atmospheric effects, the electronic device detects (1052) via one or more input devices a movement of the electronic device corresponding to changing the orientation of a viewpoint of a user of the electronic device from a first orientation to the three-dimensional environment, such as in Figures 9E-9H (e.g., detecting that the user and / or device has changed orientation within its physical environment, whereby, optionally, a representation of the physical environment changes orientation in accordance with the change in orientation of the user and / or device).

[0269] For example, if a user rotates the device 45 degrees to the left, the device shifts the display of the three-dimensional environment 45 degrees to the left, exposing the portion of the three-dimensional environment to the left of what was previously displayed, and ceasing to display the portion of the three-dimensional environment to the right of what was previously displayed. In this way, the environment is adjusted so that the user appears to be looking around the three-dimensional environment in the same way that the user looks around their physical environment. In some embodiments, as the user and / or the device change orientation, the user can see the boundary between the physical environment and the virtual environment. In some embodiments, the virtual environment expands in the direction of the device rotation (e.g., follows the user's rotation, optionally with a delay). In some embodiments, the virtual environment does not expand in the direction of the device rotation.

[0270] In some embodiments, in response to detecting movement of the electronic device, the electronic device maintains (1054) the presentation of audio corresponding to the atmospheric effect, regardless of the amount by which the user's viewpoint orientation changes into the three-dimensional environment from a first orientation, such as in FIG. 9F (e.g., continues to provide the audio component of the atmospheric effect as the viewpoint orientation changes). In some embodiments, the atmospheric effect is applied throughout the three-dimensional environment around the user, and changing the viewpoint orientation causes the user to view a different perspective of the three-dimensional environment, with the atmospheric effect still being applied to the new perspective. Thus, in some embodiments, changing the orientation causes visual and / or audio effects to continue to be displayed / presented. In some embodiments, audio effects are reduced or eliminated.

[0271] The above-described method of displaying spatial effects (e.g., by maintaining the display of atmospheric effects when a user changes the orientation of the device) provides a fast and efficient way to reduce the immersion of a three-dimensional environment while maintaining continuity (e.g., by maintaining the display of atmospheric effects and avoiding the abrupt transitions that occur when atmospheric effects are disabled), which simplifies the interaction between a user and an electronic device, improves the usability of the electronic device, and makes the user-device interface more efficient, thereby further reducing power usage and improving the battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently, as well as reducing errors during use.

[0272] In some embodiments, while displaying the three-dimensional environment with the individual spatial effects at a first immersion level, the electronic device detects user input via one or more input devices corresponding to a request to change the current immersion level of the individual spatial effects (e.g., receives input to change the current immersion level, such as selecting an affordance or rotating a mechanical dial corresponding to a request to increase or decrease the immersion level (1058) by a individual amount (e.g., increment or decrement the immersion level) or set the immersion level to a individual level).

[0273] In some embodiments, in response to detecting 1060 a user input corresponding to a request to change the immersion level of an individual spatial effect, in accordance with a determination that the individual spatial effect is a first spatial effect, the electronic device changes 1062 the current immersion level to an immersion level from a first set of immersion levels (e.g., if the individual spatial effect displayed in the three-dimensional environment is a first spatial effect and / or a first type of spatial effect, changes the immersion level from among the first set of available immersion levels for the first spatial effect and / or the first type of spatial effect), and in accordance with a determination that the individual spatial effect is a second spatial effect, the electronic device changes 1064 the current immersion level to an immersion level from a second set of immersion levels that is different from the first set (e.g., if there are more available immersion levels for the virtual environment 922, such as in FIG. 9A , than for the atmospheric effects, such as in FIG. 9B ).

[0274] In some embodiments, the virtual and / or simulated environment...

Claims

1. An electronic device in communication with a display generation component and one or more input devices, While displaying the individual environment via the display generation component, receiving input via the one or more input devices corresponding to a request to display a first simulated environment; transitioning from displaying the individual environment to displaying the first simulated environment in response to receiving the input corresponding to the request to display the first simulated environment, wherein the transitioning includes: transitioning from displaying the individual environment to displaying the first mimicked environment using a first type of transition in accordance with a determination that the physical environment around the user's viewpoint satisfies one or more first criteria; transitioning, including transitioning from displaying the individual environment to displaying the first mimicked environment using a second type of transition different from the first type of transition in accordance with a determination that the physical environment around the user's viewpoint satisfies one or more second criteria different from the one or more first criteria; A method comprising:

2. 2. The method of claim 1 , wherein the one or more first criteria include a criterion that is met when a portion of the physical environment surrounding the viewpoint of the user in a discrete direction relative to a reference orientation of the electronic device is greater than a threshold distance from the viewpoint of the user of the electronic device.

3. 3. The method of claim 1 or 2, wherein the one or more second criteria include a criterion that is met when a portion of the physical environment around the user's viewpoint in a discrete direction relative to a reference orientation of the electronic device is less than a threshold distance from the electronic device.

4. 4. The method of claim 1, wherein the individual environment includes a representation of a portion of the physical environment surrounding the user's viewpoint, and transitioning from displaying the individual environment to displaying the first simulated environment includes replacing the display of the representation of the portion of the physical environment with a display of a representation of the portion of the first simulated environment.

5. the input corresponding to the request to display the first simulated environment includes a user input, and transitioning from displaying the individual environment to displaying the first simulated environment is performed. transitioning from displaying the individual environment to displaying a first portion of the first simulated environment in accordance with a determination that the user input corresponds to a request to transition into the first simulated environment by a first amount; and transitioning from displaying the individual environment to displaying a second portion of the first simulated environment that is larger than the first portion in accordance with a determination that the user input corresponds to a request to transition into the first simulated environment by a second amount that is larger than the first amount.

6. 6. The method of claim 5, wherein the user input comprises a rotation of a rotatable input element in communication with the electronic device, the user input corresponding to the request to transition to the first simulated environment by the first amount comprises a rotation of the rotatable input element by a first discrete amount, and the user input corresponding to the request to transition to the first simulated environment by the second discrete amount comprises a rotation of the rotatable input element by a second discrete amount greater than the first discrete amount.

7. The method of any one of claims 1 to 6, wherein the display of the first simulated environment via the display generation component after transitioning from displaying the individual environment to displaying the first simulated environment using the first type of transition is the same as the display of the first simulated environment via the display generation component after transitioning from displaying the individual environment to displaying the first simulated environment using the second type of transition.

8. the first type of transition proceeds in a manner based on a difference between 1) a distance between the electronic device and a first portion of the physical environment surrounding the user's viewpoint and 2) a distance between the electronic device and a second portion of the physical environment surrounding the user's viewpoint; the second type of transition proceeds in a manner independent of the difference between 1) the distance between the electronic device and the first portion of the physical environment surrounding the user's viewpoint and 2) the distance between the electronic device and the second portion of the physical environment surrounding the user's viewpoint. The method according to any one of claims 1 to 7.

9. 9. The method of claim 1, wherein the second type of transition comprises a cross-dissolve between the individual environment and the first mimicked environment, the cross-dissolve beginning in a part of the physical environment that is furthest from the electronic device.

10. transitioning from displaying the individual environment to displaying the first emulated environment using the first type of transition; During a first portion of the transition, replacing, via the display generation component, a display of a representation of the respective portion of the physical environment with a display of at least a first portion of the first mimicked environment in accordance with a determination that the respective portion of the physical environment surrounding the user's viewpoint is greater than a first distance from the user's viewpoint; cease, via the display generation component, replacing the display of the representation of the distinct portion of the physical environment with a display of at least the first portion of the first mimicked environment in accordance with a determination that the distinct portion of the physical environment surrounding the user's viewpoint is less than the first distance from the user's viewpoint; During a second portion of the transition, replacing, via the display generation component, the display of the representation of the distinct portion of the physical environment with a display of at least a second portion of the first mimicked environment in accordance with a determination that the distinct portion of the physical environment around the user's viewpoint is greater than a second distance from the user's viewpoint, the second distance being different from the first distance; The method of any one of claims 1 to 9, comprising: ceasing, via the display generation component, to replace the display of the representation of the individual portion of the physical environment with a display of at least the second portion of the first mimicked environment in accordance with a determination that the individual portion of the physical environment surrounding the user's viewpoint is less than the second distance from the user's viewpoint.

11. The first type of transition is During the first portion of the transition, In response to the determination that the distinct portion of the physical environment surrounding the viewpoint of the user is greater than the first distance from the viewpoint of the user, displaying, via the display generation component, a representation of the second distinct portion of the physical environment between the first portion of the first mimicked environment and the user's viewpoint in accordance with determining that the second distinct portion of the physical environment surrounding the user's viewpoint is less than the first distance from the user's viewpoint; and ceasing to display, via the display generation component, the representation of the second distinct portion of the physical environment surrounding the user's viewpoint in accordance with a determination that the second distinct portion of the physical environment surrounding the user's viewpoint is greater than the first distance from the user's viewpoint.

12. the individual environment includes a representation of the physical environment around the viewpoint of the user, the first simulated environment includes a first simulated light source, and the method further comprises: The method of any one of claims 1 to 11, further comprising displaying, via the display generation component, the portion of the representation of the physical environment with lighting effects based on the first simulated light source while simultaneously displaying at least a portion of the representation of the physical environment around the user's viewpoint and at least a portion of the first simulated environment.

13. The method of claim 12 , wherein the lighting effects are based on the geometry of the physical environment around the viewpoint of the user.

14. The method of claim 12 or 13, wherein the lighting effects are based on the architecture of the physical environment around the viewpoint of the user.

15. the individual environment includes a representation of the physical environment around the viewpoint of the user, the first mimicked environment includes a first volumetric effect, and the method further comprises: The method of any one of claims 1 to 14, further comprising displaying, via the display generation component, a second volumetric effect based on the first volumetric effect within a volume of the portion of the representation of the physical environment while simultaneously displaying at least a portion of the representation of the physical environment around the user's viewpoint and at least a portion of the first mimicked environment.

16. The method of claim 15 , wherein the first volumetric effect and the second volumetric effect comprise lighting effects.

17. The method of claim 15 or 16, wherein the first volumetric effect and the second volumetric effect comprise particle effects.

18. while simultaneously displaying at least a portion of the individual environment and at least a portion of the first simulated environment via the display generation component; removing, via the display generation component, from the display in accordance with a determination that a boundary between the portion of the individual environment and the portion of the first mimicked environment intersects with a representation of the individual object in the individual environment; The method of any one of claims 1 to 17, further comprising:

19. one or more processors; Memory and One or more programs; An electronic device comprising: The one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: receiving, while displaying the individual environment via the display generation component, input via one or more input devices corresponding to a request to display the first mimicked environment; transitioning from displaying the individual environment to displaying the first simulated environment in response to receiving the input corresponding to the request to display the first simulated environment, said transitioning comprising: transitioning from displaying the individual environment to displaying the first mimicked environment using a first type of transition in accordance with a determination that the physical environment around the user's viewpoint satisfies one or more first criteria; an electronic device comprising instructions, comprising: transitioning from displaying the individual environment to displaying the first simulated environment using a second type of transition different from the first type of transition in accordance with a determination that the physical environment around the user's viewpoint satisfies one or more second criteria different from the one or more first criteria.

20. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: receiving, while displaying the individual environment via the display generation component, input via one or more input devices corresponding to a request to display the first mimicked environment; transitioning from displaying the individual environment to displaying the first simulated environment in response to receiving the input corresponding to the request to display the first simulated environment, said transitioning comprising: transitioning from displaying the individual environment to displaying the first mimicked environment using a first type of transition in accordance with a determination that the physical environment around the user's viewpoint satisfies one or more first criteria; and transitioning from displaying the individual environment to displaying the first mimicked environment using a second type of transition different from the first type of transition in accordance with a determination that the physical environment around the user's viewpoint satisfies one or more second criteria different from the one or more first criteria.

21. one or more processors; Memory and An electronic device comprising: means for receiving, via one or more input devices, input corresponding to a request to display the first mimicked environment while displaying the individual environment via the display generation component; transitioning from displaying the individual environment to displaying the first simulated environment in response to receiving the input corresponding to the request to display the first simulated environment, wherein the transitioning includes: transitioning from displaying the individual environment to displaying the first mimicked environment using a first type of transition in accordance with a determination that the physical environment around the user's viewpoint satisfies one or more first criteria; and means for transitioning, comprising transitioning from displaying the individual environment to displaying the first simulated environment using a second type of transition different from the first type of transition in accordance with a determination that the physical environment around the user's viewpoint satisfies one or more second criteria different from the one or more first criteria.

22. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: means for receiving, via one or more input devices, input corresponding to a request to display the first mimicked environment while displaying the individual environment via the display generation component; transitioning from displaying the individual environment to displaying the first simulated environment in response to receiving the input corresponding to the request to display the first simulated environment, wherein the transitioning includes: transitioning from displaying the individual environment to displaying the first mimicked environment using a first type of transition in accordance with a determination that the physical environment around the user's viewpoint satisfies one or more first criteria; and a means for transitioning, comprising transitioning from displaying the individual environment to displaying the first simulated environment using a second type of transition different from the first type of transition in accordance with a determination that the physical environment around the user's viewpoint satisfies one or more second criteria different from the one or more first criteria.

23. one or more processors; Memory and One or more programs; An electronic device comprising:

19. An electronic device, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any one of claims 1 to 18.

24. 19. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform the method of any one of claims 1 to 18.

25. one or more processors; Memory and means for carrying out the method according to any one of claims 1 to 18; An electronic device comprising:

26. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: An electronic device comprising means for carrying out the method according to any one of claims 1 to 18.

27. An electronic device in communication with a display generation component and one or more input devices, detecting, while displaying via the display generation component a three-dimensional environment without a discrete spatial effect, via the one or more input devices, a first input corresponding to a request to initiate the discrete spatial effect; displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a first immersion level in response to detecting the first input; detecting a discrete input via the one or more input devices while displaying the three-dimensional environment with the discrete spatial effect at the first immersion level; In response to detecting the individual input, displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a second immersion level greater than the first immersion level in accordance with determining that the individual input is a first input; displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a third immersion level lower than the first immersion level in accordance with determining that the individual input is a second input different from the first input; A method comprising:

28. 28. The method of claim 27, wherein the individual spatial effect includes displaying at least a portion of a virtual environment, displaying the three-dimensional environment without the individual spatial effect includes displaying a representation of a portion of a physical environment around a user's viewpoint, and displaying the three-dimensional environment with the individual spatial effect includes replacing display of at least a portion of the representation of the portion of the physical environment with a display of the portion of the virtual environment.

29. wherein displaying the three-dimensional environment with the individual spatial effects includes presenting audio corresponding to the virtual environment, the method comprising: detecting, via the one or more input devices, a movement of the electronic device corresponding to changing an orientation of a viewpoint of a user of the electronic device toward the three-dimensional environment from a first orientation while displaying the three-dimensional environment with the individual spatial effect and presenting the audio corresponding to the virtual environment; In response to detecting the movement of the electronic device, 29. The method of claim 28, further comprising: dehighlighting at least one component of the audio corresponding to the movement in accordance with a determination that the virtual environment of the electronic device satisfies one or more criteria, including a criterion that is satisfied when the orientation of the viewpoint of the user toward the three-dimensional environment changes from the first orientation by more than a threshold amount.

30. 30. The method of claim 29, wherein dehighlighting at least the components of the audio corresponding to the virtual environment comprises reducing the extent to which the audio is presented as originating from different directions within the three-dimensional environment.

31. 31. The method of claim 29 or 30, wherein dehighlighting at least the component of the audio corresponding to the virtual environment comprises reducing an amount of noise cancellation with which the audio corresponding to the virtual environment is presented.

32. 32. The method of any one of claims 29 to 31, wherein dehighlighting at least the components of the audio corresponding to the virtual environment comprises reducing a number of audio components in which the audio corresponding to the virtual environment is presented.

33. While displaying the three-dimensional environment with the individual spatial effects at the first immersion level, detecting a user input via the one or more input devices corresponding to a request to display the three-dimensional environment with the individual spatial effects at a maximum immersion level; In response to detecting the user input, displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a first individual immersion level in accordance with a determination that the user-defined setting independent of the user input has a first value; displaying, via the display generation component, the three-dimensional environment with the individual spatial effects at a second individual immersion level higher than the first individual immersion level in accordance with determining that the user-defined setting has a second value different from the first value; The method of any one of claims 28 to 32, further comprising:

34. The three-dimensional environment is associated with an application, and the method comprises: While displaying the three-dimensional environment with the individual spatial effects at the first immersion level, detecting a user input via the one or more input devices corresponding to a request to display the three-dimensional environment with the individual spatial effects at a maximum immersion level; In response to detecting the user input, displaying, via the display generation component, the three-dimensional environment with the individual spatial effects at a first individual immersion level in accordance with a determination that the application satisfies one or more first criteria; 34. The method of claim 28, further comprising: displaying, via the display generation component, the three-dimensional environment with the individual spatial effects at a second individual immersion level that is higher than the first individual immersion level, in accordance with a determination that the application satisfies one or more second criteria that are different from the first criteria.

35. 35. The method of any one of claims 27 to 34, wherein the individual spatial effect includes an atmospheric effect, and wherein displaying the three-dimensional environment without the individual spatial effect includes displaying a representation of a portion of the physical environment surrounding the user's viewpoint without the atmospheric effect, and wherein displaying the three-dimensional environment with the individual spatial effect includes displaying the representation of the portion of the physical environment surrounding the user's viewpoint with the atmospheric effect.

36. and displaying the three-dimensional environment with the individual spatial effects includes presenting audio corresponding to the atmospheric effects, the method comprising: detecting, via the one or more input devices, a movement of the electronic device corresponding to changing an orientation of a viewpoint of a user of the electronic device from a first orientation to the three-dimensional environment while displaying the three-dimensional environment with the individual spatial effect and presenting the audio corresponding to the atmospheric effect; 36. The method of claim 35, further comprising: in response to detecting the movement of the electronic device, maintaining the presentation of the audio corresponding to the atmospheric effect regardless of an amount by which the orientation of the viewpoint of the user into the three-dimensional environment changes from the first orientation.

37. detecting, while displaying the three-dimensional environment with the individual spatial effect at the first immersion level, a user input via the one or more input devices corresponding to a request to change a current immersion level of the individual spatial effect; in response to detecting the user input corresponding to the request to change the immersion level of the individual spatial effect; changing the current immersion level to an immersion level from a first set of immersion levels in accordance with determining that the individual spatial effect is a first spatial effect; changing the current immersion level to an immersion level from a second set of immersion levels different from the first set in accordance with determining that the individual spatial effect is a second spatial effect; The method of any one of claims 28 to 34, further comprising:

38. the physical environment around the viewpoint of the user includes a first area accessible to a user of the electronic device and a second area inaccessible to the user of the electronic device; displaying the three-dimensional environment without the separate spatial effect includes displaying, via the display generation component, a representation of the second region of the physical environment surrounding the viewpoint of the user; displaying the three-dimensional environment with the personalized spatial effect replaces the display of the representation of the second region of the physical environment surrounding the user's viewpoint with a personalized virtual environment; The method of any one of claims 35 to 37, comprising:

39. 39. The method of any one of claims 35 to 38, wherein displaying the three-dimensional environment without the atmosphere effect comprises displaying the representation of the portion of the physical environment around the user's viewpoint having a first lighting level, and wherein displaying the three-dimensional environment with the atmosphere effect comprises displaying the representation of the portion of the physical environment around the user's viewpoint having a second lighting level that is higher than the first lighting level.

40. 40. The method of any one of claims 35 to 39, wherein displaying the three-dimensional environment without the atmosphere effect comprises displaying the representation of the portion of the physical environment surrounding the user's viewpoint having a first lighting level, and wherein displaying the three-dimensional environment with the atmosphere effect comprises displaying the representation of the portion of the physical environment surrounding the user's viewpoint having a second lighting level that is less than the first lighting level.

41. 41. The method of any one of claims 27 to 40, wherein displaying the three-dimensional environment with the individual spatial effects at individual immersion levels comprises displaying, via the display generation component, an immersion scale visual indicator that indicates a current immersion level at which the individual spatial effects are being displayed.

42. 42. The method of claim 41 , wherein the immersion scale visible indicators include a first set of one or more indicators corresponding to major immersion levels and a second set of one or more indicators corresponding to minor immersion levels between the major immersion levels.

43. displaying the three-dimensional environment with the individual spatial effect at a first primary immersion level lower than a second primary immersion level comprises presenting audio corresponding to the individual spatial effect without displaying a visual effect corresponding to the individual spatial effect; 43. The method of claim 41 or 42, comprising:

44. Displaying the three-dimensional environment with the individual spatial effect at a first primary immersion level higher than a second primary immersion level includes presenting audio corresponding to the individual spatial effect and displaying a first portion of the visual effect corresponding to the individual spatial effect without displaying a second portion of the visual effect corresponding to the individual spatial effect.

44. The method of any one of claims 41 to 43, comprising:

45. Displaying the three-dimensional environment with the individual spatial effect at a third primary immersion level higher than the first primary immersion level includes presenting the audio corresponding to the individual spatial effect and displaying the first and second portions of the visual effect corresponding to the individual spatial effect.

45. The method of claim 44, comprising:

46. the individual spatial effect includes a display of at least a portion of a virtual environment, displaying the three-dimensional environment without the individual spatial effect includes displaying a representation of a portion of a physical environment around the viewpoint of the user, and displaying the three-dimensional environment with the individual spatial effect includes replacing a display of at least a portion of the representation of the portion of the physical environment with a display of the portion of the virtual environment, the method comprising: detecting, via the one or more input devices, movement of the electronic device away from a first location in the physical environment around the user's viewpoint while displaying the three-dimensional environment with the individual spatial effect, including displaying the portion of the virtual environment; in response to detecting the movement of the electronic device away from the first location; 46. ​​The method of claim 27, further comprising: ceasing, via the display generation component, displaying the portion of the virtual environment in accordance with a determination that the movement of the electronic device away from the first location is greater than a threshold distance.

47. in response to detecting the movement of the electronic device away from the first location; dehighlighting a display of the portion of the virtual environment by a first amount in accordance with determining that the movement of the electronic device away from the first location is a first distance that is less than the threshold distance; dehighlighting the display of the portion of the virtual environment by a second amount greater than the first amount in accordance with a determination that the movement of the electronic device away from the first location is a second distance greater than the first distance and less than the threshold distance; 47. The method of claim 46, further comprising:

48. detecting, via the one or more input devices, movement of the electronic device away from a first location within the physical environment surrounding the viewpoint of the user while displaying the three-dimensional environment with the individual spatial effect; in response to detecting the movement of the electronic device away from the first location; in response to a determination that the movement of the electronic device away from the first location is greater than a threshold distance; ceasing display of the portion of the virtual environment in accordance with a determination that displaying the three-dimensional environment with the individual spatial effect includes displaying at least a portion of the virtual environment via the display generation component; and maintaining, in accordance with a determination that displaying the three-dimensional environment with the individual spatial effects includes displaying, via the display generation component, a representation of the portion of the physical environment surrounding the user's viewpoint with the atmospheric effects; and 48. The method of any one of claims 27 to 47, further comprising:

49. wherein displaying the three-dimensional environment with the individual spatial effects includes displaying at least a portion of a virtual environment, the method comprising: detecting, via the one or more input devices, a movement of the electronic device corresponding to changing an orientation of the user's viewpoint from a first orientation while displaying the three-dimensional environment with the individual spatial effect, including displaying the portion of the virtual environment, and while a representation of a physical environment around the user's viewpoint is not being displayed via the display generation component; In response to detecting the movement of the electronic device, 49. The method of claim 27, further comprising: de-highlighting a display of the portion of the movement in accordance with a determination that the virtual environment of the electronic device satisfies one or more criteria, including a criterion that is met when the orientation of the user's viewpoint toward the three-dimensional environment changes horizontally from the first orientation by more than a threshold amount; and displaying, via the display generation component, at least a portion of the representation of the physical environment.

50. wherein displaying the three-dimensional environment with the individual spatial effects includes displaying at least a portion of a virtual environment, the method comprising: detecting, while displaying the three-dimensional environment with the individual spatial effect, including displaying the portion of the virtual environment, and while a representation of a physical environment is not being displayed via the display generation component, a movement of the electronic device corresponding to changing an orientation of a viewpoint of a user of the electronic device toward the three-dimensional environment from a first orientation via the one or more input devices; In response to detecting the movement of the electronic device, 50. The method of claim 27, further comprising: de-highlighting a display of the portion of the movement in accordance with a determination that the virtual environment of the electronic device satisfies one or more criteria, including a criterion that is satisfied when the orientation of the user's viewpoint with respect to the three-dimensional environment changes vertically from the first orientation to a second orientation that is directed toward a ground surface of the physical environment; and displaying, via the display generation component, at least a portion of the representation of the ground surface of the physical environment.

51. displaying the three-dimensional environment with the individual spatial effects includes displaying at least a portion of a virtual environment; displaying the three-dimensional environment with the individual spatial effect at the first immersion level includes displaying the portion of the virtual environment within the three-dimensional environment that is a first distance from a user's viewpoint; displaying the three-dimensional environment with the individual spatial effect at the second immersion level greater than the first immersion level includes displaying the portion of the virtual environment at a second distance less than the first distance from the viewpoint of the user to the three-dimensional environment; 51. The method of any one of claims 27 to 50, comprising:

52. wherein displaying the three-dimensional environment with the individual spatial effects includes displaying at least a portion of a virtual environment, the method comprising: detecting, via the one or more input devices, a movement of the electronic device corresponding to changing an orientation of a viewpoint of a user of the electronic device toward the three-dimensional environment from a first orientation while displaying the three-dimensional environment with the individual spatial effect, the movement of the electronic device corresponding to changing an orientation of a viewpoint of a user of the electronic device toward the three-dimensional environment from a first orientation while displaying the three-dimensional environment with the individual spatial effect, the portion of the virtual environment being displayed; In response to detecting the movement of the electronic device, changing a location within the three-dimensional environment where the virtual environment is displayed in accordance with the movement of the electronic device in accordance with a determination that the movement of the electronic device satisfies one or more criteria, including criteria that are satisfied based on movement of a predetermined portion of the user; 52. The method of any one of claims 27 to 51, further comprising: maintaining the location within the three-dimensional environment in which the virtual environment is displayed in accordance with a determination that the movement of the electronic device does not satisfy the one or more criteria.

53. one or more processors; Memory and One or more programs; An electronic device comprising: The one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: While displaying, via the display generation component, a three-dimensional environment without a discrete spatial effect, detecting, via one or more input devices, a first input corresponding to a request to initiate the discrete spatial effect; displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a first immersion level in response to detecting the first input; detecting a discrete input via the one or more input devices while displaying the three-dimensional environment with the discrete spatial effect at the first immersion level; In response to detecting the individual input, displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a second immersion level greater than the first immersion level in accordance with determining that the individual input is a first input; and instructions for displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a third immersion level that is lower than the first immersion level, in accordance with a determination that the individual input is a second input that is different from the first input.

54. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: detecting, while displaying, via a display generation component, a three-dimensional environment without a discrete spatial effect, via one or more input devices, a first input corresponding to a request to initiate the discrete spatial effect; displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a first immersion level in response to detecting the first input; detecting a discrete input via the one or more input devices while displaying the three-dimensional environment with the discrete spatial effect at the first immersion level; In response to detecting the individual input, displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a second immersion level greater than the first immersion level in accordance with determining that the individual input is a first input; and displaying, via the display generation component, the three-dimensional environment with the distinct spatial effect at a third immersion level that is lower than the first immersion level in accordance with a determination that the distinct input is a second input that is different from the first input.

55. one or more processors; Memory and An electronic device comprising: means for detecting, while displaying, via the display generation component, a three-dimensional environment without the discrete spatial effect, via one or more input devices, a first input corresponding to a request to initiate the discrete spatial effect; means for displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a first immersion level in response to detecting the first input; means for detecting a discrete input via the one or more input devices while displaying the three-dimensional environment with the discrete spatial effect at the first immersion level; In response to detecting the individual input, displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a second immersion level greater than the first immersion level in accordance with determining that the individual input is a first input; means for displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a third immersion level lower than the first immersion level in accordance with a determination that the individual input is a second input different from the first input.

56. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: means for detecting, while displaying, via the display generation component, a three-dimensional environment without the discrete spatial effect, via one or more input devices, a first input corresponding to a request to initiate the discrete spatial effect; means for displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a first immersion level in response to detecting the first input; means for detecting a discrete input via the one or more input devices while displaying the three-dimensional environment with the discrete spatial effect at the first immersion level; In response to detecting the individual input, displaying, via the display generation component, the three-dimensional environment with the individual spatial effect at a second immersion level greater than the first immersion level in accordance with determining that the individual input is a first input; and means for displaying the three-dimensional environment with the individual spatial effect at a third immersion level lower than the first immersion level via the display generation component in accordance with a determination that the individual input is a second input different from the first input.

57. one or more processors; Memory and One or more programs; An electronic device comprising:

53. An electronic device, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any one of claims 27 to 52.

58. 53. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform the method of any one of claims 27 to 52.

59. one or more processors; Memory and means for carrying out the method according to any one of claims 27 to 52; An electronic device comprising:

60. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: An information processing device comprising means for carrying out the method according to any one of claims 27 to 52.

61. An electronic device in communication with a display generation component and one or more input devices, displaying, via said display generation component, a three-dimensional environment including a representation of a portion of a physical environment; detecting, while displaying the three-dimensional environment, a first input via the one or more input devices corresponding to a request to display a respective user interface of a respective application; displaying, via the display generation component, the respective user interface of the respective application within the portion of the physical environment within the three-dimensional environment in response to detecting the first input; detecting a second input via the one or more input devices while displaying the distinct user interface in the portion of the physical environment within the three-dimensional environment; In response to detecting the second input, displaying the distinct user interface in the portion of the virtual environment within the three-dimensional environment in accordance with a determination that the second input corresponds to a request to move the distinct user interface to the portion of the virtual environment that is displayed with the portion of the physical environment within the three-dimensional environment; A method comprising:

62. 62. The method of claim 61 , wherein the portion of the virtual environment is displayed within the three-dimensional environment when the first input is detected, and in response to detecting the first input, the individual user interfaces of the individual applications are displayed within the three-dimensional environment simultaneously with the portion of the virtual environment.

63. 62. The method of claim 61 , wherein the portion of the virtual environment is not displayed within the three-dimensional environment when the first input is detected, and the portion of the virtual environment is displayed within the three-dimensional environment when the second input is detected.

64. while displaying the respective user interfaces of the respective applications within the portion of the physical environment within the three-dimensional environment, displaying, via the display generation component, at least a respective portion of the portion of the physical environment with lighting effects based on the respective user interfaces of the respective applications; 64. The method of any one of claims 61 to 63, further comprising:

65. while displaying the individual user interface of the individual application within the portion of the virtual environment within the three-dimensional environment, displaying via the display generation component at least a separate portion of the portion of the virtual environment with lighting effects based on the separate user interface of the individual application; 65. The method of any one of claims 61 to 64, further comprising:

66. while displaying the individual user interface of the individual application within the portion of the physical environment within the three-dimensional environment, displaying, via the display generation component, at least the individual portion of the individual user interface of the individual application with lighting effects based on the portion of the physical environment; 66. The method of any one of claims 61 to 65, further comprising:

67. While displaying the individual user interface of the individual application within the portion of the virtual environment within the three-dimensional environment, displaying, via the display generation component, at least the individual portion of the individual user interface of the individual application with lighting effects based on the portion of the virtual environment; 67. The method of any one of claims 61 to 66, further comprising:

68. In response to detecting the second input, increasing a size of the individual user interface of the individual application within the three dimensional environment in accordance with movement of the individual user interface into the portion of the virtual environment and away from a user's viewpoint; 68. The method of any one of claims 61 to 67, further comprising:

69. Prior to detecting the second input and when the second input is detected, at least a portion of the three-dimensional environment is displayed with a first lighting effect corresponding to a first time, the method comprising:

69. The method of any one of claims 61 to 68, further comprising, in response to detecting the second input, displaying, via the display generation component, the portion of the three-dimensional environment with a second lighting effect that corresponds to a second time that is different from the first time, the second lighting effect being different from the first lighting effect.

70. While displaying the portion of the three-dimensional environment with a first lighting effect corresponding to a first time via the display generation component, detecting a third input at the electronic device via the one or more input devices corresponding to a request to display the portion of the three-dimensional environment with a distinct lighting effect corresponding to a distinct time; In response to detecting the third input, displaying, via the display generation component, the portion of the three-dimensional environment with a second lighting effect in accordance with determining that the distinct time at the electronic device is a second time; displaying, via the display generation component, the portion of the three-dimensional environment with a third lighting effect that is different from the second lighting effect in accordance with a determination that the distinct time at the electronic device is a third time that is different from the second time; 70. The method of any one of claims 61 to 69, further comprising:

71. While displaying, via the display generation component, a portion of the three-dimensional environment with a first lighting effect corresponding to a first time, detecting, via the one or more input devices, a third input corresponding to a request to display the portion of the three-dimensional environment with a distinct lighting effect corresponding to a distinct time; In response to detecting the third input, displaying, via the display generation component, the portion of the three-dimensional environment with a second lighting effect in accordance with determining that the third input indicates a second time instant; displaying, via the display generation component, the portion of the three-dimensional environment with a third lighting effect that is different from the second lighting effect in accordance with determining that the third input indicates a third time that is different from the second time; and 71. The method of any one of claims 61 to 70, further comprising:

72. displaying, via the display generation component, at least a separate portion of the portion of the physical environment with lighting effects based on the portion of the virtual environment while displaying the portion of the virtual environment together with the portion of the physical environment within the three-dimensional environment; 72. The method of any one of claims 61 to 71, further comprising:

73. 73. The method of claim 72, wherein the lighting effect based on the portion of the virtual environment illuminates at least the respective portion of the physical environment.

74. 73. The method of claim 72, wherein the lighting effect based on the portion of the virtual environment darkens at least the respective portion of the physical environment.

75. detecting a third input via the one or more input devices while displaying the respective user interface of the respective application within the portion of the virtual environment within the three-dimensional environment; In response to detecting the third input, displaying, via the display generation component, the individual virtual environment for the individual application without displaying the portion of the virtual environment, in accordance with determining that the third input corresponds to a request to display the individual virtual environment for the individual application; 75. The method of any one of claims 61 to 74, further comprising:

76. prior to displaying the respective virtual environments of the respective applications, the portions of the virtual environments within the three-dimensional environment having one or more visual characteristics; after displaying the individual virtual environment of the individual application, the individual virtual environment has the one or more visual characteristics; 76. The method of claim 75.

77. when the third input is detected, the respective user interface of the respective application is displayed within the portion of the virtual environment together with the portion of the physical environment within the three-dimensional environment via the display generation component; in response to detecting the third input and in accordance with the determination that the third input corresponds to the request to display the individual virtual environment of the individual application, displaying the individual virtual environment of the individual application via the display generation component without displaying the portion of the virtual environment.

77. The method of claim 75 or 76.

78. when the third input is detected, the portion of the virtual environment is displayed via the display generation component at a first immersion level; in response to detecting the third input, and in accordance with the determination that the third input corresponds to the request to display the individual virtual environment of the individual application, displaying the individual virtual environment of the individual application via the display generation component at a second immersion level greater than the first immersion level.

78. The method of any one of claims 75 to 77.

79. one or more processors; Memory and One or more programs; An electronic device comprising: The one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: displaying, via a display generation component, a three-dimensional environment including a representation of a portion of the physical environment; detecting, while displaying the three-dimensional environment, a first input via one or more input devices corresponding to a request to display a respective user interface of a respective application; responsive to detecting the first input, displaying, via the display generation component, the respective user interface of the respective application within the portion of the physical environment within the three-dimensional environment; detecting a second input via the one or more input devices while displaying the distinct user interface in the portion of the physical environment within the three-dimensional environment; In response to detecting the second input, an electronic device including instructions for displaying the individual user interface in the portion of the virtual environment within the three-dimensional environment in accordance with a determination that the second input corresponds to a request to move the individual user interface to the portion of the virtual environment that is displayed together with the portion of the physical environment within the three-dimensional environment.

80. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: displaying, via a display generation component, a three-dimensional environment including a representation of a portion of the physical environment; detecting, while displaying the three-dimensional environment, a first input via one or more input devices corresponding to a request to display a respective user interface of a respective application; responsive to detecting the first input, displaying, via the display generation component, the respective user interface of the respective application within the portion of the physical environment within the three-dimensional environment; detecting a second input via the one or more input devices while displaying the distinct user interface in the portion of the physical environment within the three-dimensional environment; In response to detecting the second input, and displaying the individual user interface in the portion of the virtual environment within the three-dimensional environment in accordance with a determination that the second input corresponds to a request to move the individual user interface to the portion of the virtual environment that is displayed together with the portion of the physical environment within the three-dimensional environment.

81. 1. An electronic device comprising: one or more processors; Memory and An electronic device comprising: means for displaying, via a display generation component, a three-dimensional environment including a representation of a portion of the physical environment; means for detecting, while displaying the three-dimensional environment, a first input via one or more input devices corresponding to a request to display a respective user interface of a respective application; means for displaying, via the display generation component, the respective user interface of the respective application within the portion of the physical environment within the three-dimensional environment in response to detecting the first input; means for detecting a second input via the one or more input devices while displaying the distinct user interface in the portion of the physical environment within the three-dimensional environment; In response to detecting the second input, means for displaying the individual user interface in the portion of the virtual environment within the three-dimensional environment in accordance with a determination that the second input corresponds to a request to move the individual user interface to the portion of the virtual environment that is displayed together with the portion of the physical environment within the three-dimensional environment.

82. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: means for displaying, via a display generation component, a three-dimensional environment including a representation of a portion of the physical environment; means for detecting, while displaying the three-dimensional environment, a first input via one or more input devices corresponding to a request to display a respective user interface of a respective application; means for displaying, via the display generation component, the respective user interface of the respective application within the portion of the physical environment within the three-dimensional environment in response to detecting the first input; means for detecting a second input via the one or more input devices while displaying the distinct user interface in the portion of the physical environment within the three-dimensional environment; In response to detecting the second input, and means for displaying the individual user interface in the portion of the virtual environment within the three-dimensional environment in accordance with a determination that the second input corresponds to a request to move the individual user interface to the portion of the virtual environment that is displayed together with the portion of the physical environment within the three-dimensional environment.

83. one or more processors; Memory and One or more programs; An electronic device comprising:

79. An electronic device, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing a method according to any one of claims 61 to 78.

84. 80. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform a method according to any one of claims 61 to 78.

85. one or more processors; Memory and means for carrying out the method according to any one of claims 61 to 78; An electronic device comprising:

86. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: An electronic device comprising means for carrying out the method of any one of claims 61 to 78.

87. An electronic device in communication with a display generation component and one or more input devices, displaying, via the display generation component, a three-dimensional environment from a viewpoint of a user of the electronic device while an object corresponding to the viewpoint of the user is at a first location within an area within a physical environment of the electronic device, the three-dimensional environment including a mimicked environment and a boundary between a first portion of the three-dimensional environment including the mimicked environment and a second portion of the three-dimensional environment, the boundary being at a first distance from the viewpoint of the user of the electronic device within the three-dimensional environment; detecting, via the one or more input devices, movement of the object corresponding to the user's viewpoint from the first location to a second location within the physical environment while displaying the three-dimensional environment including the mimicked environment and the boundary at the first distance from the user's viewpoint; In response to detecting the movement of the object corresponding to the viewpoint of the user from the first location to the second location, updating the three-dimensional environment to change an amount of the three-dimensional environment occupied by the mimicked environment so that the boundary of the mimicked environment is at a second distance from the user's viewpoint that is greater than the first distance, in accordance with a determination that the movement of the object corresponding to the user's viewpoint from the first location to the second location within the physical environment satisfies one or more criteria, wherein updating the three-dimensional environment includes replacing a display of a portion of the mimicked environment with a display of a representation of a portion of the physical environment on the electronic device; A method comprising:

88. the boundary between the first portion of the three-dimensional environment and the second portion of the three-dimensional environment includes an area between the simulated environment and the second portion of the three-dimensional environment; at least a distinct portion of the simulated environment and at least a distinct portion of the second portion of the three-dimensional environment are simultaneously displayed at a particular location within the region; 88. The method of claim 87.

89. reducing an immersion level at which the mimicked environment is displayed in response to detecting the movement of the object corresponding to the user's viewpoint from the first location to the second location and in accordance with the determination that the movement of the object corresponding to the user's viewpoint from the first location to the second location in the physical environment satisfies the one or more criteria; 89. The method of claim 87 or 88, further comprising:

90. 90. The method of any one of claims 87 to 89, wherein updating the three-dimensional environment to change the amount of the three-dimensional environment occupied by the simulated environment so that the boundary of the simulated environment is at the second distance from the user's viewpoint is independent of whether an obstacle is present in the physical environment of the electronic device.

91. after detecting the movement of the object corresponding to the user's viewpoint from the first location to the second location, detecting, via the one or more input devices, a second movement of the object corresponding to the user's viewpoint away from the second location in the physical environment while displaying the three-dimensional environment including the mimicked environment and the boundary at the second distance from the user's viewpoint; in response to detecting the second movement of the object corresponding to the viewpoint of the user away from the second location in the physical environment; updating the three-dimensional environment to change the amount of the three-dimensional environment occupied by the mimicked environment, including replacing the representation of the portion of the physical environment with the portion of the mimicked environment, according to a determination that the second movement of the object corresponding to the viewpoint of the user is away from the boundary; 91. The method of any one of claims 87 to 90, further comprising:

92. updating the three-dimensional environment to change the amount of the three-dimensional environment occupied by the simulated environment; replacing a representation of a first amount of the mimicked environment with a representation of the first amount of the physical environment of the electronic device in accordance with determining that the second location is a first discrete distance from the boundary; and replacing a representation of a second amount of the mimicked environment, the second amount being different from the first amount, with a representation of the second amount of the physical environment of the electronic device in accordance with a determination that the second location is a second discrete distance from the boundary, the second discrete distance being different from the first discrete distance.

93. updating the three-dimensional environment to change the amount of the three-dimensional environment occupied by the simulated environment; replacing a representation of a first amount of the mimicked environment with a representation of the first amount of the physical environment of the electronic device in accordance with a determination that a speed of the movement of the object corresponding to the viewpoint of the user is a first speed; and replacing a display of a second amount of the mimicked environment, different from the first amount, with a display of the second amount of the physical environment of the electronic device in accordance with a determination that the speed of the movement of the object corresponding to the viewpoint of the user is a second speed different from the first speed.

94. updating the three-dimensional environment to change the amount of the three-dimensional environment occupied by the simulated environment; replacing a representation of a first amount of the mimicked environment with a representation of the first amount of the physical environment of the electronic device in accordance with a determination that an acceleration of the movement of the object corresponding to the viewpoint of the user is a first acceleration; and replacing a representation of the second amount of the physical environment of the electronic device with a representation of the second amount of the mimicked environment, the second amount being different from the first amount, in accordance with a determination that the acceleration of the movement of the object corresponding to the user's viewpoint is a second acceleration different from the first acceleration.

95. after detecting the movement of the object corresponding to the user's viewpoint from the first location to the second location, detecting, via the one or more input devices, a second movement of the object corresponding to the user's viewpoint away from the second location in the physical environment while displaying the three-dimensional environment including the mimicked environment and the boundary at the second distance from the user's viewpoint; in response to detecting the second movement of the object corresponding to the viewpoint of the user away from the second location in the physical environment; updating the three-dimensional environment to cease displaying the mimicked environment within the three-dimensional environment according to a determination that the second movement of the object corresponding to the viewpoint of the user is toward the boundary; 95. The method of any one of claims 87 to 94, further comprising:

96. detecting, via the one or more input devices, a change in orientation of the object corresponding to the user's viewpoint without detecting movement of the object corresponding to the user's viewpoint away from the first location in the physical environment while displaying the three-dimensional environment including the mimicked environment and the boundary at the first distance from the user's viewpoint; In response to detecting the change in the orientation of the object corresponding to the user's viewpoint, updating the display of the three-dimensional environment according to the change in the orientation of the object corresponding to the user's viewpoint without replacing the display of the portion of the mimicked environment with a display of the representation of the portion of the physical environment of the electronic device while maintaining the boundary at the first distance from the user's viewpoint; 96. The method of any one of claims 87 to 95, further comprising:

97. 97. The method of any one of claims 87 to 96, wherein the one or more criteria include a criterion that is met when the second location is within a threshold distance of a location corresponding to the boundary and is not met when the second location is farther than the threshold distance from the location corresponding to the boundary.

98. 98. The method of any one of claims 87 to 97, wherein updating the three-dimensional environment to change the amount of the three-dimensional environment occupied by the simulated environment comprises changing which portion of the three-dimensional environment is occupied by the simulated environment based on a distance of one or more physical features in the physical environment of the electronic device from the object corresponding to the viewpoint of the user.

99. detecting, via the one or more input devices, a second movement of the object corresponding to the viewpoint of the user from the first location to the second location within the physical environment while displaying the three-dimensional environment, including displaying a representation of a third portion of the physical environment with atmospheric effects; 99. The method of any one of claims 87 to 98, further comprising: in response to detecting a second movement of the object corresponding to the user's viewpoint from the first location to the second location, updating the display of the three-dimensional environment according to the second movement of the object corresponding to the user's viewpoint without changing the atmosphere effect in which the representation of the third portion of the physical environment is displayed.

100. Prior to detecting the movement of the object corresponding to the viewpoint of the user from the first location to the second location, the three-dimensional environment includes a user interface of an application located within the mimicked environment, and the method further comprises: in response to detecting the movement of the object corresponding to the viewpoint of the user from the first location to the second location, and in accordance with the determination that the movement of the object corresponding to the viewpoint of the user from the first location to the second location in the physical environment satisfies the one or more second criteria; updating the three-dimensional environment so that it no longer includes the simulated environment; The method of any one of claims 87 to 99, further comprising ceasing to display the user interface of the application in the three-dimensional environment.

101. Prior to detecting the movement of the object corresponding to the viewpoint of the user from the first location to the second location, the three-dimensional environment includes a user interface of an application located outside the mimicked environment within the three-dimensional environment, and the method further comprises: in response to detecting the movement of the object corresponding to the viewpoint of the user from the first location to the second location, in accordance with a determination that the movement of the object corresponding to the viewpoint of the user from the first location to the second location in the physical environment satisfies one or more second criteria; updating the three-dimensional environment so that it no longer includes the simulated environment; The method of any one of claims 87 to 100, further comprising: maintaining a representation of the user interface of the application in the three-dimensional environment.

102. Prior to detecting the movement of the object corresponding to the viewpoint of the user from the first location to the second location, the three-dimensional environment includes a user interface of an application located within the mimicked environment, and the method further comprises: in response to detecting the movement of the object corresponding to the viewpoint of the user from the first location to the second location, in accordance with a determination that the movement of the object corresponding to the viewpoint of the user from the first location to the second location in the physical environment satisfies one or more second criteria; updating the first portion of the three-dimensional environment so that it no longer includes the simulated environment; The method of any one of claims 87 to 99 and 101, further comprising: displaying the user interface of the application in the second portion of the three-dimensional environment.

103. the user interface of the application has a first size in the three-dimensional environment while the user interface of the application is located in the first portion of the three-dimensional environment; 103. The method of claim 102, wherein the user interface of the application has a second size in the first portion of the three-dimensional environment that is smaller than the first size while the user interface of the application is located in the second portion of the three-dimensional environment.

104. one or more processors; Memory and One or more programs; An electronic device comprising: The one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: while an object corresponding to a viewpoint of a user of the electronic device is at a first location within an area within a physical environment of the electronic device, displaying, via a display generation component, a three-dimensional environment from the viewpoint of the user, the three-dimensional environment including a mimicked environment and a boundary between a first portion of the three-dimensional environment including the mimicked environment and a second portion of the three-dimensional environment, the boundary being at a first distance from the viewpoint of the user of the electronic device within the three-dimensional environment; detecting, via one or more input devices, movement of the object corresponding to the user's viewpoint from the first location to a second location within the physical environment while displaying the three-dimensional environment including the mimicked environment and the boundary at the first distance from the user's viewpoint; In response to detecting the movement of the object corresponding to the viewpoint of the user from the first location to the second location, An electronic device comprising instructions for updating the three-dimensional environment to change the amount of the three-dimensional environment occupied by the simulated environment so that the boundary of the simulated environment is at a second distance from the user's viewpoint that is farther than the first distance in accordance with a determination that the movement of the object corresponding to the user's viewpoint from the first location to the second location within the physical environment satisfies one or more criteria, wherein updating the three-dimensional environment comprises replacing a display of a portion of the simulated environment with a display of a representation of a portion of the physical environment of the electronic device.

105. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: displaying, via a display generation component, a three-dimensional environment from a viewpoint of a user of the electronic device while an object corresponding to the viewpoint of the user is at a first location within an area within a physical environment of the electronic device, the three-dimensional environment including a mimicked environment and a boundary between a first portion of the three-dimensional environment including the mimicked environment and a second portion of the three-dimensional environment, the boundary being at a first distance from the viewpoint of the user of the electronic device within the three-dimensional environment; detecting, via one or more input devices, movement of the object corresponding to the user's viewpoint from the first location to a second location within the physical environment while displaying the three-dimensional environment including the mimicked environment and the boundary at the first distance from the user's viewpoint; In response to detecting the movement of the object corresponding to the viewpoint of the user from the first location to the second location, and updating the three-dimensional environment to change an amount of the three-dimensional environment occupied by the simulated environment so that the boundary of the simulated environment is at a second distance from the user's viewpoint that is greater than the first distance, in accordance with a determination that the movement of the object corresponding to the user's viewpoint from the first location to the second location within the physical environment satisfies one or more criteria, wherein updating the three-dimensional environment includes replacing a display of a portion of the simulated environment with a display of a representation of a portion of the physical environment on the electronic device.

106. one or more processors; Memory and An electronic device comprising: means for displaying, via a display generation component, a three-dimensional environment from a viewpoint of a user of the electronic device while an object corresponding to the viewpoint of the user is at a first location within an area within a physical environment of the electronic device, the three-dimensional environment including a mimicked environment and a boundary between a first portion of the three-dimensional environment including the mimicked environment and a second portion of the three-dimensional environment, the boundary being at a first distance from the viewpoint of the user of the electronic device within the three-dimensional environment; means for detecting, via one or more input devices, movement of the object corresponding to the user's viewpoint from the first location to a second location within the physical environment while displaying the three-dimensional environment including the mimicked environment and the boundary at the first distance from the user's viewpoint; In response to detecting the movement of the object corresponding to the viewpoint of the user from the first location to the second location, and updating the three-dimensional environment to change the amount of the three-dimensional environment occupied by the simulated environment so that the boundary of the simulated environment is at a second distance from the user's viewpoint that is farther than the first distance, in accordance with a determination that the movement of the object corresponding to the user's viewpoint from the first location to the second location within the physical environment satisfies one or more criteria, wherein updating the three-dimensional environment includes replacing a display of a portion of the simulated environment with a display of a representation of a portion of the physical environment of the electronic device.

107. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: means for displaying, via a display generation component, a three-dimensional environment from a viewpoint of a user of the electronic device while an object corresponding to the viewpoint of the user is at a first location within an area within a physical environment of the electronic device, the three-dimensional environment including a mimicked environment and a boundary between a first portion of the three-dimensional environment including the mimicked environment and a second portion of the three-dimensional environment, the boundary being at a first distance from the viewpoint of the user of the electronic device within the three-dimensional environment; means for detecting, via one or more input devices, movement of the object corresponding to the user's viewpoint from the first location to a second location within the physical environment while displaying the three-dimensional environment including the mimicked environment and the boundary at the first distance from the user's viewpoint; In response to detecting the movement of the object corresponding to the viewpoint of the user from the first location to the second location, and updating the three-dimensional environment to change the amount of the three-dimensional environment occupied by the simulated environment so that the boundary of the simulated environment is at a second distance from the user's viewpoint that is farther than the first distance, in accordance with a determination that the movement of the object corresponding to the user's viewpoint from the first location to the second location within the physical environment satisfies one or more criteria, wherein updating the three-dimensional environment includes replacing a display of a portion of the simulated environment with a display of a representation of a portion of the physical environment on the electronic device.

108. one or more processors; Memory and One or more programs; An electronic device comprising:

104. An electronic device, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any one of claims 87 to 103.

109. 104. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform a method according to any one of claims 87 to 103.

110. one or more processors; Memory and means for carrying out the method according to any one of claims 87 to 103; An electronic device comprising:

111. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: An electronic device comprising means for carrying out the method of any one of claims 87 to 103.

112. An electronic device in communication with a display generation component and one or more input devices, a virtual object in a three-dimensional environment visible through the display generation component; the three-dimensional environment includes a simulated environment having a first visual appearance; displaying a virtual object within the three-dimensional environment, the virtual object being located outside the simulated environment; While displaying the virtual object within the three-dimensional environment, receiving, via the one or more input devices, a first input corresponding to a request to move the virtual object within the three-dimensional environment; In response to receiving the first input, moving the virtual object within the three-dimensional environment according to the first input; displaying the simulated environment having a second visual appearance different from the first visual appearance in response to a determination that the first input corresponds to a request to move the virtual object within the simulated environment; maintaining a display of the simulated environment having the first visual appearance in accordance with a determination that the first input corresponds to a request to move the virtual object within the three-dimensional environment without moving the virtual object within the simulated environment; A method comprising:

113. displaying the simulated environment having the first visual appearance includes displaying the simulated environment occupying a first portion of the three-dimensional environment; displaying the simulated environment having the second visual appearance includes displaying the simulated environment occupying a second portion of the three-dimensional environment, the second portion of the three-dimensional environment having a different size than the first portion of the three-dimensional environment. The method of claim 112.

114. displaying the simulated environment with the first visual appearance includes displaying the simulated environment at a first opacity level in the three-dimensional environment; displaying the simulated environment with the second visual appearance comprises displaying the simulated environment with a second opacity level in the three-dimensional environment, the second opacity level being different from the first opacity level; 114. The method of claim 112 or 113, comprising:

115. displaying the simulated environment having the first visual appearance includes displaying the simulated environment at a first location within the three-dimensional environment; displaying the simulated environment having the second visual appearance comprises displaying the simulated environment at a second location within the three-dimensional environment that is different from the first location; The method of any one of claims 112 to 114, comprising:

116. displaying the simulated environment with the first visual appearance includes displaying the simulated environment with a first immersion level within the three-dimensional environment; displaying the simulated environment with the second visual appearance comprises displaying the simulated environment at a second level of immersion in the three-dimensional environment, the second level of immersion being different from the first level of immersion; 116. The method of any one of claims 112 to 115, comprising:

117. 117. The method of any one of claims 112 to 116, wherein the first input corresponds to the request to move the virtual object within the simulated environment in accordance with a determination that the first input corresponds to the request to throw the virtual object within the simulated environment.

118. 118. A method according to any one of claims 112 to 117, wherein the first input corresponds to the request to move the virtual object within the simulated environment in accordance with a determination that the first input corresponds to a selection of a selectable option displayed within the three-dimensional environment, the selectable option being selectable to move the virtual object within the simulated environment.

119. and displaying the simulated environment having the first visual appearance includes displaying the simulated environment at a first location within the three-dimensional environment, the method comprising: in response to receiving the first input, in accordance with the determination that the first input corresponds to the request to move the virtual object within the simulated environment; displaying the simulated environment at a third location within the three-dimensional environment in accordance with a determination that the first input corresponds to a request to move the virtual object to a second location within the three-dimensional environment; and The method of any one of claims 112 to 118, further comprising: displaying the mimicked environment at a fifth location within the three-dimensional environment that is different from the third location in accordance with a determination that the first input corresponds to a request to move the virtual object to a fourth location within the three-dimensional environment that is different from the second location.

120. 120. The method of claim 119, wherein displaying the simulated environment at the third location includes displaying a predetermined portion of the simulated environment at the second location.

121. 121. The method of any one of claims 119 to 120, wherein displaying the simulated environment at the fifth location comprises displaying a predetermined portion of the simulated environment at the fourth location.

122. and displaying the simulated environment having the first visual appearance includes displaying the simulated environment at a first location within the three-dimensional environment, the method comprising: in response to receiving the first input, in accordance with the determination that the first input corresponds to the request to move the virtual object within the simulated environment; displaying the simulated environment at a second location within the three-dimensional environment in accordance with a determination that a current viewpoint of a user of the electronic device is a first viewpoint, the simulated environment at the second location within the three-dimensional environment having a distinct spatial arrangement relative to the first viewpoint; 119. The method of any one of claims 112 to 118, further comprising: in accordance with a determination that the current viewpoint of the user is a second viewpoint different from the first viewpoint, displaying the simulated environment at a third location different from the second location within the three-dimensional environment, the simulated environment at the third location within the three-dimensional environment having the individual spatial arrangement relative to the second viewpoint.

123. When the first input is received, the virtual object has a first size in the three-dimensional environment, and the method further comprises:

123. The method of any one of claims 112 to 122, further comprising, in response to receiving the first input, displaying the virtual object in the simulated environment in accordance with the determination that the first input corresponds to the request to move the virtual object within the simulated environment, the virtual object in the simulated environment having a second size in the three-dimensional environment that is larger than the first size.

124. in response to receiving the first input, displaying the virtual object at a first location within the simulated environment in accordance with the determination that the first input corresponds to the request to move the virtual object within the simulated environment, the first location being a predetermined location not based on the first input; 124. The method of any one of claims 112 to 123, further comprising:

125. In response to receiving the first input, displaying the virtual object at a first location within the simulated environment in accordance with the determination that the first input corresponds to the request to move the virtual object within the simulated environment; receiving, while displaying the virtual object at the first location within the simulated environment, a second input via the one or more input devices directed toward the virtual object, the second input comprising a movement corresponding to movement away from the first location within the simulated environment; While receiving the second input, moving the virtual object away from the first location within the simulated environment in accordance with the second input; After receiving the second input, displaying, via the display generation component, the virtual object at a second location within the three-dimensional environment in accordance with the second input, the second location being outside the simulated environment, in accordance with a determination that the second input satisfies one or more criteria; moving the virtual object back to the first location within the simulated environment in accordance with a determination that the second input does not satisfy the one or more criteria; 125. The method of any one of claims 112 to 124, further comprising:

126. 126. The method of claim 125, wherein the one or more criteria include a criterion that is met when the magnitude of the movement at the second input is a first magnitude and is not met when the magnitude of the movement at the second input is a second magnitude different from the first magnitude.

127. 127. The method of claim 125 or 126, wherein the one or more criteria include a criterion that is met when the direction of the movement at the second input is a first direction and that is not met when the direction of the movement at the second input is a second direction different from the first direction.

128. one or more processors; Memory and One or more programs; An electronic device comprising: The one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: via a display generation component, a virtual object within the three-dimensional environment that is visible through said display generation component; the three-dimensional environment includes a simulated environment having a first visual appearance; displaying a virtual object located outside the simulated environment within the three-dimensional environment; While displaying the virtual object within the three-dimensional environment, receiving, via one or more input devices, a first input corresponding to a request to move the virtual object within the three-dimensional environment; In response to receiving the first input, moving the virtual object within the three-dimensional environment according to the first input; displaying the simulated environment having a second visual appearance different from the first visual appearance in response to a determination that the first input corresponds to a request to move the virtual object within the simulated environment; and instructions for maintaining a display of the simulated environment having the first visual appearance in accordance with a determination that the first input corresponds to a request to move the virtual object within the three-dimensional environment without moving the virtual object within the simulated environment.

129. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: via a display generation component, a virtual object within the three-dimensional environment that is visible through said display generation component; the three-dimensional environment includes a simulated environment having a first visual appearance; displaying a virtual object located outside the simulated environment within the three-dimensional environment; While displaying the virtual object within the three-dimensional environment, receiving, via one or more input devices, a first input corresponding to a request to move the virtual object within the three-dimensional environment; In response to receiving the first input, moving the virtual object within the three-dimensional environment according to the first input; displaying the simulated environment having a second visual appearance different from the first visual appearance in response to a determination that the first input corresponds to a request to move the virtual object within the simulated environment; and maintaining a display of the simulated environment having the first visual appearance in accordance with a determination that the first input corresponds to a request to move the virtual object within the three-dimensional environment without moving the virtual object within the simulated environment.

130. one or more processors; Memory and An electronic device comprising: via a display generation component, a virtual object within the three-dimensional environment that is visible through said display generation component; the three-dimensional environment includes a simulated environment having a first visual appearance; means for displaying a virtual object, the virtual object being located outside the simulated environment within the three-dimensional environment; means for receiving, while displaying the virtual object within the three-dimensional environment, a first input via one or more input devices corresponding to a request to move the virtual object within the three-dimensional environment; In response to receiving the first input, moving the virtual object within the three-dimensional environment according to the first input; displaying the simulated environment having a second visual appearance different from the first visual appearance in response to a determination that the first input corresponds to a request to move the virtual object within the simulated environment; means for maintaining a display of the simulated environment having the first visual appearance in accordance with a determination that the first input corresponds to a request to move the virtual object within the three-dimensional environment without moving the virtual object within the simulated environment.

131. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: via a display generation component, a virtual object within the three-dimensional environment that is visible through said display generation component; the three-dimensional environment includes a simulated environment having a first visual appearance; means for displaying a virtual object, the virtual object being located outside the simulated environment within the three-dimensional environment; means for receiving, while displaying the virtual object within the three-dimensional environment, a first input via one or more input devices corresponding to a request to move the virtual object within the three-dimensional environment; In response to receiving the first input, moving the virtual object within the three-dimensional environment according to the first input; displaying the simulated environment having a second visual appearance different from the first visual appearance in response to a determination that the first input corresponds to a request to move the virtual object within the simulated environment; means for maintaining a display of the simulated environment having the first visual appearance in accordance with a determination that the first input corresponds to a request to move the virtual object within the three-dimensional environment without moving the virtual object within the simulated environment.

132. one or more processors; Memory and One or more programs; An electronic device comprising: An electronic device, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any one of claims 112 to 127.

133. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform a method according to any one of claims 112 to 127.

134. one or more processors; Memory and means for carrying out the method according to any one of claims 112 to 127; An electronic device comprising:

135. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: An electronic device comprising means for performing the method of any one of claims 112 to 127.

Images