Devices, methods, and graphical user interfaces for three-dimensional user experience sessions in augmented reality environments

Abstract

Devices, methods, and graphical user interfaces are provided for a three-dimensional user experience session in an augmented reality environment. The present disclosure relates to techniques for providing computer-generated user experience sessions in an augmented reality environment. In some embodiments, a computer system provides particles that move based on breathing characteristics of a user to a computer-generated user experience session. In some embodiments, a computer system provides a computer-generated user experience session with options selected based on characteristics of an XR environment. In some embodiments, a computer system provides a computer-generated user experience session with a sound scene having planned sound components that are randomly selected.

Description

[0001] This application is a divisional application of the invention patent application with the application date of March 21, 2023, application number 202380029272.1, and invention name “Device, method and graphical user interface for three-dimensional user experience session in an extended reality environment”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims priority to U.S. Provisional Application No. 63 / 322,502, filed on March 22, 2022, entitled “DEVICES, METHODS, AND GRAPHICALUSER INTERFACES FOR THREE-DIMENSIONAL USER EXPERIENCE SESSIONS IN AN EXTENDED REALITY ENVIRONMENT,” and U.S. Patent Application No. 18 / 108,852, filed on February 13, 2023, entitled “DEVICES, METHODS, AND GRAPHICAL USER INTERFACES FOR THREE-DIMENSIONAL USEREXPERIENCE SESSIONS IN AN EXTENDED REALITY ENVIRONMENT,” the entire contents of each of these U.S. applications are hereby incorporated by reference in their entirety. Technical Field

[0004] The present disclosure generally relates to computer systems in communication with display generation components, one or more sensors, and optionally one or more audio generation components that provide computer-generated experiences, including but not limited to electronic devices that provide virtual reality and mixed reality experiences via a display. Background Art

[0005] In recent years, the development of computer systems for augmented reality has increased significantly. Example augmented reality environments include at least some virtual elements that replace or augment the physical world. Input devices (such as cameras, controllers, joysticks, touch-sensitive surfaces, and touch screen displays) for computer systems and other electronic computing devices are used to interact with virtual / augmented reality environments. Example virtual elements include virtual objects such as digital images, videos, text, icons, and control elements (such as buttons and other graphics). Summary of the invention

[0006] Some methods and interfaces for providing computer-generated user experience sessions in extended reality environments are cumbersome, inefficient, and limited. For example, systems that provide insufficient feedback for performing actions associated with virtual objects, systems that require a series of inputs to achieve desired results in augmented reality environments, and systems where virtual object manipulation is complex, cumbersome, and error-prone can impose a significant cognitive burden on users and detract from the experience of the virtual / augmented reality environment. In addition, these methods take longer than necessary, wasting energy from the computer system. This latter consideration is particularly important in battery-powered devices.

[0007] Therefore, there is a need for a computer system with an improved method and interface for providing a computer-generated user experience session in an extended reality environment, so that the interaction of the computer-generated user experience session with the computer system is more efficient and more intuitive for the user. Such methods and interfaces optionally supplement or replace conventional methods for providing extended reality experiences to users. Such methods and interfaces reduce the amount, degree and / or nature of inputs from the user by helping the user understand the connection between the inputs provided and the device's response to those inputs, thereby forming a more effective human-computer interface.

[0008] The above-mentioned defects and other problems associated with the user interface of the computer system are reduced or eliminated by the disclosed system. In some embodiments, the computer system is a desktop computer with an associated display. In some embodiments, the computer system is a portable device (e.g., a notebook computer, a tablet computer, or a handheld device). In some embodiments, the computer system is a personal electronic device (e.g., a wearable electronic device, such as a watch or a head-mounted device). In some embodiments, the computer system has a touch pad. In some embodiments, the computer system has one or more cameras. In some embodiments, the computer system has a touch-sensitive display (also referred to as a "touch screen" or "touch screen display"). In some embodiments, the computer system has one or more eye tracking components. In some embodiments, the computer system has one or more hand tracking components. In some embodiments, in addition to the display generation component, the computer system also has one or more output devices, which include one or more tactile output generators and / or one or more audio output devices. In some embodiments, the computer system has a graphical user interface (GUI), one or more processors, a memory, and one or more modules, a program or instruction set stored in the memory for performing multiple functions. In some embodiments, the user interacts with the GUI through contact and gestures of a stylus and / or finger on a touch-sensitive surface, movement of the user's eyes and hands in space relative to the GUI (and / or computer system) or the user's body (as captured by a camera and other motion sensors), and / or voice input (as captured by one or more audio input devices). In some embodiments, the functions performed by interaction optionally include image editing, drawing, presentations, word processing, spreadsheet creation, playing games, making and receiving calls, video conferencing, sending and receiving emails, instant messaging, testing support, digital photography, digital video recording, web browsing, digital music playback, note-taking, and / or digital video playback. Executable instructions for performing these functions are optionally included in a transient and / or non-transient computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0009] There is a need for electronic devices with improved methods and interfaces for providing computer-generated user experience sessions in extended reality environments. Such methods and interfaces can supplement or replace conventional methods for interacting with three-dimensional environments. Such methods and interfaces reduce the amount, degree, and / or nature of input from the user and produce a more efficient human-computer interface. For battery-powered computing devices, such methods and interfaces save power and increase the time interval between battery charges. Such methods and interfaces also provide a more realistic user experience while saving storage space for the visual and audio components of the user experience session.

[0010] According to some embodiments, a method is described. The method is performed at a computer system that communicates with a display generation component and one or more sensors. The method includes: displaying a user interface for a user experience session via the display generation component, including: when the user experience session is active: detecting one or more breathing characteristics of a user of the computer system via the one or more sensors; and displaying a user interface object as having a plurality of particles that move based on the one or more breathing characteristics of the user of the computer system, including: based on a determination that a first breathing event of the user of the computer system meets a first set of criteria, displaying these particles of the user interface object as moving in a first manner during the first breathing event of the user of the computer system; and based on a determination that the first breathing event of the user of the computer system meets a second set of criteria, displaying these particles of the user interface object as moving in a second manner different from the first manner during the first breathing event of the user of the computer system.

[0011] According to some embodiments, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more sensors, the one or more programs including instructions for the following operations: displaying a user interface for a user experience session via the display generation component, including: when the user experience session is active: detecting one or more respiratory characteristics of a user of the computer system via the one or more sensors; and displaying a user interface object as having a plurality of particles that move based on the one or more respiratory characteristics of the user of the computer system, including: based on a determination that a first respiratory event of the user of the computer system meets a first set of criteria, displaying the particles of the user interface object as moving in a first manner during the first respiratory event of the user of the computer system; and based on a determination that the first respiratory event of the user of the computer system meets a second set of criteria, displaying the particles of the user interface object as moving in a second manner different from the first manner during the first respiratory event of the user of the computer system.

[0012] According to some embodiments, a transient computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more sensors, the one or more programs including instructions for the following operations: displaying a user interface for a user experience session via the display generation component, including: when the user experience session is active: detecting one or more respiratory characteristics of a user of the computer system via the one or more sensors; and displaying a user interface object as having a plurality of particles that move based on the one or more respiratory characteristics of the user of the computer system, including: based on a determination that a first respiratory event of the user of the computer system meets a first set of criteria, displaying the particles of the user interface object as moving in a first manner during the first respiratory event of the user of the computer system; and based on a determination that the first respiratory event of the user of the computer system meets a second set of criteria, displaying the particles of the user interface object as moving in a second manner different from the first manner during the first respiratory event of the user of the computer system.

[0013] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component and one or more sensors. The computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: displaying a user interface for a user experience session via the display generation component, including: when the user experience session is active: detecting one or more breathing characteristics of a user of the computer system via the one or more sensors; and displaying a user interface object as having a plurality of particles moving based on the one or more breathing characteristics of the user of the computer system, including: based on a determination that a first breathing event of the user of the computer system meets a first set of criteria, displaying these particles of the user interface object as moving in a first manner during the first breathing event of the user of the computer system; and based on a determination that the first breathing event of the user of the computer system meets a second set of criteria, displaying these particles of the user interface object as moving in a second manner different from the first manner during the first breathing event of the user of the computer system.

[0014] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component and one or more sensors. The computer system includes: a component for displaying a user interface for a user experience session via the display generation component, the display including: when the user experience session is active: detecting one or more breathing characteristics of a user of the computer system via the one or more sensors; and displaying a user interface object as having a plurality of particles that move based on the one or more breathing characteristics of the user of the computer system, including: based on a determination that a first breathing event of the user of the computer system meets a first set of criteria, displaying these particles of the user interface object as moving in a first manner during the first breathing event of the user of the computer system; and based on a determination that the first breathing event of the user of the computer system meets a second set of criteria, displaying these particles of the user interface object as moving in a second manner different from the first manner during the first breathing event of the user of the computer system.

[0015] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more sensors, the one or more programs including instructions for the following operations: displaying a user interface for a user experience session via the display generation component, including: while the user experience session is active: detecting one or more respiratory characteristics of a user of the computer system via the one or more sensors; and displaying a user interface object as having a plurality of particles that move based on the one or more respiratory characteristics of the user of the computer system, including: based on a determination that a first respiratory event of the user of the computer system satisfies a first set of criteria, displaying the particles of the user interface object as moving in a first manner during the first respiratory event of the user of the computer system; and based on a determination that the first respiratory event of the user of the computer system satisfies a second set of criteria, displaying the particles of the user interface object as moving in a second manner different from the first manner during the first respiratory event of the user of the computer system.

[0016] According to some embodiments, a method is described. The method is performed at a computer system that communicates with a display generation component and one or more sensors. The method includes: when displaying an XR environment having one or more characteristics, detecting a request to initiate a user experience session in the XR environment via the one or more sensors; and in response to detecting the request to initiate the user experience session in the XR environment, initiating the user experience session in the XR environment, including: displaying a user interface for the user experience session via the display generation component, wherein displaying the user interface for the user experience session includes: based on the determination that the one or more characteristics of the XR environment meet a first set of standards, displaying the user interface for the user experience session as having a first set of one or more options enabled for the user experience session; and based on the determination that the one or more characteristics of the XR environment meet a second set of standards different from the first set of standards, displaying the user interface for the user experience session as having a second set of one or more options enabled for the user experience session, wherein the second set of one or more options is different from the first set of one or more options.

[0017] According to some embodiments, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system that communicates with a display generation component and one or more sensors, the one or more programs including instructions for the following operations: when displaying an XR environment having one or more characteristics, detecting a request to initiate a user experience session in the XR environment via the one or more sensors; and in response to detecting the request to initiate the user experience session in the XR environment, initiating the user experience session in the XR environment, including: displaying a user interface for the user experience session via the display generation component, wherein displaying the user interface for the user experience session includes: displaying the user interface for the user experience session as having a first set of one or more options enabled for the user experience session based on a determination that the one or more characteristics of the XR environment meet a first set of criteria; and displaying the user interface for the user experience session as having a second set of one or more options enabled for the user experience session based on a determination that the one or more characteristics of the XR environment meet a second set of criteria different from the first set of criteria, wherein the second set of one or more options is different from the first set of one or more options.

[0018] According to some embodiments, a transient computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system that communicates with a display generation component and one or more sensors, the one or more programs including instructions for the following operations: when displaying an XR environment having one or more characteristics, detecting a request to initiate a user experience session in the XR environment via the one or more sensors; and in response to detecting the request to initiate the user experience session in the XR environment, initiating the user experience session in the XR environment, including: displaying a user interface for the user experience session via the display generation component, wherein displaying the user interface for the user experience session includes: displaying the user interface for the user experience session as having a first set of one or more options enabled for the user experience session based on a determination that the one or more characteristics of the XR environment meet a first set of criteria; and displaying the user interface for the user experience session as having a second set of one or more options enabled for the user experience session based on a determination that the one or more characteristics of the XR environment meet a second set of criteria different from the first set of criteria, wherein the second set of one or more options is different from the first set of one or more options.

[0019] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component and one or more sensors. The computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: when displaying an XR environment having one or more characteristics, detecting a request to initiate a user experience session in the XR environment via the one or more sensors; and in response to detecting the request to initiate the user experience session in the XR environment, initiating the user experience session in the XR environment, including: displaying a user interface for the user experience session via the display generation component, wherein displaying the user interface for the user experience session includes: displaying the user interface for the user experience session as having a first set of one or more options enabled for the user experience session according to the determination that the one or more characteristics of the XR environment meet a first set of standards different from the first set of standards; and displaying the user interface for the user experience session as having a second set of one or more options enabled for the user experience session according to the determination that the one or more characteristics of the XR environment meet a second set of standards different from the first set of standards, wherein the second set of one or more options is different from the first set of one or more options.

[0020] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component and one or more sensors. The computer system includes: a component for detecting a request to initiate a user experience session in the XR environment via the one or more sensors when displaying an XR environment having one or more characteristics; and a component for initiating the user experience session in the XR environment in response to detecting the request to initiate the user experience session in the XR environment, including: a component for displaying a user interface for the user experience session via the display generation component, wherein displaying the user interface for the user experience session includes: displaying the user interface for the user experience session as having a first set of one or more options enabled for the user experience session according to a determination that the one or more characteristics of the XR environment meet a first set of standards; and displaying the user interface for the user experience session as having a second set of one or more options enabled for the user experience session according to a determination that the one or more characteristics of the XR environment meet a second set of standards different from the first set of standards, wherein the second set of one or more options is different from the first set of one or more options.

[0021] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system that communicates with a display generation component and one or more sensors, and the one or more programs include instructions for the following operations: when displaying an XR environment having one or more characteristics, detecting a request to initiate a user experience session in the XR environment via the one or more sensors; and in response to detecting the request to initiate the user experience session in the XR environment, initiating the user experience session in the XR environment, including: displaying a user interface for the user experience session via the display generation component, wherein displaying the user interface for the user experience session includes: based on a determination that the one or more characteristics of the XR environment meet a first set of criteria, displaying the user interface for the user experience session as having a first set of one or more options enabled for the user experience session; and based on a determination that the one or more characteristics of the XR environment meet a second set of criteria different from the first set of criteria, displaying the user interface for the user experience session as having a second set of one or more options enabled for the user experience session, wherein the second set of one or more options is different from the first set of one or more options.

[0022] According to some embodiments, a method is described. The method is performed at a computer system that communicates with a display generation component, an audio generation component, and one or more sensors. The method includes: detecting a request to initiate a user experience session of a corresponding type in an XR environment via the one or more sensors at a first time; in response to detecting the request to initiate the user experience session in the XR environment, initiating a first user experience session of the corresponding type in the XR environment, including: displaying a user interface for the first user experience session via the display generation component; and outputting a first audio soundscape for the first user experience session via the audio generation component, wherein the first audio soundscape is output concurrently with displaying the user interface for the first user experience session, and outputting the first audio soundscape includes: outputting the first audio soundscape having a first group of two or more audio components randomly or pseudo-randomly selected from a group of available audio components; in communicating with the detecting, via the one or more sensors, a request to initiate a user experience session of the corresponding type in the XR environment at a second time different from the first time; and in response to detecting the request to initiate the user experience session in the XR environment, initiating a second user experience session of the corresponding type in the XR environment, including: displaying a user interface for the second user experience session via the display generation component; and outputting a second audio soundscape for the second user experience session via the audio generation component, wherein the second audio soundscape is output concurrently with displaying the user interface for the second user experience session, and outputting the second audio soundscape includes: outputting the second audio soundscape having a second group of two or more audio components randomly or pseudo-randomly selected from the group of available audio components.

[0023] According to some embodiments, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system that communicates with a display generation component, an audio generation component, and one or more sensors, and the one or more programs include instructions for the following operations: detecting a request to initiate a user experience session of a corresponding type in an XR environment via the one or more sensors at a first time; in response to detecting the request to initiate the user experience session in the XR environment, initiating the first user experience session of the corresponding type in the XR environment, including: displaying a user interface for the first user experience session via the display generation component; and outputting a first audio soundscape for the first user experience session via the audio generation component, wherein the first audio soundscape is output concurrently with displaying the user interface for the first user experience session, and outputting the first audio soundscape includes: outputting a first audio soundscape having a plurality of audio elements selected from a set of available audio groups; The method of claim 1 further comprising: generating a first audio soundscape having a first group of two or more audio components selected partially randomly or pseudo-randomly; detecting, via the one or more sensors, a request to initiate a user experience session of the corresponding type in an XR environment at a second time different from the first time; and in response to detecting the request to initiate the user experience session in the XR environment, initiating a second user experience session of the corresponding type in the XR environment, comprising: displaying, via the display generating component, a user interface for the second user experience session; and outputting, via the audio generating component, a second audio soundscape for the second user experience session, wherein the second audio soundscape is output concurrently with displaying the user interface for the second user experience session, and outputting the second audio soundscape comprises: outputting the second audio soundscape having a second group of two or more audio components selected randomly or pseudo-randomly from the group of available audio components.

[0024] According to some embodiments, a transient computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system that communicates with a display generation component, an audio generation component, and one or more sensors, the one or more programs including instructions for the following operations: detecting a request to initiate a user experience session of a corresponding type in an XR environment via the one or more sensors at a first time; in response to detecting the request to initiate the user experience session in the XR environment, initiating a first user experience session of the corresponding type in the XR environment, including: displaying a user interface for the first user experience session via the display generation component; and outputting a first audio soundscape for the first user experience session via the audio generation component, wherein the first audio soundscape is output concurrently with displaying the user interface for the first user experience session, and outputting the first audio soundscape includes: outputting a first audio soundscape having a plurality of audio elements selected from a set of available audio groups; The method of claim 1 further comprising: generating a first audio soundscape having a first group of two or more audio components selected partially randomly or pseudo-randomly; detecting, via the one or more sensors, a request to initiate a user experience session of the corresponding type in an XR environment at a second time different from the first time; and in response to detecting the request to initiate the user experience session in the XR environment, initiating a second user experience session of the corresponding type in the XR environment, comprising: displaying, via the display generating component, a user interface for the second user experience session; and outputting, via the audio generating component, a second audio soundscape for the second user experience session, wherein the second audio soundscape is output concurrently with displaying the user interface for the second user experience session, and outputting the second audio soundscape comprises: outputting the second audio soundscape having a second group of two or more audio components selected randomly or pseudo-randomly from the group of available audio components.

[0025] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component, an audio generation component, and one or more sensors. The computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: detecting a request to initiate a user experience session of a corresponding type in an XR environment via the one or more sensors at a first time; in response to detecting the request to initiate the user experience session in the XR environment, initiating a first user experience session of the corresponding type in the XR environment, including: displaying a user interface for the first user experience session via the display generation component; and outputting a first audio soundscape for the first user experience session via the audio generation component, wherein the first audio soundscape is output concurrently with displaying the user interface for the first user experience session, and outputting the first audio soundscape includes: outputting a first audio soundscape having a random or pseudo audio component selected from a set of available audio components; The method of claim 1 , wherein the first audio soundscape comprises a first set of two or more audio components randomly selected; detecting, via the one or more sensors, a request to initiate a user experience session of the corresponding type in the XR environment at a second time different from the first time; and in response to detecting the request to initiate the user experience session in the XR environment, initiating a second user experience session of the corresponding type in the XR environment, comprising: displaying, via the display generating component, a user interface for the second user experience session; and outputting, via the audio generating component, a second audio soundscape for the second user experience session, wherein the second audio soundscape is output concurrently with displaying the user interface for the second user experience session, and outputting the second audio soundscape comprises: outputting the second audio soundscape having a second set of two or more audio components randomly or pseudo-randomly selected from the set of available audio components.

[0026] According to some embodiments, a computer system is described. The computer system is configured to communicate with a display generation component, an audio generation component, and one or more sensors. The computer system includes: a component for detecting a request to initiate a user experience session of a corresponding type in an XR environment via the one or more sensors at a first time; a component for initiating a first user experience session of the corresponding type in the XR environment in response to detecting the request to initiate the user experience session in the XR environment, including: a component for displaying a user interface for the first user experience session via the display generation component; and a component for outputting a first audio soundscape for the first user experience session via the audio generation component, wherein the first audio soundscape is output concurrently with displaying the user interface for the first user experience session, and outputting the first audio soundscape includes: outputting the first audio soundscape having a first group of two or more audio components randomly or pseudo-randomly selected from a group of available audio components; for A component for detecting, via the one or more sensors, a request to initiate the corresponding type of user experience session in the XR environment at a second time different from the first time; and a component for initiating a second user experience session of the corresponding type in the XR environment in response to detecting the request to initiate the user experience session in the XR environment, including: a component for displaying a user interface for the second user experience session via the display generation component; and a component for outputting a second audio soundscape for the second user experience session via the audio generation component, wherein the second audio soundscape is output concurrently with displaying the user interface for the second user experience session, and outputting the second audio soundscape includes: outputting the second audio soundscape having a second group of two or more audio components randomly or pseudo-randomly selected from the group of available audio components.

[0027] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system that communicates with a display generation component, an audio generation component, and one or more sensors, and the one or more programs include instructions for the following operations: detecting a request to initiate a user experience session of a corresponding type in an XR environment via the one or more sensors at a first time; in response to detecting the request to initiate the user experience session in the XR environment, initiating a first user experience session of the corresponding type in the XR environment, including: displaying a user interface for the first user experience session via the display generation component; and outputting a first audio soundscape for the first user experience session via the audio generation component, wherein the first audio soundscape is output concurrently with displaying the user interface for the first user experience session, and outputting the first audio soundscape includes: outputting a first audio soundscape having a plurality of audio components selected from a set of available audio components; The method comprises: generating a first audio soundscape having a first group of two or more audio components randomly or pseudo-randomly selected from the set of available audio components; detecting, via the one or more sensors, a request to initiate the corresponding type of user experience session in the XR environment at a second time different from the first time; and in response to detecting the request to initiate the user experience session in the XR environment, initiating a second user experience session of the corresponding type in the XR environment, including: displaying a user interface for the second user experience session via the display generating component; and outputting a second audio soundscape for the second user experience session via the audio generating component, wherein the second audio soundscape is output concurrently with displaying the user interface for the second user experience session, and outputting the second audio soundscape includes: outputting the second audio soundscape having a second group of two or more audio components randomly or pseudo-randomly selected from the set of available audio components.

[0028] It should be noted that the various embodiments described above can be combined with any other embodiments described herein. The features and advantages described in this specification are not comprehensive, and in particular, many additional features and advantages will be apparent to those of ordinary skill in the art based on the drawings, the specification, and the claims. In addition, it should be noted that the language used in this specification is selected in principle for readability and instructional purposes, and may not be selected to describe or define the subject matter of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, wherein like reference numerals designate corresponding parts throughout the several views.

[0030] Figure 1 is a block diagram illustrating an operating environment of a computer system for providing an XR experience according to some embodiments.

[0031] Figure 2 is a block diagram illustrating a controller of a computer system configured to manage and coordinate a user's XR experience according to some embodiments.

[0032] Figure 3 is a block diagram illustrating display generation components of a computer system configured to provide a visual component of an XR experience to a user according to some embodiments.

[0033] Figure 4 is a block diagram illustrating a hand tracking unit of a computer system configured to capture gesture input from a user according to some embodiments.

[0034] Figure 5 is a block diagram illustrating an eye tracking unit of a computer system configured to capture gaze input of a user according to some embodiments.

[0035] Figure 6 is a flow chart illustrating a flash-assisted gaze tracking pipeline according to some embodiments.

[0036] FIG. 7A to FIG. 7L Example techniques for providing a computer-generated user experience session in an extended reality environment are shown according to some embodiments.

[0037] Figure 8 is a flow chart of a method of providing particles that move based on a user's breathing characteristics to a computer-generated user experience session, according to various embodiments.

[0038] Fig. 9 is a flowchart of a method for providing options selected based on characteristics of an XR environment to a computer-generated user experience session according to various embodiments.

[0039] FIG. 10A to FIG. 10B is a flow chart of a method of providing a soundscape having randomly selected curated sound components to a computer-generated user experience session, according to various embodiments. DETAILED DESCRIPTION

[0040] According to some embodiments, the present disclosure relates to a user interface for providing an extended reality (XR) experience to a user.

[0041] The systems, methods, and GUIs described herein improve user interface interactions with virtual / augmented reality environments in a number of ways.

[0042] In some embodiments, a computer system provides particles that move based on a user's breathing characteristics to a computer-generated user experience session. The computer system displays a user interface for the user experience session. While the user experience session is active, the computer system detects one or more breathing characteristics of a user of the computer system and displays a user interface object as having a plurality of particles that move based on the one or more breathing characteristics of the user. When a first breathing event of the user meets a first set of criteria, the computer system displays the particles of the user interface object as moving in a first manner during the first breathing event of the user. When the first breathing event of the user meets a second set of criteria, the computer system displays the particles of the user interface object as moving in a second manner different from the first manner during the first breathing event of the user.

[0043] In some embodiments, a computer system provides options selected based on characteristics of an XR environment to a computer-generated user experience session. When the computer system displays an XR environment having one or more characteristics, the computer system detects a request to initiate a user experience session in the XR environment. In response to detecting the request to initiate a user experience session in the XR environment, the computer system initiates a user experience session in the XR environment, including displaying a user interface for the user experience session. When one or more characteristics of the XR environment meet a first set of criteria, the computer system displays the user interface for the user experience session as having a first set of one or more options enabled for the user experience session. When one or more characteristics of the XR environment meet a second set of criteria that is different from the first set of criteria, the computer system displays the user interface for the user experience session as having a second set of one or more options enabled for the user experience session, wherein the second set of one or more options is different from the first set of one or more options.

[0044] In some embodiments, a computer system provides a soundscape with randomly selected curated sound components to a computer-generated user experience session. The computer system detects a request to initiate a user experience session of a corresponding type in an XR environment at a first time. In response to detecting the request to initiate a user experience session in the XR environment, the computer system initiates a first user experience session of the corresponding type in the XR environment. Initiating the first user experience session includes: displaying a user interface for the first user experience session; and outputting a first audio soundscape for the first user experience session. The first audio soundscape is output concurrently with displaying the user interface for the first user experience session. Outputting the first audio soundscape includes: outputting a first audio soundscape with a first set of two or more audio components randomly or pseudo-randomly selected from a set of available audio components. The computer system detects a request to initiate a user experience session of the corresponding type in the XR environment at a second time different from the first time. In response to detecting the request to initiate a user experience session in the XR environment, the computer system initiates a second user experience session of the corresponding type in the XR environment. Initiating the second user experience session includes: displaying a user interface for the second user experience session; and outputting a second audio soundscape for the second user experience session. The second audio soundscape is output concurrently with displaying the user interface for the second user experience session. Outputting the second audio soundscape includes outputting the second audio soundscape having a second set of two or more audio components randomly or pseudo-randomly selected from the set of available audio components.

[0045] Figures 1 to 6 A description of an example computer system for providing an XR experience to a user is provided. FIG. 7A to FIG. 7L Example techniques for providing a computer-generated user experience session in an extended reality environment are shown according to some embodiments. Figure 8 is a flow chart of a method of providing particles that move based on a user's breathing characteristics to a computer-generated user experience session, according to various embodiments. Fig. 9 is a flowchart of a method for providing options selected based on characteristics of an XR environment to a computer-generated user experience session according to various embodiments. FIG. 10A to FIG. 10B is a flow chart of a method of providing a soundscape having randomly selected curated sound components to a computer-generated user experience session, according to various embodiments. FIG. 7A to FIG. 7L The user interface in Figure 8 , Fig. 9 , Fig. 10A and Fig. 10B process.

[0046] The process described below enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device) through various techniques, including by providing improved visual feedback to the user, reducing the number of inputs required to perform an operation, providing additional control options without cluttering the user interface with additional display controls, performing an operation without further user input when a set of conditions have been met, improving privacy and / or security, providing a richer, more detailed and / or more realistic user experience while saving storage space, and / or additional techniques. These techniques also reduce power usage and extend the battery life of the device by enabling the user to use the device faster and more efficiently. Saving battery power, and therefore saving weight, improves the ergonomics of the device. These techniques also enable real-time communication, allow the use of fewer and / or less accurate sensors, thereby producing a more compact, lighter and cheaper device, and enable the device to be used under various lighting conditions. These techniques reduce energy usage and thereby reduce the heat emitted by the device, which is particularly important for wearable devices where if the device generates too much heat well within the operating parameters of the device components, it may become uncomfortable for the user to wear the device.

[0047] The process described below enhances the operability of the device and makes the user-device interface more efficient by providing a richer, more detailed and / or more realistic user experience while saving storage space. For example, these technologies allow devices (e.g., computer systems, tablet devices and / or HMDs) to provide user experience while saving space by reducing the amount of visual and / or audio components required to store for generating user experience sessions. For example, in some embodiments, the device stores a superset of audio characteristics, and the device can select various combinations of audio characteristics from the superset (randomly or pseudo-randomly) to create a corresponding soundscape for the user experience session. This technology saves space because the device can generate many different soundscapes from a subset of audio characteristics without having to store a fully assembled soundscape for the user experience session. Similarly, the device stores a superset of visual characteristics, and the device can select various combinations of visual characteristics from the superset (randomly or pseudo-randomly) to create visual components and visual effects for the user experience session. This technology saves space because the device can generate many different visuals from a subset of visual characteristics without having to store the fully rendered visual components of the user experience session. Additional examples of these technologies are described below with reference to the accompanying drawings.

[0048] In addition, in the method described herein where one or more steps depend on one or more conditions being met, it should be understood that the method can be repeated in multiple repetitions so that in the process of repetition, all conditions for determining the steps in the method have been met in different repetitions of the method. For example, if the method needs to perform the first step (if the condition is met), and perform the second step (if the condition is not met), then the ordinary technician will know that the steps stated are repeated until both the condition is met and the condition is not met (in no particular order). Therefore, the method described as having one or more steps depending on one or more conditions being met can be rewritten as a method of repeating until each condition described in the method is met. However, this does not require the system or computer-readable medium to declare that the system or computer-readable medium contains instructions for performing a contingent operation based on the satisfaction of the corresponding one or more conditions, and is therefore able to determine whether the possible situation has been met without explicitly repeating the steps of the method until all conditions for determining the steps in the method have been met. It will also be understood by ordinary technicians in the art that, similar to the method with the contingent step, the system or computer-readable storage medium can repeat the steps of the method as many times as needed to ensure that all the contingent steps have been performed.

[0049] In some embodiments, such as Figure 1 As shown in , an XR experience is provided to a user via an operating environment 100 including a computer system 101. The computer system 101 includes a controller 110 (e.g., a processor of a portable electronic device or a remote server), a display generation component 120 (e.g., a head mounted device (HMD), a display, a projector, a touch screen, etc.), one or more input devices 125 (e.g., an eye tracking device 130, a hand tracking device 140, other input devices 150), one or more output devices 155 (e.g., a speaker 160, a tactile output generator 170, and other output devices 180), one or more sensors 190 (e.g., an image sensor, a light sensor, a depth sensor, a tactile sensor, an orientation sensor, a proximity sensor, a temperature sensor, a position sensor, a motion sensor, a velocity sensor, a speed sensor, etc.), and optionally one or more peripheral devices 195 (e.g., a household appliance, a wearable device, etc.). In some embodiments, one or more of the input device 125, output device 155, sensor 190, and peripheral device 195 are integrated with the display generation component 120 (e.g., in a head-mounted device or a handheld device).

[0050] In describing an XR experience, various terms are used to distinguishably refer to several related but distinct environments that a user can sense and / or with which the user can interact (e.g., using inputs detected by the computer system 101 generating the XR experience, which inputs cause the computer system generating the XR experience to generate audio, visual, and / or tactile feedback corresponding to the various inputs provided to the computer system 101). The following is a subset of these terms:

[0051] Physical environment: The physical environment refers to the physical world that people can sense and / or interact with without the help of electronic systems. A physical environment, such as a physical park, includes physical objects, such as physical trees, physical buildings, and physical people. People can directly sense and / or interact with the physical environment, such as through vision, touch, hearing, taste, and smell.

[0052] Extended Reality: In contrast, an extended reality (XR) environment refers to a fully or partially simulated environment that people sense and / or interact with via an electronic system. In XR, a subset of a person's physical movements, or a representation thereof, is tracked, and in response, one or more characteristics of one or more virtual objects simulated in the XR environment are adjusted in a manner that complies with at least one physical law. For example, an XR system may detect a person's head turn, and in response, adjust the graphical content and sound field presented to the person in a manner similar to the way such views and sounds change in a physical environment. In some cases (e.g., for accessibility reasons), adjustments to the characteristics of virtual objects in the XR environment may be made in response to a representation of physical movement (e.g., a voice command). People can sense and / or interact with XR objects using any of their senses, including vision, hearing, touch, taste, and smell. For example, people can sense and / or interact with audio objects, which create a 3D or spatial audio environment that provides the perception of a point audio source in 3D space. As another example, an audio object can enable audio transparency that selectively introduces ambient sounds from the physical environment with or without computer-generated audio. In some XR environments, a person can sense and / or interact only with an audio object.

[0053] Examples of XR include virtual reality and mixed reality.

[0054] Virtual Reality: A virtual reality (VR) environment refers to a simulated environment designed to be based entirely on computer-generated sensory input to one or more senses. A VR environment includes a plurality of virtual objects that a person can sense and / or interact with. For example, trees, buildings, and computer-generated images representing avatars of people are examples of virtual objects. A person can sense and / or interact with virtual objects in a VR environment through a simulation of the person's presence within the computer-generated environment and / or through a simulation of a subset of the person's physical movement within the computer-generated environment.

[0055] Mixed Reality: In contrast to VR environments, which are designed to be based entirely on computer-generated sensory input, a mixed reality (MR) environment refers to a simulated environment that is designed to include sensory input from the physical environment, or representations thereof, in addition to computer-generated sensory input (e.g., virtual objects). On the virtuality continuum, a mixed reality environment is anything between, but not including, a fully physical environment at one end and a virtual reality environment at the other end. In some MR environments, computer-generated sensory input can respond to changes in sensory input from the physical environment. In addition, some electronic systems used to render MR environments can track position and / or orientation relative to the physical environment to enable virtual objects to interact with real objects (i.e., physical items from the physical environment, or representations thereof). For example, the system can cause motion so that virtual trees appear stationary relative to the physical ground.

[0056] Examples of mixed reality include augmented reality and augmented virtuality.

[0057] Augmented reality: An augmented reality (AR) environment refers to a simulated environment in which one or more virtual objects are superimposed on a physical environment or a representation of a physical environment. For example, an electronic system for presenting an AR environment may have a transparent or translucent display through which a person can directly view the physical environment. The system may be configured to present virtual objects on a transparent or translucent display so that a person uses the system to perceive virtual objects superimposed on the physical environment. Alternatively, the system may have an opaque display and one or more imaging sensors that capture images or videos of the physical environment, which are representations of the physical environment. The system combines the image or video with the virtual object and presents the composition on an opaque display. A person uses the system to indirectly view the physical environment via an image or video of the physical environment and perceives virtual objects superimposed on the physical environment. As used herein, a video of a physical environment displayed on an opaque display is referred to as a "transparent video," meaning that the system uses one or more image sensors to capture images of the physical environment and uses those images when presenting the AR environment on an opaque display. Further alternatively, the system may have a projection system that projects virtual objects into a physical environment, such as as a hologram or on a physical surface, so that a person using the system perceives virtual objects superimposed on the physical environment. An augmented reality environment also refers to a simulated environment in which a representation of a physical environment is transformed by computer-generated sensory information. For example, in providing a pass-through video, the system may transform one or more sensor images to apply a selected perspective (e.g., a viewpoint) that is different from the perspective captured by the imaging sensor. For another example, the representation of the physical environment may be transformed by graphically modifying (e.g., enlarging) a portion thereof so that the modified portion may be a representative but not real version of the original captured image. For another example, the representation of the physical environment may be transformed by graphically eliminating a portion thereof or blurring a portion thereof.

[0058] Augmented Virtual: An augmented virtual (AV) environment refers to a simulated environment in which a virtual environment or a computer-generated environment incorporates one or more sensory inputs from the physical environment. The sensory input may be a representation of one or more characteristics of the physical environment. For example, an AV park may have virtual trees and virtual buildings, but the faces of people are realistically reproduced from images taken of physical people. For another example, a virtual object may take the shape or color of a physical object imaged by one or more imaging sensors. For another example, a virtual object may take a shadow that conforms to the positioning of the sun in the physical environment.

[0059] Viewpoint-locked virtual objects: When a computer system displays a virtual object at the same position and / or location in a user's viewpoint, the virtual object is viewpoint-locked even if the user's viewpoint shifts (e.g., changes). In embodiments where the computer system is a head-mounted device, the user's viewpoint is locked to the forward direction of the user's head (e.g., when the user is looking straight ahead, the user's viewpoint is at least a portion of the user's field of view); thus, without moving the user's head, the user's viewpoint remains fixed even when the user's gaze shifts. In embodiments where the computer system has a display generation component (e.g., a display screen) that is repositionable relative to the user's head, the user's viewpoint is an augmented reality view presented to the user on the display generation component of the computer system. For example, a viewpoint-locked virtual object that is displayed in the upper left corner of the user's viewpoint when the user's viewpoint is in a first orientation (e.g., the user's head is facing north) continues to be displayed in the upper left corner of the user's viewpoint even when the user's viewpoint changes to a second orientation (e.g., the user's head is facing west). In other words, the position and / or location of the viewpoint-locked virtual object displayed in the user's viewpoint is independent of the user's position and / or orientation in the physical environment. In embodiments where the computer system is a head-mounted device, the user's viewpoint is locked to the orientation of the user's head, such that the virtual object is also referred to as a "head-locked virtual object."

[0060] Environment-locked visual objects: A virtual object is environment-locked (alternatively, "world-locked") when a computer system displays a virtual object at a position and / or location in a user's viewpoint that is based on (e.g., selected with reference to and / or anchored to) a location and / or object in a three-dimensional environment (e.g., a physical environment or a virtual environment). As the user's viewpoint moves, the location and / or objects in the environment change relative to the user's viewpoint, which causes the environment-locked virtual object to be displayed at a different location and / or location in the user's viewpoint. For example, an environment-locked virtual object locked to a tree immediately in front of the user is displayed at the center of the user's viewpoint. When the user's viewpoint shifts to the right (e.g., the user's head turns to the right) such that the tree is now to the left of center in the user's viewpoint (e.g., the tree location in the user's viewpoint shifts), the environment-locked virtual object locked to the tree is displayed to the left of center in the user's viewpoint. In other words, the position and / or orientation of the virtual object that is locked to the environment in the user's viewpoint depends on the position and / or orientation of the object in the environment to which the virtual object is locked. In some embodiments, the computer system uses a stationary reference system (e.g., a coordinate system anchored to a fixed position and / or object in the physical environment) to determine the position of the virtual object that is locked to the user's viewpoint. The virtual object that is locked to the environment can be locked to a stationary part of the environment (e.g., a floor, wall, table, or other stationary object), or can be locked to a movable part of the environment (e.g., a vehicle, animal, person, or even a representation of a part of the user's body that moves independently of the user's viewpoint, such as a hand, wrist, arm, or foot of the user) so that the virtual object moves as the viewpoint or that part of the environment moves to maintain a fixed relationship between the virtual object and that part of the environment.

[0061] In some embodiments, an environment-locked or viewpoint-locked virtual object exhibits an inertial following behavior that reduces or delays the movement of the environment-locked or viewpoint-locked virtual object relative to the movement of a reference point followed by the virtual object. In some embodiments, when exhibiting inertial following behavior, when detecting the movement of a reference point (e.g., a portion of the environment, a viewpoint, or a point fixed relative to the viewpoint, such as a point between 5 cm and 300 cm from the viewpoint) that the virtual object is following, the computer system intentionally delays the movement of the virtual object. For example, when the reference point (e.g., a portion of the environment or a viewpoint) moves at a first speed, the virtual object is moved by the device to remain locked to the reference point, but moves at a second speed that is slower than the first speed (e.g., until the reference point stops moving or slows down, at which time the virtual object begins to catch up with the reference point). In some embodiments, when the virtual object exhibits inertial following behavior, the device ignores small amounts of movement of the reference point (e.g., ignoring movements of the reference point below a threshold movement amount, such as moving 0 to 5 degrees or moving 0 to 50 cm). For example, when a reference point (e.g., a portion or viewpoint of an environment to which a virtual object is locked) moves a first amount, the distance between the reference point and the virtual object increases (e.g., because the virtual object is being displayed so as to maintain a fixed or substantially fixed position relative to a viewpoint or portion of the environment different from the reference point to which the virtual object is locked), and when the reference point (e.g., the portion or viewpoint of the environment to which the virtual object is locked) moves a second amount greater than the first amount, the distance between the reference point and the virtual object first increases (e.g., because the virtual object is being displayed so as to maintain a fixed or substantially fixed position relative to a viewpoint or portion of the environment different from the reference point to which the virtual object is locked), and then decreases when the amount of movement of the reference point increases above a threshold (e.g., a “lazy follow” threshold) because the virtual object is moved by the computer system to maintain a fixed or substantially fixed position relative to the reference point. In some embodiments, maintaining a substantially fixed position of the virtual object relative to the reference point includes displaying the virtual object within a threshold distance (e.g., 1 cm, 2 cm, 3 cm, 5 cm, 15 cm, 20 cm, 50 cm) of the reference point in one or more dimensions (e.g., up / down, left / right, and / or forward / backward relative to the position of the reference point).

[0062] In some embodiments, spatial media includes spatial visual media and / or spatial audio. In some embodiments, spatial capture is the capture of spatial media. In some embodiments, spatial visual media (also referred to as stereoscopic media) (e.g., spatial images and / or spatial videos) is a media including two different images or sets of images representing two perspectives with the same or overlapping fields of view for simultaneous display. A first image representing a first perspective is presented to a first eye of a viewer, and a second image representing a second perspective different from the first perspective is presented concurrently to a second eye of the viewer. The first image and the second image have the same or overlapping fields of view. In some embodiments, a computer system displays a first image via a first display positioned for viewing by a first eye of a viewer, and displays a second image concurrently via a second display positioned for viewing by a second eye of a viewer different from the first display. In some embodiments, when viewed together, the first image and the second image create a depth effect and provide a depth perception of the content of the image to the viewer. In some embodiments, a first video representing a first perspective is presented to a first eye of a viewer, and a second video representing a second perspective different from the first perspective is presented concurrently to a second eye of the viewer. The first video and the second video have the same or overlapping fields of view. In some embodiments, when viewed together, the first video and the second video create a depth effect and provide the viewer with a depth perception of the content of the video. In some embodiments, a spatial audio experience in the headphones is produced by manipulating the sounds in the two audio channels (e.g., left and right) of the headphones so that they resemble directional sounds arriving at the ear canal. For example, the headphones may reproduce a spatial audio signal that simulates the soundscape around the listener (also referred to as the user). Effective spatial sound reproduction may present sounds so that the listener perceives the sounds as coming from a location within the soundscape outside the listener's head, just as the listener would experience the sounds if they were encountering the sounds in the real world.

[0063] The geometry of the listener's ear, and in particular the outer ear (pinna), has a significant effect on the sound that reaches the listener's eardrum from the sound source. By taking into account the effect of the listener's pinna, the listener's head, and the listener's torso on the sound that enters the listener's ear canal, a spatial audio sound experience may be achieved. The geometry of the user's ear is optionally determined using a three-dimensional scanning device that produces a three-dimensional model of at least a portion of the visible portion of the user's ear. The geometry is optionally used to generate filters for producing a spatial audio experience. In some embodiments, spatial audio is audio that has been filtered so that a listener of the audio perceives the audio as coming from one or more directions and / or locations in three-dimensional space (e.g., from above, below, and / or in front of the listener).

[0064] An example of such a filter is a head-related transfer function (HRTF) filter. These filters are used to provide an effect similar to how the human ear, head, and torso filter sound. When the geometry of the listener's ear is known, a personalized filter (e.g., a personalized HRTF filter) can be generated so that the sound experienced by the listener through headphones (e.g., in-ear headphones, on-ear headphones, and / or over-ear headphones) is more realistic. In some embodiments, two filters are generated - one filter for each ear - so that each ear of the listener has a corresponding personalized filter (e.g., a personalized HRTF filter), because the listener's ears can have different geometries.

[0065] In some embodiments, the HRTF filter includes some (or all) of the acoustic information needed to describe how sound is reflected or diffracted around the listener's head before entering the listener's auditory system. In some embodiments, a personalized HRTF filter may be selected from a database of previously determined HRTFs for users with similar anatomical characteristics. In some embodiments, a personalized HRTF filter may be generated by digital modeling based on the geometry of the listener's ear. One or more processors of the computer system optionally apply a personalized HRTF filter for the listener to an audio input signal to generate a spatial input signal for playback by headphones connected (e.g., wirelessly or wired) to the computer system.

[0066] Hardware: There are many different types of electronic systems that enable people to sense various XR environments and / or interact with various CGR environments. Examples include head-mounted systems, projection-based systems, heads-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays formed as lenses designed to be placed on people's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without tactile feedback), smartphones, tablet devices, and desktop / laptop computers. A head-mounted system may include speakers and / or other audio output devices integrated into the head-mounted system for providing audio output. A head-mounted system may have one or more speakers and an integrated opaque display. Alternatively, a head-mounted system may be configured to accept an external opaque display (e.g., a smartphone). A head-mounted system may incorporate one or more imaging sensors for capturing images or video of a physical environment and / or one or more microphones for capturing audio of a physical environment. Instead of an opaque display, a head-mounted system may have a transparent or translucent display. A transparent or translucent display can have a medium through which light representing an image is directed to a person's eyes. The display can utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser scanning light sources, or any combination of these technologies. The medium can be an optical waveguide, a hologram medium, an optical combiner, an optical reflector, or any combination thereof. In one embodiment, a transparent or translucent display can be configured to selectively become opaque. A projection-based system can employ retinal projection technology that projects graphic images onto a person's retina. The projection system can also be configured to project virtual objects into a physical environment, such as as a hologram or on a physical surface. In some embodiments, the controller 110 is configured to manage and coordinate the user's XR experience. In some embodiments, the controller 110 includes a suitable combination of software, firmware, and / or hardware. The following description with respect to Figure 2Controller 110 is described in more detail. In some embodiments, controller 110 is a computing device that is located locally or remotely relative to scene 105 (e.g., physical environment). For example, controller 110 is a local server located within scene 105. As another example, controller 110 is a remote server (e.g., cloud server, central server, etc.) located outside scene 105. In some embodiments, controller 110 is communicatively coupled to display generation component 120 (e.g., HMD, display, projector, touch screen, etc.) via one or more wired or wireless communication channels 144 (e.g., Bluetooth, IEEE802.11x, IEEE 802.16x, IEEE 802.3x, etc.). In another example, the controller 110 is included within a housing (e.g., a physical housing) of the display generating component 120 (e.g., an HMD or a portable electronic device including a display and one or more processors, etc.), one or more input devices of the input devices 125, one or more output devices of the output devices 155, one or more sensors of the sensors 190, and / or one or more peripheral devices of the peripheral devices 195, or shares the same physical housing or support structure with one or more of the above devices.

[0067] In some embodiments, the display generation component 120 is configured to provide an XR experience (e.g., at least the visual component of the XR experience) to the user. In some embodiments, the display generation component 120 includes a suitable combination of software, firmware, and / or hardware. Figure 3 Display generation component 120 is described in further detail. In some embodiments, the functionality of controller 110 is provided by and / or combined with display generation component 120.

[0068] According to some embodiments, display generation component 120 provides an XR experience to the user when the user is virtually and / or physically present within scene 105.

[0069] In some embodiments, the display generation component is worn on a part of the user's body (e.g., on his / her head, on his / her hand, etc.). In this way, the display generation component 120 includes one or more XR displays provided for displaying XR content. For example, in various embodiments, the display generation component 120 surrounds the user's field of view. In some embodiments, the display generation component 120 is a handheld device (such as a smart phone or tablet device) configured to present XR content, and the user holds a device with a display facing the user's field of view and a camera facing the scene 105. In some embodiments, the handheld device is optionally placed in a shell worn on the user's head. In some embodiments, the handheld device is optionally placed on a support (e.g., a tripod) in front of the user. In some embodiments, the display generation component 120 is an XR room, shell, or room configured to present XR content, wherein the user does not wear or hold the display generation component 120. Many user interfaces described with reference to one type of hardware for displaying XR content (e.g., a handheld device or a device on a tripod) may be implemented on another type of hardware for displaying XR content (e.g., an HMD or other wearable computing device). For example, a user interface showing interactions with XR content triggered based on interactions occurring in the space in front of a handheld device or a tripod-mounted device may similarly be implemented with an HMD, where the interactions occur in the space in front of the HMD and responses to the XR content are displayed via the HMD. Similarly, a user interface showing interactions with XR content triggered based on movement of a handheld device or a tripod-mounted device relative to a physical environment (e.g., scene 105 or a part of a user's body (e.g., the user's eyes, head, or hands)) may similarly be implemented with an HMD, where the movement is caused by movement of the HMD relative to the physical environment (e.g., scene 105 or a part of a user's body (e.g., the user's eyes, head, or hands)).

[0070] Despite Figure 1 Relevant features of operating environment 100 are shown, but those skilled in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure more relevant aspects of the example embodiments disclosed herein.

[0071] Figure 21 is a block diagram of an example of a controller 110 in some embodiments. Although some specific features are shown, those skilled in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure more relevant aspects of the embodiments disclosed herein. To this end, as a non-limiting example, in some embodiments, the controller 110 includes one or more processing units 202 (e.g., a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a graphics processing unit (GPU), a central processing unit (CPU), a processing core, etc.), one or more input / output (I / O) devices 206, one or more communication interfaces 208 (e.g., a universal serial bus (USB), IEEE802.3x, IEEE 802.11x, IEEE 802.16x, Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Global Positioning System (GPS), infrared (IR), Bluetooth, ZIGBEE and / or similar types of interfaces), one or more programming (e.g., I / O) interfaces 210, a memory 220, and one or more communication buses 204 for interconnecting these components and various other components.

[0072] In some embodiments, the one or more communication buses 204 include circuits that interconnect and control communications between system components. In some embodiments, the one or more I / O devices 206 include at least one of a keyboard, a mouse, a touch pad, a joystick, one or more microphones, one or more speakers, one or more image sensors, one or more displays, and the like.

[0073] The memory 220 includes a high-speed random access memory, such as a dynamic random access memory (DRAM), a static random access memory (SRAM), a double data rate random access memory (DDR RAM), or other random access solid-state memory devices. In some embodiments, the memory 220 includes a non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 220 optionally includes one or more storage devices located away from the one or more processing units 202. The memory 220 includes a non-transitory computer-readable storage medium. In some embodiments, the memory 220 or the non-transitory computer-readable storage medium of the memory 220 stores the following programs, modules, and data structures or subsets thereof, including an optional operating system 230 and an XR experience module 240.

[0074] The operating system 230 includes instructions for handling various basic system services and for performing hardware-related tasks. In some embodiments, the XR experience module 240 is configured to manage and coordinate single or multiple XR experiences of one or more users (e.g., a single XR experience of one or more users, or multiple XR experiences of corresponding groups of one or more users). To this end, in various embodiments, the XR experience module 240 includes a data acquisition unit 241, a tracking unit 242, a coordination unit 246, and a data transmission unit 248.

[0075] In some embodiments, the data acquisition unit 241 is configured to obtain Figure 1 120, and optionally acquires data (e.g., presentation data, interaction data, sensor data, location data, etc.) from one or more of input device 125, output device 155, sensor 190, and / or peripheral device 195. To this end, in various embodiments, data acquisition unit 241 includes instructions and / or logic for instructions as well as heuristics and metadata for the heuristics.

[0076] In some embodiments, tracking unit 242 is configured to map scene 105 and track at least display generation component 120 relative to Figure 1 105, and optionally tracks the position / location of one or more of the input device 125, the output device 155, the sensor 190, and / or the peripheral device 195. To this end, in various embodiments, the tracking unit 242 includes instructions and / or logic for the instructions and heuristics and metadata for the heuristics. In some embodiments, the tracking unit 242 includes a hand tracking unit 244 and / or an eye tracking unit 243. In some embodiments, the hand tracking unit 244 is configured to track the position / location of one or more parts of the user's hand and / or the position of one or more parts of the user's hand relative to the user's hand. Figure 1 The movement of the scene 105 relative to the display generation component 120 and / or relative to a coordinate system (the coordinate system is defined relative to the user's hand). Figure 4 The hand tracking unit 244 is described in more detail. In some embodiments, the eye tracking unit 243 is configured to track the position or movement of the user's gaze (or more generally, the user's eyes, face, or head) relative to the scene 105 (e.g., relative to the physical environment and / or relative to the user (e.g., the user's hands)) or relative to the XR content displayed via the display generation component 120. Figure 5 The eye tracking unit 243 is described in more detail.

[0077] In some embodiments, coordination unit 246 is configured to manage and coordinate the XR experience presented to the user by display generation component 120, and optionally by one or more of output device 155 and / or peripherals 195. To this end, in various embodiments, coordination unit 246 includes instructions and / or logic for instructions and heuristics and metadata for the heuristics.

[0078] In some embodiments, data transfer unit 248 is configured to transfer data (e.g., presentation data, position data, etc.) to at least display generation component 120, and optionally to one or more of input device 125, output device 155, sensor 190, and / or peripheral device 195. To this end, in various embodiments, data transfer unit 248 includes instructions and / or logic for instructions as well as heuristics and metadata for the heuristics.

[0079] Although the data acquisition unit 241, the tracking unit 242 (e.g., including the eye tracking unit 243 and the hand tracking unit 244), the coordination unit 246, and the data transmission unit 248 are illustrated as residing on a single device (e.g., the controller 110), it should be understood that in other embodiments, any combination of the data acquisition unit 241, the tracking unit 242 (e.g., including the eye tracking unit 243 and the hand tracking unit 244), the coordination unit 246, and the data transmission unit 248 may be located in separate computing devices.

[0080] also, Figure 2 It serves more as a functional description of various features that may be present in a particular implementation, as opposed to a block diagram of the embodiments described herein. As one of ordinary skill in the art will recognize, items shown separately may be combined, and some items may be separated. For example, Figure 2 Some functional modules shown separately in the figure can be implemented in a single module, and the various functions of a single functional block can be implemented by one or more functional blocks in various embodiments. The actual number of modules and the division of specific functions and how the features are distributed among them will vary depending on the specific implementation, and in some embodiments, depends in part on the specific combination of hardware, software and / or firmware selected for a specific implementation.

[0081] Figure 31 is a block diagram of an example of a display generation component 120 in some embodiments. Although some specific features are shown, those skilled in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure more relevant aspects of the embodiments disclosed herein. For this purpose, as a non-limiting example, in some embodiments, the display generation component 120 (e.g., an HMD) includes one or more processing units 302 (e.g., a microprocessor, an ASIC, an FPGA, a GPU, a CPU, a processing core, etc.), one or more input / output (I / O) devices and sensors 306, one or more communication interfaces 308 (e.g., USB, FIREWIRE, THUNDERBOLT, IEEE 802.3x, IEEE 802.11x, IEEE 802.16x, GSM, CDMA, TDMA, GPS, IR, BLUETOOTH, ZIGBEE and / or similar types of interfaces), one or more programming (e.g., I / O) interfaces 310, one or more XR displays 312, one or more optional internal-facing and / or external-facing image sensors 314, a memory 320, and one or more communication buses 304 for interconnecting these components and various other components.

[0082] In some embodiments, one or more communication buses 304 include circuits for interconnecting and controlling communications between various system components. In some embodiments, one or more I / O devices and sensors 306 include an inertial measurement unit (IMU), an accelerometer, a gyroscope, a thermometer, one or more physiological sensors (e.g., a blood pressure monitor, a heart rate monitor, a blood oxygen sensor, a blood glucose sensor, etc.), one or more microphones, one or more speakers, a haptic engine, and / or one or more depth sensors (e.g., structured light, time of flight, etc.), etc.

[0083] In some embodiments, one or more XR displays 312 are configured to provide an XR experience to the user. In some embodiments, one or more XR displays 312 correspond to holographic, digital light processing (DLP), liquid crystal display (LCD), liquid crystal on silicon (LCoS), organic light emitting field effect transistor (OLET), organic light emitting diode (OLED), surface conduction electron emission display (SED), field emission display (FED), quantum dot light emitting diode (QD-LED), microelectromechanical system (MEMS) and / or similar display types. In some embodiments, one or more XR displays 312 correspond to diffraction, reflection, polarization, holographic and other waveguide displays. For example, the display generation component 120 (e.g., HMD) includes a single XR display. In another example, the display generation component 120 includes an XR display for each eye of the user. In some embodiments, one or more XR displays 312 can present MR and VR content. In some embodiments, one or more XR displays 312 can present MR or VR content.

[0084] In some embodiments, the one or more image sensors 314 are configured to acquire image data corresponding to at least a portion of the user's face, including the user's eyes (and may be referred to as an eye tracking camera). In some embodiments, the one or more image sensors 314 are configured to acquire image data corresponding to the user's hands and, optionally, at least a portion of the user's arms (and may be referred to as a hand tracking camera). In some embodiments, the one or more image sensors 314 are configured to face forward so as to acquire image data corresponding to a scene that the user would see in the absence of the display generating component 120 (e.g., an HMD) (and may be referred to as a scene camera). The one or more optional image sensors 314 may include one or more RGB cameras (e.g., with a complementary metal oxide semiconductor (CMOS) image sensor or a charge coupled device (CCD) image sensor), one or more infrared (IR) cameras, and / or one or more event-based cameras, etc.

[0085] The memory 320 includes a high-speed random access memory, such as a DRAM, SRAM, DDR RAM, or other random access solid-state memory device. In some embodiments, the memory 320 includes a non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 320 optionally includes one or more storage devices located away from the one or more processing units 302. The memory 320 includes a non-transitory computer-readable storage medium. In some embodiments, the memory 320 or the non-transitory computer-readable storage medium of the memory 320 stores the following programs, modules, and data structures or a subset thereof, including an optional operating system 330 and an XR rendering module 340.

[0086] The operating system 330 includes processes for handling various basic system services and for performing hardware-related tasks. In some embodiments, the XR rendering module 340 is configured to present XR content to a user via one or more XR displays 312. To this end, in various embodiments, the XR rendering module 340 includes a data acquisition unit 342, an XR rendering unit 344, an XR map generation unit 346, and a data transmission unit 348.

[0087] In some embodiments, the data acquisition unit 342 is configured to at least Figure 1 The controller 110 acquires data (e.g., presentation data, interaction data, sensor data, location data, etc.). For the purposes described, in various embodiments, the data acquisition unit 342 includes instructions and / or logic for instructions and heuristics and metadata for the heuristics.

[0088] In some embodiments, the XR rendering unit 344 is configured to render XR content via one or more XR displays 312. For such purposes, in various embodiments, the XR rendering unit 344 includes instructions and / or logic for the instructions and heuristics and metadata for the heuristics.

[0089] In some embodiments, the XR map generation unit 346 is configured to generate an XR map (e.g., a 3D map of a mixed reality scene or a map of a physical environment in which computer-generated objects can be placed to generate an extended reality) based on the media content data. For such purposes, in various embodiments, the XR map generation unit 346 includes instructions and / or logic for the instructions and heuristics and metadata for the heuristics.

[0090] In some embodiments, the data transmission unit 348 is configured to transmit data (e.g., presentation data, position data, etc.) to at least the controller 110, and optionally one or more of the input device 125, the output device 155, the sensor 190, and / or the peripheral device 195. For such purposes, in various embodiments, the data transmission unit 348 includes instructions and / or logic for the instructions and heuristics and metadata for the heuristics.

[0091] Although the data acquisition unit 342, the XR rendering unit 344, the XR map generation unit 346, and the data transfer unit 348 are shown as residing on a single device (e.g., Figure 1 , but it should be understood that in other embodiments, any combination of the data acquisition unit 342, the XR rendering unit 344, the XR mapping generation unit 346, and the data transmission unit 348 may be located in a separate computing device.

[0092] also, Figure 3 More as a functional description of various features that may be present in a particular embodiment, rather than a schematic diagram of the structures of the embodiments described herein. As one of ordinary skill in the art will recognize, items shown separately may be combined, and some items may be separated. For example, Figure 3 Some functional modules shown separately in the figure may be implemented in a single module, and the various functions of a single functional block may be implemented by one or more functional blocks in various embodiments. The actual number of modules and the division of specific functions and how the features are distributed among them will vary depending on the specific implementation, and in some embodiments, depends in part on the specific combination of hardware, software and / or firmware selected for a specific implementation.

[0093] Figure 4 is a schematic illustration of an example implementation of the hand tracking device 140. In some implementations, the hand tracking device 140 ( Figure 1 ) is controlled by the hand tracking unit 244 ( Figure 2 ) to track the position / location of one or more parts of a user's hand and / or one or more parts of a user's hand relative to Figure 1The hand tracking device 140 can be used to track movement of the scene 105 (e.g., relative to a portion of the physical environment surrounding the user, relative to the display generation component 120, or relative to a portion of the user (e.g., the user's face, eyes, or head), and / or relative to a coordinate system (which is defined relative to the user's hands)). In some embodiments, the hand tracking device 140 is part of the display generation component 120 (e.g., embedded in or attached to the head-mounted device). In some embodiments, the hand tracking device 140 is separate from the display generation component 120 (e.g., located in a separate housing or attached to a separate physical support structure).

[0094] In some embodiments, the hand tracking device 140 includes an image sensor 404 (e.g., one or more IR cameras, 3D cameras, depth cameras, and / or color cameras, etc.) that captures three-dimensional scene information including at least a human user's hand 406. The image sensor 404 captures hand images with sufficient resolution to enable the fingers and their corresponding positioning to be distinguished. The image sensor 404 typically captures images of other parts of the user's body, and also or possibly all parts of the body, and may have zoom capabilities or a dedicated sensor with increased magnification to capture images of the hand with a desired resolution. In some embodiments, the image sensor 404 also captures 2D color video images of the hand 406 and other elements of the scene. In some embodiments, the image sensor 404 is used in conjunction with other image sensors to capture the physical environment of the scene 105, or is used as an image sensor to capture the physical environment of the scene 105. In some embodiments, the image sensor is positioned relative to the user or the user's environment in a manner that the field of view of the image sensor 404 or a portion thereof is used to define an interaction space, in which the hand movements captured by the image sensor are considered as input to the controller 110.

[0095] In some embodiments, the image sensor 404 outputs a sequence of frames containing 3D image data (and, in addition, possible color image data) to the controller 110, which extracts high-level information from the image data. The high-level information is typically provided to an application running on the controller via an application program interface (API), which drives the display generation component 120 accordingly. For example, a user can interact with the software running on the controller 110 by moving his hand 406 and changing his hand posture.

[0096] In some embodiments, the image sensor 404 projects a speckled pattern onto a scene containing the hand 406 and captures an image of the projected pattern. In some embodiments, the controller 110 calculates the 3D coordinates of points in the scene (including points on the surface of the user's hand) by triangulation based on the lateral offset of the spots in the pattern. This method is advantageous because it does not require the user to hold or wear any kind of beacon, sensor, or other marker. The method gives the depth coordinates of points in the scene at a specific distance from the image sensor 404 relative to a predetermined reference plane. In the present disclosure, it is assumed that the image sensor 404 defines an orthogonal set of x-axis, y-axis, and z-axis, so that the depth coordinates of points in the scene correspond to the z component measured by the image sensor. Alternatively, the image sensor 404 (e.g., a hand tracking device) may use other 3D mapping methods based on a single or multiple cameras or other types of sensors, such as stereo imaging or time-of-flight measurement.

[0097] In some embodiments, the hand tracking device 140 captures and processes a time series of depth maps containing the user's hand as the user moves his hand (e.g., the entire hand or one or more fingers). Software running on the image sensor 404 and / or the processor in the controller 110 processes the 3D map data to extract image patch descriptors of the hand in these depth maps. The software can match these descriptors with image patch descriptors stored in the database 408 based on a previous learning process to estimate the pose of the hand in each frame. The pose typically includes the 3D position of the user's hand joints and finger tips.

[0098] The software can also analyze the trajectory of the hand and / or finger over multiple frames in the sequence to identify gestures. The pose estimation function described herein can be alternated with the motion tracking function so that only image block-based pose estimation is performed once every two (or more) frames, and tracking is used to find changes in pose that occur on the remaining frames. The pose, motion, and gesture information is provided to the application running on the controller 110 via the above-mentioned API. The program can, for example, move and modify the image presented on the display generation component 120 in response to the pose and / or gesture information, or perform other functions.

[0099] In some embodiments, gestures include air gestures. An air gesture is a gesture detected without the user touching an input element that is part of a device (e.g., computer system 101, one or more input devices 125, and / or hand tracking device 140) (or independent of an input element that is part of a device) and based on detected movement of a part of the user's body (e.g., head, one or more arms, one or more hands, one or more fingers, and / or one or more legs) through the air (including movement of the user's body relative to an absolute reference (e.g., the angle of the user's arm relative to the ground or the distance of the user's hand relative to the ground), movement relative to another part of the user's body (e.g., movement of the user's hand relative to the user's shoulder, movement of one of the user's hands relative to the user's other hand, and / or movement of a user's finger relative to another finger or part of the user's hand), and / or absolute movement of a part of the user's body (e.g., a tap gesture in which a hand moves a predetermined amount and / or speed in a predetermined posture, or a shake gesture including a predetermined speed or amount of rotation of a part of the user's body)).

[0100] In some embodiments, the input gestures used in the various examples and embodiments described herein include air gestures for interacting with an XR environment (e.g., a virtual or mixed reality environment) performed in some embodiments by movement of a user's fingers relative to other fingers (or parts of a user's hand). In some embodiments, an air gesture is a gesture detected without the user touching an input element that is part of the device (or independent of an input element that is part of the device) and based on detected movement of a part of the user's body through the air (including movement of the user's body relative to an absolute reference (e.g., the angle of the user's arm relative to the ground or the distance of the user's hand relative to the ground), movement relative to another part of the user's body (e.g., movement of the user's hand relative to the user's shoulder, movement of one of the user's hands relative to the user's other hand, and / or movement of the user's fingers relative to another finger or part of the user's hand), and / or absolute movement of a part of the user's body (e.g., a tap gesture in which the hand moves a predetermined amount and / or speed in a predetermined posture, or a shake gesture including a predetermined speed or amount of rotation of a part of the user's body)).

[0101] In some embodiments where the input gesture is an in-air gesture (e.g., in the absence of physical contact with an input device that provides information to the computer system about which user interface element is the target of the user input, such as contact with a user interface element displayed on a touch screen, or contact with a mouse or trackpad to move a cursor to a user interface element), the gesture takes into account the user's attention (e.g., gaze) to determine the target of the user input (e.g., for direct input, as described below). Thus, in embodiments involving in-air gestures, for example, the input gesture is combined (e.g., simultaneously) with movement of the user's fingers and / or hand to detect attention (e.g., gaze) toward a user interface element to perform a pinch and / or tap input, as described below.

[0102] In some embodiments, an input gesture pointing to a user interface object is performed directly or indirectly with reference to the user interface object. For example, when an input gesture is performed with the user's hand at a location corresponding to the location of the user interface object in a three-dimensional environment (e.g., as determined based on the user's current viewpoint), the user input is performed directly on the user interface object. In some embodiments, when the user's attention (e.g., gaze) to the user interface object is detected, the input gesture is indirectly performed on the user interface object based on the position of the user's hand not being at the location corresponding to the location of the user interface object in the three-dimensional environment while the user performs the input gesture. For example, for a direct input gesture, the user can guide the user's input to the user interface object by initiating a gesture at or near a location corresponding to the display location of the user interface object (e.g., within 0.5 cm, 1 cm, 5 cm, or a distance between 0 and 5 cm measured from the outer edge of the option or the center portion of the option). For an indirect input gesture, the user can guide the user's input to the user interface object by focusing on the user interface object (e.g., by gazing at the user interface object), and while focusing on the option, the user initiates an input gesture (e.g., at any location detectable by the computer system) (e.g., at a location that does not correspond to the display location of the user interface object).

[0103] In some embodiments, the input gestures (e.g., air gestures) used in the various examples and embodiments described herein include pinch inputs and tap inputs for interacting with a virtual or mixed reality environment in some embodiments. For example, the pinch inputs and tap inputs described below are performed as air gestures.

[0104] In some embodiments, the pinch input is part of an air gesture that includes one or more of the following: a pinch gesture, a long pinch gesture, a pinch and drag gesture, or a double pinch gesture. For example, a pinch gesture as an air gesture includes movement of two or more fingers of a hand to contact each other, that is, optionally followed by an immediate (e.g., within 0 to 1 second) interruption of contact with each other. A long pinch gesture as an air gesture includes movement of two or more fingers of a hand in contact with each other for at least a threshold amount of time (e.g., at least 1 second) before an interruption of contact with each other is detected. For example, a long pinch gesture includes the user maintaining a pinch gesture (e.g., in which two or more fingers are in contact), and the long pinch gesture continues until an interruption of contact between the two or more fingers is detected. In some embodiments, a double pinch gesture as an air gesture includes two (e.g., or more) pinch inputs (e.g., performed by the same hand) that are detected consecutively immediately (e.g., within a predefined time period) with each other. For example, the user performs a first pinch input (e.g., a pinch input or a long pinch input), releases the first pinch input (e.g., interrupts contact between two or more fingers), and performs a second pinch input within a predefined time period after releasing the first pinch input (e.g., within 1 second or within 2 seconds).

[0105] In some embodiments, a pinch and drag gesture as an air gesture includes a pinch gesture (e.g., a pinch gesture or a long pinch gesture) performed in conjunction with (e.g., following) a drag input that changes the position of the user's hand from a first position (e.g., a starting position for dragging) to a second position (e.g., an ending position for dragging). In some embodiments, the user maintains the pinch gesture while performing the drag input, and releases the pinch gesture (e.g., opens their two or more fingers) to end the drag gesture (e.g., at the second position). In some embodiments, the pinch input and the drag input are performed by the same hand (e.g., the user pinches two or more fingers to contact each other and moves the same hand to a second position in the air using a drag gesture). In some embodiments, the pinch input is performed by the user's first hand, and the drag input is performed by the user's second hand (e.g., the user's second hand moves from the first position to the second position in the air while the user continues the pinch input with the user's first hand). In some embodiments, an input gesture as an air gesture includes an input performed using both of the user's hands (e.g., a pinch and / or tap input). For example, the input gesture includes two (e.g., or more) pinch inputs that are performed in conjunction with each other (e.g., concurrently or within a predefined time period). For example, a first pinch gesture (e.g., a pinch input, a long pinch input, or a pinch and drag input) is performed using a first hand of a user, and a second pinch input is performed using another hand (e.g., a second hand of the user's two hands) in conjunction with the pinch input performed using the first hand. In some embodiments, movement between the user's two hands (e.g., increasing and / or decreasing the distance or relative orientation between the user's two hands).

[0106] In some embodiments, a tap input performed as an air gesture (e.g., pointing to a user interface element) includes movement of a user's finger toward the user interface element, movement of a user's hand toward the user interface element (optionally, extension of the user's finger toward the user interface element), downward movement of a user's finger (e.g., mimicking a mouse click motion or a tap on a touch screen), or other predefined movement of a user's hand. In some embodiments, a tap input performed as an air gesture is detected based on movement characteristics of the finger or hand performing the tap gesture movement of the finger or hand, which is a movement of the finger or hand away from the user's viewpoint and / or toward an object that is the target of the tap input, followed by an end of the movement. In some embodiments, the end of the movement is detected based on a change in the movement characteristics of the finger or hand performing the tap gesture (e.g., the end of the movement away from the user's viewpoint and / or toward an object that is the target of the tap input, a reversal of the direction of movement of the finger or hand, and / or a reversal of the acceleration direction of the movement of the finger or hand).

[0107] In some embodiments, it is determined that the user's attention is directed to a portion of the three-dimensional environment based on detection of a gaze directed to that portion of the three-dimensional environment (optionally, no other conditions are required). In some embodiments, it is determined that the user's attention is directed to a portion of the three-dimensional environment based on detection of a gaze directed to that portion of the three-dimensional environment using one or more additional conditions, such as requiring the gaze to be directed to the portion of the three-dimensional environment for at least a threshold duration (e.g., a dwell duration) and / or requiring the gaze to be directed to the portion of the three-dimensional environment when the user's viewpoint is within a distance threshold from the portion of the three-dimensional environment, so that the device determines that the user's attention is directed to the portion of the three-dimensional environment, wherein if one of these additional conditions is not met, the device determines that the attention is not directed to the portion of the three-dimensional environment to which the gaze is directed (e.g., until the one or more additional conditions are met).

[0108] In some embodiments, the detection of a ready state configuration of a user or a portion of a user is detected by a computer system. The detection of the ready state configuration of a hand is used by the computer system as an indication that the user may be preparing to interact with the computer system using one or more air gesture inputs (e.g., pinch, tap, pinch and drag, double pinch, long pinch, or other air gestures described herein) performed by the hand. For example, the ready state of the hand is determined based on whether the hand has a predetermined hand shape (e.g., a pre-pinch shape with the thumb and one or more fingers extended and spaced apart to prepare for a pinch or grab gesture, or a pre-tap with one or more fingers extended and the palm facing away from the user), based on whether the hand is in a predetermined position relative to the user's viewpoint (e.g., below the user's head and above the user's waist and extending at least 15 cm, 20 cm, 25 cm, 30 cm, or 50 cm from the body), and / or based on whether the hand has moved in a particular manner (e.g., toward an area in front of the user above the user's waist and below the user's head or away from the user's body or legs). In some embodiments, the ready state is used to determine whether an interactive element of a user interface responds to attention (e.g., gaze) input.

[0109] In scenarios where input is described with reference to in-air gestures, it should be understood that similar gestures may be detected using a hardware input device attached to or held by one or more hands of a user, where the positioning of the hardware input device in space may be tracked using optical tracking, one or more accelerometers, one or more gyroscopes, one or more magnetometers, and / or one or more inertial measurement units, and the positioning and / or movement of the hardware input device is used instead of the positioning and / or movement of the one or more hands in the corresponding in-air gesture. In scenarios where input is described with reference to an air gesture, it should be understood that similar gestures may be detected using a hardware input device attached to or held by one or more hands of a user, user input may be detected using controls contained in the hardware input device, such as one or more touch-sensitive input elements, one or more pressure-sensitive input elements, one or more buttons, one or more knobs, one or more dials, one or more joysticks, one or more hand or finger overlays that can detect the positioning or changes in positioning of parts of a hand and / or finger relative to each other, relative to the user's body and / or relative to the user's physical environment, and / or other hardware input device controls, where user input performed using controls contained in the hardware input device is used in place of a hand and / or finger gesture such as an air tap or air pinch in a corresponding air gesture. For example, a selection input described as being performed using an air tap or air pinch input may alternatively be detected using a button press, a tap on a touch-sensitive surface, a press on a pressure-sensitive surface, or other hardware input. As another example, movement input described as being performed using an air pinch and drag may be optionally detected based on interaction with a hardware input control, such as a button press and hold, a touch on a touch-sensitive surface, a press on a pressure-sensitive surface, or other hardware input following movement of a hardware input device (e.g., along with a hand associated with the hardware input device) through space. Similarly, two-handed input involving movement of hands relative to each other may be performed using one air gesture and one hardware input device in which the hand is not performing the air gesture, two hardware input devices held in different hands, or two air gestures performed by different hands using various combinations of air gestures and / or inputs detected by one or more of the above hardware input devices.

[0110] In some embodiments, the software may be downloaded to the controller 110 in electronic form, for example, over a network, or may alternatively be provided on a tangible, non-transitory medium such as an optical, magnetic, or electronic memory medium. In some embodiments, the database 408 is also stored in a memory associated with the controller 110. Alternatively or in addition, some or all of the described functions of the computer may be implemented in dedicated hardware, such as a custom or semi-custom integrated circuit or a programmable digital signal processor (DSP). Although in Figure 4Controller 110 is shown in FIG. 1 , but some or all of the processing functions of the controller may be performed by a suitable microprocessor and software or by dedicated circuitry within the housing of the image sensor 404 (e.g., a hand tracking device) or other device associated with the image sensor 404, for example, as a separate unit from the image sensor 404. In some embodiments, at least some of these processing functions may be performed by a suitable processor integrated with the display generation component 120 (e.g., in a television receiver, handheld device, or head-mounted device) or with any other suitable computerized device (such as a game console or media player). The sensing functions of the image sensor 404 may also be integrated into a computer or other computerized device to be controlled by the sensor output.

[0111] Figure 4 Also included is a schematic representation of a depth map 410 captured by the image sensor 404 in some embodiments. As described above, the depth map includes a matrix of pixels with corresponding depth values. Pixels 412 corresponding to the hand 406 have been segmented from the background and wrist in the figure. The brightness of each pixel within the depth map 410 is inversely proportional to its depth value (i.e., the measured z distance from the image sensor 404), where gray shades become darker as depth increases. The controller 110 processes these depth values ​​in order to identify and segment components of the image that have human hand characteristics (i.e., a group of adjacent pixels). These characteristics may include, for example, overall size, shape, and movement from frame to frame in the depth map sequence.

[0112] Figure 4 Also schematically shown is a hand skeleton 414 that the controller 110 ultimately extracts from the depth map 410 of the hand 406 in some embodiments. Figure 4 In the hand skeleton 414, a hand background 416 that has been segmented from the original depth map is superimposed. In some embodiments, key feature points of the hand and optionally on the wrist or arm connected to the hand (e.g., points corresponding to knuckles, finger tips, palm center, end of the hand connected to the wrist, etc.) are identified and located on the hand skeleton 414. In some embodiments, the controller 110 uses the position and movement of these key feature points over multiple image frames to determine, in some embodiments, the gesture performed by the hand or the current state of the hand.

[0113] Figure 5 The eye tracking device 130 ( Figure 1 ). In some embodiments, the eye tracking device 130 is composed of an eye tracking unit 243 ( Figure 2) controls to track the position and movement of the user's gaze relative to the scene 105 or relative to the XR content displayed via the display generation component 120. In some embodiments, the eye tracking device 130 is integrated with the display generation component 120. For example, in some embodiments, when the display generation component 120 is a head-mounted device (such as headphones, helmets, goggles, or glasses) or a handheld device placed in a wearable frame, the head-mounted device includes both components for generating XR content for viewing by the user and components for tracking the user's gaze relative to the XR content. In some embodiments, the eye tracking device 130 is separate from the display generation component 120. For example, when the display generation component is a handheld device or an XR room, the eye tracking device 130 is optionally a device separate from the handheld device or the XR room. In some embodiments, the eye tracking device 130 is a head-mounted device or a part of a head-mounted device. In some embodiments, the head-mounted eye tracking device 130 is optionally used in conjunction with a display generation component that is also head-mounted or a display generation component that is not head-mounted. In some embodiments, the eye tracking device 130 is not a head mounted device, and is optionally used in conjunction with a head mounted display generation component. In some embodiments, the eye tracking device 130 is not a head mounted device, and is optionally part of a non-head mounted display generation component.

[0114] In some embodiments, the display generation component 120 uses a display mechanism (e.g., a left near-eye display panel and a right near-eye display panel) to display a frame including a left image and a right image in front of the user's eyes, thereby providing a 3D virtual view to the user. For example, the head-mounted display generation component may include a left optical lens and a right optical lens (referred to herein as eye lenses) located between the display and the user's eyes. In some embodiments, the display generation component may include or be coupled to one or more external cameras that capture a video of the user's environment for display. In some embodiments, the head-mounted display generation component may have a transparent or translucent display, and display a virtual object on the transparent or translucent display, and the user can directly view the physical environment through the transparent or translucent display. In some embodiments, the display generation component projects the virtual object into the physical environment. The virtual object may, for example, be projected on a physical surface or projected as a hologram so that an individual using the system observes the virtual object superimposed on the physical environment. In this case, separate display panels and image frames for the left and right eyes may not be required.

[0115] like Figure 5As shown in , in some embodiments, the eye tracking device 130 (e.g., gaze tracking device) includes at least one eye tracking camera (e.g., an infrared (IR) or near infrared (NIR) camera), and an illumination source (e.g., an IR or NIR light source, such as an array or ring of LEDs) that emits light (e.g., IR or NIR light) toward the user's eyes. The eye tracking camera can be pointed at the user's eyes to receive the IR or NIR light reflected directly from the eyes by the light source, or alternatively can be pointed at "hot" mirrors located between the user's eyes and the display panel, which reflect the IR or NIR light from the eyes to the eye tracking camera while allowing visible light to pass through. The eye tracking device 130 optionally captures images of the user's eyes (e.g., as a video stream captured at 60-120 frames per second (fps)), analyzes the images to generate gaze tracking information, and transmits the gaze tracking information to the controller 110. In some embodiments, the user's two eyes are tracked separately by corresponding eye tracking cameras and illumination sources. In some embodiments, only one eye of the user is tracked by the corresponding eye tracking camera and illumination source.

[0116] In some embodiments, the eye tracking device 130 is calibrated using a device-specific calibration process to determine the parameters of the eye tracking device for a specific operating environment 100, such as the 3D geometric relationships and parameters of the LED, camera, thermal mirror (if present), eye lens, and display screen. The device-specific calibration process can be performed at a factory or another facility before the AR / VR equipment is delivered to the end user. The device-specific calibration process can be an automatic calibration process or a manual calibration process. The user-specific calibration process may include estimating the eye parameters of a specific user, such as pupil position, foveal position, optical axis, visual axis, eye spacing, etc. In some embodiments, once the device-specific parameters and user-specific parameters are determined for the eye tracking device 130, a flash-assisted method can be used to process the images captured by the eye tracking camera to determine the user's current visual axis and gaze point relative to the display.

[0117] like Figure 5As shown in , the eye tracking device 130 (e.g., 130A or 130B) includes an eye lens 520 and a gaze tracking system, which includes at least one eye tracking camera 540 (e.g., an infrared (IR) or near infrared (NIR) camera) positioned on the side of the user's face on which eye tracking is performed, and an illumination source 530 (e.g., an IR or NIR light source, such as an array or ring of NIR light emitting diodes (LEDs)) that emits light (e.g., IR or NIR light) toward the user's eye 592. The eye tracking camera 540 can be directed toward a mirror 550 (which reflects IR or NIR light from the eye 592 while allowing visible light to pass) located between the user's eye 592 and a display 510 (e.g., a left display panel or a right display panel of a head-mounted display, or a display of a handheld device, a projector, etc.) Figure 5 ), or alternatively may be directed toward the user's eye 592 to receive reflected IR or NIR light from the eye 592 (e.g., as shown in the top portion of Figure 5 ).

[0118] In some embodiments, the controller 110 renders AR or VR frames 562 (e.g., left and right frames for left and right display panels) and provides the frames 562 to the display 510. The controller 110 uses the gaze tracking input 542 from the eye tracking camera 540 for various purposes, such as for processing the frames 562 for display. The controller 110 optionally estimates the user's gaze point on the display 510 based on the gaze tracking input 542 obtained from the eye tracking camera 540 using a flash-assisted method or other suitable method. The gaze point estimated from the gaze tracking input 542 is optionally used to determine the direction in which the user is currently looking.

[0119] Several possible use cases for the user's current gaze direction are described below and are not intended to be limiting. As an example use case, the controller 110 may render virtual content differently based on the determined direction of the user's gaze. For example, the controller 110 may generate virtual content at a higher resolution in the foveal area determined according to the user's current gaze direction than in the peripheral area. As another example, the controller may position or move virtual content in a view based at least in part on the user's current gaze direction. As another example, the controller may display specific virtual content in a view based at least in part on the user's current gaze direction. As another example use case in an AR application, the controller 110 may guide an external camera for capturing the physical environment of the XR experience to focus in the determined direction. The autofocus mechanism of the external camera may then focus on an object or surface in the environment that the user is currently looking at on the display 510. As another example use case, the eye lens 520 may be a focusable lens, and the controller uses gaze tracking information to adjust the focus of the eye lens 520 so that the virtual object that the user is currently looking at has an appropriate degree of convergence to match the convergence of the user's eye 592. The controller 110 can use the gaze tracking information to guide the eye lens 520 to adjust the focus so that nearby objects that the user is looking at appear at the correct distance.

[0120] In some embodiments, the eye tracking device is part of a head mounted device that includes a display (e.g., display 510), two eye lenses (e.g., eye lenses 520), an eye tracking camera (e.g., eye tracking camera 540), and a light source (e.g., light source 530 (e.g., IR or NIR LED)) mounted in a wearable housing. The light source emits light (e.g., IR or NIR light) toward the user's eye 592. In some embodiments, the light sources may be arranged in a ring or circle around each of the lenses, such as Figure 5 In some embodiments, for example, eight light sources 530 (eg, LEDs) are arranged around each lens 520. However, more or fewer light sources 530 may be used, and other arrangements and locations of the light sources 530 may be used.

[0121] In some embodiments, the display 510 emits light in the visible range and does not emit light in the IR or NIR range, and therefore does not introduce noise into the gaze tracking system. Note that the positions and angles of the eye tracking cameras 540 are given by way of example and are not intended to be limiting. In some embodiments, a single eye tracking camera 540 is located on each side of the user's face. In some embodiments, two or more NIR cameras 540 may be used on each side of the user's face. In some embodiments, a camera 540 with a wider field of view (FOV) and a camera 540 with a narrower FOV may be used on each side of the user's face. In some embodiments, a camera 540 operating at one wavelength (e.g., 850 nm) and a camera 540 operating at a different wavelength (e.g., 940 nm) may be used on each side of the user's face.

[0122] like Figure 5 Embodiments of the gaze tracking system shown may be used, for example, in computer-generated reality, virtual reality, and / or mixed reality applications to provide a user with a computer-generated reality, virtual reality, augmented reality, and / or augmented virtual experience.

[0123] Figure 6 The flash-assisted gaze tracking pipeline in some embodiments is shown. In some embodiments, the gaze tracking pipeline is implemented by a flash-assisted gaze tracking system (e.g., Figure 1 and Figure 5 The flash-assisted gaze tracking system can maintain a tracking state. Initially, the tracking state is off or "no". When in the tracking state, the flash-assisted gaze tracking system uses previous information from previous frames when analyzing the current frame to track the pupil outline and the flash in the current frame. When not in the tracking state, the flash-assisted gaze tracking system attempts to detect the pupil and the flash in the current frame, and if successful, initializes the tracking state to "yes" and continues with the next frame in the tracking state.

[0124] like Figure 6 As shown in , the gaze tracking camera can capture left and right images of the user's left and right eyes. The captured images are then input to the gaze tracking pipeline for processing starting at 610. As indicated by the arrow returning to element 600, the gaze tracking system can continue to capture images of the user's eyes at a rate of, for example, 60 to 120 frames per second. In some embodiments, each group of captured images can be input to the pipeline for processing. However, in some embodiments or under some conditions, not all captured frames are processed by the pipeline.

[0125] At 610, for the currently captured image, if the tracking status is yes, the method proceeds to element 640. At 610, if the tracking status is no, the image is analyzed to detect the user's pupil and glint in the image, as indicated at 620. At 630, if the pupil and glint are successfully detected, the method proceeds to element 640. Otherwise, the method returns to element 610 to process the next image of the user's eye.

[0126] At 640, if advancing from element 610, the current frame is analyzed to track pupils and glints based in part on previous information from previous frames. At 640, if advancing from element 630, the tracking state is initialized based on the pupils and glints detected in the current frame. The processing results at element 640 are checked to verify that the results of tracking or detection can be credible. For example, the results can be checked to determine whether the pupil and a sufficient number of glints for performing gaze estimation are successfully tracked or detected in the current frame. At 650, if the results are not likely to be credible, at element 660, the tracking state is set to no, and the method returns to element 610 to process the next image of the user's eye. At 650, if the results are credible, the method advances to element 670. At 670, the tracking state is set to YES (if not already yes), and the pupil and glint information is passed to element 680 to estimate the user's gaze point.

[0127] Figure 6 It is intended to be used as an example of eye tracking technology that may be used for a particular implementation. As recognized by one of ordinary skill in the art, in some embodiments, other eye tracking technologies currently existing or developed in the future may be used in computer system 101 in place of or in combination with the flash-assisted eye tracking technology described herein for providing an XR experience to a user.

[0128] In the present disclosure, various input methods are described with respect to interaction with a computer system. When an input device or input method is used to provide an example, and another input device or input method is used to provide another example, it should be understood that each example is compatible with the input device or input method described with respect to another example and optionally utilizes the input device or input method. Similarly, various output methods are described with respect to interaction with a computer system. When an output device or output method is used to provide an example, and another output device or output method is used to provide another example, it should be understood that each example is compatible with the output device or output method described with respect to another example and optionally utilizes the output device or output method. Similarly, various methods are described with respect to interaction with a virtual environment or a mixed reality environment by a computer system. When an example is provided using interaction with a virtual environment, and another example is provided using a mixed reality environment, it should be understood that each example is compatible with the method described with respect to another example and optionally utilizes these methods. Therefore, the present disclosure discloses an embodiment as a combination of features of multiple examples, without exhaustively listing all features of the embodiment in the description of each example embodiment.

[0129] User interface and associated processes

[0130] Attention is now focused on embodiments of a user interface ("UI") and associated processes that may be implemented on a computer system, such as a portable multifunction device or a head-mounted device, that communicates with a display generating component, one or more sensors, and (optionally) one or more audio generating components.

[0131] FIG. 7A to FIG. 7L An example of providing a computer-generated user experience session in an extended reality (XR) environment is shown. Figure 8 is a flow chart of an exemplary method 800 for providing particles that move based on a user's breathing characteristics to a computer-generated user experience session. Fig. 9 is a flow chart of an exemplary method 900 for providing a computer-generated user experience session with options selected based on characteristics of an XR environment. FIG. 10A to FIG. 10B is a flow chart of an exemplary method 1000 for providing a soundscape having randomly selected curated sound components to a computer-generated user experience session. FIG. 7A to FIG. 7L The user interface in is used to illustrate the process described below, including Figure 8 , Fig. 9 , Fig. 10A and Fig. 10B process.

[0132] The accompanying drawings and accompanying descriptions are provided to describe various embodiments of a computer-generated user experience session in an XR environment. Various embodiments are described with respect to an example user experience session provided by a meditation application operating at a computer system. The user experience session is referred to as a "meditation session". The meditation session is described herein as having various stages or portions having corresponding visual and audio characteristics of the meditation session and / or corresponding portions of the meditation session. For example, the meditation session is described as having an introductory portion, a guided breathing portion, a reaction portion, and an ending portion. In addition, the meditation session is described as having various visual characteristics, such as, for example, virtual objects (e.g., shapes, particles, and / or user interfaces), virtual overlays, virtual wallpapers, and visual effects. The meditation session is also described as having various audio characteristics, such as, for example, a curated soundscape that may include sound effects, music, and guiding instructions. The meditation session is described as being provided in an XR environment. In some specific implementations, the XR environment is an AR environment. In some specific implementations, the XR environment is a VR environment. In some embodiments, various options, settings, parameters, and / or characteristics of the meditation session are determined based on whether the meditation session is performed in an AR environment or in a VR environment. For example, in some embodiments, audio and / or visual characteristics for a meditation session in a VR environment are selected from a subset of audio and / or visual characteristics available for a meditation session in an AR environment. In some embodiments, visual and / or audio characteristics of a meditation session may be selected randomly (or pseudo-randomly) in a manner that is biased toward not repeating the audio and / or visual characteristics of a previous meditation session, so as to provide a unique user experience for each session. In some embodiments, audio and / or visual characteristics are selected pseudo-randomly. For example, a random number generator is used to select audio characteristics from a superset of audio characteristics. Similarly, a random number generator is used to select visual characteristics from a superset of visual characteristics. In some embodiments, when an audio and / or visual characteristic is pseudo-randomly (or randomly) selected and the selected audio and / or visual characteristic has been previously used (or has been previously used with other specific audio and / or visual characteristics), the selected audio and / or visual characteristic is ignored and a different audio and / or visual characteristic is pseudo-randomly (or randomly) selected. It should be understood that aspects of the various embodiments described herein may be combined, rearranged, and / or omitted according to the examples described and illustrated in the corresponding figures. Selecting elements randomly or pseudo-randomly from a set of elements requires selecting elements with apparent randomness based on numbers generated from a true randomness source such as radioactive decay or another non-deterministic source or based on numbers generated from a deterministic source that is statistically random and produces results that appear to be random, even if the results are not actually truly random. Selecting elements from a set of elements with apparent randomness produces results that do not have a discernible deterministic pattern to a typical user of the device.

[0133] Fig. 7A Depicted is a physical environment 700, which is a physical room that includes a physical table 700-1, a physical chair 700-2, and a physical painting 700-3. A user 701 is in the physical environment 700, holding a device 702 (e.g., a tablet device or a smart phone) and wearing an audio output device 703 (e.g., headphones or earbuds) that is connected (e.g., wirelessly or via a wired connection) to the device 702. The device 702 includes a display 702-1 and one or more cameras 702-2. The device cameras are collectively referred to as cameras 702-2, and may include a camera located on the display side of the device (front-facing camera) and / or a camera located on a side of the device different from the display (rear-facing camera). In Fig. 7A In the depicted embodiment, the physical table 700-1, chair 700-2, and painting 700-3 are within the field of view of the rear camera of the device 702, and the view of the physical environment is displayed on the device display 702-1, while the user experience session is not active. The view of the physical environment is a representation 700a of the physical environment 700, including a representation 700-1a of the physical table 700-1, a representation 700-2a of the physical chair 700-2, and a representation 700-3a of the physical painting 700-3. In the embodiments described herein, the device 702 is used to provide a user experience session (also referred to as a meditation session) in an XR environment displayed on the display 702-1 of the device 702. The audio of the user experience session is output at the audio output device 703. However, in some embodiments, different audio sources such as one or more speakers of the device 702 can be used to output the audio. Display 702-1 is a touch screen display that can be used to display an XR environment and detect user input (e.g., touch input, tap gestures, swipe gestures, and / or text input) for interacting with device 702, a user experience session, and / or an XR environment. In some embodiments, camera 702-2 can be used to detect user input (e.g., hand gestures, biometric input, and / or breathing movements or gestures) for interacting with device 702, a user experience session, and / or an XR environment. In some embodiments, device 702 includes a microphone that can be used to detect user input (e.g., voice commands and / or ambient sounds) for interacting with device 702, a user experience session, and / or an XR environment.

[0134] exist FIG. 7A to FIG. 7L In the depicted embodiment, device 702 is a computer system (similar to Figure 1701). However, it should be understood that different types of computer systems may be used to provide a user experience session in an XR environment. For example, instead of using device 702 (or in addition to using the device), the computer system may be a head mounted device (HMD) worn by user 701. In such embodiments, the HMD includes a display component similar to display 702-1 and one or more sensors similar to camera 702-2. For example, the display may be an opaque display screen with a display component and / or a transparent or translucent display through which user 701 can directly view the physical environment 700 and virtual elements of the user experience session may be displayed or projected on the display. The HMD may also include speakers and / or other audio output devices integrated into the HMD for providing audio output, as well as one or more cameras, microphones, and / or other sensors for capturing images (e.g., videos and / or pictures) of the physical environment 700 (e.g., for display at the HMD and / or for detecting input) and receiving user input in the form of hand gestures, voice gestures, gaze gestures, and / or other inputs discussed herein. Although methods for providing a user experience session in an XR environment are discussed herein with respect to device 702 , it should be understood that these methods may be performed using other computer systems including, for example, an HMD.

[0135] FIG. 7B to FIG. 7L Depicted is a user interface at device 702 for an embodiment in which a meditation application is active and, in some instances, a meditation user session is ongoing. FIG. 7B to FIG. 7I Depicting a user interface for a meditation session in an AR environment. Figures 7J to 7L Depicting a user interface for a meditation session in a VR environment.

[0136] exist Figure 7B, device 702 is providing the first part of a meditation session (an introductory part, a starting part, and / or a part before a guided breathing part), and a virtual interface 705 overlaying a displayed representation 700a of a physical environment 700 is displayed via display 702-1. In some embodiments, the virtual interface is displayed as part of an AR environment for the user to experience the session. In such embodiments, the virtual interface includes a dimming effect 704 displayed on top of the representation 700a of the physical environment 700. The dimming effect allows, for example, 98% visibility of the physical environment by displaying the dimming effect at 98% transparency (or 2% opacity), thereby dimming the user's view of the physical environment. In some embodiments, the dimming effect may be larger or smaller, and may have varying amounts of dimming at different portions of the device display (e.g., the opacity of the dimming effect is less near the center of the display, and the opacity of the dimming effect increases toward the periphery of the device display). The dimming effect can indicate to user 701 that the user is viewing an AR environment, and in some embodiments, helps focus the user on aspects of the user experience session by reducing distractions that may be present in the physical environment. In some embodiments, the virtual interface is displayed as part of the VR environment for the user experience session. In such embodiments, the virtual interface is opaque and the physical environment 700 is not visible through the virtual interface. Examples of such interfaces are shown in FIG. Figure 7J and Figure 7L Unless otherwise specifically noted, the values ​​and ranges discussed herein for various visual effects, such as dimming effects, are intended to be non-limiting examples.

[0137] In some embodiments, the virtual interface 705 is displayed as part of a user experience session in an XR environment. Figure 7B , device 702 displays virtual interface 705 as part of a meditation session provided by a meditation application operating at device 702, and thus includes additional virtual elements associated with the meditation session. Figure 7B , device 702 displays virtual objects 710, virtual particles 712, and virtual menu 715 as part of an introductory portion or introductory phase of a meditation session in an XR environment. In some embodiments, in response to launching a meditation application in the XR environment (e.g., by detecting a selection of a virtual icon for a meditation application in the XR environment and / or by detecting an audio instruction to launch a meditation application in the XR environment), a virtual interface 705 including a dimming effect 704, virtual objects 710, particles 712, and menu 715 is displayed by device 702. In some embodiments, the virtual interface 705 may include a virtual wallpaper.

[0138] In some embodiments, the virtual interface is displayed as a portion of the XR environment that is separate from or disassociated from the user experience session. For example, the virtual interface may be displayed as part of a virtual staging room or a virtual "home" environment, and the device displays the virtual staging room or virtual "home" environment as part of the XR environment before starting the user experience session (e.g., before launching the meditation application and / or after launching the meditation application and before starting the user experience session). Sometimes, in such embodiments, the device 702 does not display virtual objects 710, particles 712, or menus 715, but optionally displays icons or other virtual elements that can be selected to manage the user's experience of the XR environment and / or device 702. For example, the device 702 may display icons that can be selected to launch applications operating at the device and / or access various settings of the XR environment and / or device 702. In some embodiments, the user 701 can interact with other users and share the XR environment with other users, including applications launched in the XR environment.

[0139] Virtual objects 710 are graphical elements that move and change appearance when the meditation application is active (e.g., during a meditation session or not during a meditation session). Figure 7B As shown, virtual object 710 has a roughly triangular macro-shape formed by a collection of particles 712. During the introductory portion of the meditation session, device 702 displays particles 712 as moving in a rhythmic pattern, such that virtual object 710 has pulsating, swaying, and / or other rhythmic movements (represented by lines 714 in the figure) that convey a relaxing or soothing environment to user 701. In some embodiments, device 702 displays particles 712 as moving closer together and then farther apart, causing virtual object 710 to expand and contract in a rhythmic pattern that mimics a predetermined breathing rhythm (or other biometric rhythm, such as heartbeat, brain waves, or walking rate) while maintaining a roughly triangular macro-shape. In some embodiments, virtual object 710 pulsates (expands and contracts) at a rate that is faster or slower than the predetermined breathing rhythm. Additionally, virtual object 710 may have a rhythmic pattern that mimics a predetermined breathing rhythm. Figure 7B In some embodiments, the virtual object 710 may be a two-dimensional object having a macroscopic shape that is a circle, square, triangle, rectangle, or an abstract two-dimensional shape. In some embodiments, the virtual object 710 may be a three-dimensional object having a macroscopic shape that is a sphere, a ball, a cube, a cuboid, a pyramid, or an abstract three-dimensional shape (such as a cloud), or a moving arrangement of particles having the appearance of floating mist or swirling brush strokes.

[0140] exist FIG. 7B to FIG. 7IIn the illustrated embodiment, particles 712 are shown as having a triangular shape. However, particles may have different two-dimensional and / or three-dimensional shapes or appearances, such as circles, spheres, pyramids, petals, leaves, squares, cubes, clouds, droplets, brush strokes, or any combination thereof. In some embodiments, some of the particles have uniform spacing between adjacent particles. In some embodiments, some of the particles have uneven spacing between adjacent particles. In some embodiments, some of the particles have overlapping arrangements. In some embodiments, as virtual object 710 moves in a rhythmic pattern, particles move between various appearances and spacing arrangements. For example, in the expanded state of virtual object 710, particles 712 have a spaced-apart arrangement, and in the contracted state of virtual object 710, particles 712 have an overlapping or smaller spacing arrangement. In addition, the virtual objects 710 and / or particles 712 may have different visual characteristics, such as animation effects or appearances, translucency of particles (e.g., partial translucency or full translucency), simulated reflections of light detected in the physical environment (e.g., simulated reflections from particles and / or macroscopic shapes and / or simulated bending of light), simulated reflections of virtual lighting, or combinations thereof. In some embodiments, the visual characteristics of the particles 712 and / or virtual objects 710 change based on various criteria (such as, for example, the physical environment, the XR environment, the state of the meditation session, the movement of the user and / or device 702, user input, and / or any combination thereof). In some embodiments, the visual characteristics of the user experience session are randomly (or pseudo-randomly) generated by the device 702. For example, in some embodiments, each time a meditation application is launched, the device 702 generates virtual objects and / or particles with different, randomly selected visual characteristics. In some embodiments, the visual characteristics are randomly selected in a manner that is biased toward not repeating a previously displayed set of visual characteristics of the user experience session.

[0141] exist Figure 7B , device 702 displays menu 715, which is a virtual menu user interface for selecting or modifying various aspects of a meditation session. Menu 715 includes text 715-1 that provides context for the meditation session to user 701 (e.g., notifying the user of meditation application activity, indicating that the user experience session is a meditation session, and / or commanding the user to prepare for a meditation session). Menu 715 also includes option element 720, which is selectable to display an options menu for the meditation session. Option element 720 also includes a duration indication 720-1 indicating a currently selected duration of the meditation session (e.g., the duration of one or more portions of the meditation session and / or the combined duration of the guided breathing portion and reaction portion of the meditation session, and optionally, the ending portion) and a coach indication 720-2 indicating a currently selected coach for the meditation session. As shown Figure 7B720-1 indicates that the duration of the meditation session is selected as 10 minutes.Menu 715 also includes a start element 725, which can be selected (e.g., via input 724 and / or via audio input) to start a meditation session by transitioning from the introductory part of the meditation session to the breathing part of the guidance.In some embodiments, the meditation application is active, but during the introductory part and the ending part, the meditation session is not considered to be ongoing.In such embodiments, the meditation session is considered to be started by selecting the start element 725.In some embodiments, during the introductory part and the ending part, the meditation session is considered to be ongoing.In such embodiments, the meditation session is started by starting the meditation application, and the start element 725 is selected to transition from one part of the meditation session to a different part of the meditation session (e.g., transitioning from the introductory part to the breathing part of the guidance or transitioning to another part of the meditation session).

[0142] Figure 7B Also depicted is an audio schematic 707, which is a schematic representation of some characteristics of the audio output by device 702 for a user experience session. Audio schematic 707 is not part of the user interface, but is provided to provide a better understanding of the described techniques. Audio schematic 707 illustrates the relative positioning of user 701 and audio output device 703 in physical environment 700 (showing a top-down view of the user's head). Audio schematic 707 also includes an audio indicator 711 representing a particular soundscape output during the meditation session and spatial audio indicators 709-1 and 709-2 indicating the perceived location of the audio for the meditation session relative to the location of the user's head (the location of the audio perceived by user 701 when the audio is played using audio output device 703). In Figure 7B In the depicted embodiment, the audio is output in stereo. Thus, spatial audio indicator 709-1 represents the perceived position of an audio channel (e.g., the left channel and / or the first channel) adjacent to the left side of the user's head, and spatial audio indicator 709-2 represents the perceived position of another audio channel (e.g., the right channel and / or the second channel) adjacent to the right side of the user's head. Thus, as Figure 7B As depicted, user 701 is facing forward (e.g., facing Fig. 7A The physical environment 700 shown is or / and facing the device 702), and the device 702 is causing the soundscape of the meditation session to be played in stereo using the audio output device 703. In some embodiments, Figure 7BThe audio output in the depicted embodiment is a preview of the soundscape that will be played during the meditation session (e.g., after the introductory portion, during the guided breathing portion, during the reaction portion, and / or during the ending portion). In some embodiments, a soundscape with different audio characteristics is played during the preview than during the meditation session. For example, the audio is output at a lower volume, such as Figure 7B In (with Fig.7D As another example, as indicated by the smaller size of the audio indicator 711 in FIG. Fig.7D As shown and discussed below, the soundscape is output in stereo rather than in full audio immersion with three-dimensional perceived spatial location (referred to herein as "spatial audio").

[0143] In some embodiments, device 702 uses a collection of curated sound components to create a soundscape for a specific user experience session. The device assembles the sound components to provide a harmonious audio experience for the user. In some embodiments, the soundscape for a specific user experience session is created to convey a specific mood or theme. For example, for a meditation session, a soundscape is created to provide a relaxing audio that helps users focus on their breathing rhythm or a specific theme or topic. In some embodiments, device 702 randomly (or pseudo-randomly) selects sound components to create a soundscape. In some embodiments, the sound components are randomly selected, but tend not to repeat the soundscape previously created or previously used. In some embodiments, the sound components available for creating a corresponding soundscape are selected to be harmonious when played together at a randomly selected time. In some embodiments, the sound components selected for the soundscape are repeated for a user experience session or for a portion of the user experience session. In some embodiments, device 702 introduces a randomly (or pseudo-randomly) generated sound component, which is played at various (in some embodiments, random) moments throughout the soundscape to introduce diversity to the soundscape. In some embodiments, the soundscape has a common start and end sound. In some embodiments, the audio characteristics of the soundscape change based on the animation of the virtual object 710. For example, the audio volume is modulated with the pulsing animation of the virtual object 710, increasing as the virtual object 710 expands, and decreasing when the virtual object 710 contracts. In some embodiments, the spatial position of the audio changes with the pulsing animation of the virtual object 710. For example, when the object expands, the audio sounds like it is moving closer to the user, and when the object contracts, the audio sounds like it is moving away from the user.

[0144] exist Figure 7B , device 702 detects input 722 on option element 720 and, in response, displays option menu 730, such as Figure 7CAs shown. Options menu 730 is a user interface for customizing various settings for a meditation session. Options menu 730 includes a duration option 732, a rhythm option 734, and a guidance option 736. Duration option 732 may be selected to set the duration of the meditation session. Figure 7C As shown, duration option 732-2 is currently selected and the duration of the meditation session is set to 10 minutes. Other available duration options include a five minute duration option 732-1 and a fifteen minute duration option 732-3. Rhythm option 732 may be selected to set a defined breathing rhythm for the meditation session. In some embodiments, animation effects (such as the pulsating rhythm of virtual object 710) are based on the selected breathing rhythm. Figure 7C As shown, rhythm option 734-2 is currently selected and the breathing rhythm is set to seven breaths per minute (bpm). Other available rhythm options include five bpm rhythm option 734-1 and ten bpm rhythm option 734-3. Guidance option 736 is selectable to select a coach for the meditation session. The selected coach is associated with one or more audio recordings (e.g., audio guidance 740 and / or other audio recordings) output by device 702 as part of the soundscape during the meditation session, thereby providing instructions, encouragement and / or guidance for the meditation session to user 701. In some embodiments, the audio recording consists of different scripts recorded by the coach for playback during various parts of the meditation session. Each coach has their own different voice, which has a speaking rhythm, pitch, intonation, accent, and other speech characteristics unique to the selected coach. As shown in FIG. Figure 7C As shown, the coach option 736-1 is currently selected, which indicates that a male coach is selected for the meditation session. The other available coach option is a female coach option 736-2.

[0145] While device 702 is displaying options menu 730, device 702 continues to display virtual object 710 with an animated, pulsating effect represented by line 714. Figure 7C , the virtual object 710 is shown as having an expanded state, as indicated by the increased spacing between particles 712 and the increased size of particles 712 (when compared to Figure 7B In some embodiments, the spacing of the particles is changed without changing the size of the particles. Device 702 continues to output a preview of the soundscape in stereo at audio output device 703.

[0146] exist Figure 7C, device 702 detects input 731 selecting five minute duration option 736-1, input 733 selecting five bpm rhythm option 732-1, and input 735 selecting female coach option 734-2. In response to the respective inputs, device 702 updates the settings of the meditation session based on the selected options. Specifically, the duration of the meditation session is changed from ten minutes to five minutes, the breathing rhythm is changed from 7 bpm to 5 bpm, and the coach is changed from a male coach to a female coach. Then, device 702 detects input 739 selecting completion affordance 738. In response to detecting input 739, device 702 displays menu 715, which is similar to Figure 7B The menu shown in the figure is updated based on the selected option. For example, the option element 720 is updated to show the selected duration of five minutes and is updated to include a representation of a female coach. The device 702 continues to display the animated movement of the virtual object 710, which pulsates with a rhythm having the selected breathing rhythm, which is now changed to five bpm.

[0147] In having Figure 7C In case of changes in the selection, Fig.7D and Fig. 7E An embodiment is shown in which device 702 outputs a second portion of the meditation session (e.g., a guided breathing portion, a portion after the introductory portion, and / or a non-introductory portion) in response to an input (e.g., input 724 and / or an audio input) selecting start element 725. When the user selects start element 725, device 702 transitions from the introductory portion of the meditation session to the second portion of the meditation session. Fig.7D and Fig. 7EThe guided breathing portion of the meditation session shown. When transitioning from the introductory portion of the meditation session to the guided breathing portion, the device 702 increases the user's visual and audio immersion by displaying a virtual object 710 that grows to a larger size, increasing the dimming effect 704, and switching from outputting audio in stereo to outputting spatial audio. During the transition, the device 702 increases the dimming effect 704 by gradually increasing the opacity of the dimming effect from, for example, 2% to 95% or from 5% to 90%, and displays the virtual object 710 and optionally the particles 712 as growing to a larger average size than in the introductory portion. In some embodiments, the virtual object 710 and the particles 712 are output as three-dimensional objects. In some embodiments, the transition from the introductory portion to the guided breathing portion includes: displaying virtual objects and particles that transition from two-dimensional objects to three-dimensional objects. In some embodiments, other visual characteristics of the virtual object 710 and the particles 712 change during the transition. For example, the object is displayed as having increased brightness, translucency, and / or having a reflection of a simulated or detected light that changes. Device 702 increases the user's audio immersion by outputting a soundscape with audio effects at audio output device 703, whereby the volume of the audio gradually increases (as indicated by the larger size of audio indicator 711) and sounds as if it is moving from the right stereo position and the left stereo position to different perceived locations around the user, as discussed in more detail below. In some embodiments, device 702 outputs a start sound (such as, for example, a ding or chime) to indicate the transition from the introductory portion to the guided breathing portion via audio output device 703. In some embodiments, device 702 outputs the same ding or chime via audio output device 703 when transitioning to the ending portion of the meditation session.

[0148] In the breathing part of the guidance, the device 702 provides a combination of visual and audio effects to help the user 701 focus their breathing on the controlled breathing rate. The device 702 displays virtual objects 710 and optionally particles 712 as having a larger average size than in the introductory part. In some embodiments, the device 702 displays particles 712 as being spaced apart by a larger amount (e.g., a larger average amount and / or a larger instantaneous amount) than during the introductory part. During the breathing part of the guidance of the meditation session, the device 702 displays the virtual object 710 as expanding and contracting at a rate set by the selected breathing rhythm, and outputs (e.g., using the audio output device 703 and / or the audio generation component integrated into the HMD) an audio guide 740 that commands the user 701 to control their breathing to match the selected breathing rhythm. For example, because the user changes the breathing rhythm to 5 bpm using input 733, device 702 displays virtual object 710 as expanding and contracting at a rate matching the breathing rhythm of five breaths per minute, and outputs a soundscape with audio characteristics that encourage the user to conform their own breathing rate to the selected breathing rhythm matching five breaths per minute. In some embodiments, virtual object 710 changes to a different (e.g., expands to a larger or contracts to a smaller) size in the guided breathing portion than during the introductory portion of the meditation session.

[0149] Fig.7D and Fig. 7E Depicts various moments of the guided breathing portion of a meditation session. Specifically, Fig.7D The virtual object 710 is depicted in a contracted state, which is consistent with the expectation that the user 701 is exhaling or has just completed an exhalation and is about to inhale. When the virtual object is displayed in the contracted state, the device 702 outputs a soundscape with audio guidance 740-1 instructing the user to inhale in the voice of a female coach (selected via input 735). The device 702 then displays the virtual object 710 as expanding at a steady rate (presumably, while the user is inhaling) until it reaches an expanded state, as shown in FIG. Fig. 7E As depicted, this is consistent with the expectation that user 701 has inhaled and is about to exhale. Device 702 then outputs audio guidance 740-2 at audio output device 703 commanding user 701 to begin exhaling. Device 702 then displays virtual object 710 as contracting at a steady rate (presumably, while the user is exhaling) until it reaches a contracted state, as shown in FIG. Fig.7D The process is repeated during the guided breathing portion of the meditation session. In some embodiments, the process is repeated for a predetermined amount of time (e.g., Figure 7CIn some embodiments, the process is repeated for a predetermined number of breathing cycles (e.g., seven inhalations and seven exhalations or ten inhalations and ten exhalations). In some embodiments, the process is repeated until the device 702 determines that the user 701 has matched their breathing rhythm with the selected breathing rhythm. In some embodiments, the device 702 includes a sensor and / or camera for detecting the user's breathing rhythm to determine whether the user's breathing rhythm matches the selected breathing rhythm.

[0150] During the guided breathing portion of the meditation session, as the device outputs audio guidance 740, the device 702 continues to output the soundscape (represented at least in part by the audio indicator 711) and displays the virtual object 710 as expanding and contracting at the selected breathing rhythm. In some embodiments, one or more components of the soundscape audio are output as spatial audio. For example, Fig.7D As shown, the audio diagram 707 includes spatial audio indicators 709-3 through 709-9, each of which represents the perceived position of the audio (e.g., soundscape and / or audio guidance) of the meditation session relative to the position of the user's head. Thus, during the guided breathing portion of the meditation session, the user 701 perceives the audio as being located at multiple points around the user's head. In some embodiments, the spatial position of the audio changes during the meditation session. For example, as the virtual object 710 is expanding, the device 702 adjusts the spatial position of the audio so that it sounds like the particles 712 are moving closer to the user 701, and as the virtual object 710 is shrinking, the device 702 adjusts the spatial position of the audio so that it sounds like the particles 712 are moving away from the user 701. This movement of the spatial audio is represented by the audio indicators 709-3 through 709-9, which are shown in FIG. Fig. 7E has a position closer to the user's head and is Fig.7D In some embodiments, the device 702 outputs sound effects as part of the soundscape. In some embodiments, portions of the soundscape are selected based on animation effects of virtual objects and / or particles. For example, as the virtual object 710 expands and the particles 712 move closer to the user 701, the soundscape may include an increasing hissing sound, and as the particles move away from the user 701 when the virtual object shrinks, the hissing sound may be reduced.

[0151] FIG. 7F to FIG. 7H Depicts an embodiment in which the device 702 has transitioned from the guided breathing portion of a meditation session to a third portion (e.g., a reaction portion and / or a portion subsequent to the guided breathing portion). Figure 7FAs depicted, device 702 transitions from the guided breathing portion of the meditation session to the reaction portion and displays the virtual object in an expanded state in which particles 712 are displayed in a three-dimensional arrangement. In some embodiments, device 702 displays the transition as having a shape depicting particles 712 expanding from a configuration forming virtual object 710 to Figure 7F Animation of a proliferating three-dimensional arrangement depicted.

[0152] In the reaction part, device 702 prompts user 701 to focus on or consider a specific topic or theme, and provides a combination of visual and audio effects to provide a soothing, relaxing environment to help the user focus on the topic. In some embodiments, the reaction part is based on the relaxation of the user 701 realized during the breathing part of the guidance of the meditation session. During the reaction part, device 702 continues to output soundscape and display particles 712 moving based on detecting user breathing. When device 702 detects that user 701 inhales, the device displays particle 712 as moving towards user 701 and / or around it. When device 702 detects that user 701 exhales, the device displays particle 712 as moving away from user 701. Figure 7F and Figure 7G Depicting an example of a display state of a meditation session based on detecting the user's breathing.In some embodiments, device 702 displays particles that move based on detecting different biometric inputs, such as heart rate, brain waves, or walking speed.

[0153] For example, Figure 7F depicts particles 712 in the XR environment before the user inhales (or after exhaling), and Figure 7G Particles 712 are depicted in the XR environment after the user inhales (or before exhaling). In response to detecting the user inhaling, device 702 displays particles 712 as moving toward user 701, e.g., from Figure 7F Move the layout in Figure 7G Conversely, when device 702 detects the user exhaling, the device displays particles 712 as moving away from user 701, for example, from Figure 7G Move the layout in Figure 7F Generally speaking, the process continues, moving the particles based on the user's detected breathing, for the duration of the responsive portion of the meditation session.

[0154] Figure 7F and Figure 7GThe depicted embodiment is provided as a non-limiting example of movement of particles 712 in response to detecting the breathing of user 701. In some embodiments, device 702 displays particles as moving in different ways based on the user's breathing. For example, in some embodiments, the amplitude of movement of particles 712 is based on the amplitude and / or duration of the user's breathing (inward and / or outward). Thus, if the device detects that the user inhales with a greater amplitude and / or duration than in the examples discussed above, device 702 displays particles 712 as moving in different ways based on the user's breathing. Figure 7G Similarly, if the device detects that the user inhales with a smaller amplitude and / or duration than in the examples discussed above, the device 702 displays the particles 712 as having a greater amount of movement toward the user 701. Figure 7G The particles are shown having less movement toward user 701. In some embodiments, the displayed movement of the particles away from the user similarly varies based on the amplitude and / or duration of the user's exhalation as detected by device 702.

[0155] In some embodiments, when the user exhales, device 702 displays particles 712 as moving at a greater speed than when the user inhales (or vice versa). In some embodiments, device 702 displays particles 712 in a three-dimensional arrangement, with some particles displayed as appearing closer to user 701 and some particles displayed as appearing farther away from user 701. For example, in Figure 7F In some embodiments, particles 712-1 are displayed as appearing to be closer to user 701 than particles 712-2. In some embodiments, when the user breathes, particles displayed as being closer to user 701 are shown as having a greater amount of movement (e.g., distance and / or amplitude) in the XR environment than particles displayed as being farther away from user 701. In some embodiments, when particles 712 do not move based on the user's breathing (e.g., between the user's breathing and / or when the user's breathing is below a threshold amplitude), device 702 displays these particles as having floating or swaying movement. In some embodiments, device 702 displays particles 712 that move according to simulated physical parameters (such as simulated inertia, spring constants, friction, etc.). In some embodiments, when the user breathes, device 702 displays particles 712 as leaving the screen or moving on the screen. For example, when device 702 detects that the user inhales, the device displays particles 712 as moving toward the user, with some of the particles or portions thereof moving away from the screen as the user is inhaling, simulating the movement of particles past the user and / or leaving the user's field of view. Similarly, when device 702 detects the user exhaling, the device displays particles 712 as moving away from the user, with some particles or portions of particles moving on the screen, simulating the particles moving into the user's field of view.

[0156] During the reactive portion of the meditation session, as the device outputs audio guidance 740, the device 702 continues to output a soundscape (represented at least in part by the audio indicator 711) and displays particles 712 that move based on detected breathing of the user. In some embodiments, when the reactive portion begins, the device 702 outputs audio guidance 740-3 that prompts the user (in the voice of a coach) to focus on a particular topic or theme. The device 702 outputs the soundscape in spatial audio, and adjusts the audio characteristics of the soundscape during the reactive portion of the meditation session. For example, as the particles 712 move toward the user, the device 702 adjusts the spatial position of the audio so that it sounds as if the particles 712 are moving closer to the user 701, and as the particles 712 move away from the user, the device 702 adjusts the spatial position of the audio so that it sounds as if the particles 712 are moving away from the user. For example, during Figure 7F , the audio diagram 707 shows spatial audio indicators 709-3 through 709-9 positioned to the sides and front of the representation of the user's head, which indicate where the audio is perceived to originate from the front and sides of the user's head. Figure 7G As shown, when particle 712 is shown moving toward and around user 701, device 702 adjusts the spatial position of the audio so as to sound as if particle 712 is moving around (including behind) the user's head, as shown in FIG. Figure 7G As the user exhales, particles 712 are shown moving away from the user 701 to Figure 7F In the arrangement shown, device 702 adjusts the spatial position of the audio so that it sounds as if the particles are moving back to positions to the sides and in front of the user, such as Figure 7F Depicted.

[0157] In some embodiments, device 702 continues the response portion of the meditation session for a predetermined amount of time (e.g., Figure 7C 1 / 2 or 1 / 3 of the duration selected in the meditation session), thereby continuing to modify the display state of the particles based on the detected user breathing. In some embodiments, the reaction portion continues for a predetermined number of breathing cycles (e.g., seven inhalations and seven exhalations or ten inhalations and ten exhalations). In some embodiments, the process continues until the device 702 determines that the user 701 is no longer focused on the meditation session or has indicated that the user wishes to end the meditation session.

[0158] Figure 7H Depicts an embodiment in which the device 702 has detected that the user is not focusing on the meditation session. Figure 7HAs shown, user 701 has shifted their focus from the meditation session, rotating their head and the positioning of device 702 to look at person 745 located in physical environment 700. In some embodiments, device 702 determines whether the user is focused on the meditation session by detecting different attention- and focus-based indicators, such as the user's gaze, breathing rhythm, heart rate, brain waves, body movement, movement of device 702, detected sounds (e.g., the voice of another person nearby, background sounds, and / or sounds indicating movement or restlessness of the user), and / or other inputs detected by device 702 and indicating that the user is distracted. In some embodiments, device 702 detects that the user is distracted when the user's focus has drifted from the meditation session for at least a threshold amount of time (e.g., a non-zero amount of time, 2 seconds, and / or 5 seconds). In some embodiments, device 702 detects that the user has regained focus based on detecting the attention- and focus-based indicators. For example, in some embodiments, device 702 determines that the user has regained focus when the user has returned to the position they were in before the interruption (e.g., facing forward and / or with eyes focused on particles in the XR environment) and their breathing has resumed the breathing rate from before the interruption.

[0159] In some embodiments, when device 702 detects that user 701 is distracted, device 702 prompts the user to focus on the meditation session. Figure 7H As shown, the device prompts the user to focus on their breathing, as indicated by audio guidance 740-4. In some embodiments, when the user is distracted, device 702 pauses the meditation session (e.g., pauses the movement of particles 712 and / or modifies or pauses the output of the soundscape). In some embodiments, when the device determines that the user has regained focus, device 702 continues the current portion of the meditation session. In some embodiments, device 702 returns to the guided breathing portion of the meditation session to help the user regain focus. Figure 7H The user is shown to have been distracted during the reaction portion, however, the device is able to detect that the user was distracted in other portions of the meditation session, such as, for example, the guided breathing portion. In such embodiments, the device pauses the guided breathing portion and encourages the user to focus on the meditation session in a manner similar to that described above for distractions occurring during the reaction portion.

[0160] In some embodiments, the user's position (e.g., approximated by the position and / or orientation of device 702) and virtual objects in the XR environment (e.g., virtual object 710 and / or particle 712) are world-locked. Figure 7H, as the user rotates the orientation of device 702, device 702 displays particles 712 from a changing perspective of device 702 because the device has been rotated from Figure 7G Positioning rotation in Figure 7H For example, device 702 is located in Figure 7G and Figure 7H Particle 712-3 is shown in both. However, in Figure 7H Because the user has already Figure 7G , and because the positioning of particle 712 is world locked, particle 712-3 is in Figure 7H In relative to Figure 7G 702 is displayed in an orientation rotated from a previously displayed orientation of particle 712-3 in the XR environment. Thus, as the user rotates or moves device 702, the displayed view of the XR environment changes based on the world-locked configuration. In some embodiments, the world-locked configuration is demonstrated by device 702 modifying the output of the soundscape as the user moves so as to preserve the spatial audio position of the soundscape relative to the XR environment. For example, Figure 7H As shown, the user's head is rotated to the left. Instead of keeping the spatial audio locked to the position of the user's head, device 702 adjusts the spatial audio to remain fixed relative to the XR environment, such as by maintaining the spatial audio from Figure 7G 709 - 3 through 709 - 9 of the XR environment. Thus, as the user moves their head, the user perceives the audio to have the same relative position in the XR environment. In some embodiments, such as, for example, when device 702 is an HMD, particles 712 are spatially arranged around the viewpoint of user 701 and as the particles move in the XR environment (e.g., as described with respect to Fig.7D , Fig. 7E , Figure 7F , Figure 7G , Figure 7H and / or Fig.7I described) and moves relative to the user's viewpoint.

[0161] In some embodiments, user 701 can interact with virtual objects in the XR environment. For example, particles 712 can react to the position or touch of the user detected in the XR environment. In some embodiments, device 702 determines that the user's body (e.g., hand, wrist, and / or arm) is co-located with particles 712 in the XR environment, and in response, particles 712 are displayed as changing appearance (e.g., changing color, moving based on the user's touch, emitting light, and / or becoming more or less translucent). In some embodiments, user 701 can interact with other users in the XR environment. For example, in some embodiments, representation 745a is a virtual representation of person 745 who is sharing the XR environment with user 701. In some embodiments, representation 745a can interact with user 701 and, optionally, view and / or manipulate virtual objects displayed as part of a meditation session.

[0162] Fig.7I The depiction device 702 transitions from the reaction portion of the meditation session to a fourth portion (e.g., an ending portion and / or a portion after the reaction portion). In some embodiments, the ending portion transitions the user outside the meditation session and brings the user's focus back to the physical world (or back to a different experience in the XR environment). In some embodiments, the device 702 displays the particles 712 as moving together (e.g., with an animated flying effect and / or with coordinated movement) to form a virtual object 710. In some embodiments, the device 702 reduces the dimming effect 704 (e.g., reducing the opacity of the dimming effect and / or otherwise reducing the dimming effect), thereby increasing the visibility of the representation 700a of the physical environment 700 on the device 702. In some embodiments, the transition to the ending portion is a reversal of the audio and visual effects provided in the transition from the introductory portion to the guided breathing portion. In some embodiments, the device 702 darkens the appearance of the particles 712 and reduces the movement of the particles 712 from the movement in the guided breathing portion and the reaction portion of the meditation session. In some embodiments, the device 702 displays the virtual object 710 as pulsating in a manner similar to that in the introductory portion. In some embodiments, the device 702 displays the virtual object 710 as having less movement than in the introductory portion.

[0163] like Fig.7IAs shown, when transitioning to the ending part, the device 702 converts the soundscape audio from the spatial audio output to stereo, gradually reduces the volume of the soundscape (as indicated by the smaller size of the audio indicator 711), and outputs the audio guidance 740-5 that prompts the user to increase their awareness of their surroundings with the coach voice. In some embodiments, the device 702 outputs a sound effect via the audio output device 703, such as a ding or ringing sound similar to when transitioning from the introductory part to the breathing part of the guidance. In some embodiments, when transitioning to the ending part, the device 702 outputs the same audio as when transitioning from the introductory part to the breathing part of the guidance. In some embodiments, the device 702 outputs the same start sound for each meditation session (for example, when transitioning from the introductory part to the breathing part of the guidance, when ending the introductory part, and / or when starting the breathing part of the guidance). In some embodiments, the device 702 outputs the same end sound for each meditation session (for example, when transitioning to the reaction part and / or when transitioning to the ending part). In some embodiments, the start sound is different from the end sound of the corresponding meditation session. In some embodiments, the beginning sound is the same as the ending sound, and the sound is different for multiple respective meditation sessions or unique to each respective meditation session.

[0164] like Fig.7I As depicted, device 702 displays menu 750 in the ending portion of a meditation session. Menu 750 includes text that provides historical data for meditation sessions that the user has experienced. Text 754 indicates that user 701 has spent five minutes in a meditation session today, and text 756 indicates that the user has experienced at least one meditation session in three days in the past week. Other historical data may be provided, such as, for example, the number of meditation sessions, the specific amount of time spent for each meditation session, and / or the average amount of time spent for a meditation session. Menu 750 also includes a coach indicator 752 indicating that a female coach provides audio guidance 740 for a meditation session. Menu 750 includes a continue option 758, which can be selected to continue or extend a meditation session. For example, in some embodiments, in response to detecting a selection of the continue option 758, device 702 resumes the meditation session for approximately a predetermined time or until the user indicates that they wish to end the meditation session. In some embodiments, when device 702 resumes the meditation session, device 702 returns to the breathing portion of the guidance of the meditation session. In some embodiments, when the device 702 resumes the meditation session, the device 702 returns to the reaction portion of the meditation session. In some embodiments, the device 702 provides the user 701 with the option to select whether to return to the guided breathing portion or to return to the reaction portion. The menu 750 also includes a Done option 760. In some embodiments, when the user selects Done option 760, the device 702 returns to the introductory portion of the meditation session (e.g., displaying the Figure 7B The UI and / or display depicted Figure 7B ) or exit the meditation application (e.g., displaying a UI similar to the UI depicted in FIG. Fig. 7A A UI similar to the one depicted or returning to a staging room or virtual home environment in an XR environment).

[0165] In some embodiments, the meditation session may be performed in an AR environment or a VR environment. FIG. 7B to FIG. 7I depicting various embodiments of meditation sessions provided in an AR environment, and Figures 7J to 7L Various embodiments of different meditation sessions provided in a VR environment are depicted. Device 702 can provide a meditation session in a VR environment in a manner similar to a meditation session in an AR environment, but with various differences, such as, for example, using a virtual environment instead of an AR environment. In some embodiments, the visual effects and soundscapes provided for the meditation session in the VR environment are a subset of the visual effects and soundscapes available for meditation sessions in an AR environment. In some embodiments, device 702 automatically selects visual and audio characteristics based on the characteristics of the XR environment. For example, if the XR environment is an AR environment (e.g., providing a meditation session in an AR environment and / or providing a portion of a meditation session in an AR environment), device 702 automatically selects visual and audio characteristics based on various aspects of the AR environment (such as, for example, the detected lighting conditions, a history of previously used audio and / or visual characteristics of an AR meditation session) or simply based on the fact that the XR environment is AR. As another example, if the XR environment is a VR environment (e.g., providing a virtual wallpaper for a meditation session and / or providing a meditation session in a virtual environment), device 702 automatically selects visual and audio characteristics based on various aspects of the VR environment (such as, for example, the scene in the virtual wallpaper, a history of previously used audio and / or visual characteristics for VR meditation sessions) or simply based on the fact that the XR environment is VR.

[0166] Figure 7J The depiction device 702 displays different meditation sessions to be performed in the VR environment (e.g., FIG. 7B to FIG. 7I A session that occurs after or independently of the meditation session depicted in FIG. 7B to FIG. 7I During the VR meditation session, device 702 displays virtual wallpaper 765 that provides an opaque virtual background for the meditation session. Device 702 displays virtual wallpaper 765 over a portion of representation 700a of physical environment 700. Virtual wallpaper 765 is a virtual background depicting Figure 7J However, the virtual wallpaper may have different images or background scenes for different meditation sessions. For example, Figure 7LDepicting a different meditation session, where the virtual wallpaper 787 has a mountain view. In some embodiments, the user can select a background scene from a collection of background scenes for the virtual wallpaper. In some embodiments, the device 702 randomly selects a background scene from the collection of background scenes (optionally, in combination with other visual characteristics such as virtual objects and / or particles, with a bias to not repeat previously used background scenes).

[0167] exist Figure 7J In the depicted embodiment, device 702 displays virtual object 770 with particles 772. Virtual object 770 and particles 772 have different visual characteristics than virtual object 710 and particles 712. For example, virtual object 770 has a generally circular or rounded macroscopic shape. In addition, particles 772 are circles or spheres that form virtual object 770 and move around within the shape of virtual object 770. In some embodiments, particles 772 have different material properties than particles 712. For example, particles 772 may have different simulated optical properties, shapes, and / or curvatures, and may have different light properties (e.g., direction, color, and / or intensity) for simulated light reflected on the particles, e.g., due to the rounded shape of particles 772, while particles 712 are triangular.

[0168] Although virtual objects 770 and particles 772 have different visual characteristics than virtual objects 710 and particles 712, in some embodiments, virtual objects and particles may exhibit similar behavior based on various portions of the meditation session being output by device 702. For example, device 702 displays virtual objects 770 and particles 772 as moving rhythmically (e.g., pulsating and / or swaying) in an introductory portion and expanding to a larger arrangement that moves rhythmically based on a predetermined breathing rhythm for a guided breathing portion, similar to virtual objects 710 and particles 712. Additionally, device 702 displays particles 772 as moving to a surge state for a reaction portion and moving based on the user's detected breathing, similar to particles 712. In an ending portion, device 702 displays particles 772 as reassembling to form virtual object 770, similar to particles 712 forming virtual object 710, as described above with respect to Fig.7I Discussed.

[0169] In some embodiments, device 702 selects visual and audio characteristics of the meditation session to provide a varying user experience for multiple meditation sessions and / or to provide a unique user experience for each meditation session. Figure 7J and Figure 7K In the depicted embodiment, device 702 outputs the same information as described above with respect to FIG. 7B to FIG. 7I The different soundscapes in the discussed embodiments, such as Figure 7JThe audio indicator 774 is depicted as indicated by the audio indicator 774. The audio indicator 774 is similar to the audio indicator 711, but has a different appearance (e.g., a different set of notes and / or a different placement) to indicate the audio being played. FIG. 7B to FIG. 7I The output of different soundscapes for the meditation sessions depicted. In some embodiments, device 702 outputs different soundscapes for various meditation sessions. In some embodiments, device 702 selects or creates a soundscape to be in harmony with the visual characteristics. For example, device 702 may select or create a relaxing soundscape to be in harmony with the relaxing beach scene and the gentle swaying of particles 772.

[0170] When device 702 generates or selects different combinations of visual and audio features for respective meditation sessions, in some embodiments, some visual and / or audio features may be repeated. For example, a virtual object or particle may have the same appearance in two different meditation sessions, but exhibit different movement characteristics (e.g., swaying in one session and pulsating in another session and / or moving faster / further in one session than in another session). As another example, the same virtual wallpaper may be used in two different meditation sessions, but the virtual objects, particles, and / or soundscapes are different for those sessions.

[0171] exist Figure 7J 7, device 702 displays menu 767, which is similar to menu 715. Menu 767 includes option element 773, which is similar to option element 720 and can be selected to display the Figure 7C The options menu depicted is similar to the options menu. Figure 7J In the depicted embodiment, the meditation session is set to a ten minute duration with the female instructor selected for the previous meditation session, as indicated by duration indicator 773-1 and instructor indicator 773-2, respectively. Menu 767 also includes a start element 776, which is similar to start element 725. In response to detecting selection of start element 776 via input 778, device 702 transitions from the introductory portion of the meditation session to the guided breathing portion, as shown in FIG. Figure 7K Depicted.

[0172] exist Figure 7K , device 702 has transitioned to the guided breathing portion and is displaying a virtual wallpaper 765 having an expanded size and is displaying virtual objects 770 and particles 772 that move based on a predefined breathing rhythm, similar to the above description of FIG. Fig.7D and Fig. 7E Additionally, device 702 has transitioned the soundscape from stereo audio to spatial audio, including gradually increasing the volume, as indicated by the displayed larger size of audio indicator 774 .

[0173] In some embodiments, a different audio guide is used for each meditation session. Figure 7J and Figure 7K The depicted embodiment of the VR meditation session uses the same female instructor's voice as in the AR meditation session, but the audio guidance output by device 702 is different for various portions of the meditation session. In some embodiments, the audio guidance is selected from a subset of the selected instructor's audio recordings available for the respective portion of the meditation session. Figure 7K , device 702 outputs instructions 780-1 that provide similar instructions as instructions 740-1, but use different words (and, in some instances, different speaking characteristics such as intonation) because it is a different audio recording than instructions 740-1.

[0174] Figure 7L Examples of different audio and visual features of different meditation sessions are shown, showing the guided breathing portion of different meditation sessions. Figure 7L , device 702 displays virtual wallpaper 787, which is similar to virtual wallpaper 765, but has a mountain scene instead of a beach scene. Device 702 also displays virtual objects 790 and particles 792 (similar to virtual objects 710 and 770 and particles 712 and 772), which show additional examples of different visual characteristics of different meditation sessions. Device 702 also outputs different soundscapes as indicated by audio indicators 794 and audio guidance 785-1, which are similar to audio guidance 740-1 and 780-1, but are different audio recordings as indicated by different words included in audio guidance 785-1. In some embodiments, the audio guidance is a different audio recording that uses the same words but has different audio characteristics such as pitch, accent, or rhythm.

[0175] The following reference is relative to FIG. 7A to FIG. 7L The described methods 800, 900 and 1000 provide for FIG. 7A to FIG. 7L Additional description of .

[0176] Figure 8 800 is a flow chart of an exemplary method 800 for providing particles that move based on a user's breathing characteristics to a computer-generated user experience session according to some embodiments. In some embodiments, the method 800 is performed on a computer system (e.g., Figure 1 1 and / or device 702) (e.g., a smart phone, device, and / or head-mounted display generation component), which is executed at the computer system 101 and / or device 702 in the display generation component (e.g., Figure 1 , Figure 3 and Figure 4The display generation component 120 and / or display 702-1 in (e.g., a display, a touch screen, a visual output device, a 3D display, a display having at least a portion that is transparent or translucent on which an image can be projected (e.g., a see-through display and / or a transparent display), a projector, a head-up display, and / or a display controller) and one or more sensors (e.g., a camera 702-2) (e.g., a gyroscope, an accelerometer, a motion sensor, a movement sensor, a microphone, an infrared sensor, a camera sensor, a depth camera, a visible light camera, an eye tracking sensor, a gaze tracking sensor, a physiological sensor, an image sensor, a camera pointing downward at the user's hand (e.g., a color sensor, an infrared sensor, and other depth sensing cameras) and / or a camera pointing forward from the user's head) communicate. In some embodiments, method 800 is performed by storing in a non-transitory (or transient) computer-readable storage medium and by one or more processors of a computer system (such as one or more processors 202 of computer system 101) (e.g., Figure 1 Some operations in method 800 are optionally combined, and / or the order of some operations is optionally changed.

[0177] In method 800, a computer system (e.g., 702) displays (802) a user interface (e.g., 705, 765, and / or 787) for a user experience session (e.g., a UI for an application for an XR environment, the UI optionally including guided instructions for breathing to relax and / or focus the user) via a display generation component (e.g., 702-1) (e.g., in an XR environment, in a non-XR environment). While the user experience session is active (e.g., after the user experience session is initiated and before the user experience session has ended), the computer system detects (804) via one or more sensors (e.g., 702-2) one or more breathing characteristics of a user (e.g., 701) of the computer system (e.g., whether the user is currently exhaling, whether the user is currently inhaling, inhalation rate, exhalation rate, inhalation duration, exhalation duration, pause duration (e.g., after inhalation, after exhalation, during inhalation, during exhalation), changes in inhalation and / or exhalation rate, and / or inhalation and / or exhalation pattern), and displays (806) a user interface object (e.g., 710, 770, and / or 790) (e.g., a cube, a sphere, a spherical object, a cloud, a pyramid, and / or an abstract object) having a plurality of particles (e.g., 712, 772, and / or 792) that move based on the one or more breathing characteristics of the user of the computer system.

[0178] As part of displaying a user interface for a user experience session, and based on a determination that a first respiratory event of a user of the computer system (e.g., when the user inhales, exhales, and / or pauses while breathing) satisfies a first set of criteria (e.g., characteristics of the respiratory event indicate that the user is inhaling), the computer system (e.g., 702) displays (808) particles (e.g., 712, 772, and / or 792) of a user interface object (e.g., 710, 770, and / or 790) as being displayed in a first manner (e.g., as the user inhales, exhales, and / or pauses while breathing) during the first respiratory event of the user of the computer system. Figure 7G The particles may be depicted as described above (e.g., having a first direction of movement (e.g., expanding away from a fixed point (e.g., in a user's field of view)), a first rate or speed of movement, and / or a first pattern of movement (e.g., expanding from a fixed point when a user inhales, wherein the particles move at a rate determined based on a rate at which the user inhales).

[0179] As part of displaying a user interface for a user experience session, and based on a determination that a first breathing event of a user of the computer system satisfies a second set of criteria (e.g., characteristics of the breathing event indicate that the user is exhaling), the computer system (e.g., 702) displays (810) particles (e.g., 712, 772, and / or 792) of a user interface object (e.g., 710, 770, and / or 790) as being displayed during the first breathing event of the user of the computer system in a second manner that is different from the first manner (e.g., as Figure 7F The particles of the user interface object may be depicted as moving (e.g., having a second direction of movement (e.g., contracting toward a fixed point (e.g., in the user's field of view)), a second rate or speed of movement, and / or a second pattern of movement (e.g., contracting toward a fixed point as the user exhales, wherein the particles move at a rate determined based on the rate at which the user exhales). Displaying particles of the user interface object as moving in a first manner based on a determination that a first respiratory event of the user satisfies a first set of criteria and displaying particles of the user interface object as moving in a second manner based on a determination that a first respiratory event of the user satisfies a second set of criteria provides feedback about a state of the computer system (e.g., providing a state of a user experience session).

[0180] In some embodiments, the computer system (e.g., 702) communicates with an audio generation component (e.g., 703) (e.g., a speaker, a bone conduction audio output device, and / or an audio generation component integrated into an HMD). In some embodiments, as part of displaying a user interface for a user experience session, prior to the user experience session activity (e.g., after launching an application available for providing the user experience session and prior to initiating (e.g., starting) the user experience session (e.g., by a user of the computer system), the computer system concurrently displays multiple particles of a user interface object (e.g., such as particles of a plurality of ... Figure 7Band / or Figure 7C The particles 712 depicted, such as Figure 7J The particles 772 depicted and / or Figure 7L The user experience session may be performed by generating a representation of the particles 792 depicted (e.g., a static and / or animated view of the particles before the user experience session begins), and outputting an audio soundscape for the user experience session via an audio generation component (e.g., as shown in FIG. 1 ). Figure 7B and / or Figure 7C The audio 711 depicted, Figure 7J The audio 774 and / or Figure 7L Depicted audio 794) (e.g., a set of curated sound components automatically and / or manually selected to create an audio environment for the user experience session). Displaying representations of multiple particles of user interface objects before the user experience session is active and outputting an audio soundscape for the user experience session provides feedback about the state of the computer system. For example, the audio and visual feedback indicates to the user what the user experience session will include when active.

[0181] In some embodiments, the computer system outputs a portion of the audio soundscape (e.g., a preview of the audio soundscape) before the user experiences the activity, and outputs the complete audio soundscape (e.g., a preview of the audio soundscape) while the user experiences the activity. Fig.7D , Fig. 7E , Figure 7F , Figure 7G , Figure 7H , Figure 7K and / or Figure 7L In some embodiments, outputting the audio soundscape comprises: outputting the audio soundscape (or a portion of the audio soundscape) having a set of two or more audio components randomly or pseudo-randomly selected from a set of available audio components.

[0182] In some embodiments, as part of displaying a user interface for a user experience session, a computer system (e.g., 702) displays a user interface prior to a user experience session activity (e.g., Figure 7B and Figure 7C ) displays a dimmed appearance (e.g., 704) (e.g., a fade effect and / or a dimming effect) (e.g., dimming at least a portion of the environment by 0.5%, 1%, 2%, 3%, 5%, 7%, 10%, 15%, 20%, and / or another amount greater than 0%) of an environment (e.g., 700a) for a user experience (e.g., a virtual environment and / or a physical environment (e.g., visible via pass-through video and / or due to the transparent nature of the display)). Displaying the dimmed appearance of the environment prior to the user experience session activity encourages a user of the computer system to focus on the user experience session.

[0183] In some embodiments, the dimmed appearance of the display environment includes: visually obscuring a view of the physical environment (e.g., 700a) visible to a user (e.g., 701) of the computer system (e.g., 702) (e.g., via pass-through video and / or due to the transparent nature of the display). In some embodiments, the dimmed appearance of the display environment includes: displaying a partially transparent virtual overlay (e.g., 704) to show the physical environment through the virtual overlay. In some embodiments, the dimmed appearance is uniform. In some embodiments, the dimmed appearance is variable.

[0184] In some embodiments, displaying a user interface for a user experience session includes: before the user experience session is active, the computer system (e.g., 702) displays a start option (e.g., 725 and / or 776) (e.g., an affordance, a graphical user interface object, and / or a graphical element) that can be selected (e.g., via input 724 and / or input 778) (e.g., via a pinch gesture, a tap input, a gaze gesture, a gaze rest gesture, and / or other input gestures) to initiate (e.g., start, begin, and / or activate) the user experience session, and displays an indication (e.g., 720-1, 732-1, 732-2, 732-3, and / or 773-1) of the duration of the user experience session (e.g., 3 minutes, 5 minutes, 10 minutes, 12 minutes, 15 minutes, 20 minutes) (e.g., when the user experience session is active). Displaying the start affordance and the indication of the duration of the user experience session before the user experience session is active provides feedback about the state of the computer system. For example, the indication of the duration provides feedback to the user about the selected length of the user experience session. In some embodiments, in response to detecting input (e.g., 724 and / or 778) directed to a start option (e.g., a pinch gesture, a tap input, a gaze gesture, a gaze rest gesture, and / or other input gestures), the computer system initiates a user experience session.

[0185] In some embodiments, displaying a user interface for a user experience session includes: before the user experience session is active, the computer system (e.g., 720) displays a set of one or more duration options (e.g., 732, 732-1, 732-2, and / or 732-3), which are selectable (e.g., via a pinch gesture, a tap input, a gaze gesture, a gaze dwell gesture, and / or other input gestures) to modify the duration of the user experience session (e.g., the duration of at least a portion of the user experience session such as a breathing portion and / or a reaction portion of the guidance). The computer system detects input (e.g., 731) directed to a first duration option (e.g., 732-1) of the set of one or more duration options that is selectable to modify the duration of the user experience session (e.g., 732-1), and in response to detecting the input directed to the first duration option of the set of one or more duration options, the computer system selects (e.g., modifies and / or sets based on the detected input) the duration of the user experience session to a first duration (e.g., 5 minutes or 10 minutes) (e.g., modified from a default duration of 1 minute or from a previously selected duration of 20 minutes). In some embodiments, detecting the input directed to the second duration option causes the computer system to set the duration to a second duration that is different from the first duration. Modifying the duration of the user experience session to the first duration in response to detecting the input directed to the first duration option causes the computer system to automatically optimize the user experience session to meet a requested duration provided by a user of the computer system.

[0186] In some embodiments, displaying a user interface for a user experience session includes: before the user experience session is active, the computer system (e.g., 702) displays a set of one or more audio options (e.g., 736, 736-1, and / or 736-2), which are selectable (e.g., via a pinch gesture, a tap input, a gaze gesture, a gaze rest gesture, and / or other input gestures) to select audio guidance (e.g., an audio source, a narrator, a speaker, a coach, and / or a person) from multiple audio guides for the user experience session. The computer system detects an input (e.g., 735) pointing to a first audio option (e.g., 736-2) in the set of one or more audio options that can be selected to select audio guidance from multiple audio guides for the user experience session. In response to detecting the input pointing to the first audio option, the computer system selects the first audio guidance from multiple audio guides for the user experience session (e.g., and optionally does not select the second audio guidance for the user experience session). In response to detecting the input pointing to the first audio option that can be selected to select the audio guidance, selecting the first audio guidance for the user experience session causes the computer system to automatically optimize the user experience session to provide the requested audio guidance for the user experience session. In some embodiments, in response to detecting input directed to a second audio option (e.g., 736-1), the computer system selects a second audio guidance different from the first audio guidance for the user experience session. In some embodiments, the selected audio guidance provides verbal instructions, encouragement, teaching, prompts, and / or guidance to the user in order to experience the user experience session.

[0187] In some embodiments, displaying a user interface for a user experience session includes: while the user experience session is active (e.g., after (in response to) starting the user experience session; and / or in a first phase or portion of the user experience session (e.g., a phase or portion of the user experience session prior to a subsequent phase or portion of the user experience session)), the computer system (e.g., 702) displays the user interface objects (e.g., 710, 770, and / or 790) as having an animation effect (e.g., animated movement of user interface objects and / or particles of user interface objects) (e.g., a pulsating animation (e.g., a repeated alternating pattern of increasing the size of the user interface object (e.g., expanding the user interface object) and decreasing the size of the user interface object (e.g., contracting the user interface object))), the animation effect being based on a predetermined biometric rhythm (e.g., a breathing rhythm (e.g., an inhalation and exhalation pattern)) (e.g., 3 breaths per minute, 5 breaths per minute, 7 breaths per minute, 10 breaths per minute) (e.g., Fig.7D , Fig. 7E and / or Figure 7KDepicted). In some embodiments, the predetermined biometric rhythm is a default setting set by an application used to provide the user experience session and / or by an operating system of the computer system. In some embodiments, the predetermined biometric rhythm is a setting selected by the user. In some embodiments, the predetermined biometric rhythm includes heart rate. In some embodiments, the predetermined biometric rhythm includes walking speed. Displaying a user interface object as having an animation effect based on a predetermined biometric rhythm includes: based on a determination (e.g., based on one or more settings of the user experience session) that the predetermined biometric rhythm is a first biometric rhythm (e.g., 5 breaths per minute (e.g., inhaling and then exhaling (optionally pausing in between) five times in one minute) or 6 breaths per minute), the computer system animates the user interface object based on a first pattern corresponding to the first biometric rhythm (e.g., expanding the user interface object using the inhalation portion of the breathing rhythm and contracting the user interface object using the exhalation portion of the breathing rhythm, and / or rotating the user interface object in a first direction using the inhalation portion of the breathing rhythm and exhaling using the breathing rhythm The computer system may also provide a computer system that animates the user interface object based on a first pattern corresponding to the second biometric rhythm (e.g., utilizing the inhalation portion of the breathing rhythm to expand the user interface object and utilizing the exhalation portion of the breathing rhythm to contract the user interface object, and / or utilizing the inhalation portion of the breathing rhythm to rotate the user interface object in the first direction and utilizing the exhalation portion of the breathing rhythm to rotate the user interface object in the second direction). The computer system may also provide a computer system that animates the user interface object based on a first pattern corresponding to the first biometric rhythm based on a determination that the predetermined biometric rhythm is the first biometric rhythm, and may provide a computer system that animates the user interface object based on a second pattern corresponding to the second biometric rhythm based on a determination that the predetermined biometric rhythm is the second biometric rhythm, providing feedback about the state of the computer system.

[0188] In some embodiments, the predetermined biometric rhythm is a biometric rhythm (e.g., a user-selectable or customizable setting) selected (e.g., via input 733) by a user (e.g., 701) of a computer system (e.g., 702). In some embodiments, the first biometric rhythm is a default biometric rhythm (e.g., the biometric rhythm is set by an application for providing the user experience session and / or by an operating system of the computer system), and the second biometric rhythm is a biometric rhythm selected by a user of the computer system (e.g., a user-selectable or customizable setting). In some embodiments, both the first biometric rhythm and the second biometric rhythm are user-selectable settings.

[0189] In some embodiments, the computer system (e.g., 702) communicates with an audio generating component (e.g., 703) (e.g., a speaker, a bone conduction audio output device, and / or an audio generating component integrated into an HMD). In some embodiments, while the user experience the session is active, the computer system outputs audio components (e.g., 711, 774, and / or 794) having perceptual spatial locations (e.g., 709-3 to 709-9) that move (e.g., automatically) (e.g., relative to the user's location) based on one or more breathing characteristics of a user of the computer system (e.g., one or more audio components of a soundscape (automatically) selected for the user experience session (e.g., such as Fig.7D , Fig. 7E , Figure 7F , Figure 7G , Figure 7H and / or Figure 7K Based on a determination that a second respiratory event (in some embodiments, the second respiratory event is a first respiratory event) of a user of the computer system (e.g., 701) satisfies a third set of criteria (in some embodiments, the third set of criteria is the first set of criteria), the computer system outputs a signal having a first perceived spatial location relative to the user of the computer system (e.g., as Figure 7G Based on a determination that a second respiratory event of the user of the computer system satisfies a fourth set of criteria (in some embodiments, the fourth set of criteria is the second set of criteria), the computer system outputs an audio component having a second perceptual spatial location different from the first perceptual spatial location relative to the user of the computer system (e.g., Figure 7FBased on a determination that the second respiratory event satisfies a third set of criteria, outputting the audio component having a first perceived spatial location relative to a user of the computer system and based on a determination that the second respiratory event satisfies a fourth set of criteria, outputting the audio component having a second perceived spatial location relative to the user causes the computer system to automatically adjust the spatial location of the audio component based on the detected respiratory event of the user of the computer system.

[0190] In some embodiments, the spatial position of the audio component changes in conjunction with changes in respiratory events. In some embodiments, the perceived spatial audio position of the audio component changes as particles of the user interface object move. For example, as the particles move in a first manner (e.g., expanding away from a fixed point), the perceived spatial audio position moves toward a user of the computer system, and as the particles move in a second manner (e.g., contracting toward a fixed point), the perceived spatial audio position moves away from the user of the computer system.

[0191] In some embodiments, a portion of the physical environment (e.g., 700) of a user (e.g., 701) of a computer system (e.g., 702) is visible (e.g., 700a) prior to the user experience session activity (e.g., via a pass-through video display and / or due to the transparent nature of the display). In some embodiments, the computer system initiates (e.g., starts, begins, and / or activates) the user experience session. In some embodiments, as part of initiating the user experience session, the computer system displays, via a display generation component (e.g., 702-1), a gradually decreasing visibility of the physical environment (e.g., as shown in FIG. 700a ). Figure 7B , Figure 7C and / or Fig.7D A dimming effect (e.g., 704) (e.g., a fade-in / fade-out effect) (e.g., dimming 99.9%, 99%, 98%, 97%, 95%, 90%, 85%, 80%, and / or another amount less than 100%) is depicted (e.g., visibility of the physical environment is reduced at a first time (e.g., 50% visible), further reduced at a second time after the first time (e.g., 25% visible), further reduced at a third time after the second time (e.g., 10% visible), and further reduced at a fourth time after the third time (e.g., 2% visible)). Displaying a dimming effect that gradually reduces visibility of the physical environment reduces distraction and encourages a user of the computer system to focus on the user experience session.

[0192] In some embodiments, displaying a dimming effect (e.g., 704) includes visually blurring a view of a physical environment (e.g., 700a) visible to a user of a computer system (e.g., 702) (e.g., via pass-through video and / or due to the transparent nature of the display). In some embodiments, displaying a dimming effect includes displaying a partially opaque virtual overlay with increased opacity to reduce visibility of the physical environment through the virtual overlay, resulting in an effect where the physical environment appears to fade out of view (or appears to be barely visible). In some embodiments, the dimming effect is uniform. In some embodiments, the dimming effect is variable.

[0193] In some embodiments, while a user experience session is active, a computer system (e.g., 702) detects, via one or more sensors (e.g., 702-2), one or more focus-based characteristics of a user (e.g., 701) of the computer system (e.g., 701) (e.g., one or more biometric characteristics (e.g., head movement, body movement, breathing rate, heart rate, and / or eye gaze) and / or user input indicating whether the user is focusing their attention on the user experience session (e.g., breathing and / or listening to audio guidance) and / or user input indicating a particular level of focus on the user experience session), and outputs feedback (e.g., 740-4) (e.g., audio feedback and / or visual feedback) based on the one or more focus-based characteristics of the user of the computer system (e.g., such as Figure 7H Outputting feedback based on one or more concentration-based characteristics of a user of the computer system enables the computer system to automatically encourage the user of the computer system to participate in a user experience session and provide feedback to the user regarding the detected concentration-based characteristics.

[0194] In some embodiments, outputting feedback based on one or more focus-based characteristics of a user (e.g., 701) of a computer system (e.g., 702) includes: displaying visual feedback based on one or more focus-based characteristics of the user of the computer system via a display generation component (e.g., movement of particles of a user interface object (e.g., 712, 772, and / or 792) and / or text indicating a level of focus of the user). Displaying visual feedback based on one or more focus-based characteristics of the user of the computer system enables the computer system to automatically encourage the user of the computer system to participate in a user experience session and provide visual feedback to the user about the detected focus-based characteristics. In some embodiments, when one or more focus-based characteristics of the user indicate that the user is focusing on the user experience session, the visual feedback includes movement of particles of the user interface object. In some embodiments, when one or more focus-based characteristics of the user indicate that the user is not focusing on the user experience session (e.g., as Figure 7H), the visual feedback includes a pause (e.g., a temporary pause) in the movement of the particle.

[0195] In some embodiments, displaying visual feedback based on one or more focus-based characteristics of a user (e.g., 701) of a computer system (e.g., 702) includes: based on a determination that the one or more focus-based characteristics of the user of the computer system satisfy a first set of focus criteria (e.g., one or more biometric characteristics of the user (e.g., head movement, body movement, breathing rate, heart rate, and / or eye gaze) indicate that the user is focusing their attention on the user experience session (e.g., breathing and / or listening to audio guidance)) (in some embodiments, and based on a determination that a first breathing event of the user satisfies the first set of criteria), displaying particles of a user interface object (e.g., 712, 772, and / or 792) as moving away from a reference location (e.g., a viewpoint or a predefined location of the user) during an outward breath (e.g., exhalation) of the user of the computer system (e.g., as Figure 7F Displaying particles of a user interface object as moving away from a reference position during an outgoing breath by a user of the computer system provides feedback about a state of the computer system (e.g., a state in which the displayed particles are responsive to a detected outgoing breath by the user). In some embodiments, displaying particles of a user interface object as moving in a first manner during a first respiratory event of the user includes displaying the particles as moving away from a reference position during an outgoing breath.

[0196] In some embodiments, displaying visual feedback based on one or more focus-based characteristics of the user of the computer system includes: based on a determination that the one or more focus-based characteristics of the user of the computer system satisfy a first set of focus criteria (e.g., one or more biometric characteristics of the user (e.g., head movement, body movement, breathing rate, heart rate, and / or eye gaze) indicate that the user is focusing their attention on the user experience session (e.g., breathing and / or listening to audio guidance)) (and in some embodiments, based on a determination that a first breathing event of the user satisfies a second set of criteria), displaying particles of a user interface object (e.g., 712, 772, and / or 792) as moving toward a reference location (e.g., a viewpoint of the user or a predefined location) during an inward breathing (e.g., inhalation) of the user of the computer system (e.g., as Figure 7G Displaying particles of the user interface object as moving toward a reference position during inward breathing of a user of the computer system provides feedback about a state of the computer system (e.g., a state in which the displayed particles are responsive to the user's detected inward breathing). In some embodiments, displaying particles of the user interface object as moving in a second manner during a first breathing event of the user includes displaying the particles as moving toward a reference position during inward breathing.

[0197] In some embodiments, the computer system (e.g., 702) displays particles of a user interface object (e.g., 712, 772, and / or 792) as moving toward a reference position at a first rate during inward breathing. In some embodiments, the computer system displays particles of a user interface object as moving away from a reference position at a second rate during outward breathing. In some embodiments, the second rate is different from (e.g., slower than; faster than) the first rate. In some embodiments, the second rate is different from the first rate regardless of the characteristics of the inward breathing and the outward breathing (e.g., speed, volume, and / or duration).

[0198] In some embodiments, the reference location is a world-locked (e.g., environment-locked) location corresponding to a viewpoint of a user (e.g., 701) of a computer system (e.g., 702) in a physical environment (e.g., 700) (e.g., the user's viewpoint is based on (e.g., selected with reference to and / or anchored to) a location and / or object in a three-dimensional environment (e.g., a physical environment or a virtual environment) (e.g., Figure 7H depicted).

[0199] In some embodiments, the computer system (e.g., 702) communicates with an audio generation component (e.g., 703). In some embodiments, outputting feedback based on one or more focus-based characteristics of a user of the computer system includes: outputting audio feedback based on one or more focus-based characteristics of a user of the computer system (e.g., 740-4) via an audio generation component (e.g., audio guidance commanding the user of the computer system to focus on one or more elements of the user experience session (e.g., breathing and / or listening to audio) and / or audio tones). Outputting audio feedback based on one or more focus-based characteristics of a user of the computer system enables the computer system to automatically encourage the user of the computer system to participate in the user experience session.

[0200] In some embodiments, one or more focus-based characteristics of a user (e.g., 701) of a computer system (e.g., 702) include multiple biometric indicators (e.g., gaze (e.g., increased movement / glances), head posture (significant changes in head orientation (e.g., rotation and / or tilt)), and / or breathing (e.g., increased or irregular frequency)).

[0201] In some embodiments, one or more focus-based characteristics of a user (e.g., 701) of a computer system (e.g., 702) include an indication of whether the user's focus has failed to meet a focus criteria (e.g., the user's focus has ceased to be focused on the user experience session) for a threshold amount of time (e.g., 7 seconds, 8 seconds, 9 seconds, or 10 seconds).

[0202] In some embodiments, displaying particles of a user interface object (e.g., 712, 772, and / or 792) as moving in a first manner or a second manner includes: a computer system (e.g., 702) displaying a first group of one or more particles (e.g., 712-2) as having a first distance from a location corresponding to a viewpoint of a user of the computer system and having a first amount of movement during a first respiratory event (e.g., as Figure 7F or Figure 7G The computer system may further include displaying a second set of one or more particles (e.g., 712-1) that is different from the first set of one or more particles as having a second distance from a position corresponding to the viewpoint of the computer system user that is different from (e.g., smaller than; larger than) the first distance and having a second amount of movement during the first respiratory event that is different from (e.g., smaller than; larger than) the first amount of movement (e.g., as depicted in FIG. 1 ). Figure 7F or Figure 7G Displaying the second set of one or more particles as having a second distance from the user that is different from the first distance and having a second amount of movement that is different from the first amount of movement during the first respiratory event provides feedback about the state of the computer system.

[0203] In some embodiments, a first group of one or more particles (e.g., 712-2) are displayed as having a perceived distance from the user that is farther than a second group of one or more particles (e.g., 712-1), and the first group of one or more particles move a smaller amount during a corresponding respiratory event than the second group of one or more particles. In some embodiments, from the user's perspective, the shorter the distance of the corresponding particles, the greater the amount of movement of the corresponding particles during the corresponding respiratory event. For example, as the user inhales or exhales, particles that are displayed closer to the user appear to move a larger amount (to the user) than particles that are displayed farther away from the user, and particles that are displayed farther away from the user appear to move a smaller amount than particles that are displayed closer to the user.

[0204] In some embodiments, the movement of the plurality of particles (eg, 712, 772, and / or 792) is based on a set of one or more simulated physical parameters (eg, inertia, spring constant, and / or friction).

[0205] In some embodiments, displaying a user interface for a user experience session includes: while the user experience session is active, a computer system (e.g., 702) detecting, via one or more sensors (e.g., 702-2), gaze data indicating a gaze of a user of the computer system (e.g., 701). In some embodiments, while a user interface object is displayed as having a plurality of particles (e.g., 712, 772, and / or 792) that move based on one or more breathing characteristics of a user of the computer system, the computer system detects updated gaze data. In some embodiments, in response to detecting updated gaze data, and based on a determination that the updated gaze data indicates that the user's gaze exceeds a gaze deviation threshold (e.g., the user's gaze is not focused on or looking at one or more elements (e.g., particles) of the user experience session for at least a threshold amount of time (e.g., 7 seconds, 8 seconds, 9 seconds, 10 seconds, or 12 seconds)), the computer system pauses the user experience session (e.g., temporarily stops the user experience session) (e.g., as Figure 7H In some embodiments, in response to detecting updated gaze data, and based on a determination that the updated gaze data does not indicate that the user's gaze exceeds a gaze deviation threshold, the computer system abandons pausing the user experience session (e.g., continuing or resuming the user experience session). Selectively pausing the user experience session based on a determination of whether the updated gaze data indicates that the user's gaze exceeds a gaze deviation threshold enables the computer system to automatically suspend the user experience session when a user of the computer system is disturbed. In some embodiments, pausing the user experience session includes: stopping displaying a user interface object. In some embodiments, pausing the user experience session includes: displaying a user interface object in a static state. In some embodiments, pausing the user experience session includes: outputting an audio instruction for resuming focus on the user experience session.

[0206] In some embodiments, displaying a user interface for a user experience session includes: a computer system (e.g., 702) displays the user interface (e.g., 705, 710, 712, 770, 772, 790, and / or 794) as having a set of visual characteristics (e.g., shapes (e.g., cubes, spheres, spheres, clouds, pyramids, and / or abstract objects), components (e.g., particles), and / or visual appearances (e.g., animation effects, translucency, moving characteristics, and / or displayed background)) randomly or pseudo-randomly selected from a set of available visual characteristics (e.g., a superset of visual characteristics). In some embodiments, the computer system communicates with an audio generation component (e.g., 703). In some embodiments, a computer system outputs an audio soundscape (e.g., 711, 774, and / or 794) for a user experience session (e.g., a set of curated sound components selected to create an audio environment for the user experience session) via an audio generation component, wherein the audio soundscape is output concurrently with displaying a user interface for the user experience session, and outputting the audio soundscape includes: outputting an audio soundscape having a first set of two or more audio components (e.g., a set of sound components) randomly or pseudo-randomly selected from a set of available audio components. Displaying the user interface as having a set of visual characteristics randomly or pseudo-randomly selected from a set of available visual characteristics and outputting an audio soundscape having a first set of two or more audio components randomly or pseudo-randomly selected from a set of available audio components enables the computer system to provide a more realistic user experience while saving storage space by not requiring multiple different complete audio tracks and / or visual components to be stored and selected for playback / display. In some embodiments, two or more audio components in the first set of audio components are a subset of curated sound components selected from a superset of curated sound components for the audio soundscape. Additional aspects of the soundscape are discussed in more detail below with respect to method 1000.

[0207] In some embodiments, displaying a user interface for a user experience session includes, when displaying a user interface object (e.g., 710, 770, and / or 790) to have a first display state in which a plurality of particles (e.g., 712, 772, and / or 792) are displayed to have a first amount of spacing (e.g., dense, compact, and / or close grouping of particles; forming shapes (e.g., spheres, clouds, triangles, and / or cubes)) for a first portion (e.g., an introductory portion) of the user experience session. Figure 7B , Figure 7C and / or Figure 7JIn one embodiment, the computer system (e.g., 702) detects a transition from a first portion of the user experience session to a second portion of the user experience session (e.g., a guided breathing portion of the user experience session). In response to detecting the transition from the first portion of the user experience session to the second portion of the user experience session, the computer system displays the user interface object to have a second display state that is different from the first display state (e.g., transitions to the second display state), wherein a plurality of particles are displayed for the second portion of the user experience session to have a second spacing amount that is different from (e.g., greater than) the first spacing amount (e.g., a loose grouping of particles; a more spaced-apart positioning of particles, but still forming respective shapes (e.g., spheres, clouds, triangles, and / or cubes)) (e.g., such as Fig.7D , Fig. 7E , Figure 7K and / or Figure 7L In response to detecting a transition from a first portion of the user experience session to a second portion of the user experience session, displaying the user interface object with a second display state different from the first display state, wherein the plurality of particles are displayed with a second spacing amount different from the first spacing amount for the second portion of the user experience session, provides feedback regarding a state of the computer system (e.g., a state of a particular portion of the user experience session that the computer system is providing).

[0208] In some embodiments, displaying a user interface for a user experience session includes: when a computer system (e.g., 702) displays a user interface object (e.g., 710, 770, and / or 790) as having a second display state (e.g., as Fig.7D , Fig. 7E , Figure 7K and / or Figure 7L In response to detecting the transition from the second portion of the user experience session to the third portion of the user experience session, the computer system displays the user interface object as having (e.g., transitioning to) a third display state that is different from the first display state and the second display state, wherein a plurality of particles (e.g., 712, 772, and / or 792) are displayed for the third portion of the user experience session as having a third spacing amount that is different from (e.g., greater than) the second spacing amount (e.g., the spaced-apart positioning of the particles and / or the spacing of the particles that no longer form different shapes) that is different from (e.g., greater than) the second spacing amount (e.g., such as Figure 7F , Figure 7G and / or Figure 7HIn response to detecting a transition from the second portion of the user experience session to the third portion of the user experience session, displaying the user interface object with a third display state different from the first display state and the second display state, wherein the plurality of particles are displayed with the third spacing amount different from the second spacing amount for the third portion of the user experience session, provides feedback regarding a state of the computer system (e.g., a state of a particular portion of the user experience session that the computer system is providing).

[0209] In some embodiments, displaying a user interface for a user experience session includes: displaying, on a computer system (e.g., 702), user interface objects (e.g., 710, 770, and / or 790) as having an arrangement (e.g., such as Figure 7F , Figure 7G and / or Figure 7H When the computer system detects a termination of the user experience session (e.g., a transition to an ending portion (e.g., an end) of the user experience session or an immediate termination) when the computer system displays a plurality of particles (e.g., 712, 772, and / or 792) having a plurality of particles (e.g., as depicted) (e.g., spaced apart positioning of particles, spacing of particles that do not form different shapes, uniform spacing of particles, irregularly spaced positioning of particles, and / or a non-uniform manner of spacing of particles), the computer system detects a termination of the user experience session (e.g., a transition to an ending portion (e.g., an end) of the user experience session or an immediate termination). In response to detecting the termination of the user experience session, the computer system animates the plurality of particles to move to an arrangement having a second average spacing between the particles, wherein the second average spacing is less than the first average spacing (e.g., as depicted). Fig.7I In some embodiments, the computer system animates the plurality of particles as moving to the grouped arrangement in response to detecting the termination of the user experience session (e.g., dense, compact, and / or close grouping of particles, forming shapes (e.g., spheres, clouds, triangles, pyramids, and / or cubes), uniform spacing of particles, irregularly spaced positioning of particles, and / or a non-uniform manner of spacing of particles) (e.g., and terminates the user experience session (e.g., initiates termination of the user experience session)). Animating the plurality of particles as moving to the grouped arrangement in response to detecting the termination of the user experience session provides feedback about the state of the computer system (e.g., the state that the computer system is terminating the user experience session). In some embodiments, in response to detecting the termination of the user experience session, the computer system terminates the user experience session (or initiates termination of the user experience session) while (or after) the plurality of particles are animated as moving to an arrangement having a second average spacing between particles. In some embodiments, terminating the user experience session includes:

[0210] Transition to the end of the user experience session.

[0211] In some embodiments, displaying a user interface for a user experience session includes: while the user experience session is active and an environment (e.g., 700a) (e.g., elements in a physical environment or representations of virtual elements in an XR environment) of a user (e.g., 701) of a computer system (e.g., 702) is visually blurred (e.g., via a dimming effect 704) (e.g., dimmed (e.g., 1% visible or 2% visible) or not displayed), the computer system detects termination of the user experience session (e.g., a transition to an ending portion (e.g., the end) of the user experience session. In response to detecting termination of the user experience session, the computer system initiates termination of the user experience session (e.g., transitions to an ending portion (e.g., the end) of the user experience session and / or ends the user experience session) and gradually increases visibility of the user's environment (e.g., fades the environment into view) (e.g., as shown in FIG. 7 ). Fig.7I ). Initiating termination of the user experience session in response to detecting termination of the user experience session and gradually increasing visibility of the user's environment provides feedback about the state of the computer system (e.g., the state of the computer system terminating the user experience session). In some embodiments, increasing the visibility of the environment includes: reducing a dimming effect displayed (e.g., overlaid) on the environment (e.g., a physical environment). In some embodiments, increasing the visibility of the environment includes: displaying a larger portion of the environment (e.g., a virtual or XR environment). In some embodiments, the visibility of the environment increases at a first time (e.g., 10% visible), further increases at a second time after the first time (e.g., 25% visible), further increases at a third time after the second time (e.g., 50% visible), and further increases at a fourth time after the third time (e.g., 98% visible).

[0212] In some embodiments, displaying a user interface for a user experience session includes: while the user experience session is active, the computer system (e.g., 702) detects termination of the user experience session (e.g., transition to an ending portion (e.g., the end) of the user experience session). In response to detecting termination of the user experience session, the computer system initiates termination of the user experience session (e.g., transitions to an ending portion (e.g., the end) of the user experience session and / or ends the user experience session) and displays an option (e.g., 758) that is selectable to continue the user experience session (e.g., resume the user experience session after termination of the session; resume the user experience session before termination of the session). Initiating termination of the user experience session and displaying an option that is selectable to continue the user experience session reduces the number of inputs required to continue the user experience session by automatically displaying an option to continue the user experience session without requiring additional user input. In some embodiments, in response to detecting a selection of an option that is selectable to continue the user experience session, the computer system resumes the user experience session and stops displaying the option that is selectable to continue the user experience session.

[0213] In some embodiments, displaying a user interface for a user experience session includes: while the user experience session is active, the computer system (e.g., 702) detecting termination of the user experience session (e.g., transitioning to an ending portion (e.g., the end) of the user experience session. In response to detecting the termination of the user experience session, the computer system initiates termination of the user experience session (e.g., transitioning to an ending portion (e.g., the end) of the user experience session and / or ending the user experience session) and displays a history of data associated with (e.g., based on or corresponding to) one or more previous user experience sessions (e.g., 754 and / or 756) (in some embodiments, the one or more previous user experience sessions include the current user experience session). Initiating termination of the user experience session and displaying a history of data associated with one or more previous user experience sessions reduces the amount of input required to view historical data by automatically displaying the historical data without requiring additional user input.

[0214] In some embodiments, displaying a user interface for a user experience session includes causing at least a portion of an environment (e.g., 700a) (e.g., elements in the physical environment and / or representations of virtual elements in the XR environment) of a user (e.g., 701) of a computer system (e.g., 702) to be visible (e.g., via a pass-through video display and / or due to a transparent property of the display) while the user experience session is active (e.g., as shown in FIG. Fig.7D , Fig. 7E , Figure 7F , Figure 7G and / or Figure 7H). Causing at least a portion of the environment of a user of the computer system to be visible while the user experience session is active provides feedback about the state of the computer system. In some embodiments, a user interface for a user experience session is displayed with an amount of opacity or transparency such that the user's environment is visible through (or behind) the user interface. In some embodiments, the user interface is displayed with an opacity of 99.9%, 99%, 98%, 97%, 95%, 90%, 85%, 80%, and / or another amount of opacity that is less than 100%.

[0215] In some embodiments, while the user is experiencing session activity and while the computer system (e.g., 702) displays a user interface object (e.g., 710, 770, and / or 790) as having a first display orientation relative to a user (e.g., 701) of the computer system (e.g., 702) (e.g., the user interface object is displayed at a first angle relative to the user), the computer system receives a signal indicating that the user of the computer system is positioned from a first position in the environment (e.g., 700) (e.g., Figure 7G A change in position (e.g., a change in the position of the user relative to the user's physical environment and / or the XR environment) to a second position in the environment that is different from the first position (e.g., as depicted) Figure 7H In response to receiving data indicating a change in the positioning of a user of the computer system, the computer system displays the user interface object with a second display orientation relative to the user of the computer system that is different from the first display orientation (e.g., the user interface object is displayed at a second angle relative to the user) (e.g., as shown in FIG. Figure 7H In response to receiving data indicating a change in the position of a user of the computer system, displaying the user interface object with a second display orientation relative to the user of the computer system that is different from the first display orientation causes the computer system to automatically adjust a displayed view of the user experience session based on the detected change in the position of the user of the computer system.

[0216] In some embodiments, displaying the user interface object as having the second display orientation includes: displaying at least a portion of the user interface object that is not visible (e.g., displayed) when the user interface object is displayed as having the first orientation. In some embodiments, displaying the user interface object as having the second display orientation includes: hiding (stopping displaying) at least a portion of the user interface object that is visible (e.g., displayed) when the user interface object is displayed as having the first orientation.

[0217] In some embodiments, a user interface for a user experience session can be displayed at one or more external computer systems (e.g., devices or computer systems associated with other users (e.g., users in the XR environment)).

[0218] In some embodiments, while a user is experiencing an active session and while a computer system (e.g., 702) is displaying a user interface object as having multiple particles (e.g., 712, 772, and / or 792) that move based on one or more breathing characteristics of a user of the computer system, the computer system receives data indicating a change in posture of a portion of a user of the computer system (e.g., 701) (e.g., the user's hands and / or arms are moving in the XR environment). In response to receiving data indicating a change in the posture of a portion of a user of the computer system, the computer system updates a display of a plurality of particles, including: modifying display characteristics of the corresponding particles (e.g., changing the color, position, and / or orientation of the corresponding particles, and / or displaying an animation effect based on the intersection of the user's hand with the corresponding particles) based on a determination that the data indicating the change in the posture of the portion of the user includes an indication that the portion of the user intersects (or intersects during movement) with a displayed location of the corresponding particles; and abandoning modifying the display characteristics of the corresponding particles (e.g., maintaining the display color, position, and / or orientation of the corresponding particles and / or abandoning displaying an animation effect based on the intersection of the user's hand with the corresponding particles) based on a determination that the data indicating the change in the posture of the portion of the user does not include an indication that the portion of the user intersects (or intersects during movement) with the displayed location of the corresponding particles. Modifying the display characteristics of the corresponding particles based on a determination that the data indicating the change in the posture of the portion of the user includes an indication that the portion of the user intersects with the displayed location of the corresponding particles causes the computer system to automatically adjust the displayed corresponding particles based on the detected position of the portion of the user intersecting with the corresponding particles. In some embodiments, some of the particles move and / or change color based on movement on the part of a user (eg, the user can interact with the particles).

[0219] In some embodiments, aspects / operations of methods 900 and / or 1000 may be interchanged, replaced, and / or added between these methods. For the sake of brevity, these details are not repeated here.

[0220] Fig. 9 900 is a flow chart of an exemplary method 900 for providing a computer-generated user experience session with options selected based on characteristics of an XR environment, according to some embodiments. In some embodiments, the method 900 is performed on a computer system (e.g., Figure 1 1 and / or device 702) (e.g., a smart phone, a tablet device, and / or a head-mounted display generating component), which is executed at the computer system 101 and / or device 702 in the display generating component (e.g., Figure 1 , Figure 3 and Figure 4The display generation component 120 and / or display 702-1 in (e.g., a display, a touch screen, a visual output device, a 3D display, a display having at least a portion that is transparent or semi-transparent on which an image can be projected (e.g., a see-through display), a projector, a head-up display, and / or a display controller) and one or more sensors (e.g., camera 702-2) (e.g., a gyroscope, an accelerometer, a motion sensor, a movement sensor, a microphone, an infrared sensor, a camera sensor, a depth camera, a visible light camera, an eye tracking sensor, a gaze tracking sensor, a physiological sensor, an image sensor, a camera pointing downward at the user's hand (e.g., a color sensor, an infrared sensor, and other depth sensing cameras) and / or a camera pointing forward from the user's head) communicate. In some embodiments, method 900 is performed by storing in a non-transitory (or transient) computer-readable storage medium and by one or more processors of a computer system (such as one or more processors 202 of computer system 101) (e.g., Figure 1 Some operations in method 900 are optionally combined, and / or the order of some operations is optionally changed.

[0221] In method 900, while displaying an XR environment (e.g., 705, 765, and / or 787) having one or more characteristics (e.g., lighting characteristics, display of virtual objects, pass-through display of physical objects, and / or a history of user activity in the XR environment), a computer system (e.g., 702) detects (902) via one or more sensors (e.g., 702-1 and / or 702-2) a request to initiate a user experience session in the XR environment (e.g., via input 724 and / or input 778) (e.g., a request to begin a user experience in the XR environment that optionally includes guided instructions for breathing exercises to relax and / or focus the user and / or reaction exercises to help the user react to scenes, topics, ideas, concepts, etc.).

[0222] In response to detecting a request to initiate a user experience session in the XR environment, the computer system (e.g., 702) initiates (904) a user experience session in the XR environment (e.g., starting a user experience in the XR environment that optionally includes guided instructions for breathing exercises to relax and / or focus the user and / or reaction exercises to help the user react to scenes, topics, ideas, concepts, etc.). In some embodiments, starting the user experience in the XR environment includes: launching an application for providing the user experience.

[0223] As part of initiating a user experience session in an XR environment, a computer system (e.g., 702) displays (906) (e.g., in the XR environment) via a display generation component (e.g., 702-1) a user interface (e.g., 705, 765, and / or 787) (e.g., a UI of an application for breathing exercises and / or reaction exercises) for the user experience session. Displaying the user interface for the user experience session includes: based on a determination that one or more characteristics of the XR environment meet a first set of criteria (e.g., the XR environment is an AR environment; the XR environment includes a first environment; the XR environment has a first set of lighting conditions; one or more user experience sessions have been previously initiated in the XR environment), the computer system displays (908) the user interface for the user experience session as having a first set of one or more options enabled for the user experience session (e.g., a first set of visual and / or audio characteristics for the session, displaying a first virtual environment, and / or displaying one or more virtual objects as having a first appearance).

[0224] Based on a determination that one or more characteristics of the XR environment satisfy a second set of criteria that is different from the first set of criteria (e.g., the XR environment is a VR environment; the XR environment includes a second environment; the XR environment has a second set of lighting conditions; a user experience session has not been previously initiated in the XR environment), the computer system (e.g., 702) displays (910) a user interface for the user experience session as having a second set of one or more options enabled for the user experience session (e.g., a second set of visual and / or audio characteristics for the session, displaying a second virtual environment, and / or displaying one or more virtual objects as having a second appearance), wherein the second set of one or more options is different from the first set of one or more options. Displaying the user interface for the user experience session as having the first set of one or more options enabled for the user experience session based on a determination that the one or more characteristics of the XR environment satisfy the first set of criteria, and displaying the user interface for the user experience session as having the second set of one or more options enabled for the user experience session based on a determination that the one or more characteristics of the XR environment satisfy the second set of criteria, reduces the number of inputs required to display the user interface for the user experience session as having a particular option enabled for the user experience session.

[0225] In some embodiments, displaying a user interface for the user experience session with a first set of one or more options enabled for the user experience session (e.g., a first set of visual and / or audio characteristics for the session) includes: a computer system (e.g., 702) displaying an augmented reality (AR) environment (e.g., 705) for the user experience session (e.g., Figure 7B , Figure 7C , Fig.7D , Fig. 7E , Figure 7F , Figure 7G, Figure 7H and / or Fig.7I In some embodiments, displaying a user interface for the user experience session with a second set of one or more options enabled for the user experience session (e.g., a second set of visual and / or audio features for the session) includes: the computer system displays a virtual reality (VR) environment (e.g., 765 and / or 787) for the user experience session (e.g., as shown in FIG. 1 ). Figure 7J , Figure 7K and / or Figure 7L Displaying an AR environment or a VR environment for a user experience session when one or more characteristics of the XR environment meet a first set of criteria or a second set of criteria causes the device to automatically enable the AR environment or the VR environment based on the characteristics of the XR environment without displaying additional controls.

[0226] In some embodiments, the first set of one or more options includes options for the AR environment (e.g., such as the following) selected (e.g., randomly or pseudo-randomly) from the first set of available options (e.g., a superset of options for the AR environment). Figure 7B , Figure 7C , Fig.7D , Fig. 7E , Figure 7F , Figure 7G , Figure 7H and / or Fig.7I In some embodiments, the second set of one or more options includes a first subset of options for the VR environment (e.g., such as those depicted in the example above) selected (e.g., randomly or pseudo-randomly) from the second set of available options (e.g., a superset of the options for the VR environment). Figure 7J , Figure 7K and / or Figure 7L A second subset of options (e.g., visual and / or audio characteristics for the user experience session) is selected from the first set of available options (as depicted). In some embodiments, the first set of available options includes a different number of options (fewer or more) than the second set of available options. Selecting a first subset of options for an AR environment from the first set of available options and selecting a second subset of options for a VR environment from the second set of available options, wherein the first set of available options includes a different number of options than the second set of available options, enables the computer system to provide a more realistic user experience while saving storage space by not requiring multiple different complete audio tracks and / or visual components to be stored and selected for playback / display. In some embodiments, the options available for a VR environment are more limited than the options available for an AR environment. In some embodiments, the options available for a VR environment can be used to create a smaller set of recipes for creating a VR environment, or include a smaller range of variables than would otherwise be available for creating an AR environment.

[0227] In some embodiments, displaying a user interface for a user experience session as having a first set of one or more options enabled for the user experience session (e.g., a first set of visual and / or audio characteristics for the session) includes: a computer system (e.g., 702) displays a first environment (e.g., 765) (e.g., a first virtual environment) for the user experience session. In some embodiments, displaying a user interface for a user experience session as having a second set of one or more options enabled for the user experience session (e.g., a second set of visual and / or audio characteristics for the session) includes: displaying a second environment (e.g., 787) (e.g., a second virtual environment) for the user experience session that is different from the first environment. Displaying the first environment or the second environment for the user experience session when one or more characteristics of the XR environment meet the first set of criteria or the second set of criteria causes the device to automatically display the first environment or the second environment based on the characteristics of the XR environment without displaying additional controls.

[0228] In some embodiments, the virtual environment is a curtain or wallpaper of a three-dimensional environment. In some embodiments, the virtual environment can be displayed as outside the user experience session (e.g., before and / or after). In some embodiments, the virtual environment provides a virtual three-dimensional space, wherein the user uses a computer system such as using an application, playing a game, communicating with other users, experiencing co-presence with other users and / or interacting with elements of the operating system of the computer system to perform activities. In some embodiments, the computer system provides corresponding spatial audio soundscapes for various virtual environments. In some embodiments, the spatial audio soundscape is unique for each corresponding virtual environment. In some embodiments, for example, in order to convey the specific emotions and / or themes of the corresponding virtual environment, the spatial audio soundscape is planned for the corresponding virtual environment.

[0229] In some embodiments, the first set of one or more options includes a first set of variables (e.g., 770, 772, 774, and / or 780-1) for a first environment based on one or more characteristics of the first environment (e.g., 765) (e.g., visual and / or audio characteristics for the user experience session) (e.g., variables customized for the first environment or optimized for the first environment). In some embodiments, the second set of one or more options includes a second set of variables (e.g., 790, 792, 794, and / or 785-1) for a second environment (e.g., 787) based on one or more characteristics of the second environment (e.g., visual and / or audio characteristics for the user experience session) (e.g., different from the first set of variables) (e.g., variables customized for the second environment or optimized for the second environment).

[0230] In some embodiments, the first set of criteria includes a first criterion that is satisfied when a previous user interface (e.g., 765) for a previous user experience session (e.g., a previous instance of the user experience session) has been displayed with a previous set of visual characteristics (e.g., 770 and / or 772) for the previous user experience session (e.g., visual characteristics of displayed components of the previous user experience session, colors, shapes, sizes, graphics, and / or animation effects). In some embodiments, displaying the user interface for the user experience session with a first set of one or more options enabled for the user experience session (e.g., the current instance of the user experience session) includes: the computer system (e.g., 702) displays the user interface (e.g., 787) with a first set of visual characteristics for the user experience session (e.g., 790 and / or 792) that is different from the previous set of visual characteristics for the previous user experience session (e.g., visual characteristics of displayed components of the user experience session, colors, shapes, sizes, graphics, and / or animation effects). Displaying the user interface with a first set of visual characteristics for the user experience session that is different from the previous set of visual characteristics for the previous user experience session enables the computer system to generate a unique or fresh user interface for the user experience session based on the history of the user experience session without displaying additional controls. In some embodiments, subsequent instances of the user experience session (e.g., repeated user experience sessions) have different visual characteristics that are selected to be unique or different from previous instances of the user experience session. In some embodiments, the third set of criteria includes the first criterion. In some embodiments, the second set of criteria includes a second criterion that is satisfied when a second previous user interface for a second previous user experience session has been displayed with a second set of previous visual characteristics for the second previous user experience session. In some embodiments, displaying the user interface for the user experience session as having a second set of one or more options enabled for the user experience session includes: displaying the user interface as having a second set of visual characteristics for the user experience session that is different from the second set of previous visual characteristics for the second previous user experience session (and optionally, different from the set of previous visual characteristics for the previous user experience session).

[0231] In some embodiments, a first set of visual characteristics for a user experience session is randomly or pseudo-randomly selected by a computer system (e.g., 702) from a set of available visual characteristics (e.g., a superset of visual characteristics). Randomly or pseudo-randomly selecting a first set of visual characteristics for a user experience session from the set of available visual characteristics enables the computer system to provide a more realistic user experience while saving storage space by not requiring multiple different complete visual components to be stored and selected for display. In some embodiments, the set of visual characteristics for the user experience session is selected with a preference for not repeating one or more visual characteristics that have been used in a previous user experience session. For example, the set of available visual characteristics includes the set of previous visual characteristics, and the first set of visual characteristics is selected from a subset of the available visual characteristics that does not include the set of previous visual characteristics.

[0232] In some embodiments, the set of prior visual characteristics includes a first light property (e.g., direction, color, and / or intensity of light) for a simulated lighting effect (e.g., simulated light reflection) for a user interface object (e.g., 710, 712, 770, 772, 790, and / or 792) displayed in a prior user experience session (e.g., one or more particles forming a cube, sphere, orb, cloud, pyramid, and / or abstract object). In some embodiments, the first set of visual characteristics includes a second light property that is different from the first light property for the simulated lighting effect for the user interface object displayed in the user experience session (e.g., the user interface for the user experience session includes a user interface object that includes a plurality of particles having a set of light properties for simulated light reflected from the particles). Displaying the user interface as having a first set of visual properties that includes a second light property that is different from the first light property for the simulated lighting effect for the user interface object displayed in the user experience session enables the computer system to provide a more realistic user experience while saving storage space by not requiring multiple different complete visual components to be stored and selected for display.

[0233] In some embodiments, the set of prior visual properties includes a first material property (e.g., a simulated optical property, shape, and / or curvature) for a user interface object (e.g., 710, 712, 770, 772, 790, and / or 792) displayed in a prior user experience session (e.g., one or more particles forming a cube, sphere, orb, cloud, pyramid, and / or abstract object). In some embodiments, the first set of visual properties includes a second material property that is different from the first material property for a user interface object displayed in the user experience session (e.g., a user interface for the user experience session includes a user interface object that includes a plurality of particles having a set of material properties). Displaying the user interface as having a first set of visual properties that includes a second material property that is different from the first material property for a user interface object displayed in the user experience session enables the computer system to provide a more realistic user experience while saving storage space by not requiring multiple different complete visual components to be stored and selected for display.

[0234] In some embodiments, displaying a user interface for a user experience session as having a first set of one or more options enabled for the user experience session (e.g., a first set of visual and / or audio characteristics for the session) includes: a computer system (e.g., 702) displaying a VR environment (e.g., 765 and / or 787) for the user experience session as having a first emphasis amount (e.g., a low dimming amount, no dimming; no dimming effect; no fading of the VR environment). In some embodiments, displaying the user interface for the user experience session as having a second set of one or more options enabled for the user experience session (e.g., a second set of visual and / or audio features for the session) includes: displaying the AR environment (e.g., 705) for the user experience session as having a second amount of emphasis (e.g., 704) that is less than the first amount of emphasis (e.g., the AR environment is displayed as having a greater fade effect or dimming effect (e.g., the AR environment is dimmed 99.9%, 99%, 98%, 97%, 95%, 90%, 85%, 80%, 50%, 20%, 15%, 10%, 7%, 5%, 3%, 2%, 1%, 0.5%, and / or another amount greater than 0%). Displaying the AR environment for the user experience session as having a second amount of emphasis that is less than the first amount of emphasis for the VR environment reduces interference with the user experience session.

[0235] In some embodiments, the AR environment is displayed with a greater amount of dimming than the VR environment. In some embodiments, displaying the dimming effect includes: visually blurring the view (e.g., 700a) of the physical environment visible to the user (e.g., 701) of the computer system (e.g., 702) (e.g., via pass-through video or due to the transparent nature of the display). In some embodiments, displaying the dimming effect (e.g., 704) includes: displaying a partially transparent virtual overlay to show the physical environment through the virtual overlay. In some embodiments, the AR environment is dimmed to reduce interference in a different manner than in the VR environment. For example, a computer system can harmonize a user experience session in a VR environment by displaying a particular virtual environment, thereby reducing potential interference to the user, while interference in the AR environment is reduced by the computer system dimming the visibility of objects in the physical environment.

[0236] In some embodiments, displaying a user interface for a user experience session includes: displaying multiple user interface objects (e.g., 710, 712, 770, 772, 790, and / or 792) (e.g., one or more particles forming a cube, sphere, orb, cloud, pyramid, and / or abstract object) as having a translucent appearance based on an environment (e.g., 700a, 705, 765, and / or 787) for the user experience session (e.g., physical objects, virtual objects, detected lighting, and / or virtual lighting in an XR environment) (e.g., the translucent appearance of the particles is affected by elements of the environment for the user experience session). Displaying multiple user interface objects as having a translucent appearance based on the environment for the user experience session enables the computer system to provide a more realistic user experience based on the environment for the user experience session while saving storage space by not requiring multiple different complete visual components to be stored and selected for display.

[0237] In some embodiments, displaying multiple user interface objects (e.g., 710, 712, 770, 772, 790, and / or 792) as having a translucent appearance based on an environment for the user experience session includes: based on a determination that the environment for the user experience session includes a first virtual lighting effect (e.g., a simulated positioning, direction, color, and / or intensity of light in the XR environment), the computer system (e.g., 702) displays the multiple user interface objects as having a first translucent appearance (e.g., translucency of particles having a first positioning and / or a first translucency amount on corresponding particles) (e.g., Figure 7J and / or Figure 7KIn some embodiments, based on a determination that an environment for the user experience session includes a second virtual lighting effect that is different from the first virtual lighting effect (e.g., a simulated position, direction, color, and / or intensity of light in the XR environment), multiple user interface objects are displayed as having a second translucent appearance that is different from the first translucent appearance (e.g., the translucency of particles has a second position and / or a second amount of translucency on the respective particles) (e.g., in Figure 7L Displaying the plurality of user interface objects as having a second translucent appearance in accordance with a determination that the environment includes a second virtual lighting effect enables the computer system to provide a more realistic user experience based on the virtual lighting effects in the environment for the user experience session while saving storage space by not requiring a plurality of different complete visual components to be stored and selected for display. In some embodiments, the visual appearance of the user interface objects having a translucent appearance is affected by the virtual lighting of the AR / VR environment.

[0238] In some embodiments, initiating a user experience session in an XR environment includes: a computer system (e.g., 702) increasing one or more immersive aspects of the user experience session (e.g., increasing the proportion of the user's field of view occupied by the environment, and / or increasing the spatial immersion of the environment's audio) (e.g., Fig.7D and / or Figure 7K Increasing one or more immersive aspects of a user experience session provides feedback about the state of the computer system and eliminates distractions from the user experience session.

[0239] In some embodiments, increasing one or more immersive aspects of the user experience session includes: the computer system (e.g., 702) increasing the proportion of the user's field of view (e.g., the field of view of the display generation component (e.g., 702-1)) occupied by the user interface for the user experience session (e.g., increasing Figure 7K 765, virtual objects 770, and / or particles 772 in the image) (e.g., increasing the displayed size of the virtual wallpaper 765, virtual objects 770, and / or particles 772 in the image) Fig.7D Increasing the proportion of the user's field of view occupied by the user interface for the user experience session provides feedback about the state of the computer system and eliminates interference with the user experience session.

[0240] In some embodiments, increasing one or more immersive aspects of a user experience session includes: a computer system (e.g., 702) increasing the spatial immersion of audio generated for the user experience session (e.g., causing the perceived spatial position of the audio to move from a first position perceived as a distance away from a user of the computer system to a second position perceived (by the user) as being near and / or around the user) (e.g., Fig.7D and / or Figure 7K Increasing the spatial immersion of the audio generated for the user experience session provides feedback about the state of the computer system and eliminates distractions from the user experience session.

[0241] In some embodiments, aspects / operations of methods 800 and 1000 may be interchanged, replaced, and / or added between these methods. For the sake of brevity, these details are not repeated here.

[0242] FIG. 10A to FIG. 10B is a flow chart of an exemplary method 900 for providing a soundscape having randomly selected curated sound components to a computer-generated user experience session according to some embodiments. In some embodiments, the method 1000 is performed on a computer system (e.g., Figure 1 1 and / or device 702) (e.g., a smart phone, a tablet device, and / or a head-mounted display generating component), which is executed at the computer system 101 and / or device 702 in the display generating component (e.g., Figure 1 , Figure 3 and Figure 4 The display generating components 120 and / or display 702-1 in the HMD (e.g., a display, a touch screen, a visual output device, a 3D display, a display having at least a portion that is transparent or semi-transparent onto which an image can be projected (e.g., a see-through display), a projector, a head-up display, and / or a display controller), an audio generating component (e.g., an audio output device 703 and / or a speaker at device 702, or an audio generating component integrated into the HMD) (e.g., a speaker and / or a bone conduction audio output device), and one or more sensors (e.g., a camera 702-2) (e.g., a gyroscope, an accelerometer, a motion sensor, a movement sensor, a microphone, an infrared sensor, a camera sensor, a depth camera, a visible light camera, an eye tracking sensor, a gaze tracking sensor, a physiological sensor, an image sensor, a camera at a hand pointing downward to the user (e.g., a color sensor, an infrared sensor, and other depth sensing cameras) and / or a camera pointing forward from the user's head) communicate. In some embodiments, method 1000 is performed by storing in a non-transitory (or transient) computer-readable storage medium and executed by one or more processors of a computer system (such as one or more processors 202 of computer system 101) (e.g., Figure 1 Some operations in method 1000 are optionally combined, and / or the order of some operations is optionally changed.

[0243] In method 1000, a computer system (702) at a first time (e.g., Figure 7B Depicted or Figure 7JAs depicted, the device detects (1002) via one or more sensors (e.g., 702-1 and / or 702-2) a request (e.g., input 724, input 778, or the like) to initiate a corresponding type of user experience session in the XR environment (e.g., a request to begin a user experience in the XR environment that optionally includes guided instructions for breathing exercises to relax and / or focus the user and / or reaction exercises to help the user react to scenarios, topics, ideas, concepts, etc.).

[0244] In response to detecting the request to initiate a user experience session in the XR environment, the computer system (e.g., 702) initiates (1004) a first user experience session of a corresponding type (e.g., Fig.7D Depicted and Figure 7K In some embodiments, starting a user experience in an XR environment includes launching an application for providing the user experience.

[0245] As part of initiating a first user experience session of a corresponding type in an XR environment, a computer system (e.g., 702) displays (1006) via a display generation component (e.g., 702-1) a user interface (e.g., 705, 710, 712, 765, 770, and / or 772) (e.g., a UI of an application for breathing exercises and / or reaction exercises) for the first user experience session (e.g., in the XR environment). The computer system also outputs (1008) via an audio generation component (e.g., 703) a first audio soundscape (e.g., 711 or 774) (e.g., a set of curated sound components selected to create an audio environment for the user experience session) for the first user experience session (e.g., when displaying the user interface for the first user experience session). The computer system outputs the first audio soundscape concurrently with displaying the user interface for the first user experience session. Outputting the first audio soundscape includes: outputting the first audio soundscape having a first set of two or more audio components randomly or pseudo-randomly selected from a set of available audio components. In some embodiments, two or more of the first set of audio components are a subset of curated sound components selected from a superset of curated sound components for the audio soundscape.

[0246] In method 1000, a computer system (e.g., 702) performs a process of executing a command at a second time (e.g., Figure 7B Depicted or Figure 7J) detects (1010) via one or more sensors (e.g., 702-1 and / or 702-2) a request (e.g., input 724, input 778, or similar input) to initiate a corresponding type of user experience session in the XR environment (e.g., a request to start a user experience in the XR environment that optionally includes guided instructions for breathing exercises to relax and / or focus the user and / or reaction exercises to help the user react to scenarios, topics, ideas, concepts, etc.).

[0247] In response to detecting the request to initiate a user experience session in the XR environment, the computer system (e.g., 702) initiates (1012) a second user experience session of a corresponding type (e.g., Figure 7L In some embodiments, starting a user experience in an XR environment includes launching an application for providing the user experience.

[0248] As part of initiating a second user experience session of a corresponding type in an XR environment, a computer system (e.g., 702) displays (1014) via a display generation component (e.g., 702-1) a user interface (e.g., 787, 790, and / or 792) (e.g., a UI of an application for breathing exercises and / or reaction exercises) for the second user experience session (e.g., in the XR environment). The computer system also outputs (1016) via an audio generation component (e.g., 703) a second audio soundscape (e.g., 794) (e.g., a set of curated sound components selected to create an audio environment for the user experience session) for the second user experience session (e.g., while displaying the user interface for the second user experience session). The computer system outputs the second audio soundscape concurrently with displaying the user interface for the second user experience session. Outputting the second audio soundscape includes: outputting a second audio soundscape having a second set of two or more audio components randomly or pseudo-randomly selected from the set of available audio components (e.g., different from the first set of two or more audio components). Outputting a second audio soundscape for a second user experience session, wherein the second audio soundscape is output concurrently with displaying a user interface for the second user experience session, and outputting the second audio soundscape includes outputting a second audio soundscape having a second set of two or more audio components randomly or pseudo-randomly selected from the set of available audio components, enabling the computer system to provide a more realistic user experience while saving storage space by not requiring multiple different complete audio tracks to be stored and selected for playback. In some embodiments, two or more of the second set of audio components are a subset of curated sound components selected from a superset of curated sound components for the audio soundscape. In some embodiments, at least one of the two or more of the second set of audio components is selected to be different from an audio component in the first set of audio components.

[0249] In some embodiments, outputting a first audio soundscape (e.g., 711 or 774) for a first user experience session includes: the computer system (e.g., 702) repeating (e.g., continuously repeating on a loop) a first set of two or more audio components during the first user experience session. In some embodiments, outputting a second audio soundscape for a second user experience session includes: repeating (e.g., continuously repeating on a loop) a second set of two or more audio components during the second user experience session. Repeating the first set of two or more audio components during the first user experience session provides feedback about the state of the computer system (e.g., outputting the state of the first user experience session).

[0250] In some embodiments, a first set of two or more audio components of a first audio soundscape (e.g., 711 or 774) is different from a second set of two or more audio components of a second audio soundscape (e.g., 794). Outputting the first soundscape and / or the second soundscape, wherein the first set of two or more audio components is different from the second set of two or more audio components, enables the computer system to provide a more realistic user experience while saving storage space by not requiring multiple different complete audio tracks to be stored and selected for playback.

[0251] In some embodiments, outputting a first audio soundscape (e.g., 711 or 774) for the first user experience session concurrently with displaying a user interface (e.g., 705, 710, 712, 765, 770, and / or 772) for the first user experience session includes the following: When the user interface for the first user experience session includes a first predetermined animation effect (e.g., an animation effect of a visual component displayed for a particular phase or portion of the first user experience session) (e.g., a pulsating animation (e.g., a repeating alternating pattern of increasing the size of the visual component and decreasing the size of the visual component) of a visual component (e.g., a sphere, a ball, a cube, a vortex, and / or a cloud) for a breathing portion of the guidance of the first user experience), the computer system (e.g., 702) outputs a first group of two or more audio components having a first set of audio characteristics (e.g., the first group of two or more audio components having a set of audio characteristics determined based on the predetermined animation effect) (e.g., increasing the volume of two or more audio components in the first group when the size of the visual component increases, and decreasing the volume of two or more audio components in the first group when the size of the visual component decreases). When the user interface for the first user experience session includes a second predetermined animation effect different from the first predetermined animation effect (e.g., a floating / swaying animated appearance of visual components (e.g., particles, triangles, circles, squares, and / or small cubes) for a reactive portion of the first user experience), the computer system outputs a first set of two or more audio components having a second set of audio characteristics different from the first set of audio characteristics (e.g., the audio is output with spatial position changes based on the floating / swaying movement of the visual components). Outputting the first set of two or more audio components having the first set of audio characteristics when the user interface for the first user experience session includes the first predetermined animation effect and outputting the first set of two or more audio components having the second set of audio characteristics different from the first set of audio characteristics when the first user experience session includes the second predetermined animation effect different from the first predetermined animation effect causes the computer system to automatically modify the first audio soundscape based on the animation effect output in the user interface for the first user experience session.

[0252] In some embodiments, outputting a second audio soundscape (e.g., 794) for the second user experience session concurrently with displaying a user interface (e.g., 787, 790, and / or 792) for the second user experience session includes the following. When the user interface for the second user experience session includes a third predetermined animation effect (in some embodiments, the third predetermined animation effect is the same as the first predetermined animation effect), the computer system (e.g., 702) outputs a second set of two or more audio components having a third set of audio characteristics (in some embodiments, the third set of audio characteristics is the same as the first set of audio characteristics). When the user interface for the second user experience session includes a fourth predetermined animation effect different from the third predetermined animation effect (in some embodiments, the fourth predetermined animation effect is the same as the second predetermined animation effect), the computer system outputs a second set of two or more audio components having a fourth set of audio characteristics different from the third set of audio characteristics (in some embodiments, the fourth set of audio characteristics is the same as the second set of audio characteristics).

[0253] In some embodiments, both the visual effect (e.g., a predetermined animation effect) and the audio component (e.g., 711, 774, and / or 794) (e.g., a corresponding audio soundscape) are selected based on some criteria (e.g., the same criteria). For example, when a particular soundscape is selected (e.g., by a user and / or by a computer system), the corresponding visual effect is automatically selected (e.g., by the computer system) to accompany the selected soundscape, or when a particular visual effect is selected, the corresponding soundscape is automatically selected (e.g., by the computer system) to accompany the selected visual effect. As another example, a particular virtual environment, theme, and / or mood is selected, and a particular soundscape and / or visual effect is automatically selected (e.g., by the computer system) for the selected environment, theme, and / or mood.

[0254] In some embodiments, while outputting a first audio soundscape (e.g., 711 or 774) for a first user experience session, a computer system (e.g., 702) detects (e.g., determines) (e.g., using a sensor) a biometric input (e.g., inward breathing, outward breathing, heartbeat, and / or body movement). In some embodiments, in response to detecting the biometric input, the computer system modifies the first audio soundscape, including: based on a determination that the biometric input includes the first biometric input (e.g., inward breathing), the computer system modifies the first set of two or more audio components in a first manner (e.g., outputting the first soundscape as having a perceived spatial location of the audio components that is moving toward the user) (e.g., as Figure 7GBased on a determination that the biometric input includes a second biometric input different from the first biometric input (e.g., breathing out), the computer system modifies the first set of two or more audio components in a second manner different from the first manner (e.g., outputting the first soundscape as having perceived spatial locations of the audio components moving away from the user) (e.g., as shown in FIG. 1 ). Figure 7F 9 in FIG. 1 ). Modifying a first set of two or more audio components in a first manner when the biometric input includes a first biometric input and modifying the first set of two or more audio components in a second manner different from the first manner when the biometric input includes a second biometric input different from the first biometric input causes the computer system to automatically modify the first audio soundscape based on the biometric input detected during the first user experience session.

[0255] In some embodiments, when outputting a second audio soundscape (e.g., 774 or 794) for a second user experience session, the computer system (e.g., 702) detects a biometric input. In response to detecting the biometric input, the computer system modifies the second audio soundscape, including: modifying the second set of two or more audio components in a third manner (e.g., in a first manner) based on a determination that the biometric input includes a third biometric input (e.g., the first biometric input) (e.g., breathing inward) (e.g., outputting the second soundscape as a perceived spatial location with audio components moving toward the user); and modifying the second set of two or more audio components in a fourth manner (e.g., in a second manner) different from the third manner (e.g., outputting the second soundscape as a perceived spatial location with audio components moving away from the user) based on a determination that the biometric input includes a fourth biometric input (e.g., the second biometric input) that is different from the third biometric input (e.g., breathing outward). In some embodiments, the characteristics of the audio (e.g., the spatial location of the audio and / or the spatial movement of the audio) are based on a user biometric me...

Claims

1. A method, comprising: At a computer system in communication with a display generating component and one or more sensors: Displaying a user interface for a user experience session via the display generation component includes: While the user is experiencing a session activity: detecting, via the one or more sensors, one or more breathing characteristics of a user of the computer system; and displaying a user interface object having a plurality of particles that move based on the one or more breathing characteristics of a user of the computer system, include: Based on determining that a first respiratory event of the user of the computer system satisfies a first set of criteria, displaying the particles of the user interface object as moving in a first manner during the first respiratory event of the user of the computer system; and Based on determining that the first breathing event of the user of the computer system satisfies a second set of criteria, displaying the particles of the user interface object as moving in a second manner different from the first manner during the first breathing event of the user of the computer system, wherein displaying the particles of the user interface object as moving in the first manner or the second manner comprises: displaying a first set of one or more particles having a first distance from a location corresponding to the viewpoint of a user of the computer system and having a first amount of movement during the first respiratory event; and A second group of one or more particles is displayed, the second group of one or more particles being different from the first group of one or more particles and having a second distance from the location corresponding to the viewpoint of a user of the computer system that is different from the first distance, and the second group of one or more particles having a second amount of movement during the first respiratory event that is different from the first amount of movement.

2. The method of claim 1, wherein the computer system is in communication with an audio generation component, and wherein displaying the user interface for the user experience session comprises: Prior to the user experience session activity, concurrently: displaying, via the display generation component, a representation of the plurality of particles of the user interface object; as well as An audio soundscape for the user experience session is output via the audio generation component.

3. The method of claim 2, wherein displaying the user interface for the user experience session further comprises: A dimmed appearance of the user experience's environment is displayed prior to the user experience session activity.

4. The method of any one of claims 1 to 3, wherein displaying the user interface for the user experience session comprises: Prior to the User Experience Session activity: displaying a start option selectable to initiate the user experience session; as well as An indication of a duration of the user experience session is displayed.

5. The method of claim 4, wherein displaying the user interface for the user experience session comprises: Prior to the User Experience Session activity: displaying a set of one or more duration options selectable to modify the duration of the user experience session; detecting input directed to a first duration option of the set of one or more duration options selectable to modify the duration of the user experience session; as well as In response to detecting the input directed to the first duration option of the set of one or more duration options, a duration of the user experience session is selected as a first duration.

6. The method of any one of claims 1 to 5, wherein displaying the user interface for the user experience session comprises: Prior to the User Experience Session activity: displaying a set of one or more audio options selectable to select audio guidance from a plurality of audio guidance for the user experience session; detecting input directed to a first audio option of the set of one or more audio options selectable to select audio guidance from the plurality of audio guidance for the user experience session; as well as In response to detecting the input directed to the first audio option, a first audio guide is selected from the plurality of audio guides for the user experience session.

7. The method of any one of claims 1 to 6, wherein displaying the user interface for the user experience session comprises: While the user is experiencing a session activity: Displaying the user interface object with an animation effect based on a predetermined biometric rhythm comprises: Based on determining that the predetermined biometric rhythm is a first biometric rhythm, animating the user interface object based on a first pattern corresponding to the first biometric rhythm; and Based on determining that the predetermined biometric rhythm is a second biometric rhythm different from the first biometric rhythm, the user interface object is displayed in an animated manner based on a second pattern corresponding to the second biometric rhythm.

8. The method of claim 7, wherein the predetermined biometric rhythm is a biometric rhythm selected by a user of the computer system.

9. The method according to any one of claims 1 to 8, wherein the computer system is in communication with an audio generating component, the method further comprising: While the user is experiencing a session activity: Outputting an audio component having a perceived spatial location that moves based on the one or more breathing characteristics of a user of the computer system comprises: outputting the audio component having a first perceptual spatial location relative to the user of the computer system based on determining that a second respiratory event of the user of the computer system satisfies a third set of criteria; and Based on determining that the second respiratory event of the user of the computer system satisfies a fourth set of criteria, outputting the audio component having a second perceptual spatial location relative to the user of the computer system that is different from the first perceptual spatial location.

10. The method of any one of claims 1 to 9, wherein the portion of the physical environment of the user of the computer system is visible prior to the user experience session activity, the method further comprising: Initiating the user experience session includes: A dimming effect that gradually reduces visibility of the physical environment is displayed via the display generation component.

11. The method according to any one of claims 1 to 10, further comprising: While the user is experiencing a session activity: detecting, via the one or more sensors, one or more concentration-based characteristics of the user of the computer system; as well as Feedback based on the one or more concentration-based characteristics of a user of the computer system is output.

12. The method of claim 11, wherein outputting feedback based on the one or more concentration-based characteristics of a user of the computer system comprises: Visual feedback based on the one or more concentration-based characteristics of a user of the computer system is displayed via the display generating component.

13. The method of claim 12, wherein displaying visual feedback based on the one or more focus-based characteristics of a user of the computer system comprises: Based on determining that the one or more concentration-based characteristics of the user of the computer system satisfy a first set of concentration criteria, the particles of the user interface object are displayed as moving away from a reference position during outward breathing of the user of the computer system.

14. The method of claim 13, wherein displaying visual feedback based on the one or more focus-based characteristics of a user of the computer system comprises: Based on determining that the one or more concentration-based characteristics of the user of the computer system satisfy the first set of concentration criteria, the particles of the user interface object are displayed as moving toward the reference location during inward breathing of the user of the computer system.

15. The method according to claim 14, wherein: the particles of the user interface object being displayed as moving toward the reference position at a first rate during the inward breathing; The particles of the user interface object are displayed as moving away from the reference position at a second rate during the outward breathing; and The second rate is different from the first rate.

16. The method of claim 13, wherein the reference location is a world-locked location corresponding to a viewpoint of the user of the computer system in a physical environment.

17. The method of any one of claims 11 to 16, wherein the computer system is in communication with an audio generating component, and wherein outputting feedback based on the one or more concentration-based characteristics of a user of the computer system comprises: Audio feedback based on the one or more concentration-based characteristics of a user of the computer system is output via the audio generation component.

18. The method of any one of claims 11 to 17, wherein the one or more concentration-based characteristics of the user of the computer system include a plurality of biometric indicators.

19. The method of any one of claims 11 to 18, wherein the one or more concentration-based characteristics of the user of the computer system include an indication of whether the user's concentration has failed to meet a concentration criterion for a threshold amount of time.

20. The method of any one of claims 1 to 19, wherein the movement of the plurality of particles is based on a set of one or more simulated physical parameters.

21. The method of any one of claims 1 to 20, wherein displaying the user interface for the user experience session comprises: detecting, via the one or more sensors, gaze data indicative of a gaze of the user of the computer system while the user experience session is active; detecting updated gaze data while displaying the user interface object having the plurality of particles that move based on the one or more breathing characteristics of a user of the computer system; and In response to detecting the updated gaze data: pausing the user experience session based on determining that the updated gaze data indicates that the gaze of the user exceeds a gaze deviation threshold; as well as Based on determining that the updated gaze data does not indicate that the gaze of the user exceeds the gaze deviation threshold, pausing the user experience session is abandoned.

22. The method according to any one of claims 1 to 21, wherein: Displaying the user interface for the user experience session includes: displaying the user interface with a set of visual characteristics randomly or pseudo-randomly selected from a set of available visual characteristics; and The computer system is in communication with an audio generating component, the method further comprising: Outputting, via the audio generation component, an audio soundscape for the user experience session, wherein the audio soundscape is output concurrently with displaying the user interface for the user experience session, and outputting the audio soundscape comprises: outputting the audio soundscape having a first set of two or more audio components randomly or pseudo-randomly selected from a set of available audio components.

23. The method of any one of claims 1 to 22, wherein displaying the user interface for the user experience session comprises: detecting a transition from the first portion of the user experience session to a second portion of the user experience session while displaying the user interface object having a first display state in which the plurality of particles are displayed with a first spacing amount for the first portion of the user experience session; as well as In response to detecting the transition from the first portion of the user experience session to the second portion of the user experience session, displaying the user interface object having a second display state different from the first display state, wherein the plurality of particles are displayed for the second portion of the user experience session with a second spacing amount different from the first spacing amount.

24. The method of claim 23, wherein displaying the user interface for the user experience session comprises: while displaying the user interface object having the second display state, detecting a transition from the second portion of the user experience session to a third portion of the user experience session; as well as In response to detecting the transition from the second portion of the user experience session to the third portion of the user experience session, displaying the user interface object having a third display state different from the first display state and the second display state, wherein the plurality of particles are displayed for the third portion of the user experience session with a third spacing amount different from the second spacing amount.

25. The method of any one of claims 1 to 24, wherein displaying the user interface for the user experience session comprises: detecting termination of the user experience session while displaying the user interface object having the plurality of particles including an arrangement having a first average spacing between particles; as well as In response to detecting termination of the user experience session, the plurality of particles are animated to move to an arrangement having a second average spacing between particles, wherein the second average spacing is less than the first average spacing.

26. The method of any one of claims 1 to 25, wherein displaying the user interface for the user experience session comprises: detecting termination of the user experience session while the user experience session is active and an environment of the user of the computer system is visually obscured; as well as In response to detecting termination of the user experience session, termination of the user experience session is initiated and visibility of the environment to the user is gradually increased.

27. The method of any one of claims 1 to 26, wherein displaying the user interface for the user experience session comprises: While the user experience session is active, detecting termination of the user experience session; as well as In response to detecting termination of the user experience session, termination of the user experience session is initiated and an option selectable to continue the user experience session is displayed.

28. The method of any one of claims 1 to 27, wherein displaying the user interface for the user experience session comprises: While the user experience session is active, detecting termination of the user experience session; as well as In response to detecting termination of the user experience session, termination of the user experience session is initiated and a data history associated with one or more previous user experience sessions is displayed.

29. The method of any one of claims 1 to 28, wherein displaying the user interface for the user experience session comprises: At least a portion of an environment of the user of the computer system is caused to be visible while the user experiences an active session.

30. The method according to any one of claims 1 to 29, further comprising: receiving, while the user experience session is active and while the user interface object is displayed having a first display orientation relative to the user of the computer system, data indicating a change in position of the user of the computer system from a first position in an environment to a second position in the environment that is different than the first position; as well as In response to receiving the data indicative of a change in the positioning of the user of the computer system, the user interface object is displayed having a second display orientation relative to the user of the computer system that is different from the first display orientation.

31. The method of any one of claims 1 to 30, wherein the user interface for the user experience session is displayable at one or more external computer systems.

32. The method according to any one of claims 1 to 31, further comprising: receiving data indicating a change in pose of a portion of the user of the computer system while the user experience session is active and while displaying the user interface object having the plurality of particles that move based on the one or more breathing characteristics of the user of the computer system; as well as In response to receiving the data indicating a change in the pose of the portion of the user of the computer system, updating the display of the plurality of particles comprises: modifying a display characteristic of the corresponding particle in response to determining that the data indicating a change in the pose of the portion of the user includes an indication that the portion of the user intersects a display location of a corresponding particle; and Based on determining that the data indicative of the change in pose of the portion of the user does not include an indication that the portion of the user intersects the display location of the corresponding particle, modifying the display characteristic of the corresponding particle is abandoned.

33. A computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a display generating component and one or more sensors, the one or more programs comprising instructions for executing the method according to any one of claims 1 to 32.

34. A computer system configured to communicate with a display generating component and one or more sensors, the computer system comprising: one or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs comprising instructions for executing the method according to any one of claims 1 to 32.

35. A computer system configured to communicate with a display generating component and one or more sensors, the computer system comprising: A component for carrying out the method according to any one of claims 1 to 32.

36. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a display generating component and one or more sensors, the one or more programs comprising instructions for performing the method according to any one of claims 1 to 32.

Images