System and method for phase transition in creative workflow

Abstract

The present disclosure relates to systems and methods for stage transitions in creative workflows. Some examples of the present disclosure relate to systems and methods for displaying a boundary associated with a virtual scene. In some examples, a device displays the virtual scene and the boundary simultaneously. In some examples, when a viewpoint of a user of the device satisfies one or more criteria associated with the boundary, the device presents a representation of the user's physical environment. In some examples, the device displays a visual indication of an individual physically co-located with the user of the device.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 586,949, filed on September 29, 2023, and U.S. Patent Application No. 18 / 892,109, filed on September 20, 2024, the contents of both applications are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] The present application generally relates to systems and methods for displaying a virtual environment and adding virtual content to the virtual environment. Background Art

[0004] Some computer graphics environments provide two-dimensional environments and / or three-dimensional environments in which at least some of the objects presented for viewing by a user are virtual and generated by a computer. In some examples, the virtual three-dimensional environment may be based on one or more images of the computer's physical environment. In some examples, the virtual three-dimensional environment does not include an image of the computer's physical environment. Summary of the invention

[0005] Some examples of the present disclosure relate to systems and methods for presenting a representation of a physical environment of a user of an electronic device and / or presenting a virtual three-dimensional environment scene to the user. In some examples, the electronic device may present and / or display the three-dimensional environment to the user of the three-dimensional environment. In some examples, the three-dimensional environment may include representations of physical objects and / or individuals. In some examples, the three-dimensional environment may include one or more virtual objects and / or virtual assets.

[0006] In some examples, the electronic device may display one or more representations of a physical individual. In some examples, the one or more representations include a representation of a user of another electronic device participating in a communication session with the electronic device. In some examples, the electronic device displays a virtual scene that can be shared with the other electronic device. In some examples, the virtual scene is displayed as if the user of the electronic device is present within a physical equivalent of the virtual scene. In some examples, the representation of the physical individual moves relative to the virtual scene.

[0007] In some examples, the electronic device displays an indication of the user's attention within the three-dimensional environment. In some examples, the electronic device displays an indication of attention corresponding to the attention of other users examining the virtual scene. In some examples, the electronic device displays a visual indication of a boundary within the three-dimensional environment. In some examples, the electronic device keeps displaying the virtual scene based on determining that the viewpoint of the user of the electronic device corresponds to an area of ​​the three-dimensional environment within the boundary. In some examples, the electronic device displays a representation of the user's physical environment based on determining that the viewpoint of the user does not correspond to the area of ​​the three-dimensional environment within the boundary. In some examples, the electronic device changes one or more dimensions of the boundary based on user input. In some examples, the electronic device displays a visual indication of the physical individual while keeping the virtual scene displayed. In some examples, the electronic device displays representations of those physical individuals. In some examples, the electronic device displays a representation of the user's physical environment based on determining that the viewpoint of the user meets one or more criteria.

[0008] A comprehensive description of these examples is provided in the drawings and detailed description, and it should be understood that this summary does not limit the scope of the present disclosure in any way. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] For a better understanding of the various examples described herein, reference should be made to the following detailed description and the following drawings. Throughout the drawings, like reference numerals generally refer to corresponding parts.

[0010] Figure 1 An electronic device presenting an extended reality environment according to some examples of the present disclosure is illustrated.

[0011] Figure 2 A block diagram illustrating an exemplary architecture of a device according to some examples of the present disclosure.

[0012] FIG. 3A to FIG. 3J Example interactions including virtual scenes and virtual boundaries according to some examples of the present disclosure are illustrated.

[0013] Figure 4 A flowchart illustrating an example process for displaying a virtual scene and a virtual boundary according to some examples of the present disclosure is illustrated. DETAILED DESCRIPTION

[0014] Some examples of the present disclosure relate to systems and methods for presenting a representation of a physical environment of a user of an electronic device and / or presenting a virtual three-dimensional environment scene to the user. In some examples, the electronic device may present and / or display the three-dimensional environment to the user of the three-dimensional environment. In some examples, the three-dimensional environment may include representations of physical objects and / or individuals. In some examples, the three-dimensional environment may include one or more virtual objects and / or virtual assets.

[0015] In some examples, the electronic device may display one or more representations of a physical individual. In some examples, the one or more representations include a representation of a user of another electronic device participating in a communication session with the electronic device. In some examples, the electronic device displays a virtual scene that can be shared with the other electronic device. In some examples, the virtual scene is displayed as if the user of the electronic device is present within a physical equivalent of the virtual scene. In some examples, the representation of the physical individual moves relative to the virtual scene.

[0016] In some examples, the electronic device displays an indication of the user's attention within the three-dimensional environment. In some examples, the electronic device displays an indication of attention corresponding to the attention of other users examining the virtual scene. In some examples, the electronic device displays a visual indication of a boundary within the three-dimensional environment. In some examples, the electronic device keeps displaying the virtual scene based on determining that the viewpoint of the user of the electronic device corresponds to an area of ​​the three-dimensional environment within the boundary. In some examples, the electronic device displays a representation of the user's physical environment based on determining that the viewpoint of the user does not correspond to the area of ​​the three-dimensional environment within the boundary. In some examples, the electronic device changes one or more dimensions of the boundary based on user input. In some examples, the electronic device displays a visual indication of the physical individual while keeping the virtual scene displayed. In some examples, the electronic device displays representations of those physical individuals. In some examples, the electronic device displays a representation of the user's physical environment based on determining that the viewpoint of the user meets one or more criteria.

[0017] Figure 1 An electronic device 101 is illustrated that presents an extended reality (XR) environment (e.g., a computer-generated environment) according to some examples of the present disclosure. In some examples, the electronic device 101 is a handheld device or a mobile device, such as a tablet, a laptop, a smart phone, or a head-mounted display. Figure 2 An example of the device 101 is described by using an architectural block diagram of FIG. Figure 1As shown, electronic device 101 and table 106 are located in physical environment 100. The physical environment may include physical features, such as physical surfaces (e.g., floors, walls) or physical objects (e.g., tables, lights, etc.). In some examples, electronic device 101 may be configured to capture an image of physical environment 100 including table 106 (illustrated in the field of view of electronic device 101). In some examples, in response to a trigger, electronic device 101 may be configured to capture an image of physical environment 100 including table 106 (illustrated in the field of view of electronic device 101). Figure 1 1. A virtual object 104 (e.g., two-dimensional virtual content) is displayed in the physical environment 100 (e.g., a rectangular representation illustrated), which virtual object does not exist in the physical environment 100 but is displayed in the computer-generated environment positioned on top of (e.g., anchored to) the computer-generated representation 106' of the real-world table 106. For example, in response to detecting a flat surface of the table 106 in the physical environment 100, the virtual object 104 may be displayed on the surface of the computer-generated representation 106' of the table in the computer-generated environment displayed via the electronic device 101.

[0018] It should be understood that the virtual object 104 is a representative virtual object and may include and render one or more different virtual objects (e.g., virtual objects with various dimensions, such as two-dimensional or other three-dimensional virtual objects) in a three-dimensional computer-generated environment. For example, a virtual object may represent an application or user interface displayed in a computer-generated environment. In some examples, a virtual object may represent content corresponding to an application and / or displayed via a user interface in a computer-generated environment. In some examples, the virtual object 104 is optionally configured to be interactive and responsive to user input, so that a user can virtually touch, tap, move, rotate, or otherwise interact with the virtual object 104. In some examples, the virtual object 104 may be displayed in a specific orientation in a three-dimensional computer-generated environment. In some examples, when the virtual object 104 is displayed in a three-dimensional environment, the electronic device selectively moves the virtual object 104 according to the specific orientation of the displayed virtual object in response to user input (e.g., direct input or indirect input). In addition, it should be understood that the 3D environment (or 3D virtual object) described herein may be a representation of a 3D environment (or three-dimensional virtual object) projected or presented at an electronic device.

[0019] In the following discussion, an electronic device that communicates with a display generation component and one or more input devices is described. It should be understood that the electronic device optionally communicates with one or more other physical user interface devices, such as a touch-sensitive surface, a physical keyboard, a mouse, a joystick, a hand tracking device, an eye tracking device, a stylus, etc. In addition, as described herein, it should be understood that the described electronic device, display, and touch-sensitive surface are optionally distributed between two or more devices. Therefore, as used in this disclosure, information displayed on or by an electronic device is optionally used to describe information output by the electronic device to be displayed on an independent display device (touch-sensitive or non-touch-sensitive). Similarly, as referred to herein, input received on an electronic device (e.g., touch input received on a touch-sensitive surface of an electronic device, or touch input received on the surface of a stylus) is optionally used to describe input received on an independent input device, which is communicated to and / or indicated to the electronic device.

[0020] The device typically supports a variety of applications, such as one or more of the following applications: a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, a television channel browsing application, and / or a digital video player application.

[0021] Figure 2 A block diagram of an exemplary architecture of a device 201 according to some examples of the present disclosure is illustrated. In some examples, the device 201 includes one or more electronic devices. For example, the electronic device 201 can be a portable device such as a mobile phone, a smart phone, a tablet computer, a laptop computer, an auxiliary device that communicates with another device, a head-mounted display, etc.

[0022] like Figure 2 As illustrated, the electronic device 201 optionally includes various sensors (e.g., one or more hand tracking sensors 202, one or more position sensors 204, one or more image sensors 206, one or more touch-sensitive surfaces 209, one or more motion and / or orientation sensors 210, one or more eye tracking sensors 212, one or more microphones 213 or other audio sensors, etc.), one or more display generation components 214, one or more speakers 216, one or more processors 218, one or more memories 220, and / or communication circuits 222. One or more communication buses 208 are optionally used for communication between the above components of the electronic device 201.

[0023] The communication circuit 222 optionally includes circuits for communicating with electronic devices, networks such as the Internet, intranets, wired networks and / or wireless networks, cellular networks, and wireless local area networks (LANs). The communication circuit 222 optionally includes circuits for using near field communication (NFC) and / or short-range communication such as Bluetooth. ) circuit for communication.

[0024] Processor 218 includes one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, memory 220 is a non-transitory computer-readable storage medium (e.g., flash memory, random access memory, or other volatile or non-volatile memory or storage device) storing computer-readable instructions, which are configured to be executed by processor 218 to perform the techniques, processes, and / or methods described below. In some examples, memory 220 may include more than one non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium may be any medium (e.g., excluding signals) that can tangibly contain or store computer-executable instructions for use by or in combination with an instruction execution system, device, and apparatus. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include disks, optical disks based on CD, DVD, or Blu-ray technology, and persistent solid-state memory (such as flash memory, solid-state drives, etc.).

[0025] In some examples, display generation component 214 includes a single display (e.g., a liquid crystal display (LCD), an organic light emitting diode (OLED), or other type of display). In some examples, display generation component 214 includes multiple displays. In some examples, display generation component 214 may include a display with touch capabilities (e.g., a touch screen), a projector, a holographic projector, a retinal projector, etc. In some examples, electronic device 201 includes a touch-sensitive surface 209 for receiving user inputs (such as tap inputs and swipe inputs or other gestures). In some examples, display generation component 214 and touch-sensitive surface 209 form a touch-sensitive display (e.g., a touch screen integrated with electronic device 201 or a touch screen that communicates with electronic device 201 outside electronic device 201).

[0026] The electronic device 201 optionally includes an image sensor 206. The image sensor 206 optionally includes one or more visible light image sensors (such as a charge coupled device (CCD) sensor) and / or a complementary metal oxide semiconductor (CMOS) sensor operable to obtain an image of a physical object from a real-world environment. The image sensor 206 also optionally includes one or more infrared (IR) sensors, such as a passive IR sensor or an active IR sensor, for detecting infrared light from the real-world environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real-world environment. The image sensor 206 also optionally includes one or more cameras configured to capture the movement of physical objects in the real-world environment. The image sensor 206 also optionally includes one or more depth sensors configured to detect the distance of the physical object from the electronic device 201. In some examples, information from the one or more depth sensors may allow the device to identify an object in the real-world environment and distinguish it from other objects in the real-world environment. In some examples, the one or more depth sensors may allow the device to determine the texture and / or topography of an object in the real-world environment.

[0027] In some examples, the electronic device 201 uses a combination of a CCD sensor, an event camera, and a depth sensor to detect the physical environment around the electronic device 201. In some examples, the image sensor 206 includes a first image sensor and a second image sensor. The first image sensor and the second image sensor work in tandem and are optionally configured to capture different information of physical objects in the real-world environment. In some examples, the first image sensor is a visible light image sensor and the second image sensor is a depth sensor. In some examples, the electronic device 201 uses the image sensor 206 to detect the positioning and orientation of the electronic device 201 and / or the display generation component 214 in the real-world environment. For example, the electronic device 201 uses the image sensor 206 to track the positioning and orientation of the display generation component 214 relative to one or more fixed objects in the real-world environment.

[0028] In some examples, the electronic device 201 includes a microphone 213 or other audio sensor. The electronic device 201 optionally uses the microphone 213 to detect sounds from the user and / or the user's real-world environment. In some examples, the microphone 213 includes a microphone array (multiple microphones) that optionally operate in conjunction, such as to identify ambient noise or locate a sound source in the space of the real-world environment.

[0029] The electronic device 201 includes a location sensor 204 for detecting the location of the electronic device 201 and / or the display generation component 214. For example, the location sensor 204 may include a GPS receiver that receives data from one or more satellites and allows the electronic device 201 to determine the absolute location of the device in the physical world.

[0030] The electronic device 201 includes an orientation sensor 210 for detecting the orientation and / or movement of the electronic device 201 and / or the display generation component 214. For example, the electronic device 201 uses the orientation sensor 210 to track changes in the position and / or orientation of the electronic device 201 and / or the display generation component 214, such as relative to physical objects in a real-world environment. The orientation sensor 210 optionally includes one or more gyroscopes and / or one or more accelerometers.

[0031] In some examples, the electronic device 201 includes a hand tracking sensor 202 and / or an eye tracking sensor 212. The hand tracking sensor 202 is configured to track the position / location of one or more parts of a user's hand, and / or the movement of one or more parts of a user's hand relative to an extended reality environment, relative to a display generation component 214, and / or relative to another defined coordinate system. The eye tracking sensor 212 is configured to track the position and movement of the user's gaze (more generally, eyes, face, or head) relative to the real world or extended reality environment and / or relative to the display generation component 214. In some examples, the hand tracking sensor 202 and / or the eye tracking sensor 212 are implemented together with the display generation component 214. In some examples, the hand tracking sensor 202 and / or the eye tracking sensor 212 are implemented separately from the display generation component 214.

[0032] In some examples, the hand tracking sensor 202 may use an image sensor 206 (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that captures three-dimensional information from the real world including one or more hands (e.g., one or more hands of a human user). In some examples, the hands may be resolved with sufficient resolution to distinguish fingers and their corresponding positioning. In some examples, one or more image sensors 206 are positioned relative to the user to define a field of view and an interaction space of the image sensor 206, in which the finger / hand position, orientation, and / or movement captured by the image sensor is used as input (e.g., to distinguish from the user's resting hand or other hands of other people in the real-world environment). Tracking fingers / hands for input (e.g., gestures, touches, taps, etc.) may be advantageous because it does not require the user to touch, hold, or wear any type of beacon, sensor, or other marker.

[0033] In some examples, eye tracking sensor 212 includes at least one eye tracking camera (e.g., an infrared (IR) camera) and / or an illumination source (e.g., an IR light source such as an LED) that emits light toward an eye of the user. The eye tracking camera may be pointed toward the user's eye to receive reflected IR light from the light source directly or indirectly from the eye. In some examples, both eyes are tracked separately by respective eye tracking cameras and illumination sources, and focus / gaze may be determined by tracking both eyes. In some examples, one eye (e.g., the dominant eye) is tracked by respective eye tracking cameras / illumination sources.

[0034] The electronic device 201 is not limited to Figure 2 Rather than being a single component and configuration, the device 201 may include fewer components, other components, or additional components in a variety of configurations. In some examples, the device 201 may be implemented between two electronic devices (e.g., as a system). One or more people using the electronic device 201 are optionally referred to herein as one or more users of the device.

[0035] A virtual scene can be used as a digitized background included in a movie experience. For example, a virtual scene can be used as a background for chroma key synthesis and / or a display on a light emitting diode (LED) wall array. A virtual scene can also be included in immersive virtual content, such as an immersive virtual scene for virtual reality (VR), extended reality (XR), and / or mixed reality (MR) applications, in which virtual assets consume at least a portion of the view of the viewer's physical environment. Editing a virtual scene using traditional methods, especially when the editing process is a collaborative process, can be cumbersome and unintuitive. The present disclosure contemplates methods and systems for improving the efficiency of a user's interaction with a virtual asset included in a scene and improving a user's awareness of their physical environment. One or more of the examples of the present disclosure relate to displaying a virtual boundary and / or boundary associated with a virtual scene via an electronic device. In one or more examples, when a user's position and / or orientation relative to the user's three-dimensional environment (sometimes referred to herein as the user's "viewpoint") corresponds to an area of ​​the three-dimensional environment outside the virtual boundary and / or boundary, the electronic device can display a representation of the user's physical environment. In one or more examples, the electronic device may display one or more portions of the virtual scene when the user's viewpoint corresponds to a position within the virtual boundary and / or boundary. In one or more examples, the electronic device displaying the virtual boundary annotation may communicate with one or more other devices, and users of the corresponding devices may view visual indications of the attention of other users. In one or more examples, the electronic device displays a visual indication of a physical individual within the user's physical environment and displays a representation of the physical individual in response to an event. In one or more examples, when a criterion is met (such as when the user's viewpoint changes rapidly relative to the three-dimensional environment), the electronic device displays a representation of the user's physical environment that replaces portions of the displayed virtual scene.

[0036] Attention is now directed to methods and systems for facilitating annotation of a virtual scene displayed in a three-dimensional environment presented at an electronic device (e.g., corresponding to electronic device 201). As previously described, it will be appreciated that extended reality (XR) editing of a virtual scene improves the efficiency of user interaction when editing and examining the virtual scene. In some examples, enabling multiple devices to participate in a communication session to collaboratively edit and / or examine a virtual scene improves the clarity and efficiency of communication between device users. Displaying virtual boundaries and / or borders foreshadows the display of a virtual scene and / or a user's physical environment, which may occur in response to a change in the user's viewpoint relative to the user's environment, thereby improving the user's awareness of content that may be presented via an electronic device, thereby improving human-computer interaction when viewing the user's physical environment and / or virtual scene.

[0037] FIG. 3A to FIG. 3JAn example interaction with a virtual scene and a user's physical environment according to an example of the present disclosure is illustrated, including inspection of the virtual scene. It is understood that FIG. 3A to FIG. 3J The particular order of input, determination, information presentation, and other operations described is exemplary only, and it is contemplated that the order of execution of these operations may differ from the examples explicitly described without departing from the scope of the present disclosure.

[0038] Figure 3A An example device for displaying a virtual scene according to some examples of the present disclosure is illustrated. Figure 3A In the present invention, a three-dimensional environment 300 includes a virtual scene, which itself includes a plurality of virtual objects and / or textures, and the virtual scene is displayed via a display 120 of an electronic device 101. In some examples, the electronic device displays the virtual scene that completely replaces the view of the physical environment, as if the user were physically within a physical equivalent of the virtual scene. In some examples, displaying the virtual scene that replaces the user's view of the physical environment may correspond to displaying the virtual scene at an immersion level greater than a threshold immersion level, which immersion level is further described herein. In some examples, in response to detecting a change in the user's viewpoint in the physical environment (e.g., a change in the user's position and / or orientation), the electronic device may change the perspective of the virtual scene, as if the user were changing the position within the virtual scene. FIG. 3A to FIG. 3J , the three-dimensional environment 300 is illustrated from the perspective of the electronic device 101 and also from a top-down perspective in the glyphs below the perspective of the electronic device 101 .

[0039] In some examples, the virtual scene is an immersive three-dimensional environment. For example, the user of electronic device 101 can be in its entire physical environment (including Figure 3A The user physically moves within a three-dimensional environment 300 (e.g., an area of ​​the physical environment illustrated by the end of the housing of the electronic device 101 in FIG. 1 ), and the device 101 optionally updates the simulated perspective of the virtual sky, virtual floor, objects in response to detecting a change in the user's viewpoint (e.g., the user's position and / or orientation relative to their physical environment), similar to the physical perspective of the physical sky, physical floor, and / or one or more physical objects when the user moves relative to their physical environment. In some examples, the virtual scene included in the three-dimensional environment 300 optionally includes a simulated texture covering a physical representation of the floor of the user's physical environment and / or a virtual floor having a simulated spatial contour (e.g., topography) that is different from the user's physical environment. In addition, the virtual scene may include a simulated atmosphere, such as a virtual sky (e.g., simulating the lower atmosphere at dawn, daytime, dusk, nighttime, etc.). It should be understood that the virtual scene may be any suitable computer-generated environment without departing from the scope of the present disclosure. Figure 3A , the user's physical environment is illustrated as being beyond the ends of the housing of the electronic device 101.

[0040] In some examples, the virtual scene is displayed as if occupying one or more areas of the user's physical environment. Figure 3A The illustrated physical environment outside the housing of the electronic device 101 may include a physical room occupied by the user 318. In some examples, the virtual scene may be displayed by the display 120, at least partially replacing the view of the representation of the user's physical environment, thereby "consuming" the view of the physical environment. For example, the electronic device 101 may include one or more outward-facing cameras that obtain images of the user's physical environment, and these images may be displayed via the display 120 as if the user were able to directly view the physical environment without the assistance of the electronic device 101. At least a portion or all of such views of the physical environment may be displayed at corresponding locations of display 120 with an opacity level less than a threshold level (e.g., 0%, 1%, 5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%, 45%, or 50% opacity), and the virtual scene may be displayed at those corresponding locations with an opacity level greater than the opacity threshold level (e.g., 0%, 1%, 5%, 15%, 25%, 40%, 50%, 60%, 65%, 75%, 90%, or 100% opacity). Additionally or alternatively, in some examples, portions of the physical environment may be visible through transparent portions of the display without the display actively displaying those portions of the physical environment.

[0041] In some examples, the physical environment of user 318 may include one or more physical objects in the user's environment, physical individuals in the user's environment, physical walls, physical floors, etc. In some examples, electronic device 101 may present a representation of the user's physical environment. For example, Figure 3A The virtual scene included in the three-dimensional environment 300 in the display may be displayed at least partially translucently, overlaying a representation and / or view of the user's physical environment presented via the display 120 (e.g., images collected by sensors 114a to 114c of the user's physical environment). In some examples, presenting the three-dimensional environment 300 includes displaying virtual content and / or presenting a view of the user's physical environment (e.g., via passive optical transmission including lenses and / or partially or completely transparent sheets of material such as glass). In some examples, the representation of the user's physical environment includes one or more images of the user's physical environment. In some examples, the electronic device 101 displays a real-time or near real-time image stream (e.g., video) of one or more portions of the physical environment corresponding to the "representation" of the user's physical environment.

[0042] As previously described, in some examples of the present disclosure, a virtual scene may include one or more virtual objects. The virtual objects may include digital assets that model physical objects, virtual placeholder objects (e.g., polygons, prisms, and / or simulated two-dimensional or three-dimensional shapes), virtual objects that include user interfaces for applications (e.g., stored in memory by the electronic device 101), and / or other virtual objects that may be displayed within a VR, XR, and / or MR environment. For example, the three-dimensional environment 300 includes a barrel 316, which is optionally a virtual asset displayed at a simulated position within the virtual scene that is similar to the physical position and orientation of the physical barrel relative to the viewpoint of the user 318. Similarly, the crate 312 is included at a simulated position and / or orientation in the three-dimensional environment that is different from the position and / or orientation of the barrel 316. In Figure 3A 3, building 310 is also included in the virtual scene, which is also a virtual asset (e.g., a virtual building) having a position and orientation relative to the virtual scene and / or the viewpoint of user 318. It should be understood that a greater number, fewer number, and / or alternative objects may be displayed without departing from the scope of the present disclosure.

[0043] In some examples, the electronic device 101 simultaneously displays the virtual boundaries and / or borders previously described simultaneously with the virtual scene. Figure 3A 300, the border 314 includes a plurality of lines forming a rectangular boundary covering the floor of the virtual scene. In some examples, the border 314 is displayed using visual attributes, such as color, brightness, saturation, opacity, hue, simulated lighting and / or a glow effect that mimics the visual appearance of a light source illuminating the virtual scene, and / or width to distinguish the border 314 from the three-dimensional environment 300. In some examples, the virtual boundary includes a plurality of lines that form a rectangular boundary covering the floor of the virtual scene. Figure 3A The virtual boundary 314 may be a sphere, a cube, a rectangular prism, and / or another suitable volumetric shape, optionally including one or more edges that are visually distinguishable.

[0044] In some examples, the electronic device 101 may simultaneously display the boundary 314 and one or more visual objects for changing one or more dimensions of the boundary 314. For example, object 303a is optionally a virtual "grabber" and the user 318 may direct input (e.g., gaze, voice command, air gesture (e.g., air pinch including one or more contacts of multiple fingers of the user, air pointing of one or more fingers, and air gripping of one or more fingers), contact with a touchpad, and / or selection with a stylus) to object 303a. In response to receiving input directed to object 303a, the electronic device 101 may initiate scaling of the boundary 314 in a first dimension, and scale by a first magnitude corresponding to the direction and / or magnitude of the user input. Additionally or alternatively, the electronic device 101 may initiate scaling of the boundary 314 in a different second direction in response to input directed to object 303b, and may initiate scaling along the first dimension in a second direction in response to input directed to object 303c, as described with reference to FIG. FIG. 3A to FIG. 3B In some examples, the electronic device 101 displays an additional or alternative virtual object and, in response to input directed to the additional or alternative virtual object, scales and / or translates the boundary 314 in one or more directions simultaneously or in rapid succession.

[0045] In some examples, when the position and / or orientation of the user 318 corresponds to an area of ​​the three-dimensional environment 300 corresponding to the virtual scene, the electronic device 101 displays the virtual scene in a first visual appearance. Figure 3A , the position of user 318 in three-dimensional environment 300 is completely within the area of ​​three-dimensional environment 300 defined by boundary 314, which optionally defines and corresponds to an area of ​​the user's physical environment. Thus, for example, electronic device 101 displays the virtual scene at a first immersion level (e.g., full immersion, where the virtual scene replaces any representation of the user's physical environment). Thus, for example, Figure 3A The illustrated user is currently unable to see physical objects that may be in his physical environment. In some examples, prior to initiating display of the virtual scene, the electronic device 101 displays a boundary 314 overlaying a representation (e.g., an image and / or video) of the user's physical environment, prompting the user 318 to remove physical objects from the area of ​​the three-dimensional environment 300 defined by the boundary 314. Figure 3A , the electronic device 101 displays the previously described virtual sky, virtual floor, and virtual objects at a relatively high opacity level (eg, 100% opacity).

[0046] In some examples, electronic device 101 displays one or more representations of individuals other than user 318 via display 120. For example, representation 302a and representation 302b include Figure 3A300, representing other individuals that are virtually or physically included in the user's three-dimensional environment 300. For example, representation 302a and / or representation 302b are optionally expressive avatars, such as anthropomorphic avatars, having one or more body parts that can move relative to each other. These body parts optionally include a head, hands, arms, shoulders, neck, legs, fingers, toes, facial features, etc. In some examples, representation 302a and / or representation 302b may correspond to individuals that share the user's physical environment, such as individuals in the user's physical room. In some examples, representation 302a and / or representation 302b are presented via passive optical transmission (e.g., lenses, transparent materials, and / or directly visible to the user's eyes) and correspond to the views of their respective corresponding physical users. As an example of an expressive avatar or a representation of a physical individual that shares the user's 318 physical environment, the illustrated representation 302b includes multiple body parts. As Figure 3A The illustrated representation 302a includes a partially expressive avatar or representation of a user of an electronic device that does not physically share the physical environment of user 318. It should be understood that such representations are merely exemplary, and that additional or alternative representations of users of corresponding electronic devices may be included in three-dimensional environment 300, and that representation 302a may have one or more characteristics that are similar or the same as those described with reference to representation 302b, and vice versa.

[0047] In some examples, representation 302a and / or representation 302b may correspond to an individual that is not in the user's physical environment but is represented using spatial information. In some examples, electronic device 101 uses spatial information to map portions of the user 318's physical environment to portions of the virtual scene, and / or to map portions of the physical environment of individuals corresponding to representation 302a and / or representation 302b to portions of the virtual scene. For example, a communication session between electronic device 101, a first electronic device used by a first user represented by representation 302a, and a second electronic device used by a second user represented by representation 302b may be ongoing to facilitate mapping between the physical environments of the respective users of the respective electronic devices. In some examples, the communication session includes communication of information corresponding to real-time or near real-time communication of sounds detected by the electronic device (e.g., voice, sounds emitted by the user, and / or ambient sounds). In some examples, the communication session includes communication of information corresponding to real-time or near real-time movement and / or a request to move a representation (e.g., avatar) corresponding to a user participating in the communication session.

[0048] For example, the first electronic device may detect movement of the user corresponding to representation 302a in the user's physical environment (e.g., different from the physical environment of user 318), and may communicate information indicating the movement to electronic devices participating in the communication session (including electronic device 101). Before detecting the movement, the first electronic device may display a virtual scene relative to the viewpoint of the first electronic device (e.g., a position and / or orientation relative to the virtual scene, similar to the physical position and / or orientation of the user relative to the physical equivalent of the virtual scene). In response to detecting the movement (e.g., obtaining information indicating the movement from another electronic device), the first electronic device may update the viewpoint of the user of the first electronic device to an updated viewpoint based on the physical movement (e.g., in a certain direction and / or moving a certain movement amplitude), as if the user of the first electronic device is physically moving through the physical equivalent of the virtual scene. It is understood that in addition to or as an alternative to detecting the physical movement of the user, the request for such movement may be directed to an input device (e.g., a virtual joystick, a trackpad, a physical joystick, a virtual button, a physical button, and / or another suitable control). The electronic device 101 may receive such information and, in response, may cause the representation 302a to move relative to the virtual scene, with the movement magnitude and / or direction mimicking the physical movement of the user of the first electronic device relative to the physical environment of the user of the first electronic device. It should be understood that other electronic devices (such as the electronic device corresponding to the representation 302b and / or the electronic device 101) may also detect similar inputs described with reference to the first electronic device and cause their corresponding representations to move within the virtual scene.

[0049] It should also be appreciated that the movement and / or placement of representations of users participating in the communication session may be defined relative to a shared coordinate system, rather than strictly relative to the virtual dimensions of the virtual scene. For example, electronic device 101 may present a view of the physical environment of user 318 that does not include the virtual scene, and may display representations 302a and 302b at locations within the view of the physical environment and / or display movement of representations 302a and 302b within the view of the physical environment. It should be appreciated that the movement and / or placement of representations of users participating in the communication session may be defined relative to a shared coordinate system, rather than strictly relative to the virtual dimensions of the virtual scene. For example, electronic device 101 may present a view of the physical environment of user 318 that does not include the virtual scene, and may display representations 302a and 302b at locations within the view of the physical environment and / or display movement of representations 302a and 302b within the view of the physical environment. FIG. 3A to FIG. 3J The described examples may occur during a communication session (described herein), and information conveying the location, orientation, audio, and / or other aspects of a physical user and / or information provided by the physical user may be exchanged via the communication session to devices participating in the communication session. It should be understood that, depending on the context, operations described with reference to virtual content displayed relative to a virtual scene may be displayed relative to a representation of the user's physical environment (such as a visual indication of user 318's attention described below).

[0050] In some examples, electronic device 101 displays one or more visual indications indicating user attention within three-dimensional environment 300. For example, electronic device 101 detects a virtual location of a target of user attention 306 (e.g., gaze) and displays visual indication 304c at that virtual location, thereby presenting a visual indication of the portion of three-dimensional environment 300 to which the user's attention is directed. In some examples, one or more parts of the user's body other than the eyes are used to indicate the target of the user's attention. For example, although Figure 3A Although not shown in the figure, the electronic device 101 can detect a spatial relationship between a contact point between one or more fingers included in the hand 308 (e.g., forming an air pinch gesture or an air pointing gesture) and the electronic device 101. The spatial relationship can be based on a ray projected from a portion of the electronic device 101 (such as the center of the electronic device 101), passing through parts of the user's body (e.g., passing through the air pinch gesture, passing through the fingertips arranged in the air pointing gesture), and extending toward a location within the virtual scene.

[0051] In some examples, attention of representation 302a is detected and / or information indicating a target of attention is obtained, and the electronic device 101 displays a visual indication of attention in response to the detection and / or the obtaining. Based on determining that the user's attention is directed to an invisible portion of the virtual scene (e.g., as if a physical user is staring at a portion of a physical crate in the user's physical environment corresponding to representation 302a, which is not visible to user 318 from his perspective), the electronic device 101 abandons displaying an indication of attention of representation 302a. Additionally or alternatively, the electronic device 101 may display a visual indication of attention with a modified appearance (e.g., a different spatial outline, such as a simulated glow around a portion of crate 312, an arrow with simulated depth that curves behind crate 312, and / or having visual characteristics (e.g., opacity, blur, saturation, and / or simulated lighting effects)) to convey that the target of the user's attention corresponding to representation 302a is currently not visible to user 318. It should be understood that a visual indication of attention may be displayed and sometimes omitted from the figure for convenience. For example, Figure 3A The figure shows the direction to the building 310. Figure 3B 302b, but alternatively the visual indication may not be displayed (eg, based on a determination that the user corresponding to representation 302b is inhibited from sharing its attention).

[0052] In some examples, the virtual scene has simulated depth, and visual indication 304c is displayed at a location based on the user's attention and / or the spatial relationship between device 101 and the mid-air gesture. For example, due to the user's gaze and / or a ray projected from device 101 through the mid-air pinch gesture intersecting a location on the virtual floor, Figure 3AThe electronic device 101 in the virtual scene displays a visual indication 304c at the location on the surface of the virtual floor included in the virtual scene.

[0053] In some examples, as described herein, electronic device 101 may display an indication of the attention of other users. Figure 3A , the user's attention (e.g., gaze) corresponding to representation 302a is directed toward the virtual floor. The user's electronic device may detect that the floor is the target of the user's attention and may communicate information indicating the target to electronic device 101. In response to obtaining this information, electronic device 101 may display a visual indication of attention, such as Figure 3A Similarly, the electronic device corresponding to representation 302b can detect the attention of the user corresponding to representation 302b and can display a visual indication of the user's attention, such as Figure 3A 304b overlaying building 310 in the virtual scene. It should be understood that in some examples, in response to detecting that the corresponding user's attention has changed relative to the three-dimensional environment 300 and / or the virtual scene, the corresponding electronic device may convey information that moves the attention of the updated attention target. In response to obtaining such updated information, the electronic device 101 may move the visual indication of attention (e.g., move visual indication 304a and / or visual indication 304b) to an updated position and / or orientation relative to the content included in the virtual scene based on the updated information. In some examples, the visual indication of attention is displayed as a representation of the physical environment overlaying the user 318 (e.g., not including a portion of the virtual scene).

[0054] In some examples, electronic device 101 selectively displays a visual indication of attention (e.g., Figure 3A For example, when an interactive mode (e.g., editing mode) with respect to the virtual scene is enabled, the electronic device 101 may display a visual indication of attention. In some examples, when the interactive mode is disabled, the electronic device forgoes displaying a visual indication of attention. Similarly, when the interactive mode is enabled, the electronic device 101 may display other visual indications of attention of other users (e.g., Figure 3A304a and / or visual indication 304b in the three-dimensional environment 300), and when the interactive mode is disabled, the electronic device 101 may forgo displaying visual indications of the attention of other users. In some examples, the electronic device 101 displays the visual indication of attention according to user preferences. For example, user settings specified by the electronic device 101 may allow or prohibit sharing the visual indication of the attention of the user 318 with other users participating in the communication session with the electronic device 101. In some examples, the visual indication of attention may be displayed in response to detecting a display of a visual indication within the three-dimensional environment 300 (e.g., a predefined air gesture performed by the user's body, a posture of one or more parts of the user's body, a verbal request to display a visual indication, and / or a selection of a virtual and / or physical control (e.g., a button, a slider, and / or a menu option)) and / or an explicit request to share the visual indication with other devices participating in the communication session with the electronic device 101.

[0055] Figure 3B The present invention illustrates the presentation of a virtual scene when a user of an electronic device is at a position corresponding to a boundary that partially exceeds the virtual scene according to some examples of the present disclosure. FIG. 3A to FIG. 3B , user 318 moves to the right relative to the top view of three-dimensional environment 300. As illustrated in the top view, and from the perspective of the three-dimensional environment visible via electronic device 101, optionally as a result of the aforementioned movement, the viewpoint of user 318 spans Figure 3B In some examples, the electronic device 101 starts displaying representations of portions of the physical environment of the electronic device based on determining that the viewpoint of the user 318 satisfies one or more criteria, such as criteria that are satisfied when the viewpoint of the user corresponds to a location associated with the boundary 314. For example, the one or more criteria include corresponding criteria that are satisfied when the viewpoint of the user 318 coincides with the location defining the boundary 314, when the viewpoint is within a threshold distance (e.g., 0.01m, 0.05m, 0.075m, 0.1m, 0.25m, 0.4m, 0.5m, 1m, or 1.5m) of the location defining the boundary 314, and / or when the viewpoint of the user has recently moved to the aforementioned location within a threshold time period (e.g., 0.01 seconds, 0.05 seconds, 0.1 seconds, 0.5 seconds, 1 second, or 1.5 seconds).

[0056] exist Figure 3B In FIG. 1 , boundary 320 illustrates the demarcation between the virtual scene and the representation of the user's physical environment. Figure 3B , the electronic device 101 projects a dividing line from the boundary 314 (e.g., toward the ceiling of the three-dimensional environment 300), maintaining display of the portions of the virtual scene that are within the boundary 314. For example, due to the respective positioning of the building 310 and the bucket 316 relative to the boundary 314, the building is partially displayed and the bucket is fully displayed. Figure 3B , a representation 302b of another user present in the physical environment of the electronic device 101 is presented near a physical corner of a room included in the three-dimensional environment 300. For example, although such positioning is beyond the boundary 314 and / or a projection of the boundary 314 (e.g., to the ceiling of the user's environment), the electronic device 101 presents the representation 302b because the representation 302b corresponds to a representation of the user in the physical environment of the electronic device 101 (e.g., not a physical object and / or a virtual asset included in the virtual scene). Figure 3B , table 319 corresponds to a representation (e.g., an image and / or video, such as a real-time video or a view of the table through a transparent portion of display generation component 120 and / or electronic device 101) of a physical table that was previously not visible when user 318 was within boundary 314. Figure 3B As illustrated, one or more portions of the virtual scene continue to be displayed with their respective positions and / or orientations relative to the viewpoint of the user 318 within the three-dimensional environment 300 and / or boundary 314, and in response to the user's viewpoint satisfying one or more criteria (e.g., movement of the viewpoint satisfies the one or more criteria), one or more second portions of the virtual scene beyond the boundary 314 are replaced by one or more representations of the user's physical environment at one or more second areas of the three-dimensional environment 300.

[0057] In some examples, based on determining that the viewpoint of user 318 is completely beyond boundary 314, electronic device 101 stops displaying the virtual scene. For example, in response to user 318 moving to a position that does not cross boundary 314 and / or is not within a threshold distance of the boundary, electronic device 101 optionally stops displaying all virtual objects and / or textures included in the virtual scene, and optionally starts presenting a representation of the user's physical environment (e.g., an image and / or video of the user's physical environment and / or a view of portions of the physical environment through a transparent portion of display generation component 120 and / or electronic device 101). In some examples, if the position of one or more lines defining boundary 314 is within the user's viewport, boundary 314 continues to be displayed independently of the viewpoint of user 318. In some examples, when the viewpoint of user 318 corresponds to boundary 314 and / or is within the boundary, boundary 314 is displayed. In some examples, when the viewpoint of user 318 is within a threshold distance of boundary 314, boundary 314 is displayed (e.g., and the virtual scene is displayed, or the virtual scene is not displayed). In some examples, border 314 is displayed in response to an explicit request to display the border, such as user input including a voice command, selection of a physical or virtual button, press of an electromechanical crown button, an air gesture, and / or some combination of one or more of the foregoing input modalities.

[0058] Figure 3CThe expansion of the boundaries associated with the virtual scene and the increase of the immersion of the virtual scene are illustrated. FIG. 3B to FIG. 3C , the electronic device 101 detects an input requesting to scale the boundary 314 of the virtual object 303c when the attention 306 is directed to the virtual object 303c, and in response, changes the first dimension of the boundary 314. For example, from FIG. 3B to FIG. 3C , the electronic device can detect air pinch or air pointing and / or movement of hand 308 while maintaining air pinch (e.g., contact between fingers) or air pointing (e.g., a gesture of one or more fingers). In response to detecting the air pinch or air pointing, the electronic device 101 scales the border 314 by a first magnitude in the first direction based on a second magnitude of the air gesture moving in a second direction that is optionally the same as the first direction.

[0059] For example, in response to the user's hand 308 moving to the right, the electronic device 101 expands the rectangular boundary 314 to present the location of the right side of the user's viewpoint. Figure 3C , user 318 is completely within the area of ​​3D environment 300 defined by extended boundary 314. Thus, user 318's viewpoint again satisfies one or more criteria (similar to Figure 3A 314), such as when the viewpoint of user 318 corresponds to an area of ​​three-dimensional environment 300 within boundary 314. Thus, in response to the expansion of boundary 314, electronic device 101 displays a fully consumed Figure 3C The virtual scene of the viewport of the electronic device 101 in the embodiment replaces the presentation of the representation of the user's physical environment that would otherwise be visible when the virtual scene is not displayed. Figure 3C Again, because the virtual crate is included in the virtual scene, its positioning is now consumed by the virtual content rather than by the representation of the user’s physical environment. Figure 3B The previously visible table 319 illustrated is no longer included in Figure 3C In some examples, the representation is displayed independently of the spatial relationship of the user's representation to the boundary 314. For example, in Figure 3B and 3C In both, representation 302b is displayed so that visibility of the representation to the user is maintained independent of the viewpoint of user 318 relative to boundary 314. As previously described, representation 302b optionally corresponds to a presentation of a physical user via a transparent material included in display 120 and / or electronic device 101, and / or optionally corresponds to an at least partially virtual avatar representing a user who does not share a physical environment with user 318.

[0060] In some examples, the electronic device 101 determines a component of movement of the hand 308 in a first direction, and scales the boundary 314 based on the component of movement of the hand 308 that is parallel to an allowed scaling direction associated with the object 303c (e.g., the right edge of the boundary 314 is scaled to the right or left). In such examples, the electronic device 101 may abandon scaling the boundary 314 based on the movement of the hand 308 not being parallel to the allowed scaling direction. Although not shown, in response to detecting that the hand 308 moves to the left while the user's attention 306 is directed toward the object 303c, the electronic device 101 optionally scales the boundary 314 based on the leftward movement, thereby reducing the area and / or volume of the boundary 314. In some examples, based on determining that the object associated with the scaling boundary 314 is parallel to the allowed scaling direction along multiple dimensions (e.g., relative to the object 303c), the electronic device 101 may scale the boundary 314 based on the leftward movement. Figure 3B The electronic device 101 scales the border 314 based on user input in multiple directions, in association with scaling of the border 314 (depth and / or width of the border line from the viewpoint of the user 318 in FIG. 1 ). For example, the electronic device 101 optionally displays an object in a corner of the border 314 that is interactive to scale both sides of the border line that meets at the corner, thereby enabling the user to scale the border in both directions on both sides using one or more inputs directed to the object in the corner.

[0061] exist Figure 3D According to some examples of the present disclosure, an electronic device displays visual indications of physical entities within a user's physical environment while maintaining display of a virtual scene. For example, Figure 3D The viewpoint of user 318 in Figure 3C The user's viewpoint in includes different positions and / or orientations. Figure 3D , the viewpoint of the user 318 remains corresponding to (eg, within) the boundary 314 of the virtual scene. Thus, the electronic device 101 continues to display the virtual scene (eg, Figure 3D 312) and / or a virtual sky including clouds. Figure 3D, the three-dimensional environment 300 also includes a visual indication 322. In some examples, the visual indication 322 indicates an area of ​​the user's physical environment that the electronic device 101 may expect the physical individual to occupy. For example, the area is optionally occupied by equipment, which includes a computing workstation, a laptop computer, a server, an additional media recording device, a table, a chair, and one or more people who operate such equipment. Such equipment is optionally able to communicate with the electronic device 101, other devices that communicate with the electronic device 101, and / or optionally participate in the communication session previously described including the electronic device 101. In some examples, the positioning of the visual indication 322 is fixed relative to the three-dimensional environment 300 (e.g., it cannot be moved in response to a request from the user 318). In some examples, the positioning of the visual indication 322 is not necessarily fixed, but it will not move in response to one or more user inputs that can be operated to move other virtual content (e.g., virtual objects, boundaries 314, and / or user interfaces of applications and / or settings associated with the electronic device 101). For example, the electronic device 101 updates the positioning of the visual indication 322 in response to detecting the movement of the equipment associated with the above-mentioned physical individual.

[0062] In some examples, the electronic device 101 displays a representation of a portion of the user's physical environment in response to the user input pointing to the visual indication 322. For example, in Figure 3D , the electronic device 101 detects an air gesture performed by a hand 308 and / or the user's attention 306 directed to a visual indication 322. In response to detecting the air gesture, the electronic device 101 displays a representation (e.g., an image and / or video) of an area of ​​the physical environment corresponding to the visual indication 322, such as Figure 3E Therefore, in response to pointing to Figure 3D In response to the user input of the visual indication 322, the electronic device 101 simultaneously displays a portion of the user's physical environment and the virtual scene, independent of the user's viewpoint relative to the boundary 314. Figure 3E , the representation of the physical environment includes representation 326a and representation 326b (including images and / or videos, such as real-time video of people in that portion of the user's physical environment), as well as table 328 corresponding to the representation of the physical table.

[0063] FIG. 3E to FIG. 3H Examples of electronic devices prohibiting or allowing modification of a virtual scene based on a spatial relationship between a user participating in a communication session with the electronic device and a boundary associated with the virtual scene are illustrated. In some examples, the electronic device 101 abandons modification of the virtual scene, such as abandoning insertion of a virtual asset, based on determining that the user is within the boundary 314. Figure 3EIn the example of FIG. 1 , the electronic device 101 detects a user input including an air gesture (e.g., an air pinch performed by the hand 308) requesting the insertion and / or display of a virtual asset within the virtual scene. However, it should be noted that when the Figure 3E When user input is detected in , it indicates that the positioning of 302a is within the boundary 314. FIG. 3E to FIG. 3F , the electronic device 101 abandons displaying the information requested by the user 318 Figure 3E 314. A virtual bucket corresponding to the mid-air gesture performed by hand 308 in FIG. 314. It should be appreciated that when additional or alternative input is detected, the additional or alternative input may be detected and the operation associated with modifying the virtual scene abandoned based on determining that the user's representation is within boundary 314.

[0064] exist Figure 3G In the embodiment, the electronic device 101 detects a representation 302a at a location outside the boundary 314 of the virtual scene. For example, the electronic device 101 detects that the information and / or the representation 302a and / or the user corresponding to the representation 302a is moved from Figure 3F Move to the position shown in Figure 3G , and in response, moves representation 302a to an updated position outside of boundary 314 based on the information and / or the detected movement. Figure 3G , the electronic device 101 detects a user input including an air gesture performed by the hand 308 (similar to or equivalent to the reference Figure 3E The user input described above), and in response, the electronic device 101 displays Figure 3H The barrel 330. Figure 3H In response to receiving the input, electronic device 101 allows bucket 330 to be inserted into the virtual scene based on determining that users other than 318 do not correspond to positions within boundary 314, and / or based on determining that the distance from the user to the requested location for inserting bucket 330 is greater than a threshold distance. Figure 3H From the perspective of the user corresponding to representation 302a in , the user's physical environment is visible, and when bucket 330 is initially displayed, the virtual scene shared in the communication session with electronic device 101 is not visible because the position of the user corresponding to representation 302a is not associated with boundary 314.

[0065] Fig. 3I and Figure 3J Examples of presenting a visual representation of the physical environment of a user of electronic device 101 based on determining that the user's viewpoint satisfies one or more criteria are illustrated. In some examples, based on determining that the viewpoint of user 318 changes at a rate faster than a threshold rate, electronic device 101 displays a representation of the user's physical environment. For example, FIG. 3A to FIG. 3I, user 318 pans his viewpoint to the left as illustrated in the top view. In some examples, because speed 332 is less than a threshold speed (e.g., "V=x cm / s"), and / or because user 318's viewpoint is at a location corresponding to boundary 314 (e.g., within the boundary), electronic device 101 maintains displaying the virtual scene and abandons displaying a representation of the user's physical environment. FIG. 3A to FIG. 3J , the viewpoint of user 318 moves to Fig. 3I The same positioning as shown in the example, its speed 334 is greater than that of FIG. 3A to FIG. 3I The user's speed 332 (e.g., Figure 3J The velocity is "V = 2x cm / s"). FIG. 3A to FIG. 3J , because the rate of change of the viewpoint is greater than a threshold value (e.g., 0.05 cm / s, 0.1 cm / s, 0.5 cm / s, 1 cm / s, 2 cm / s, 5 cm / s, 7.5 cm / s, 10 cm / s, 12.5 cm / s, 15 cm / s, or 20 cm / s), the electronic device 101 displays representations of the user's physical environment beyond the boundaries 336a and 336b, which may be projections of the dimensions of the boundaries 314 toward the ceiling of the user's three-dimensional environment 300. In some examples, in response to detecting that the speed of the user's positioning is higher than the threshold amount, the electronic device 101 displays portions of the physical environment even if the user's position during movement corresponds to the boundaries 314 as described herein.

[0066] In some examples, the movement of the user's viewpoint is rotational (e.g., relative to a floor extending vertically through the user's head, horizontally and / or parallel to the user in a perceived lateral direction, and / or along a depth axis extending parallel to the floor in front of / behind the user's viewpoint), and the electronic device 101 displays a representation of the user's physical environment in response to detecting the rotation at a rate greater than a threshold rate. In some examples, the movement of the user's viewpoint is a translation in a different direction (e.g., different from a vertical direction) relative to the top view of the three-dimensional environment 300. Fig. 3I and 3J318 ) and the electronic device displays a representation of the user's physical environment in response to detecting the translation at a rate greater than a threshold rate. In some examples, portions of the viewport that are no longer consumed by the virtual scene in response to the movement of the user's 318 viewpoint are not strictly associated with the dimensions of boundary 314. For example, boundary 336a and boundary 336b may correspond to portions (e.g., 5%, 10%, 15%, 25%, 40%, 50%, 60%, 75%, 80%, 85%, or 90%) of the user's viewport relative to an edge (e.g., a vertical and / or lateral edge) of the viewport. Additionally or alternatively, the virtual scene may remain in a first portion of the user's viewport in response to movement that satisfies one or more criteria (e.g., a circular, rectangular, and / or elliptical portion of the viewport), and / or a representation of the user's physical environment may be displayed in a second portion of the viewport (e.g., beyond the boundary of the first portion of the viewport).

[0067] It should be understood that similar to as further described herein with reference to displaying a representation of the user's physical environment relative to boundary 314, the visual appearance of the virtual scene and / or the representation of the user's physical environment may differ from, for example, Figure 3J and / or Fig. 3I For example, in response to the movement of the user's viewpoint satisfying one or more criteria (e.g., when the movement speed is faster than a threshold speed), the representation of the user's physical environment may increase in opacity relative to the three-dimensional environment 300, and / or the virtual scene may decrease in opacity relative to the three-dimensional environment 300. Additionally or alternatively, in response to the movement satisfying one or more criteria, the color, saturation, brightness, hue, and / or simulated lighting effects overlaying the representation of the user's physical environment and / or the virtual scene may be applied, modified, and / or ceased.

[0068] Attention is now directed to inputs and operations associated with inserting virtual objects into a virtual scene, such as virtual objects inserted into a virtual scene based in part on the determined context of a user's interaction with the virtual scene. In some examples, the electronic device 101 detects one or more user inputs as at least further described below, and in response, the electronic device 101 performs one or more operations to insert, move, modify, and / or visually distinguish one or more virtual objects. In some examples,

[0069] As reference Figure 2As described, the electronic device 101 includes one or more sensors and / or communicates with one or more sensors to detect the spatial relationship between the user's viewpoint, one or more parts of the user's body and the virtual scene. For example, the electronic device 101 optionally projects one or more rays from one or more sensors, thereby intersecting with one or more parts of the user's body, and further intersecting with one or more parts of the virtual scene. In some examples, the electronic device 101 detects that the user input is pointing to one or more parts of the virtual scene based on determining that one or more rays correspond to one or more parts of the virtual scene (e.g., virtually intersecting with one or more parts of the virtual scene). In some examples, one or more parts of the user's body include one or more fingers, a part of a finger, a palm, a hand, a wrist, a forearm and / or an arm of the user. In some examples, the electronic device 101 detects and / or determines the positioning and / or orientation of multiple body parts in the aforementioned one or more body parts, and determines the target of the user input based on the combination of the positioning and / or orientation of multiple body parts in the one or more body parts.

[0070] In some examples, one or more user inputs include the user's attention. For example, Figure 3G The electronic device 101 in 3D environment 300 uses one or more imaging sensors, such as one or more cameras included in sensors 114a-114c, to detect the position and / or orientation of one or more eyes of a user (e.g., indicating a gaze of user 318). In some examples, the electronic device 101 detects the position and / or orientation of one or more rays projected from the one or more eyes of the user to one or more corresponding locations within the three-dimensional environment 300. In some examples, the electronic device 101 detects user input to one or more corresponding locations that correspond to (e.g., intersect or approach) the one or more rays projected from the one or more eyes of the user.

[0071] In some examples, one or more user inputs include the user's voice. The audio provided by the user optionally includes one or more words spoken by the user, which are detected by one or more microphones included in the electronic device 101 and / or communicating with the electronic device. In some examples, the electronic device 101 parses information and / or communicates information to one or more external computing devices to determine the content of the user's voice, including recognizing the user's words and / or obtaining a semantic understanding of the voice. For example, the electronic device includes one or more processors configured to perform natural language processing to detect one or more words and determine the possible meaning of the sequence of the one or more words. In some examples, the electronic device 101 additionally or alternatively determines the meaning of the sequence of one or more words based on the user's determination context.

[0072] In some examples, the electronic device 101 determines the context of the user's interaction with the three-dimensional environment 300. For example, when the input is detected, the electronic device 101 determines the context of the user based in part or in whole on the location and / or orientation of the visual indication 304c. Figure 3B When an air gesture (e.g., pinching and / or pointing) is detected in the middle of the screen, the user's attention is directed to object 303c, so the electronic device 101 initiates the scaling of boundary 314. In some examples, additional or alternative factors are used to determine the user context. For example, in the absence of an air pinch gesture including contact between the user's fingers, the electronic device 101 may display a virtual object at a location corresponding to the location where the target of attention is directed in response to detecting user input requesting the placement of an annotation. For example, the electronic device 101 optionally parses Figure 3G to determine that user 318 is likely referring to crate 312 because the user refers to moving "this crate" and crate 312 is the only virtual object that resembles a physical crate. Therefore, electronic device 101 optionally determines Figure 3D The user context in corresponds to a "crate" or a crate-like virtual object, and optionally determines that crate 312 corresponds to the crate of interest. Thereafter, electronic device 101 may detect subsequent commands to move and / or modify the dimensions and / or appearance of crate 312, and in response, may alter crate 312 according to the detected commands. In some examples, the voice command may request the insertion of a virtual object, such as Figure 3H Bucket 330 in the image, and in response to the voice command, a virtual bucket 330 can be displayed.

[0073] In some examples, in response to inserting a virtual object into a virtual scene, simultaneously with inserting a virtual object into a virtual scene, and / or after inserting a virtual object into a virtual scene, the electronic device 101 detects and / or prompts the user 318 with information corresponding to the annotation and / or description of the virtual object. For example, the electronic device 101 displays a user interface that prompts the user to provide voice, air gestures, text input (e.g., via a virtual or physical keyboard), movement, attention, and / or other suitable information modalities. Such a user interface may include one or more virtual buttons to start text input, sound recording, movement recording, and / or recording of the user's attention, and / or stop such text input and / or recording. In some examples, the information provided by the user 318 includes a description associated with the virtual object, the name of the virtual object, metadata associated with the virtual object (such as the category of the virtual object), and / or other suitable information that may be of interest to future inspectors of the virtual object. After the text input and / or recording provided by the user 318 is completed, the electronic device 101 may stop displaying the user interface and / or associate the provided information with the corresponding virtual object. In some examples, the electronic device 101 begins recording and / or initiates text input in response to inserting the virtual object without displaying a dedicated user interface.

[0074] In some examples, the electronic device 101 may map the user speech to a virtual object and / or a request to insert the virtual object into a specific location within the virtual scene based on determining that the user speech describes an object similar to the virtual object and / or a location associated with the virtual object. For example, speech that mentions a box, a rectangular prism, a cuboid, a container, a basket (e.g., if the virtual object includes an opening on at least one side of the crate 312), etc. may be determined to correspond to the crate 312. In addition, speech that mentions a name assigned to the crate 312 (e.g., "Crate 1") may be detected and determined to correspond to the name of the crate 312. Figure 3H crate 312 in. In such an example, the electronic device 101 may interpret the pronoun "this" object as a virtual object to which the user 318 directs his or her attention within a threshold amount of time (e.g., 0 seconds, 0.01 seconds, 0.05 seconds, 0.1 seconds, 0.5 seconds, 1 second, 1.5 seconds, 2 seconds, 3 seconds, 5 seconds, 10 seconds, or 30 seconds), a physical object to which the user physically gestures (e.g., points, moves his or her fingers and / or hands toward the virtual object, moves his or her lips toward the virtual object, tilts his or her head toward the virtual object, moves his or her arms toward the virtual object, and / or points his or her legs and / or feet toward the virtual object), and / or a virtual object within a threshold distance of the user (e.g., 0m, 0.01m, 0.05m, 0.1m, 0.5m, 1m, 1.5m, or 3m). Similarly, speech referring to a barrel, a cylinder, a vat, a bucket, a basin, and / or a keg may be determined to correspond to a virtual object. Figure 3H330 in the bucket. It should be understood that additional or alternative factors are contemplated without departing from the scope of the present disclosure. For example, speech indicating "that," "these," "those," "the objects there," etc. may be detected by the electronic device 101 and mapped to one or more virtual objects based on a determination of the user context. Additionally, based on factors similar to or the same as those described with reference to a particular virtual object, the electronic device 101 may detect that the user 318 is requesting that a virtual object be inserted and / or displayed at a particular location within the virtual scene, and in response, may insert the virtual object into the virtual scene.

[0075] In some examples, electronic device 101 may determine user context based on movement, indication of attention, and / or other factors. Figure 3H In response to detecting movement, attention, and / or speech of a user corresponding to representation 302a, the electronic device corresponding to representation 302a may communicate information indicating the detected movement, attention, and / or speech. In response to detecting the information, the electronic device corresponding to representation 302a may initiate spatial recording, storage, and / or capture information to later present an animation and / or audio recording including an indication of the user's movement, speech, and / or attention. In such examples, the recording may be initiated without detecting an explicit input (e.g., actuation of a virtual or physical button, a voice command, an air gesture, and / or another suitable input as further described herein), or may be initiated in response to the explicit input. For example, the electronic device may detect that the user begins talking about crate 312, is looking at crate 312, and / or is moving around crate 312 and / or within a threshold distance of crate, and determines that the user's context is related to crate 312. The recording may continue until the electronic device detects a cessation of speech, a pause in speech, movement beyond a threshold distance from the crate 312, a change of attention away from the crate 312, and / or until an input (e.g., a quick input) is detected requesting that the recording be stopped. After the recording ends and / or in response to the cessation of the recording, information indicating the recording may be communicated to other electronic devices in real time. In response to obtaining such information associated with the recording, the electronic device 101 may display a virtual annotation within the virtual scene, which may be later interacted with as further described herein to optionally simultaneously initiate presentation of a spatial representation of the user's speech, an indication of attention, and / or presentation of audio included in the recording.

[0076] In some examples, one or more factors indicative of user context (e.g., voice, air gestures, air postures, gaze, previous interactions with the three-dimensional environment 300, etc.) may be used in combination to determine a possible set of one or more virtual objects that are possible annotation targets. For example, the electronic device 101 may use gaze in conjunction with voice, may ignore one or more factors associated with user context (e.g., air gestures that override gaze and / or voice), and / or may probabilistically determine user context based on previous placement (e.g., display) of virtual objects in light of previous determinations of user context.

[0077] In some examples, the user context may be determined to be general and not explicitly mention a virtual object within the three-dimensional environment. In some examples, the electronic device 101 may determine that the reminder generally refers to the virtual scene and does not explicitly mention a specific virtual object. The electronic device 101 may determine that the audio provided by the user 318 generally refers to multiple parts of the virtual scene (e.g., the scene texture may correspond to a virtual texture covering the floor of the virtual scene, one or more textures applied to one or more virtual objects, and / or a virtual texture covering the virtual sky included in the three-dimensional environment 300). Therefore, the electronic device 101 may display a virtual object within the three-dimensional environment 300 in response to determining that the user context generally refers to the virtual scene. Generally placed annotations such as virtual objects may be displayed at a predetermined virtual distance (e.g., 0m, 0.01m, 0.05m, 0.1m, 0.5m, 1m, 1.5m, or 3m) from the user 318's viewpoint and / or centered relative to the user 318's viewpoint.

[0078] In some examples, the user context is determined to be general and / or directional, such as "to my left", "to my right", "in front of me", and / or in a simulated cardinal direction specified by the user's voice. In such examples, the electronic device 101 may display a virtual object based on a determination of the meaning of the user's voice. For example, the electronic device 101 may parse the user's voice, determine the relative portion of the three-dimensional environment 300 that the voice may refer to relative to the user's 318 viewpoint when the voice is received, and in response to detecting the voice, display and / or place a virtual object at a predetermined distance from the user's 318 viewpoint toward the relative portion of the three-dimensional environment. For example, a discussion linked to a virtual scene on the left side of the user is optionally mapped to portions of the three-dimensional environment 300 to the left of the center of the user's viewpoint. In response to detecting such a discussion, the electronic device 101 may place a virtual object at a predetermined distance to the left of the center of the user's viewpoint. Additionally or alternatively, a discussion linked to a virtual scene on the right side of the user may optionally be mapped to portions of the three-dimensional environment 300 to the right of the center of the user's viewpoint. In response to detecting speech indicating that a portion of the virtual scene is "to the user's right," the electronic device 101 may place the virtual object at a predetermined distance, optionally toward the right of the center of the user's viewpoint. It should be understood that additional or alternative directions relative to the user's viewpoint and / or simulated cardinal directions may be included as factors in determining user context and / or placement of virtual objects (e.g., behind the user's viewpoint, "north" of the user's viewpoint, and / or above or below as visible via the viewport when the user 318 has a particular viewpoint).

[0079] In some examples, based on determining that information is obtained including a request to insert a virtual object at a location that is not currently visible to the user 318 in the virtual scene, the electronic device 101 presents spatial feedback indicating the location of the requested annotation. For example, the spatial feedback includes audio generated using one or more characteristics to mimic the perception of a physical audio source placed at a physical equivalent of the location of the requested virtual object in the physical environment of the user playing the audio feedback (e.g., a ringtone, voice, and / or another notification noise). In some examples, the spatial feedback additionally or alternatively includes a visual indication, such as a simulated glow effect that illuminates one or more portions of the user's viewport that are close to and / or generally toward the location of the requested virtual object.

[0080] In some examples, the location of the inserted virtual object is clearly indicated by the user's movement. For example, the electronic device 101 may detect attention and / or air gestures (e.g., pinching) pointing to an empty location within the virtual floor of the virtual scene. In response, the electronic device 101 may display the requested virtual object in an empty location on the virtual floor. In some examples, the virtual object is displayed at a location according to a user context (such as voice). In some examples, the electronic device 101 determines the location and / or orientation of the virtual annotation according to the determination of the user context and / or according to the virtual content included in the virtual scene. For example, a virtual object may be displayed as pointing downward to a part of the virtual floor and / or virtual scene, which may optionally be the default orientation of such virtual objects. In some examples, the orientation of the inserted virtual object may vary based on the surrounding virtual content. For example, according to determining that the requested virtual object will block another virtual object, the electronic device 101 may display the requested virtual object with an orientation and / or location different from the default orientation and / or an orientation and / or location translated away from the default position.

[0081] Figure 4 A flowchart illustrating an example process for interaction including annotation, editing, and inspection of a virtual scene according to an example of the present disclosure is illustrated. Figure 4 In some examples, the method 400 may be performed at an electronic device in communication with one or more input devices and a display. In some examples, within a three-dimensional environment, while a first portion of a virtual scene is displayed at a first area of ​​the three-dimensional environment and a second portion of the virtual scene is displayed at a second area of ​​the three-dimensional environment, the electronic device detects (402a) a first event via one or more input devices, wherein the first event includes a viewpoint of a user of the electronic device moving from a first viewpoint to a second viewpoint. In some examples, in response to detecting (402b) the first event, and based on determining (402c) that the viewpoint of the user satisfies one or more first criteria, the electronic device maintains (402d) displaying the first portion of the virtual scene within the first area of ​​the three-dimensional environment, and replaces (402e) display of the second portion of the virtual scene with a representation of a first portion of the user's physical environment.

[0082] A view of a three-dimensional environment is typically visible to a user through a virtual viewport via one or more display generation components (e.g., a display or a pair of display modules that provide stereoscopic content to different eyes of the same user), the virtual viewport having a viewport boundary that defines the range of the three-dimensional environment visible to the user via the one or more display generation components. In some examples, the area defined by the viewport boundary is smaller than the user's visual range in one or more dimensions (e.g., based on the user's visual range, the size, optical properties or other physical characteristics of the one or more display generation components, and / or the position and / or orientation of the one or more display generation components relative to the user's eyes). In some examples, the area defined by the viewport boundary is larger than the user's visual range in one or more dimensions (e.g., based on the user's visual range, the size, optical properties or other physical characteristics of the one or more display generation components, and / or the position and / or orientation of the one or more display generation components relative to the user's eyes). The viewport and viewport boundary typically move with the movement of one or more display generation components (e.g., with the user's head for a head-mounted device, or with the user's hand for a handheld device such as a tablet or smart phone).

[0083] The user's viewpoint determines what is visible in the viewport, and the viewpoint typically specifies a position and orientation relative to the three-dimensional environment, and as the viewpoint moves, the view of the three-dimensional environment will also move in the viewport. For head-mounted devices, the viewpoint is typically based on the position and orientation of the user's head, face, and / or eyes to provide a view of the three-dimensional environment that is perceptually accurate and provides an immersive experience when the user is using the head-mounted device. For handheld or fixed devices, the viewpoint moves as the handheld or fixed device moves and / or as the user's positioning relative to the handheld or fixed device changes (e.g., the user moves toward, away from, up, down, right, and / or left). For devices including display generation components with virtual pass-through, portions of the physical environment that are visible (e.g., displayed and / or projected) via one or more display generation components are based on the field of view of one or more cameras in communication with the display generation components, which one or more cameras typically move with movement of the display generation components (e.g., with movement of the user's head for a head-mounted device, or with movement of the user's hands for a handheld device such as a tablet or smart phone) because the user's viewpoint moves with movement of the field of view of the one or more cameras (and the appearance of one or more virtual objects displayed via the one or more display generation components is updated based on the user's viewpoint (e.g., the display position and pose of the virtual objects are updated based on movement of the user's viewpoint)). For display generation components with optical transmittance, portions of the physical environment that are visible via one or more display generation components (e.g., optically visible through one or more partially or fully transparent portions of the display generation components) are based on the user's field of view through the partially or fully transparent portions of the display generation components (e.g., moves as the user's head moves for a head-mounted device, or moves as the user's hands move for a handheld device such as a tablet or smart phone) because the user's viewpoint moves as the user moves through the field of view of the partially or fully transparent portions of the display generation components (and the appearance of one or more virtual objects is updated based on the user's viewpoint).

[0084] In some examples, a representation of the physical environment (e.g., displayed via virtual pass-through or optical pass-through) may be partially or completely obscured by the virtual environment. In some examples, the amount of the virtual environment displayed (e.g., the amount of the physical environment that is not displayed) is based on the immersion level of the virtual environment (e.g., relative to the representation of the physical environment). For example, increasing the immersion level optionally causes more of the virtual environment to be displayed, replacing and / or obscuring more of the physical environment, and decreasing the immersion level causes less of the virtual environment to be displayed, thereby revealing portions of the physical environment that were previously not displayed and / or obscured. In some examples, at a particular immersion level, one or more first background objects (e.g., in the representation of the physical environment) are visually de-emphasized more than one or more second background objects (e.g., dimmed, blurred, and / or displayed with increased transparency), and one or more third background objects cease to be displayed.

[0085] In some examples, the immersion level includes an associated degree to which virtual content (e.g., a virtual environment and / or virtual content) displayed by an electronic device obscures background content (e.g., content other than the virtual environment and / or virtual content) surrounding / behind the virtual content, optionally including the number of items of background content displayed and / or visual characteristics of the displayed background content (e.g., color, contrast, and / or opacity), the angular range of the virtual content displayed via the display generation component (e.g., 60 degrees for content displayed at low immersion, 120 degrees for content displayed at medium immersion, or 180 degrees for content displayed at high immersion), and / or the proportion of the field of view displayed via the display generation component consumed by the virtual content (e.g., 33% of the field of view consumed by the virtual content at low immersion, 66% of the field of view consumed by the virtual content at medium immersion, or 100% of the field of view consumed by the virtual content at high immersion). In some examples, the background content is included in the background in which the virtual content is displayed (e.g., background content in a representation of the physical environment). In some examples, the background content includes a user interface (e.g., a user interface corresponding to an application generated by the electronic device), virtual objects that are not associated with or included in the virtual environment and / or virtual content (e.g., files or representations of other users generated by the electronic device, etc.), and / or real objects (e.g., transparent objects representing real objects in the physical environment surrounding the user, which are visible so that they are displayed via a display generation component and / or are visible via a transparent or translucent component of the display generation component because the electronic device does not block / impede their visibility through the display generation component). In some examples, at a low immersion level (e.g., a first immersion level), the background, virtual and / or real objects are displayed in an unobstructed manner. For example, a virtual environment with a low immersion level is optionally displayed simultaneously with background content, which is optionally displayed at full brightness, color, and / or translucency.

[0086] In some examples, at a higher immersion level (e.g., a second immersion level that is higher than the first immersion level), background, virtual and / or real objects are displayed in an obstructed manner (e.g., dimmed, blurred, or removed from the display). For example, a corresponding virtual environment with a high immersion level is displayed without simultaneously displaying background content (e.g., in a full screen or fully immersive mode). As another example, a virtual environment displayed at a medium immersion level is displayed simultaneously with background content that is dimmed, blurred, or otherwise de-emphasized. In some examples, the visual characteristics of background objects differ between background objects. For example, at a particular immersion level, one or more first background objects are more visually de-emphasized (e.g., dimmed, blurred, and / or displayed with increased transparency) than one or more second background objects, and one or more third background objects cease to be displayed. In some examples, no immersion level or zero immersion level corresponds to ceasing to display the virtual environment, but instead displaying a representation of the physical environment (optionally with one or more virtual objects, such as applications, windows, or virtual three-dimensional objects), and the representation of the physical environment is not obscured by the virtual environment. Using physical input elements to adjust the immersion level provides a fast and efficient method of adjusting the immersion level, which enhances the operability of the electronic device and makes the user-device interface more efficient.

[0087] Therefore, according to the above, some examples of the present disclosure relate to a method, the method comprising: at an electronic device in communication with a display and one or more input devices, within a three-dimensional environment, when a first portion of a virtual scene is displayed at a first area of ​​the three-dimensional environment and a second portion of the virtual scene is displayed at a second area of ​​the three-dimensional environment, detecting a first event via the one or more input devices, wherein the first event includes a viewpoint of a user of the electronic device moving from a first viewpoint to a second viewpoint. In some examples, in response to detecting the first event, and based on determining that the viewpoint of the user satisfies one or more first criteria, maintaining the first portion of the virtual scene displayed within the first area of ​​the three-dimensional environment, and replacing the display of the second portion of the virtual scene with a representation of the first portion of the user's physical environment.

[0088] Additionally or alternatively, in some examples, the method may include: in response to detecting the first event, and based on determining that the user's viewpoint does not satisfy the one or more first criteria, maintaining display of the first portion of the virtual scene within the first area of ​​the three-dimensional environment, and maintaining display of the second portion of the virtual scene within the second area of ​​the three-dimensional environment.

[0089] Additionally or alternatively, in some examples, the one or more first criteria include criteria that are satisfied when the second viewpoint of the user corresponds to a boundary surrounding one or more portions of the virtual scene, where the one or more portions include the first portion of the virtual scene.

[0090] Additionally or alternatively, in some examples, the method may include: when the viewpoint of the user is the first viewpoint, displaying a first view of the virtual scene, and in response to the first event, displaying a second view of the virtual scene.

[0091] Additionally or alternatively, in some examples, the method may include: when displaying the virtual scene within the three-dimensional environment, displaying a boundary between the first area of ​​the three-dimensional environment and the second area of ​​the three-dimensional environment via the display.

[0092] Additionally or alternatively, in some examples, the boundary is displayed before the first event is detected and is displayed when the first portion of the virtual scene is displayed at the first area of ​​the three-dimensional environment and when the second portion of the virtual scene is displayed at the second area of ​​the three-dimensional environment.

[0093] Additionally or alternatively, in some examples, the method may include: while displaying the second portion of the virtual scene, detecting, via the one or more input devices, an indication of one or more inputs requesting a change in one or more dimensions of the boundary, wherein the virtual scene includes the representation of the first portion of the physical environment of the user. In some examples, the method may also include: in response to detecting the indication of the one or more inputs, updating the one or more dimensions of the boundary according to the indication of the one or more inputs; changing one or more dimensions of the first area of ​​the three-dimensional environment including the virtual scene according to the indication; and changing one or more dimensions of the second area of ​​the three-dimensional environment according to the indication, wherein the three-dimensional environment includes the representation of the physical environment of the user.

[0094] Additionally or alternatively, in some examples, the virtual scene is shared in a communication session between the electronic device and another electronic device different from the electronic device. In some examples, the method may also include: when the communication session is ongoing, obtaining an indication of a request to modify virtual content included in the virtual scene via the one or more input devices. In some examples, the method may also include: in response to obtaining the indication of the request to modify the virtual content, modifying the virtual content according to the indication of the request to modify the virtual content based on determining that one or more second criteria are met, and abandoning the modification of the virtual content based on determining that one or more second criteria are not met.

[0095] Additionally or alternatively, in some examples, the method may also include: when the first portion of the user's physical environment is visible, displaying via the display a representation of a participant corresponding to the other electronic device within the first portion of the user's physical environment, wherein the one or more second criteria include a criterion that is satisfied when the positioning of the representation of the participant corresponds to the positioning within the representation of the first portion of the user's physical environment.

[0096] Additionally or alternatively, in some examples, the method may also include: when the first portion of the user's physical environment is visible, displaying a representation of a participant corresponding to the other electronic device at a position within the first portion of the virtual scene via the display, wherein the one or more second criteria include criteria that are satisfied when the position of the representation of the participant is different from one or more positions corresponding to the location where the virtual content is modified.

[0097] Additionally or alternatively, in some examples, the virtual scene is shared in a communication session between the electronic device and another electronic device different from the electronic device. In some examples, the method may also include: when the communication session is ongoing, based on determining that one or more characteristics of the other electronic device meet one or more second criteria, including a representation of a participant of the communication session using the other electronic device in the three-dimensional environment, and based on determining that one or more characteristics of the other electronic device do not meet the one or more second criteria, abandoning including the representation of the participant in the three-dimensional environment.

[0098] Additionally or alternatively, in some examples, when the other electronic device is a head-mounted electronic device, the one or more characteristics of the other electronic device satisfy the one or more second criteria.

[0099] Additionally or alternatively, in some examples, when the other electronic device is any one of a laptop computing device, a mobile handset, a tablet computing device, and a desktop computing device, the one or more characteristics of the other electronic device do not satisfy the one or more second criteria.

[0100] Additionally or alternatively, in some examples, the representation of the user's physical environment is displayed as a portion of a viewport consuming the electronic device.

[0101] Additionally or alternatively, in some examples, the first event includes: when the viewpoint of the user changes from the first viewpoint to the second viewpoint at a rate greater than a threshold rate, the movement of the viewpoint satisfies the one or more criteria.

[0102] Additionally or alternatively, in some examples, the first portion of the physical environment of the user corresponds to a corresponding portion of the three-dimensional environment within which one or more representations of physical individuals are displayed.

[0103] Additionally or alternatively, in some examples, the method may also include: before detecting the first event and when the viewpoint of the user is the first viewpoint, displaying, via the display, a visual indication corresponding to the first portion of the physical environment of the user.

[0104] Additionally or alternatively, in some examples, the first portion of the physical environment of the user is world-locked relative to the physical environment of the user.

[0105] Additionally or alternatively, in some examples, the method may also include: when the first portion of the user's physical environment is visible, in response to the first event and based on determining that the user's viewpoint satisfies the one or more first criteria, displaying via the display one or more representations of physical individuals within the representation of the first portion of the user's physical environment.

[0106] Additionally or alternatively, in some examples, before detecting the first event, when the first portion of the virtual scene is displayed at the first area of ​​the three-dimensional environment, and when the second portion of the virtual scene is displayed at the second area of ​​the three-dimensional environment, a second event is detected via the one or more input devices, the second event comprising user input directed to the second portion of the virtual scene; and in response to detecting the second event, the display of the second portion of the virtual scene is replaced with the representation of the first portion of the user's physical environment.

[0107] Some examples of the present disclosure relate to an electronic device that communicates with a display and one or more input devices, the electronic device comprising: one or more processors; a memory; and one or more programs stored in the memory, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs comprising instructions for executing a method comprising: within a three-dimensional environment, when a first portion of a virtual scene is displayed at a first area of ​​the three-dimensional environment and a second portion of the virtual scene is displayed at a second area of ​​the three-dimensional environment, detecting a first event via the one or more input devices, wherein the first event comprises a viewpoint of a user of the electronic device moving from the first viewpoint to the second viewpoint; and in response to detecting the first event, and based on determining that the viewpoint of the user satisfies one or more first criteria, maintaining display of the first portion of the virtual scene within the first area of ​​the three-dimensional environment, and replacing display of the second portion of the virtual scene with a representation of a first portion of a physical environment of the user.

[0108] Some examples of the present disclosure relate to a non-transitory computer-readable storage medium storing one or more programs including instructions that, when executed by one or more processors of an electronic device in communication with a display and one or more input devices, cause the electronic device to perform a method comprising: within a three-dimensional environment, while a first portion of a virtual scene is displayed at a first area of ​​the three-dimensional environment and a second portion of the virtual scene is displayed at a second area of ​​the three-dimensional environment, detecting a first event via the one or more input devices, wherein the first event comprises a viewpoint of a user of the electronic device moving from the first viewpoint to the second viewpoint; and in response to detecting the first event, and based on determining that the viewpoint of the user satisfies one or more first criteria, maintaining display of the first portion of the virtual scene within the first area of ​​the three-dimensional environment, and replacing display of the second portion of the virtual scene with a representation of a first portion of a physical environment of the user.

[0109] Some examples of the present disclosure relate to an electronic device that includes one or more processors, a memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for executing a method as described herein.

[0110] Some examples of the present disclosure relate to a non-transitory computer-readable storage medium that stores one or more programs that include instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform a method as described herein.

[0111] Some examples of the present disclosure relate to an electronic device comprising one or more processors, a memory, and means for performing a method as described herein.

[0112] Some examples of the present disclosure relate to an information processing apparatus for use in an electronic device, the information processing apparatus comprising means for performing a method as described herein.

[0113] For the purpose of explanation, the foregoing description is described by reference to specific examples. However, the above illustrative discussion is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible based on the above teachings. The examples are selected and described in order to best illustrate the principles of the invention and its practical application, so as to enable others skilled in the art to best use the invention and the various described examples with various modifications suitable for the specific purpose contemplated.

Claims

1. A method comprising: At an electronic device in communication with a display and one or more input devices: within a three-dimensional environment, while a first portion of a virtual scene is displayed at a first area of ​​the three-dimensional environment and a second portion of the virtual scene is displayed at a second area of ​​the three-dimensional environment, detecting a first event via the one or more input devices, wherein the first event comprises a viewpoint of a user of the electronic device moving from a first viewpoint to a second viewpoint; as well as In response to detecting the first event, and in accordance with determining that the viewpoint of the user satisfies one or more first criteria: maintaining display of the first portion of the virtual scene within the first area of ​​the three-dimensional environment; and Display of the second portion of the virtual scene is replaced with a representation of a first portion of the user's physical environment.

2. The method according to claim 1, further comprising: In response to detecting the first event, and in accordance with determining that the viewpoint of the user does not satisfy the one or more first criteria: maintaining display of the first portion of the virtual scene within the first area of ​​the three-dimensional environment; and The second portion of the virtual scene remains displayed within the second area of ​​the three-dimensional environment.

3. A method according to claim 1, wherein the one or more first criteria include a criterion that is satisfied when the second viewpoint of the user corresponds to a boundary surrounding one or more portions of the virtual scene, the one or more portions of the virtual scene including the first portion of the virtual scene.

4. The method according to claim 1, further comprising: When the viewpoint of the user is the first viewpoint, the electronic device displays a first view of the virtual scene; as well as In response to the first event, the electronic device displays a second view of the virtual scene.

5. The method according to claim 1, further comprising, when displaying the virtual scene in the three-dimensional environment: A boundary between the first area of ​​the three-dimensional environment and the second area of ​​the three-dimensional environment is displayed via the display.

6. A method according to claim 5, wherein the boundary is displayed before the first event is detected and is displayed when the first part of the virtual scene is displayed at the first area of ​​the three-dimensional environment and when the second part of the virtual scene is displayed at the second area of ​​the three-dimensional environment.

7. The method according to claim 5, further comprising: detecting, via the one or more input devices, an indication of one or more inputs requesting a change in one or more dimensions of the bounds while displaying the second portion of the virtual scene including the representation of the first portion of the physical environment of the user; as well as In response to detecting the indication of the one or more inputs: updating the one or more dimensions of the boundary based on the indication of the one or more inputs; changing one or more dimensions of the first area of ​​the three-dimensional environment including the virtual scene according to the instruction; as well as One or more dimensions of the second area of ​​the three-dimensional environment including the representation of the physical environment of the user are altered in accordance with the indication.

8. The method according to claim 1, wherein the virtual scene is shared in a communication session between the electronic device and another electronic device different from the electronic device, the method further comprising: While the communication session is ongoing: obtaining, via the one or more input devices, an indication of a request to modify virtual content included in the virtual scene; In response to obtaining the indication of the request to modify the virtual content: modifying the virtual content in accordance with the indication of the request to modify the virtual content based on determining that one or more second criteria are satisfied; as well as Based on determining that the one or more second criteria are not satisfied, the modification of the virtual content is abandoned.

9. The method according to claim 8, further comprising: When the first portion of the user's physical environment is visible, a representation of a participant corresponding to the other electronic device is displayed via the display at a location within the first portion of the virtual scene, wherein the one or more second criteria include a criterion that is satisfied when the location of the representation of the participant is different from one or more locations corresponding to the location where the virtual content is modified.

10. The method according to claim 1, wherein the virtual scene is shared in a communication session between the electronic device and another electronic device different from the electronic device, the method further comprising: While the communication session is ongoing: In response to determining that one or more characteristics of the other electronic device satisfy one or more second criteria, including within the three-dimensional environment a representation of a participant of the communication session using the other electronic device; and Based on determining that the one or more characteristics of the other electronic devices do not satisfy the one or more second criteria, including the representation of the participant within the three-dimensional environment is foregone. 11 . The method according to claim 10 , wherein when the other electronic device is a head-mounted electronic device, the one or more characteristics of the other electronic device satisfy the one or more second criteria.

12. The method of claim 10, wherein the one or more characteristics of the other electronic device do not satisfy the one or more second criteria when the other electronic device is any one of a laptop computing device, a mobile handset, a tablet computing device, and a desktop computing device.

13. The method of claim 1, wherein the representation of the physical environment of the user is displayed to consume a portion of a viewport of the electronic device.

14. The method of claim 1, wherein the first event comprises: When the viewpoint of the user changes from the first viewpoint to the second viewpoint at a rate greater than a threshold rate, the movement of the viewpoint satisfies the one or more first criteria.

15. The method of claim 1, wherein the first portion of the physical environment of the user corresponds to a corresponding portion of the three-dimensional environment within which one or more representations of physical individuals are displayed.

16. The method according to claim 15, further comprising: Prior to detecting the first event and while the viewpoint of the user is the first viewpoint, displaying, via the display, a visual indication corresponding to the first portion of the physical environment of the user.

17. The method according to claim 15, further comprising: When the first portion of the user's physical environment is visible, in response to the first event and based on the determination that the user's viewpoint satisfies the one or more first criteria, one or more representations of physical individuals within the representation of the first portion of the user's physical environment are displayed via the display.

18. The method according to claim 15, further comprising: Prior to detecting the first event, while the first portion of the virtual scene is displayed at the first area of ​​the three-dimensional environment, and while the second portion of the virtual scene is displayed at the second area of ​​the three-dimensional environment, detecting a second event via the one or more input devices, the second event comprising a user input directed to the second portion of the virtual scene; as well as In response to detecting the second event, display of the second portion of the virtual scene is replaced with the representation of the first portion of the physical environment of the user.

19. An electronic device in communication with a display and one or more input devices, the electronic device comprising: one or more processors; Memory; and One or more programs, the one or more programs are stored in the memory, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs comprising instructions for executing the method according to any one of claims 1 to 18.

20. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device that communicates with a display and one or more input devices, cause the electronic device to perform a method according to any one of claims 1 to 18.

Images