System and method for rendering techniques for virtual platforms and scenes

Abstract

Some examples of the present disclosure relate to systems and methods for presenting virtual platforms and related virtual content from a virtual scene. In some examples, an electronic device displays virtual content using a first technique at a region corresponding to a virtual platform within a three-dimensional environment. In some examples, the electronic device displays the virtual content using a second technique at an area outside of the virtual platform.

Description

Cross-references to related applications

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 700,387, filed September 27, 2024, and U.S. Patent Application No. 19 / 303,092, filed August 18, 2025, the contents of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0002] This disclosure relates generally to systems and methods for presenting virtual three-dimensional environments, and more specifically to the presentation of virtual platforms and related virtual content from virtual scenes. Background Technology

[0003] Some computer graphics environments provide two-dimensional and / or three-dimensional environments, where at least some of the objects presented for the user to view are virtual and computer-generated. In some examples, the virtual three-dimensional environment may be based on one or more images of the computer's physical environment. In some examples, the virtual three-dimensional environment does not include images of the computer's physical environment. Summary of the Invention

[0004] This disclosure relates generally to systems and methods for presenting virtual three-dimensional environments, and more specifically to the presentation of a virtual platform and associated virtual content from a virtual scene. In some examples, an electronic device displays a virtual scene. In some examples, an electronic device displays a virtual platform within a three-dimensional environment. In some examples, a first rendering technique is used to render a first portion of the virtual scene, and the first portion is displayed within the virtual platform. In some examples, a second rendering technique, different from the first rendering technique, is used to render a second portion of the virtual scene outside the virtual platform. In some examples, rendering techniques are implemented to generate a projected image virtual background. In some examples, rendering techniques are implemented to generate a virtual hologram. In some examples, rendering techniques are implemented to present two-dimensional images corresponding to areas within and around the virtual platform. In some examples, rendering techniques are implemented to generate multi-planar images corresponding to the virtual background.

[0005] A full description of these examples is provided in the accompanying drawings and detailed embodiments, and it should be understood that the content of this invention does not limit the scope of this disclosure in any way. Attached Figure Description

[0006] To better understand the various examples described herein, reference should be made to the following detailed embodiments and accompanying drawings. Throughout the drawings, similar reference numerals generally refer to corresponding parts.

[0007] Figure 1 Examples of electronic devices that present extended real-world environments according to some examples of this disclosure are illustrated.

[0008] Figures 2A to 2B A block diagram illustrating an example architecture for an electronic device according to some examples of this disclosure is shown.

[0009] Figures 3A to 3D Examples of electronic devices using a virtual platform to present virtual scenes are illustrated according to some examples of this disclosure.

[0010] Figure 4 An electronic device for displaying a projected virtual background is illustrated according to some examples of this disclosure.

[0011] Figures 5A to 5B Examples of electronic devices displaying virtual backgrounds according to some examples of this disclosure are shown.

[0012] Figure 6 Examples of generating virtual backgrounds using two-dimensional images according to this disclosure are illustrated.

[0013] Figure 7 An example of an electronic device for generating multiplanar images according to the present disclosure is illustrated.

[0014] Figure 8 This is a flowchart of a method for presenting a virtual scene using a virtual platform, based on some examples of this disclosure. Detailed Implementation

[0015] This disclosure relates generally to systems and methods for presenting virtual three-dimensional environments, and more specifically to the presentation of a virtual platform and associated virtual content from a virtual scene. In some examples, an electronic device displays a virtual scene. In some examples, an electronic device displays a virtual platform within a three-dimensional environment. In some examples, a first rendering technique is used to render a first portion of the virtual scene, and the first portion is displayed within the virtual platform. In some examples, a second rendering technique, different from the first rendering technique, is used to render a second portion of the virtual scene outside the virtual platform. In some examples, rendering techniques are implemented to generate a projected image virtual background. In some examples, rendering techniques are implemented to generate a virtual hologram. In some examples, rendering techniques are implemented to present two-dimensional images corresponding to areas within and around the virtual platform. In some examples, rendering techniques are implemented to generate multi-planar images corresponding to the virtual background.

[0016] In some examples, a 3D object is displayed in a computer-generated 3D environment with a specific orientation that controls one or more behaviors of the 3D object (e.g., when the 3D object moves within the 3D environment). In some examples, the orientation of the 3D object displayed in the 3D environment is selected by the user of the electronic device or automatically by the electronic device. For example, when initiating the rendering of a 3D object in a 3D environment, the user may select a specific orientation of the 3D object, or the electronic device may automatically select the orientation of the 3D object (e.g., based on the type of the 3D object).

[0017] In some examples, 3D objects may be displayed in a 3D environment with world-locked orientation, body-locked orientation, tilt-locked orientation, or head-locked orientation, as described below. As used herein, an object displayed in a 3D environment with body-locked orientation has a distance and orientation offset relative to a portion of the user's body (e.g., the user's torso). Alternatively, in some examples, body-locked objects have a fixed distance from the user, and the orientation of the content does not reference any part of the user's body (e.g., it may be displayed in the same basic direction relative to the user, regardless of head and / or body movement). Additionally or alternatively, in some examples, body-locked objects may be configured to always maintain gravity or horizon (e.g., perpendicular to gravity) alignment, such that changes in the head and / or body's scrolling direction will not cause the body-locked object to move within the 3D environment. Conversely, translational movement in either configuration will cause the body-locked object to reposition within the 3D environment to maintain the distance offset.

[0018] As used in this article, objects displayed in a head-locked orientation within a 3D environment have a distance and orientation offset relative to the user's head. In some examples, the head-locked object moves within the 3D environment as the user's head moves (when the user's viewpoint changes).

[0019] As used in this article, objects displayed in a world-locked orientation in a 3D environment do not have a distance or orientation offset relative to the user.

[0020] As used herein, an object displayed in a tilt-locked orientation in a 3D environment (referred to herein as a tilt-locked object) has a distance offset relative to a user (such as a part of the user's body, e.g., the user's torso, or the user's head). In some examples, the tilt-locked object is displayed in a fixed orientation relative to the 3D environment. In some examples, the tilt-locked object moves according to a polar (e.g., spherical) coordinate system centered on a pole passing through the user (e.g., the user's head). For example, the tilt-locked object moves in the 3D environment based on the movement of the user's head within a spherical space surrounding the user's head (e.g., centered on the user's head). Therefore, if the user tilts their head relative to gravity (e.g., up or down in the pitch direction), the tilt-locked object will follow the head tilt and move radially along the sphere, such that the tilt-locked object is repositioned in the 3D environment with the same distance offset relative to the user as before the head tilt, while optionally maintaining the same orientation relative to the 3D environment. In some examples, if the user moves their head relative to gravity in a rolling direction (e.g., clockwise or counterclockwise), the tilt-locked object is not repositioned in the 3D environment.

[0021] Figure 1 An electronic device 101 is illustrated according to some examples of this disclosure, which presents a three-dimensional environment (e.g., an extended reality (XR) environment or a computer-generated reality (CGR) environment that optionally includes representations of physical and / or virtual objects). In some examples, such as Figure 1 As shown, electronic device 101 is a head-mounted display or other head-mountable device configured to be worn on the head of a user of electronic device 101. See below for reference. Figure 2A An example of an architecture block diagram to describe electronic device 101. For example... Figure 1 As shown, electronic device 101 and table 106 are located in a physical environment. The physical environment may include physical features such as physical surfaces (e.g., floor, wall) or physical objects (e.g., table, lamp, etc.). In some examples, electronic device 101 may be configured to detect and / or capture images of the physical environment including table 106 (exemplified in the field of view of electronic device 101).

[0022] In some examples, such as Figure 1 As shown, electronic device 101 includes one or more internal image sensors 114a oriented toward the user's face (e.g., referred to below). Figures 2A to 2B(The described eye-tracking camera). In some examples, an internal image sensor 114a is used for eye tracking (e.g., detecting the user's gaze). The internal image sensor 114a is optionally arranged on the left and right portions of the display 120 to enable eye tracking of the user's left and right eyes. In some examples, the electronic device 101 also includes external image sensors 114b and 114c facing outwards from the user to detect and / or capture the physical environment of the electronic device 101 and / or movement of the user's hands or other body parts.

[0023] In some examples, display 120 has a field of view visible to the user. In some examples, the user-visible field of view is the same as the field of view of external image sensors 114b and 114c. For example, when display 120 is optionally part of a head-mounted device, the field of view of display 120 may be the same as or similar to the field of view of the user's eyes. In some examples, the user-visible field of view is different from the field of view of external image sensors 114b and 114c (e.g., narrower than the field of view of external image sensors 114b and 114c). In other examples, the field of view of display 120 may be smaller than the field of view of the user's eyes. The user's viewpoint determines what is visible in the field of view, and the viewpoint typically specifies the position and orientation relative to the three-dimensional environment. As the user's viewpoint shifts, the field of view of the three-dimensional environment also shifts accordingly. In some examples, electronic device 101 may be an optically transparent device, through which display 120 is a transparent or translucent display through which parts of the physical environment can be directly viewed. In some examples, display 120 may be included within a transparent lens and may overlap entirely or only partially with the transparent lens. In other examples, the electronics may be a video pass-through device, wherein display 120 is an opaque display configured to display images of the physical environment captured by external image sensors 114b and 114c. Although Figure 1 A single display is shown, but it should be understood that display 120 may optionally include more than one display. For example, display 120 may optionally include displays that are merged (e.g., merged by the user's brain) to create Figure 1 The view of the content shown is displayed in a stereoscopic pair on the monitor (e.g., a left display panel and a right display panel for the user's left and right eyes, respectively). In some examples, see the following references. Figures 2A to 2B In more detail, the display 120 includes or corresponds to a transparent or translucent surface (e.g., a lens) that is not equipped with display capabilities (e.g., and therefore cannot generate and display the virtual object 104), and alternatively presents a direct view of the physical environment in the user's field of view (e.g., the user's eye's field of view).

[0024] In some examples, electronic device 101 is configured to display (e.g., in response to a trigger) virtual object 104 in a three-dimensional environment. Virtual object 104 is... Figure 1 The cube illustrated represents a cube that does not exist in the physical environment but is displayed in a three-dimensional environment positioned on top of table 106 (e.g., a real-world table or a representation thereof). Optionally, in response to detecting a flat surface of table 106 in the physical environment 100, virtual object 104 is displayed on the surface of table 106 in the three-dimensional environment displayed via display 120 of electronic device 101.

[0025] It should be understood that virtual object 104 is a representative virtual object and may include and render one or more different virtual objects (e.g., virtual objects with various dimensions, such as two-dimensional or other three-dimensional virtual objects) in a three-dimensional environment. For example, a virtual object may represent an application or user interface displayed in a three-dimensional environment. In some examples, a virtual object may represent content corresponding to an application and / or displayed via a user interface in a three-dimensional environment. In some examples, virtual object 104 may optionally be configured to be interactive and responsive to user input (e.g., air gestures, such as air pinch gestures, air tap gestures, and / or air touch gestures), allowing the user to virtually touch, tap, move, rotate, or otherwise interact with virtual object 104.

[0026] As discussed in this article, by the user (e.g., using Figure 1 One or more air pinch gestures performed by the hand (103) are detected by one or more input devices of the electronic device 101 and interpreted as one or more user inputs pointing to content displayed by the electronic device 101. Additionally or alternatively, in some examples, one or more user inputs interpreted by the electronic device 101 as pointing to content displayed by the electronic device 101 (e.g., virtual object 104) are detected via one or more hardware input devices (e.g., controllers, touchpads, proximity sensors, buttons, sliders, knobs, etc.) rather than via one or more input devices configured to detect air gestures (such as one or more air pinch gestures) performed by the user. This description is intended to be exemplary and not restrictive; the user may optionally use different air gestures and / or other forms of input to provide user input.

[0027] In some examples, electronic device 101 may be configured to communicate with a second electronic device, such as a companion device. For example, as Figure 1As illustrated, electronic device 101 may optionally communicate with electronic device 160. In some examples, electronic device 160 corresponds to a mobile electronic device, such as a smartphone, tablet computer, smartwatch, laptop computer, or other electronic device. In some examples, electronic device 160 corresponds to a non-mobile electronic device, which is typically stationary and not easily moved within a physical environment (e.g., a desktop computer, server, etc.). See below for reference. Figure 2B The following are additional examples of architectural diagrams used to describe electronic device 160. In some examples, electronic device 101 and electronic device 160 are associated with the same user. For example, in Figure 1 In this configuration, electronic device 101 may be positioned (e.g., mounted on) a user's head, and electronic device 160 may be positioned near electronic device 101, such as in the user's hand 103 (e.g., hand 103 is holding electronic device 160), in the user's pocket or bag, or on a surface near the user. Electronic devices 101 and 160 may optionally be associated with the same user account (e.g., the user's account logged into both electronic devices 101 and 160). See below for further details. Figures 2A to 2B Additional details regarding the communication between electronic device 101 and electronic device 160 are provided.

[0028] In some examples, the display of an object in a 3D environment is triggered by, or enables, interaction with one or more user interface objects in the 3D environment. For example, initiating the display of an object in a 3D environment may include interaction with one or more virtual option / power representations displayed in the 3D environment. In some examples, when initiating the display of an object in a 3D environment, the electronic device may track the user's gaze as input for identifying one or more virtual option / power representations as a target for selection. For example, a gaze may be used to identify one or more virtual option / power representations as a target for selection using another selection input. In some examples, a virtual option / power representation may be selected using hand-tracking input detected via an input device communicating with the electronic device. In some examples, an object displayed in a 3D environment may move and / or reorient itself in the 3D environment based on movement input detected via an input device.

[0029] In the following description, an electronic device that communicates with one or more displays and one or more input devices is described. It should be understood that the electronic device may optionally communicate with one or more other physical user interface devices, such as touch-sensitive surfaces, physical keyboards, mice, joysticks, hand-tracking devices, eye-tracking devices, styluses, etc. Furthermore, as described above, it should be understood that the described electronic device, display, and touch-sensitive surface may optionally be distributed among two or more devices. Therefore, as used in this disclosure, information on or displayed by the electronic device may optionally be used to describe information output by the electronic device for display on a separate display device (touch-sensitive or non-touch-sensitive). Similarly, as used in this disclosure, input received on the electronic device (e.g., touch input received on a touch-sensitive surface of the electronic device, or touch input received on the surface of a stylus) may optionally be used to describe input received on a separate input device from which the electronic device receives input information.

[0030] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, TV channel browsing applications, and / or digital video player applications.

[0031] Figures 2A to 2B Block diagrams illustrating example architectures for electronic devices according to some examples of this disclosure are shown. In some examples, electronic device 201 and / or device 260 include one or more electronic devices. For example, electronic device 201 may be a portable device, an auxiliary device for communicating with another device, a head-mounted display, a head-mounted speaker, etc. In some examples, electronic device 201 corresponds to the above reference. Figure 1 The described electronic device 101. In some examples, electronic device 260 corresponds to the above reference. Figure 1 The described electronic device 160.

[0032] like Figure 2A As illustrated, electronic device 201 may optionally include one or more sensors, such as one or more hand tracking sensors 202, one or more position sensors 204A, and one or more image sensors 206A (optionally corresponding to...). Figure 1The electronic device 201 may include an internal image sensor 114a and / or external image sensors 114b and 114c, one or more touch-sensitive surfaces 209A, one or more motion and / or orientation sensors 210A, one or more eye-tracking sensors 212, one or more microphones 213A or other audio sensors, one or more body tracking sensors (e.g., torso tracking sensors and / or head tracking sensors), etc. The electronic device 201 may optionally include one or more output devices, such as one or more display generation components 214A (optionally corresponding to...). Figure 1 The electronic device 201 may include a display 120, one or more speakers 216A, one or more tactile output devices (not shown), etc. The electronic device 201 may optionally include one or more processors 218A, one or more memories 220A, and / or communication circuitry 222A. One or more communication buses 208A may optionally be used for communication between the components of the electronic device 201 mentioned above.

[0033] Additionally, electronic device 260 may optionally include components that are the same as or similar to those of electronic device 201. For example, such as Figure 2B As shown, the electronic device 260 optionally includes one or more position sensors 204B, one or more image sensors 206B, one or more touch-sensitive surfaces 209B, one or more orientation sensors 210B, one or more microphones 213B, one or more display generating components 214B, one or more speakers 216B, one or more processors 218B, one or more memories 220B, and / or communication circuitry 222B. One or more communication buses 208B are optionally used for communication between the components of the electronic device 260 mentioned above.

[0034] Electronic devices 201 and 260 are optionally configured to communicate via a wired or wireless connection between the two electronic devices (e.g., via communication circuits 222A, 222B). For example, as Figure 2A As indicated, electronic device 260 can be used as an accessory device to electronic device 201. For example, in some examples, electronic device 260 processes sensor inputs from electronic devices 201 and 260, and / or uses the display generation component 214A of electronic device 201 to generate content for display.

[0035] Communication circuits 222A and 222B optionally include circuitry for communicating with electronic devices and networks such as the Internet, intranets, wired and / or wireless networks, cellular networks, and wireless local area networks (LANs). Communication circuits 222A and 222B optionally include circuitry for using near-field communication (NFC) and / or short-range communication (such as Bluetooth). ®The communication circuits 222A and 222B communicate with each other. In some examples, the communication circuits 222A and 222B include or support Wi-Fi (e.g., the 802.11 protocol), Ethernet, Ultra Wideband (“UWB”), high-frequency systems (e.g., 900MHz, 2.4GHz, and 5.6GHz communication systems) or any other communication protocol or any combination thereof.

[0036] One or more processors 218A, 218B include one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, one or more processors 218A, 218B include one or more microprocessors, one or more central processing units, one or more application-specific integrated circuits, one or more field-programmable gate arrays, one or more programmable logic devices, or combinations of such devices. In some examples, memories 220A and / or 220B are non-transitory computer-readable storage media (e.g., flash memory, random access memory, or other volatile or non-volatile memory or storage device) that store computer-readable instructions configured to be executed by one or more processors 218A, 218B to perform the techniques, processes, and / or methods described herein. In some examples, memories 220A and / or 220B may include more than one non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium can be any medium (e.g., excluding signals) that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, and device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on compact disc (CD), digital versatile optical disc (DVD), or Blu-ray technology, and persistent solid-state storage devices such as flash memory, solid-state drives, etc.

[0037] In some examples, one or more display generating components 214A, 214B include a single display (e.g., a liquid crystal display (LCD), an organic light-emitting diode (OLED), or other type of display). In some examples, one or more display generating components 214A, 214B include multiple displays. In some examples, one or more display generating components 214A, 214B may include a touch-enabled display (e.g., a touchscreen), a projector, a holographic projector, a retinal projector, a transparent or translucent display, etc. In some examples, the electronic device does not include one or more display generating components 214A or 214B. For example, instead of one or more display generating components 214A or 214B, some electronic devices include transparent or translucent lenses or other surfaces that are not configured to display or present virtual content. However, it should be understood that in such instances, electronic device 201 and / or electronic device 260 may optionally be equipped with Figure 2A and Figure 2B One or more of the other components illustrated and described herein, such as one or more hand-tracking sensors 202, one or more eye-tracking sensors 212, one or more image sensors 206A, and / or one or more motion and / or orientation sensors 210A. Alternatively, in some examples, one or more display generation components 214A or 214B are provided separately from electronic devices 201 and / or 260. For example, one or more display generation components 214A, 214B communicate with electronic device 201 (and / or electronic device 260) but are not integrated with electronic device 201 and / or electronic device 260 (e.g., within the housing of electronic devices 201, 260). In some examples, electronic devices 201 and 260 respectively include one or more touch-sensitive surfaces 209A and 209B for receiving user input, such as tap input and swipe input or other gestures (e.g., hand-based gestures or finger-based gestures)). In some examples, one or more display generating components 214A, 214B and one or more touch-sensitive surfaces 209A, 209B form one or more touch-sensitive displays (e.g., touchscreens integrated with each of the electronic devices 201 and 260 or touchscreens external to each of the electronic devices 201 and 260 that communicate with each of the electronic devices 201 and 260).

[0038] Electronic devices 201 and 260 optionally include one or more image sensors 206A and 206B, respectively. The one or more image sensors 206A and 206B optionally include one or more visible light image sensors (such as charge-coupled device (CCD) sensors) and / or complementary metal-oxide-semiconductor (CMOS) sensors operable to obtain images of physical objects from a real-world environment. The one or more image sensors 206A and 206B also optionally include one or more infrared (IR) sensors, such as passive or active IR sensors, for detecting infrared light from the real-world environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real-world environment. The one or more image sensors 206A and 206B also optionally include one or more cameras configured to capture movement of physical objects in the real-world environment. The one or more image sensors 206A and 206B also optionally include one or more depth sensors configured to detect the distance of the physical object from the electronic devices 201 and 260. In some examples, information from one or more depth sensors allows a device to identify objects in a real-world environment and distinguish them from other objects in the real-world environment. In some examples, one or more depth sensors allow a device to determine the texture and / or shape of objects in a real-world environment. In some examples, one or more image sensors 206A or 206B are included in an electronic device different from electronic devices 201 and / or 260. For example, one or more image sensors 206A, 206B communicate with electronic devices 201, 260 but are not integrated with electronic devices 201, 260 (e.g., within the housing of electronic devices 201, 260). Specifically, in some examples, one or more cameras of one or more image sensors 206A, 206B are integrated with and / or coupled to one or more devices separate from electronic devices 201 and / or 260 (e.g., but communicating with electronic devices 201 and / or 260), such as one or more input and / or output devices (e.g., one or more speakers and / or one or more microphones, such as earpieces or headphones) that include one or more image sensors 206A, 206B. In some examples, electronic device 201 or electronic device 260 corresponds to a headphone speaker (e.g., headphones or earbuds). In such instances, electronic device 201 or electronic device 260 is equipped with Figure 2A and Figure 2B A subset of other components illustrated and described herein. In some such examples, electronic device 201 or electronic device 260 is equipped with one or more image sensors 206A, 206B, one or more motion and / or orientation sensors 210A, 210B, and / or speakers 216A, 216B.

[0039] In some examples, electronic devices 201 and 260 combine a CCD sensor, an event camera, and a depth sensor to detect the physical environment surrounding them. In some examples, one or more image sensors 206A and 206B include a first image sensor and a second image sensor. The first and second image sensors work cooperatively and are optionally configured to capture different information about physical objects in the real-world environment. In some examples, the first image sensor is a visible light image sensor, and the second image sensor is a depth sensor. In some examples, electronic devices 201 and 260 use one or more image sensors 206A and 206B to detect the location and orientation of electronic devices 201 and 260 and / or one or more display generation components 214A and 214B in the real-world environment. For example, electronic devices 201 and 260 use one or more image sensors 206A and 206B to track the location and orientation of one or more display generation components 214A and 214B relative to one or more stationary objects in the real-world environment.

[0040] In some examples, electronic devices 201 and 260 include one or more microphones 213A and 213B or other audio sensors, respectively. Electronic devices 201 and 260 may optionally use one or more microphones 213A and 213B to detect sound from a user and / or the user's real-world environment. In some examples, the one or more microphones 213A and 213B include microphone arrays (e.g., multiple microphones) that optionally operate in cooperation to identify ambient noise or locate sound sources in the space of a real-world environment.

[0041] Electronic devices 201 and 260 each include one or more position sensors 204A and 204B, which are used to detect the positions of electronic device 201 and / or one or more display generating components 214A and the positions of electronic device 260 and / or one or more display generating components 214B, respectively. For example, the one or more position sensors 204A, 204B may include a Global Positioning System (GPS) receiver that receives data from one or more satellites and allows electronic devices 201, 260 to determine the absolute position of the electronic devices in the physical world.

[0042] Electronic devices 201 and 260 each include one or more orientation sensors 210A and 210B, which are used to detect the orientation and / or movement of electronic device 201 and / or one or more display generation components 214A and the orientation and / or movement of electronic device 260 and / or one or more display generation components 214B, respectively. For example, electronic devices 201 and 260 use one or more orientation sensors 210A and 210B to track changes in the positioning and / or orientation of electronic devices 201 and 260 and / or one or more display generation components 214A and 214B, such as relative to physical objects in a real-world environment. The one or more orientation sensors 210A and 210B may optionally include one or more gyroscopes and / or one or more accelerometers.

[0043] In some examples, electronic device 201 includes one or more hand tracking sensors 202 and / or one or more eye tracking sensors 212. It should be understood that although referred to as hand tracking sensors or eye tracking sensors, electronic device 201 additionally or optionally includes one or more other body tracking sensors, such as one or more leg tracking sensors, one or more torso tracking sensors, and / or one or more head tracking sensors. One or more hand tracking sensors 202 are configured to track the localization and / or position of one or more portions of a user's hand, and / or the movement of one or more portions of the user's hand relative to a three-dimensional environment, relative to one or more display generation components 214A, and / or relative to another defined coordinate system. One or more eye tracking sensors 212 are configured to track the localization and movement of a user's gaze (e.g., the user's attention, more generally including the eyes, face, or head) relative to the real-world environment or a three-dimensional environment and / or relative to one or more display generation components 214A. In some examples, one or more hand tracking sensors 202 and / or one or more eye tracking sensors 212 are implemented together with one or more display generation components 214A. In some examples, one or more hand-tracking sensors 202 and / or one or more eye-tracking sensors 212 are implemented separately from one or more display generation components 214A. In some examples, the electronic device 201 alternatively does not include one or more hand-tracking sensors 202 and / or one or more eye-tracking sensors 212. In some such examples, the electronic device 260 may utilize one or more display generation components 214A to provide a three-dimensional environment, and the electronic device 260 may use input and other data collected via other one or more sensors of the electronic device 201 (e.g., one or more position sensors 204A, one or more image sensors 206A, one or more touch-sensitive surfaces 209A, one or more motion and / or orientation sensors 210A and / or one or more microphones 213A or other audio sensors) as input and data processed by one or more processors 218B of the electronic device 260. Additionally or alternatively, the electronic device 260 may optionally not include... Figure 2B Other components shown include one or more position sensors 204B, one or more image sensors 206B, one or more touch-sensitive surfaces 209B, etc. In some such examples, the electronic device 260 may utilize one or more display generation components 214A to provide a three-dimensional environment, and the electronic device 260 may use input and other data collected via one or more motion and / or orientation sensors 210A (and / or one or more microphones 213A) of the electronic device 201 as input.

[0044] In some examples, one or more hand-tracking sensors 202 (and / or other body-tracking sensors, such as leg-tracking sensors, torso-tracking sensors, and / or head-tracking sensors) may use one or more image sensors 206 (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that capture 3D information from the real world, including one or more body parts (e.g., a human user's hand, leg, or torso). In some examples, sufficient resolution is available to distinguish the hand to differentiate the fingers and their corresponding positions. In some examples, one or more image sensors 206A are positioned relative to the user to define a field of view and interaction space for one or more image sensors 206A, in which the finger / hand positions, orientations, and / or movements captured by the image sensors are used as input (e.g., to differentiate from the user's resting hand or other hands of other people in the real-world environment). Tracking the fingers / hands used for input (e.g., gestures, touches, taps, etc.) may be advantageous because it does not require the user to touch, hold, or wear any type of beacon, sensor, or other marker.

[0045] In some examples, one or more eye-tracking sensors 212 include at least one eye-tracking camera (e.g., an IR camera) and / or an illumination source (e.g., an IR light source, such as an LED) that emits light toward the user's eyes. The eye-tracking camera may be pointed at the user's eyes to receive reflected IR light from the light source directly or indirectly from the eyes. In some examples, both eyes are tracked separately by the respective eye-tracking camera and illumination source, and focus / gaze can be determined by tracking both eyes. In some examples, one eye (e.g., the dominant eye) is tracked by one or more respective eye-tracking cameras / illumination sources.

[0046] Electronic devices 201 and 260 are not limited to Figures 2A to 2B The components and configurations may include, but are not limited to, a few components, other components, or additional components in various configurations. In some examples, electronic device 201 and / or electronic device 260 may be implemented in various ways among multiple electronic devices (e.g., as a system). In some such examples, each electronic device (or more electronic devices) in the electronic device may include one or more of the same components discussed above, such as various sensors, one or more display generation components, one or more speakers, one or more processors, one or more memories, and / or communication circuitry. One or more persons using electronic device 201 and / or electronic device 260 may optionally be referred to herein as one or more users of the device.

[0047] Some examples of this disclosure relate to electronic devices and / or computer systems configured to convey information for interaction with virtual scenes. Virtual scenes may include virtual content and / or other information rendered on a specific device for viewing and / or interaction, such as virtual campsites, virtual offices, virtual meadows, virtual towns, etc. In some examples, virtual scenes may include user- and / or computer-generated assets, such as virtual floors, skies, object groupings, and / or metadata associated with such assets.

[0048] In some examples, the virtual scene can be displayed on the device. When the virtual scene is displayed, the device's users can view, edit, and / or share comments about the content of the virtual scene. In some examples, the display device presents the virtual scene in an extended reality (XR), virtual reality (VR), and / or mixed reality (XR) environment that includes a portion of the virtual scene. In some examples, a virtual platform, as further described herein, is used to display the virtual scene. In some examples, the virtual scene is displayed as included in a user interface used for creating content. For example, the user interface can be used to facilitate the editing of a Universal Scene Description (USD) file and the virtual assets included in the USD file. The user interface and / or the virtual platform can provide the device's users with the ability to inspect the virtual scene, comment on the virtual scene, and / or quickly collaborate with other users on other devices during the inspection of the virtual scene.

[0049] In some examples, the display device may establish a virtual platform on which virtual content from a virtual scene is displayed, and / or within the virtual platform. In some examples, the virtual content within the virtual platform is displayed at a first level of detail, including the resolution, appearance, simulated lighting, and / or some combination thereof of the virtual assets displayed within the platform. In some examples, the display device additionally or alternatively displays virtual content of relatively low visual importance included in the virtual scene, such as far-field background virtual assets and / or a virtual sky, at a second level of detail, which is smaller than or different from the first level of detail. The examples in this document illustrate several operations related to how virtual scenes are presented using virtual platforms and how a user of the device can interact with the virtual scene when creating a user interface that includes content from the virtual platform.

[0050] Figure 3A An electronic device 101 is illustrated using a virtual platform to present a virtual scene according to some examples of this disclosure. In some examples, the electronic device 101 has features similar to those described above. Figure 1 The described electronic device 101 and / or the electronic device 201 described above with reference to Figure 2 have the same architecture.

[0051] In some examples, electronic device 101 may be a first electronic device used by a first user 308 to display a user interface for viewing and interacting with virtual content and / or accessing and participating in a communication session. For example, electronic device 101 may be a mobile device (e.g., a tablet, smartphone, media player, or wearable device) or a computer or other electronic device. In some examples, the display generating component is a display integrated with the electronic device (optionally a touchscreen display), an external display such as a monitor, projector, television, and / or a hardware component (optionally integrated or external) for projecting the user interface or making the user interface visible to one or more users. In some examples, one or more input devices include electronic devices or components capable of receiving user input (e.g., capturing user input, detecting user input) and sending information associated with the user input to the electronic device. Examples of input devices include touchscreens, mice (e.g., external), trackpads (optionally integrated or external), touchpads (optionally integrated or external), remote control devices (e.g., external), another mobile device (e.g., separate from the electronic device), handheld devices (e.g., external), controllers (e.g., external), cameras, depth sensors, eye-tracking devices, and / or motion sensors (e.g., hand-tracking devices, hand motion sensors). In some examples, the electronic device communicates with a hand-tracking device (e.g., one or more cameras, depth sensors, proximity sensors, touch sensors (e.g., touchscreens or trackpads)). In some examples, the hand-tracking device is a wearable device, such as a smart glove. In some examples, the hand-tracking device is a handheld input device, such as a remote control or stylus.

[0052] In some examples, computer system 312 sends and / or receives (e.g., streaming) information, such as data. In some examples, computer system 312 includes some or all of the circuitry of electronic device 101 and / or electronic device 201 (e.g., referenced). Figure 1 (As described in Figure 2). In some examples, electronic device 101 and computer system 312 are different types of devices. For example, computer system 312 may be a desktop or laptop computer, and electronic device 101 may be a wearable device, such as a head-mounted device. In some examples, computer system 312 uses one or more data formats, such as JavaScript Object Notation (JSON), Extensible Markup Language (XML), and / or Graphics Library Send Format (GLTF), to stream data. In some examples, computer system and / or electronic device 101 use the streamed data to render and / or otherwise represent scene graphs, object models, animation data, and other graphics-related information.

[0053] In some examples, electronic device 101 communicates with computer system 312 using one or more protocols, such as UDP (User Datagram Protocol) and / or TCP / IP (Transmission Control Protocol / Internet Protocol), to send data packets to and / or receive data packets from computer system 312. In some examples, computer system 312 may host and / or manage XR applications used by electronic device 101 to render the displayed virtual content. For example, the computer system may implement a client-server architecture, where a central computing unit (e.g., computer system 312) manages the state of the virtual environment and delivers updates to connected clients (e.g., electronic device 101 and additional or alternative devices).

[0054] In some examples, computer system 312 displays virtual content with a level of detail (LOD) based on the distance between the user's viewpoint and the virtual content. LOD technology may include electronic device 101 altering the displayed level of detail, which may include geometric level of detail (e.g., the number, shape, and / or arrangement of polygons representing the virtual content), the resolution of virtual textures covering and / or included in the virtual content, and / or applying simulated lighting effects to the virtual content. Generally, computer system 312 may increase the visual fidelity and / or realism of virtual content near the viewpoint of electronic device 101, and / or decrease visual fidelity, and / or abstract virtual content away from the viewpoint.

[0055] It is understood that, as further described herein, electronic device 101 and / or computer system 312 may additionally or alternatively alter the level of detail and / or additional or alternative visual characteristics of the virtual content to reduce the computational load required to render the virtual content. In some examples, the computational load is reduced by having electronic device 101 display first virtual content corresponding to a virtual platform area at a first level of detail and display second virtual content different from the first virtual content at a second level of detail. As an example, electronic device 101 may display virtual objects including virtual floors, virtual trees, virtual cars, virtual roads, etc., at a first visual fidelity level, such that the virtual objects rendered in the virtual platform area are highly detailed for the user's inspection during inspection. Simultaneously, electronic device 101 may display virtual background content that may include one or more of the aforementioned virtual objects at a second visual fidelity level, which may be lower than the first visual fidelity level. For example, the background virtual content may be rendered with a smaller number of polygons, may be rendered without simulated lighting effects and / or with less subtle simulated lighting effects, and / or may be rendered at a lower resolution level than the virtual content on the virtual platform.

[0056] In some examples, the three-dimensional environment is generated, displayed, or otherwise made viewable through a device (e.g., a computer-generated reality (CGR) environment, such as a virtual reality (VR) environment, a mixed reality (MR) environment, and / or an augmented reality (AR) environment). For example, the three-dimensional environment 302 may include the physical and / or virtual environment of the user 308 and the electronic device 101.

[0057] As described above, electronic device 101 can present a virtual scene in an area corresponding to platform 306. As described herein, the virtual platform can be part of one or more portions of a three-dimensional environment 302 in which the virtual scene is displayed. For example, as... Figure 3A As shown, platform 306 is mapped to a physical region within 302, indicated by a top-down view of the three-dimensional environment 302, as illustrated by glyph 310. As illustrated by display 120 included in electronic device 101, electronic device 101 can display virtual content, such as virtual objects, textures, and / or shapes. As further described herein and as... Figure 3A As shown, electronic device 101 can use various rendering techniques to display a virtual scene. In some examples, electronic device 101 can display platform 306, within which a first rendering technique is used to render virtual content from the virtual scene. In some examples, electronic device 101 can display background 304, which can be rendered concurrently using a second rendering technique while the virtual content is displayed within platform 306 using the first rendering technique. In some examples, the first rendering technique results in the display of virtual content having a higher degree of visual fidelity than content rendered using the second technique. In some examples, electronic device 101 initiates the display of platform 306 and / or the virtual scene in response to detecting user input that initiates the display of a user interface for creating content. In some examples, electronic device 101 and / or computer system 312 (further described herein) can detect input specifying a first rendering technique and / or a second rendering technique, and can display the virtual scene within the user interface for creating content according to the specified rendering technique.

[0058] In some examples, platform 306 and / or the virtual scene may be displayed in response to information received from computer system 312 and / or in response to detecting input initiating and / or approving the display of the virtual scene. In some examples, platform 306 may occupy (e.g., be mapped to and / or otherwise correspond to) a predetermined portion of the physical environment. For example, platform 306 may be displayed with a specific size, spatial outline, and / or coverage portion of the virtual environment determined by computer system 312 and / or electronic device 101.

[0059] In some examples, electronic device 101 may initiate the display of platform 306 at a predetermined location relative to the three-dimensional environment 302 and / or at a viewpoint of electronic device 101 relative to the three-dimensional environment 302. For example, based on determining that electronic device 101 has a first viewpoint relative to the three-dimensional environment 302 (e.g., a first location and / or orientation relative to the physical three-dimensional environment), electronic device 101 may display platform 306 centered at the first location in the three-dimensional environment 302. Based on determining that electronic device 101 has a second viewpoint relative to the three-dimensional environment 302 different from the first viewpoint, electronic device 101 may alternatively display platform 306 centered at the second location within the three-dimensional environment 302 different from the first location.

[0060] In some examples, portions of the virtual environment displayed within platform 306 may be at least partially defined by computer system 312 and / or electronic device 101. For example, a user of computer system 312 and / or electronic device 101 may specify the loading of a first area within the virtual scene (such as the street where the virtual salon is located) when initiating the display of platform 306. Additionally or alternatively, the user may specify the loading of a second, different area within the virtual scene (such as a street away from the virtual salon that includes a virtual bank). In both examples, platform 306 may correspond to the same physical location and / or position, and the virtual content displayed in platform 306 may depend on the specifications made by the user of computer system 312 and / or electronic device 101.

[0061] like Figure 3A As shown, electronic device 101 displays platform 306 including multiple virtual objects and / or assets. For example, as Figure 3A The platform 306 shown includes first content 324. First content 324 includes a portion of a campsite, which includes virtual trees 314, virtual flowers, and virtual tents 318 among multiple virtual trees, collectively comprising... Figure 3A In a virtual scene. In some examples, a portion of the virtual scene displayed on platform 306 corresponds to the foreground of the virtual scene. Therefore, electronic device 101 can display first content 324 at one or more first levels of detail. In some examples, one or more levels of detail include rendering quality, polygon count, resolution of the virtual content, application of simulated lighting effects, level of detail of the shading model, and / or some combination thereof.

[0062] In some examples, electronic device 101 uses a first rendering technique to display virtual content within platform 306. For example, electronic device 101 may use the first technique to display first content 324 at one or more first levels of detail. It should be understood that the specific rendering technique is not limited, but may include one or more of the following: multi-rendering of the target virtual content, deferred rendering and forward rendering, screen-space effects (such as ambient occlusion, subsurface scattering and / or distortion and refraction), stereo rendering, foveated rendering, asynchronous time warp, reprojection, late latching, deferred shading, ray tracing, ray casting, radiosity analysis, path tracing, neural rendering, and / or some combination thereof. It should also be understood that, in general, virtual content may be rendered as a relatively high-resolution portion of an image, video, and / or animation presented by electronic device 101 for user interaction with electronic device 101. By using a first rendering technology to display virtual content, the user of electronic device 101 can inspect the high-fidelity portion of the virtual scene, which can reduce the time and effort required to closely examine virtual assets when creating virtual content using virtual scenes (e.g., images, videos, animations, immersive virtual experiences, and / or using virtual scenes as a backdrop for traditional media such as television and / or movies).

[0063] In some examples, electronic device 101 displays virtual content outside of platform 306 using a second rendering technique different from the first rendering technique. In some examples, using the second rendering technique to display the virtual content includes displaying the virtual content outside of platform 306 at a second level of detail different from the first level of detail. It should be understood that the second rendering technique may include one or more features similar to or the same as those described with reference to the first rendering technique. In some examples, the second rendering technique may differ from the first technique by omitting one or more techniques, setting different thresholds for the algorithm used to determine how the virtual content is displayed, or by including certain one or more techniques and / or some combination thereof not included in the first rendering technique.

[0064] Generally speaking, it should be understood that by utilizing the second technology to display virtual content, electronic device 101 can reduce the computation required to display the virtual content compared to displaying the same virtual content using the first rendering technology. As an example, electronic device 101 can display the virtual cloud 316 at a lower resolution than if the virtual cloud 316 were included in platform 306. Additionally or alternatively, the virtual cloud 316 can move in response to the detection of movement of the viewpoint of electronic device 101 to simulate a parallax effect. In some examples, virtual content outside of platform 306 corresponds to the midground and / or background of the virtual scene. In some examples, electronic device 101 can render other parts of the virtual scene concurrently with platform 306 and / or background 304, such as by using a third rendering technology different from the first and / or second technologies.

[0065] exist Figure 3A In this context, user 308 is within a three-dimensional environment 302 and has a viewpoint similar to or the same as that of the electronic device 101 described herein. User 308 is sometimes referred to herein as the first user (e.g., the user of electronic device 101), and the user of computer system 312 may be the second user. However, it is understood that the names between user and device are merely illustrative examples. For instance, a single individual may be a user of both electronic device 101 and computer system 312.

[0066] Figure 3B An example is illustrated of a user interface for content creation, including a virtual platform. In some examples, the electronic device 101 updates the viewpoint of the virtual content relative to the platform 306 and / or background 304 based on the movement of the user 308's viewpoint. Figures 3A to 3B Electronic device 101 detects the movement of the viewpoint of electronic device 101 (e.g., when displaying such as Figure 3A (Motion is detected when platform 306 and / or background virtual content are shown). For example, the viewpoint of electronic device 101 moves radially around platform 306, as shown in glyph 310. In some examples, in response to the detection of viewpoint movement, electronic device 101 updates the display of the virtual scene. For example, in Figure 3B In this process, electronic device 101 maintains the display of a first portion of the virtual scene (e.g., first content 324) and thereby changes the perspective of the first portion according to the movement of the viewpoint. For example, the updated perspective can visually appear as if user 308 has physically moved relative to the physical equivalent of platform 306. Therefore, electronic device 101 can update the first portion of the virtual scene displayed in platform 306 in a direction similar to or the same as the movement direction of the viewpoint, and / or update the first portion of the virtual scene displayed in platform 306 by an amount similar to or the same as the movement amount of the viewpoint.

[0067] like Figure 3B As shown, electronic device 101 updates platform 306 to show the outlines of tent 318, virtual tree 314, and additional content included in platform 306. Similar to reference... Figure 3A The described technology involves electronic device 101 utilizing a first rendering technique to maintain the display of a first portion of a virtual scene.

[0068] exist Figure 3B In this process, electronic device 101 displays a portion of the background 304 displayed using the second rendering technique based on the movement of the viewpoint of electronic device 101. For example, from... Figures 3A to 3BElectronic device 101 stops displaying the virtual cloud 316 (except for other virtual background content) and initiates the display of virtual content that was not displayed before the moving viewpoint was detected. For example, such as Figure 3B As illustrated, electronic device 101 displays additional details of the virtual sky and virtual scene in areas behind the virtual platform. Therefore, computer system 312 and / or electronic device 101 can map the user's viewpoint movement relative to the 3D environment 302 onto platform 306, and can initiate and / or stop the display of virtual content in virtual background 304. By mapping movement relative to platform 306, and by using knowledge of the spatial arrangement between the virtual content included in platform 306 and the virtual scene, electronic device 101 can display background 304 according to the movement of the viewpoint (e.g., as shown in reference). Figures 3A to 6 (as described).

[0069] exist Figure 3B In this system, computer system 312 detects input and displays information indicating that input (e.g., "Load Scene 2"). For example, computer system 312 may detect voice commands, selection of selectable options (e.g., buttons, menu items, icons, and / or media), and / or air gestures pointing at computer system 312 (e.g., pinching two fingers together in the air, swiping one or more fingers in the air, and curling one or more fingers in the air), and may initiate a request to update the virtual scene displayed by electronic device 101. In response to the detection of input, computer system 312 may cause electronic device 101 to display a new virtual scene, as described below. Figures 3C to 3D More detailed description.

[0070] from Figures 3B to 3C The electronic device 101 updates the platform 306 to include second content 326, which may correspond to the content of the platform 306. Figure 3B The virtual scenes shown are different from the virtual scenes shown. In some examples, the virtual scene corresponds to... Figure 3B The virtual scenes shown are different areas within the same virtual environment. In some examples, the virtual scenes correspond entirely to different virtual environments. For example, because computer system 312 requests (and, although not shown, electronic device 101 may approve) the display of a new virtual scene, electronic device 101 may stop displaying the previously displayed virtual scene and initiate the display of a replacement virtual scene. When switching between virtual scenes, electronic device 101 may maintain the display of indications to platform 306 (such as the boundaries corresponding to platform 306).

[0071] For example, Figure 3C An example is shown where electronic device 101 displays a new scene (e.g., "Scene 2" indicated by visual indicator 322). Except for those not shown in... Figure 3BIn addition to the additional virtual assets (such as trees) shown, the second content 326 included in the virtual scene also includes new virtual assets, such as roads 328. Figure 3C As shown, the electronic device 101 can use the first rendering technique to display the second content 326 (for example, because the portion of the virtual scene that includes the second content 326 is defined by the platform 306).

[0072] In addition to updating the virtual content in platform 306, electronic device 101 can also update the display of background 304 based on requests to display new virtual scenes. For example, Figure 3C Examples of additional virtual assets that can be displayed using the second rendering technique include virtual trees included in background 304. Therefore, electronic device 101 can use the first rendering technique to display a first portion of the first virtual scene, and can use the second rendering technique to display a second portion of the virtual scene. Additionally or alternatively, electronic device 101 can use the first rendering technique to display a third portion of the virtual scene (or a corresponding portion of the second virtual scene), while simultaneously using the second rendering technique to display a fourth portion of the first virtual scene (or a corresponding portion of the second virtual scene).

[0073] exist Figure 3C In this context, because road 328 extends through the portion of the virtual scene defined by platform 306 and beyond that portion, electronic device 101 uses a first rendering technique and a second rendering technique to display road 328. The first rendering technique can be used to display the first portion of road 328 within platform 306, and the second rendering technique can be used to display the second portion of road 328 outside platform 306.

[0074] exist Figure 3C In this context, electronic device 101 forgoes displaying (e.g., does not display) virtual content corresponding to area 334 of display 120 (e.g., between the viewpoint of electronic device 101 and platform 306) and area 336 of display 120 (e.g., above background 304). Therefore, electronic device 101 can present the visibility of physical rooms and / or features (such as physical walls, ceilings, and / or floors of 3D environment 302) within a three-dimensional environment. As described with reference to display 120, electronic device 101 can present portions of the physical environment within the field of view of electronic device 101. Therefore, 3D environment 302 can include an optically transparent and / or video pass-through view of the physical environment in which electronic device 101 is located.

[0075] In some examples, electronic device 101 detects input requesting a change in the display of the virtual scene. In response to the detected input, electronic device 101 can change the rendering of the virtual scene according to the request. For example, the change may include one or more of the following: the display immersion level of the virtual scene, the position of platform 306, the size of platform 306, the spatial outline of platform 306, the rendering technology used to display the virtual scene within and / or outside platform 306, the specific virtual scene displayed, and / or the display of virtual annotations associated with the virtual scene.

[0076] As an example, electronic device 101 can detect one or more inputs, such as an air pinch pointing at platform 306, or selectable options covering platform 306 (e.g., a button or icon among multiple buttons or icons covering the boundary of platform 306). When maintaining an air pinch involving contact between multiple fingers, electronic device 101 can detect movement of the hand forming the air pinch and can scale platform 306 by one or more amounts in one or more directions, the one or more amounts corresponding to one or more directions and / or one or more amounts of movement of the air pinch and / or the hand. In some examples, in response to detecting an input that increases the size of platform 306, electronic device 101 initiates the display of an additional portion of the virtual scene (such as a third portion of the virtual scene) within platform 306. In some examples, in response to detecting an input that decreases the size of platform 306, electronic device 101 stops the display of corresponding virtual content included in a first portion of the virtual scene (e.g., stopping the display of virtual content included in platform 306 before the input is detected and based on the position and / or degree of shrinkage of platform 306).

[0077] As an additional or alternative example, electronic device 101 may detect one or more inputs directed at a grasper (e.g., selectable options, such as icons or buttons), and in response, may initiate movement of platform 306. In response to detecting selection input (such as a pinch of two or more fingers in the air), electronic device 101 may initiate movement of one or more amounts in one or more directions, corresponding to one or more directions and / or one or more amounts of movement of the hand forming the pinch. In response to detecting movement, electronic device 101 may update the position of platform 306. In some examples, the “platform” portion of the virtual environment may change according to movement. Therefore, in response to detecting movement of platform 306, electronic device 101 may initiate the display of virtual content and / or stop the display of virtual content based on the updated positioning of platform 306 relative to the virtual scene.

[0078] In some examples, the input may point to a physical button or a virtual button, which may be configured to perform different functions depending on the type of input directed to the button. For example, electronic device 101 may change the level of immersion in a virtual scene based on the rotation of a physical button included in electronic device 101, and / or may perform additional or alternative operations in response to the detection of a button press, such as moving platform 306 to a position at a predetermined distance away from the viewpoint of electronic device 101.

[0079] exist Figure 3C When attention 342 is directed to selectable option 320, electronic device 101 detects input from hand 340. In some examples, selectable option 320 may be selectable to change the way the virtual scene shown on presentation platform 306 is presented. For example, selectable option may represent a user interface element selectable to increase the “level of immersion” of the virtual scene relative to the 3D environment 302. In some examples, level of immersion includes or corresponds to the degree to which virtual content consumes the viewport of electronic device 101. In some examples, level of immersion additionally or alternatively includes visual characteristics of the virtual scene, such as opacity, brightness, saturation, resolution level, and / or some combination thereof.

[0080] In some examples, electronic device 101 changes the way it presents the virtual scene based on a request to change the "immersion level" of the virtual scene. Figure 3D In the middle, electronic equipment 101 is compared to Figure 3C The high level of immersion shown presents a virtual scene. For example, electronic device 101 displays a completely immersive virtual scene. Figure 3D The virtual scene is displayed in the viewport of the electronic device 101. Therefore, the electronic device 101 displays the virtual scene with a relatively high level of immersion relative to the 3D environment 302. In some examples, when... Figure 3D When displaying a virtual scene at the indicated immersion level, the electronic device 101 displays additional portions of the virtual environment in response to detecting movement of the viewpoint of the electronic device 101 while maintaining the immersion level. For example, the electronic device 101 may detect movement of the head of the user 308 wearing the electronic device 101 relative to the three-dimensional environment 302, and in response, may display portions of the virtual scene that resemble physically equivalents as if the user 308 were viewing the virtual scene.

[0081] In some examples, electronic device 101 expands the amount of virtual content displayed via display 120 according to an increase in immersion level. For example, electronic device 101 responds to detecting... Figure 3C The input shown is used to display portions of new virtual trees, virtual floor, and / or virtual sky. Specifically, it does not include inputs from... Figure 3CRegions 334 and 336 of the virtual content in the virtual scene include those from Figure 3D Virtual content within a virtual scene.

[0082] In some examples, as the level of immersion is increased, the electronic device 101 changes the rendering technique used to display portions of the 3D environment 302. For example, Figure 3D The electronic device 101 can use the first rendering technology to display the virtual scene and Figure 3C The corresponding portion of the background 304 shown is rendered using a first rendering technique, thereby increasing the visual fidelity of the background area. Therefore, the electronic device 101 can improve the realism of the background in a virtual scene by using this first rendering technique to display the background.

[0083] In some examples, electronic device 101 may utilize a second rendering technique to maintain the display of background content. For example, in response to detecting... Figure 3C The hand 340 shown provides input, and the electronic device 101 can utilize a second technology to maintain contact with the virtual scene. Figure 3C The background 304 shown corresponds to the displayed portion. Therefore, the electronic device 101 can reduce the display of background by utilizing a second rendering technique. Figure 3D The computational load required for the background of the virtual scene in the game.

[0084] exist Figure 3D In the middle, electronic device 101 stops accessing the platform (e.g., Figure 3C The display of platform 306 in the example. In some examples, in response to the detection... Figure 3C The input shown will cause execution to stop. In some examples, electronic device 101 maintains and / or changes the display of visual indications to the platform. For example, electronic device 101 may display... Figure 3D The boundary corresponding to the platform within the 3D environment 302. In some examples, the boundary is displayed with visual characteristics to distinguish it from the 3D environment 302, such as having simulated glow effects, opacity levels, color, brightness, and / or some combination of such characteristics.

[0085] In some examples, the virtual scene includes annotations pointing to virtual content included within it. For example, electronic device 101 and / or computer system 312 can communicate to render the virtual scene while editing its content. In some examples, the user of the device enters annotations within the virtual scene, such as text, simulated handwriting, comments, voice recordings, virtual markers, spatial records of the user moving throughout the virtual scene, etc. In some examples, electronic device 101 and / or computer system 312 display visual indications of the annotations at user-specified locations and / or locations that typically point to the virtual scene. For example, Figure 3DThe pin 332 in the image may correspond to a visual marker provided by the user 308 when inspecting road 328. In some examples, the annotation includes information 330 that is displayed concurrently with the visual indication 322 and describes the comment entered by the user 308. Therefore, the electronic device 101 can display a visual representation of the annotation, thereby reducing processing and context switching caused by requests to display a list of annotations and cross-referencing that list with potentially relevant parts of the virtual scene.

[0086] For reference Figures 3A to 3D As described, electronic device 101 may use one or more rendering techniques to display virtual backgrounds included in a virtual scene. In some examples, the rendering techniques implemented by electronic device 101 reduce the amount of processing and / or pre-rendering required to display the virtual background. Additionally or alternatively, using different rendering techniques may reduce the amount of data transmitted from a computer system to electronic device 101 when streaming data to display the virtual scene.

[0087] For reference Figure 4 Further described, rendering techniques may include displaying fragments of two-dimensional images, such as panoramic background images. (See reference...) Figures 5A to 5B As described, rendering techniques may include displaying a virtual background by detecting visual features in a two-dimensional image of the virtual background and extracting and / or stacking the extracted visual features to create a depth simulation. (See reference...) Figure 6 The rendering techniques described may include determining a region of a 3D environment relative to a virtual platform, and displaying a background image corresponding to that region. (See reference...) Figure 7 The rendering techniques described may include determining multiple volumes relative to a virtual platform, displaying one or more multi-plane images, determining virtual content existing in a virtual scene and intersecting with the surfaces of the volumes, and generating one or more multi-plane images by compositing the determined virtual content.

[0088] Figure 4 Examples of electronic devices for displaying projected virtual backgrounds according to some examples of this disclosure are illustrated. For example, in Figure 4 In this configuration, the viewpoint of electronic device 101 is located within virtual platform 406, and electronic device 101 projects multiple rays to determine the virtual background based on the intersection points of the rays' projections onto virtual platform 406. In some examples, electronic device 101 generates the virtual background by projecting one or more rays from its viewpoint toward the virtual platform. Figure 4As shown, electronic device 101 (and / or computer system communicating with electronic device 101) can project one or more rays of light from the viewpoint of electronic device 101 toward the periphery of virtual platform 406, and can determine the intersection points between the rays and the boundaries of virtual platform 406 and / or the projection (e.g., orthogonal to virtual platform 406 and / or orthogonal to the floor below virtual platform 406). Electronic device 101 and / or computer system can detect the intersection points and can display background 404 based on the intersection points. For example, the projection may include at least a portion of the virtual scene (e.g., with such...). Figure 3A (The background 304 shown is similar or identical). In some examples, the virtual platform corresponds to a region in the three-dimensional environment. This region may be predefined (e.g., when the virtual scene is instantiated and / or in response to input initiating the display of the virtual scene and / or virtual platform 406). In some examples, the region is at least temporarily anchored to a portion of the physical environment. In some examples, this portion of the physical environment is predetermined (e.g., in a manner similar to or identical to that described with reference to the region). In some examples, the shape of the region is circular, rectangular, polygonal, or a three-dimensional shape (e.g., dome-shaped, cylindrical, cubic, and / or prism-shaped). In some examples, the region is a hybrid shape including one or more of the shapes described above. Regardless of the shape of the region, the electronic device 101 can detect the intersection points between rays originating from the user's viewpoint and intersecting with the region (or the projection of the region) within the three-dimensional environment. These intersection points can define the size, position, and / or spatial contour of the displayed virtual background content. Using the projection of the user's viewpoint relative to the virtual platform 406 to generate images at the electronic device 101 reduces the processing required for rendering images and virtual content that does not correspond to the user's viewpoint defined relative to the virtual platform 406, thereby improving the efficiency of the electronic device 101.

[0089] In some examples, the orientation of the light is defined by electronic device 101 and / or computer system communicating with electronic device 101. For example, settings established when the computer system is started may define the aspect ratio, size, one or more dimensions, and / or angular range that the projected background (background 404) may contain.

[0090] In some examples, electronic device 101 updates the projection background 404 based on the movement of the viewpoint. For example, in response to detecting a movement of the viewpoint of electronic device 101 from a first viewpoint to a second viewpoint, electronic device 101 can detect the intersections between multiple rays projected based on the viewpoint change from the first intersection point to the second intersection point. Electronic device 101 can determine that the intersections defined by the projected rays capture a first portion of the virtual scene when the viewpoint is the first viewpoint, and can determine that the intersections defined by the projected rays capture a second portion of the virtual scene when the viewpoint is the second viewpoint. Therefore, in response to detecting a change in viewpoint, electronic device 101 can stop displaying virtual content included in the background 404 that is not defined by a projection generated relative to the second viewpoint, and can initiate the display of virtual content in the background 404 that is newly defined by a projection relative to the second viewpoint.

[0091] In some examples, the viewpoint of the electronic device 101 is outside of and typically oriented toward the virtual platform 406. In some examples, the electronic device 101 can limit the size of the projected background 404 by determining that one or more rays do not intersect the projection of the virtual platform 406. For example, the electronic device 101 can detect that a ray projected to the right relative to the electronic device 101 is on the right side of the projection of the virtual platform 406 toward the ceiling of the 3D environment and does not intersect that projection. In this example, the electronic device 101 can set a constraint on the relative position of the right edge of the background 404. For example, if the projection of the virtual platform 406 that defines the location for displaying the background virtual content corresponds to a cylindrical piece orthogonal to the extension of the virtual platform 406, the electronic device 101 can display a virtual background covering the portion of the cylinder whose inner surface is visible from the user's viewpoint.

[0092] For reference Figures 5A to 5B As described, rendering techniques may include displaying a virtual background by detecting visual features in a two-dimensional image of the virtual background and extracting and / or stacking the extracted visual features to create a depth simulation. In some examples, rendering techniques implemented to display virtual background content from a virtual scene include generating one or more images, such as stereoscopic images that exhibit parallax effects in response to the detection of movement of an electronic device's viewpoint relative to one or more images. In some examples, one or more artificial intelligence models are used to generate stereoscopic images, which take two-dimensional images as input and generate stereoscopic images and / or virtual holograms.

[0093] exist Figure 5AIn this context, image 502 may be a two-dimensional image of a virtual background from a virtual scene, as indicated by axis 508. The two-dimensional image may be generated by the user of electronic device 101 and / or by a computer system that communicates image 502. In some examples, the two-dimensional image is processed by electronic device 101, a computer system, one or more servers, and / or some combination thereof. In some examples, processing of image 502 includes using one or more artificial intelligence models to determine visual features of image 502 and / or to determine simulated depth of those visual features. For example, electronic device 101 may use one or more deep neural networks to perform monocular depth estimation to determine depth information about the visual features included in image 502.

[0094] Visual features may include, for example, content 512 to 516. Content 512 may include ground textures and multiple virtual water bodies. Content 514 may include virtual fog overlaying content 516, and content 516 may include mountains in the far field of a virtual scene. Figure 5A As shown, electronic device 101 can determine and / or receive indications of depth information obtained from processing of image 502 using one or more artificial intelligence models, as indicated by a depth axis (e.g., "Z") included in axis 510.

[0095] In some examples, electronic device 101 uses determined depth information to render a depth simulation relative to content 512 to 516. For example, Figure 5B Examples of different spatial arrangements 518 and 520 presented to user 308 in response to the detection of movement of the viewpoint of electronic device 101 are illustrated. By modifying the spatial arrangement between visual features extracted from two-dimensional images, electronic device 101 can simulate a sense of depth, thereby adding realism to the virtual background without requiring computationally more demanding depth simulation. Therefore, electronic device 101 can present a virtual hologram by causing content 512 to 516 to float in one or more directions and / or by one or more amounts based on one or more directions and / or one or more amounts of movement of the viewpoint of electronic device 101.

[0096] For example, spatial arrangement 518 illustrates an example in which the viewpoint of electronic device 101 moves to the left of the perceived lateral center of content 512 to 516. In response to detecting leftward movement, electronic device 101 can animate content 512 to 516. In some examples, animation includes laterally shifting content 512 to 516 relative to each other, thereby causing content 512 to 516 to be laterally interlaced. Figure 3BAs shown, relative to the viewpoint of electronic device 101, content 512 is on the far left, content 514 is in the middle, and content 516 is on the far right. By interleaving the components of the virtual background generated using one or more artificial intelligence models, electronic device 101 can simulate the appearance of a hologram, thereby adding a sense of depth to the virtual background including content 512 to 516.

[0097] Spatial arrangement 520 illustrates an example in which the viewpoint of electronic device 101 moves to the right of the perceptual lateral center of content 512 to 516. Specifically, spatial arrangement 520 may reflect the staggering of content 512 to 516 relative to the viewpoint of electronic device 101 (e.g., content 512 is on the far right, content 514 is in the middle, and content 516 is on the far left). Therefore, when the viewpoint of electronic device 101 changes laterally relative to content 512 to 516, electronic device 101 can simulate depth by animateting the lateral shift of content 512 to 516. Displaying content that shifts relative to each other based on changes in the viewpoint of electronic device 101 reduces the computation required for electronic device 101 to render a more detailed representation of the virtual scene, while providing feedback to the user about the movement of the viewpoint of electronic device 101 relative to the virtual scene, thereby reducing the processing required to display a more detailed representation of the virtual scene and reducing the likelihood that electronic device 101 will perform operations based on erroneous changes in the viewpoint of electronic device 101.

[0098] Figure 6 Examples of electronic devices displaying virtual background images according to some examples of this disclosure are illustrated. In some examples, electronic device 101 determines the spatial arrangement between the viewpoint of electronic device 101 and a virtual platform, and displays a two-dimensional background image according to that spatial arrangement. For example, Figure 6 The spatial arrangements 614 to 620 illustrate different example spatial arrangements between user 308 and virtual platform 606. In such examples, electronic device 101 displays a predetermined background image corresponding to the position of electronic device 101 relative to virtual platform 606. Therefore, electronic device 101 can display a background image corresponding to a "zone" within the three-dimensional environment.

[0099] In some examples, electronic device 101 maps one or more regions relative to virtual platform 606 to one or more background images. For example, relative to 3D environment 602a, image 604a may correspond to a first region, image 608a to a second region, image 610a to a third region, and image 612a to a fourth region. In some examples, the first through fourth regions are different from each other. For example, the quadrants of the 3D environment including virtual platform 606 may be determined by electronic device 101, such as quadrants formed by intersecting vertical lines at the center of virtual platform 606. When the viewpoint of electronic device 101 corresponds to a given quadrant, electronic device 101 may display a background image located opposite the quadrant.

[0100] Spatial arrangement 614 illustrates a top-down view of a three-dimensional environment including virtual platform 606, with the viewpoint of electronic device 101 corresponding to the bottom quadrant of the three-dimensional environment. Based on the determination that the viewpoint corresponds to the bottom quadrant, electronic device 101 can display a background image 604a. For example, background image 604a may correspond to virtual content from the virtual scene or a predefined image specified by a computer system communicating with electronic device 101.

[0101] Spatial arrangement 616 illustrates the viewpoint of electronic device 101 located in the top quadrant of a three-dimensional environment. In the example of spatial arrangement 616, electronic device 101 can display image 610a and, in response to detecting movement of the viewpoint into the top quadrant of the three-dimensional environment, abandon the display of other images (e.g., image 604a).

[0102] Spatial arrangement 618 illustrates the viewpoint of electronic device 101 located in the rightmost quadrant of a three-dimensional environment. In the example of spatial arrangement 618, electronic device 101 can display image 612a and, in response to detecting movement of the viewpoint into the rightmost quadrant of the three-dimensional environment, abandon the display of other images (e.g., image 604a and / or image 610a).

[0103] Spatial arrangement 620 illustrates the viewpoint of electronic device 101 located in the leftmost quadrant of a three-dimensional environment. In the example of spatial arrangement 620, electronic device 101 can display image 608a and, in response to detecting movement of the viewpoint into the leftmost quadrant of the three-dimensional environment, abandon the display of other images (e.g., image 604a, image 610a, and / or image 612a).

[0104] In some examples, electronic device 101 divides the three-dimensional environment into more or fewer segments, such as those described above. Therefore, electronic device 101 can display a varying number of images, each of which optionally corresponds to a segment of the three-dimensional environment. Additionally or alternatively, electronic device 101 can present two-dimensional images that are curved and / or bent in a third dimension, such as two-dimensional images forming a curved canvas. For example, a curved canvas can be curved along a platform. Displaying one or more images corresponding to a virtual scene can reduce the processing required to render and display a more detailed view of the depth of the virtual scene, and can reduce the likelihood that the positioning of background virtual content included in the images can remain in a predictable position and / or orientation relative to the virtual platform.

[0105] Figure 7 Examples of electronic devices for generating multi-planar images according to the present disclosure are illustrated. In some examples, electronic device 101 generates multiple planes and / or volumes (e.g., volumes 704, 708, and 710) that correspond to different depths relative to a virtual platform (e.g., virtual platform 706) in a three-dimensional environment 702. In some examples, electronic device 101 can generate a virtual background by combining virtual content from a virtual scene at depths corresponding to the depths of the multiple planes and / or volumes relative to the virtual platform 706.

[0106] like Figure 7 As shown, electronic device 101 generates and / or receives instructions to virtual platform 706. Virtual platform 706 may be similar to or the same as other virtual platforms described above. In some examples, in order to render virtual background content of a virtual scene, electronic device 101 may determine which parts of the virtual content are visible from virtual platform 706 at the depth of volumes 704, 708, 710 and additional or alternative volumes. Therefore, electronic device 101 may generate a depth map between virtual platform 706 and parts of the virtual scene (e.g., defined by the number and / or size of volumes used to generate the virtual background).

[0107] In some examples, this determination is similar to capturing cross-sectional slices of a 3D environment, where each slice corresponds to a facet of a volume. For example, volume 704 may correspond to a first range of depth relative to virtual platform 706. In such examples, electronics 101 can detect what virtual content (such as virtual objects and / or textures) is located at a position coinciding with a facet of volume 704. Electronics 101 can similarly detect virtual content coinciding with a facet of volume 708 and a facet of volume 710. By detecting which virtual content coincides with a facet, electronics 101 can determine what aspects of the virtual scene are likely visible within virtual platform 706.

[0108] Based on the detected visible portions of the virtual content that overlap with the volume, the electronic device 101 can use the visible portions of the virtual content to form one or more composite images. For example, virtual rocks may be visible at a depth corresponding to volume 708 (e.g., there is no virtual object overlapping with volume 704). In some examples, the composite image can be used as a virtual background. In the previously described examples, the virtual background may include virtual rocks (e.g., when the viewpoint of the electronic device 101 is oriented towards the virtual rocks). Using references... Figure 7 The described one or more volumes for displaying a virtual scene reduce the processing required for rendering and displaying a more detailed representation of the virtual background, thereby improving the efficiency of the electronic device 101.

[0109] In some examples, electronic device 101 may participate in a multi-user communication session. In some examples, a multi-user communication session may include electronic device 101 and additional or alternative electronic devices and / or computer systems. In some examples, devices participating in a multi-user communication session may be head-mounted computing devices and / or desktop or laptop computing devices. In some examples, mobile devices (such as head-mounted devices) may communicate spatial data to each other to simulate the sharing of a physical environment by placing representations of other users within the corresponding three-dimensional environment of the head-mounted device. In some examples, devices display a virtual platform at a shared location relative to the physical and / or virtual environment. In some examples, devices display a view of the virtual scene relative to the virtual platform based on the viewpoint of the respective device relative to the virtual platform. For example, electronic device 101 may display a first perspective based on its own viewpoint relative to the virtual platform, and another device may display a second perspective based on its own viewpoint relative to the virtual platform (e.g., the other device may perform one or more operations similar to or the same as those described herein with reference to electronic device 101). Therefore, devices participating in multi-user communication can exchange spatial data to simulate the sharing of physical space (and / or the devices can share the same physical space), and each can present a unique view of the virtual scene. In such examples, each device can use a first rendering technique and / or a second rendering technique to render content from the shared virtual scene in a similar or identical manner to that described in the various examples referred to herein. In some examples, a desktop computing device can display a two-dimensional view from the perspective of one or each device participating in the multi-user communication session. In this way, users of the multi-user communication session can dynamically and in real time discuss and / or interact with the virtual scene without waiting for the packaging and / or transmission of scene data and / or comments on the rendering of the scene data.

[0110] Figure 8This is a flowchart of a method 800 for updating a virtual seating arrangement according to some examples of this disclosure. In some examples, instructions for performing method 800 are stored using (e.g., non-transitory) computer-readable storage media, and executing the instructions causes electronic devices (e.g., electronic device 101 or electronic device 201) to perform method 800.

[0111] At 802, in some examples, when displaying a 3D environment, the electronic device initiates the display of a user interface (such as the user interface including platform 306) for creating content, which includes the physical environment of the electronic device (such as...). Figure 3C The platform 306 shown occupies a predefined three-dimensional area corresponding to the physical environment, wherein initiating the display of the user interface includes using multiple rendering techniques to display at least a portion of the virtual environment, wherein the multiple rendering techniques include a first technique and a second technique, the first technique being such as for rendering... Figure 3A The first rendering technique shown in the first content 324, and the second technique such as the second rendering technique (such as for rendering such as Figure 3B (The rendering technique shown in background 304). At 804, in some examples, when a first part of the virtual environment is rendered in an area of ​​the user interface corresponding to a predefined 3D area of ​​the physical environment, the electronic device uses a first technique to display the first part of the virtual environment and a second technique to display the second part of the virtual environment. The first technique includes techniques such as those used for rendering... Figure 3A The rendering technique of the first content 324 shown, and the second technique such as the second rendering technique (such as for rendering such as Figure 3B (The rendering technique shown for background 304).

[0112] Additionally or alternatively, in some examples, when the first technique is used to display a first portion of the virtual environment and when the second technique is used to display a second portion of the virtual environment, the user's viewpoint of the electronic device is a first viewpoint, and in some examples, method 800 includes: in response to detecting a change in viewpoint from the first viewpoint to a second viewpoint different from the first viewpoint, utilizing the second technique to display a third portion of the virtual environment different from the second portion of the virtual environment. Additionally or alternatively, in some examples, when the first technique is used to display a first portion of the virtual environment and when the second technique is used to display a second portion of the virtual environment, the user's viewpoint of the electronic device is a first viewpoint, and in some examples, method 800 also includes using the first technique to maintain the display of the first portion of the virtual environment in response to detecting a change in viewpoint from the first viewpoint to a second viewpoint different from the first viewpoint. Additionally or alternatively, in some examples, when the first technique is used to display a first portion of the virtual environment and when the second technique is used to display a second portion of the virtual environment, the user's viewpoint of the electronic device is a first viewpoint, and in some examples, method 800 further includes, in response to detecting a change in viewpoint from the first viewpoint to a second viewpoint different from the first viewpoint: stopping the display of the first portion of the virtual environment using the first technique; and initiating the display of a third portion different from the first portion of the virtual environment using the first technique. Additionally or alternatively, in some examples, the second portion of the virtual environment is a predetermined portion of the virtual environment, which is predetermined when the display of the user interface is initiated. Additionally or alternatively, in some examples, method 800 includes: when the first technique is used to display the first portion of the virtual environment and when the second technique is used to display the second portion of the virtual environment, receiving one or more inputs modifying the size of a region corresponding to a predefined three-dimensional region of the environment relative to the three-dimensional environment; and in response to receiving one or more inputs, modifying the size of the region; and using the first technique to display the third portion of the virtual environment, wherein the size of the portion of the virtual environment changes according to one or more inputs. Additionally or alternatively, in some examples, method 800 includes: receiving one or more inputs modifying the positioning of a region relative to a three-dimensional environment when displaying a first portion of a virtual environment using a first technique and when displaying a second portion of a virtual environment using a second technique; and, in response to receiving one or more inputs, stopping the display of at least a corresponding portion of the first portion of the virtual environment. Additionally or alternatively, in some examples, the second portion of the virtual environment corresponds to a projection of a user's viewpoint on an electronic device relative to a region corresponding to a predefined three-dimensional region of the physical environment.Additionally or alternatively, in some examples, displaying a second portion of the virtual environment includes generating a depth map between the user's viewpoint on the electronic device and a region corresponding to a predefined three-dimensional region of the physical environment, and displaying one or more images based on corresponding virtual content corresponding to multiple depths included in the depth map. Additionally or alternatively, in some examples, method 800 includes streaming information representing the virtual environment from a computer system other than the electronic device, including corresponding content in the first portion of the virtual environment based on that information. Additionally or alternatively, in some examples, method 800 includes: receiving from a computer system other than the electronic device an instruction to change which corresponding virtual content is included in the region of the user interface corresponding to the predefined three-dimensional region of the physical environment when displaying the first portion of the virtual environment using a first technique and when displaying the second portion of the virtual environment using a second technique; and in response to receiving the instruction, displaying a third portion of the virtual environment different from the first portion of the virtual environment in the region corresponding to the predefined three-dimensional region of the physical environment using the first technique; and displaying a fourth portion of the virtual environment different from the second portion of the virtual environment using the second technique. Additionally or alternatively, in some examples, the first and second techniques are defined by the user of the electronic device before initiating the display of a user interface for creating content. Additionally or alternatively, in some examples, method 800 includes, when displaying a first portion of the virtual environment, conveying first information corresponding to a first viewpoint of the user of the electronic device relative to the first portion of the virtual environment to a computer system other than the electronic device. Additionally or alternatively, in some examples, the multiple rendering techniques include a third technique, and initiating the display of the virtual environment includes: when displaying the first portion of the virtual environment using the first technique and when displaying the second portion of the virtual environment using the second technique, initiating the display of a third portion of the virtual environment, different from the first and second portions of the virtual environment, using the third technique among the multiple rendering techniques. Additionally or alternatively, in some examples, using the second technique to display the second portion of the virtual environment includes displaying corresponding virtual content corresponding to one or more visual features included in a two-dimensional image, the one or more visual features in the two-dimensional image being arranged in a defined first spatial arrangement, and the corresponding virtual content being displayed in a three-dimensional environment in a second spatial arrangement corresponding to the first spatial arrangement. Additionally or alternatively, in some examples, the virtual environment is shared with one or more other electronic devices, distinct from the electronic device, via a multi-user communication session. Additionally or alternatively, in some examples, method 800 includes using a first technique to display the third portion of the virtual environment and using a second technique to display a fourth portion of the virtual environment when a third portion of the virtual environment is presented in an area of ​​the user interface corresponding to a predefined three-dimensional area of ​​the physical environment.

[0113] For purposes of explanation, the foregoing description has been presented with reference to specific examples. However, the illustrative discussion above is not intended to be exhaustive or to limit this disclosure to the precise form disclosed. Many modifications and variations are possible based on the teachings above. The examples were chosen and described to best elucidate the principles of this disclosure and its practical application, thereby enabling others skilled in the art to make optimal use of this disclosure with various modifications suitable for the particular intended purpose, as well as the various described examples.

Claims

1. A method comprising: at an electronic device in communication with one or more input devices and one or more displays: while displaying a three-dimensional environment, initiating display of a user interface for creating content, the user interface including a region corresponding to a predefined three-dimensional region of a physical environment of the electronic device, wherein initiating display of the user interface includes displaying at least a portion of a virtual environment using a plurality of rendering techniques, wherein the plurality of rendering techniques includes a first technique and a second technique, the method further comprising: while presenting a first portion of the virtual environment in the region of the user interface corresponding to the predefined three-dimensional region of the physical environment, displaying the first portion of the virtual environment using the first technique and a second portion of the virtual environment using the second technique.

2. The method of claim 1, wherein a viewpoint of a user of the electronic device is a first viewpoint while the first portion of the virtual environment is displayed using the first technique and while the second portion of the virtual environment is displayed using the second technique, the method further comprising: in response to detecting a change in the viewpoint from the first viewpoint to a second viewpoint different from the first viewpoint, displaying a third portion of the virtual environment different from the second portion of the virtual environment with the second technique.

3. The method of claim 1, wherein a viewpoint of a user of the electronic device is a first viewpoint while the first portion of the virtual environment is displayed using the first technique and while the second portion of the virtual environment is displayed using the second technique, the method further comprising: in response to detecting a change in the viewpoint from the first viewpoint to a second viewpoint different from the first viewpoint, maintaining display of the first portion of the virtual environment using the first technique.

4. The method of claim 1, wherein a viewpoint of a user of the electronic device is a first viewpoint while the first portion of the virtual environment is displayed using the first technique and while the second portion of the virtual environment is displayed using the second technique, the method further comprising: in response to detecting a change in the viewpoint from the first viewpoint to a second viewpoint different from the first viewpoint: stopping display of the first portion of the virtual environment using the first technique; and initiating display of a third portion different from the first portion of the virtual environment using the first technique.

5. The method of claim 1, wherein the second portion of the virtual environment is a predetermined portion of the virtual environment, the predetermined portion being predetermined at the time of initiating display of the user interface.

6. The method of claim 1, further comprising: while displaying the first portion of the virtual environment using the first technique and while displaying the second portion of the virtual environment using the second technique, receiving one or more inputs modifying a size of the region corresponding to the predefined three-dimensional region of the environment relative to the three-dimensional environment; and in response to receiving the one or more inputs: modifying the size of the region; and displaying a third portion of the virtual environment using the first technique, wherein the size of the portion of the virtual environment is changed in accordance with the one or more inputs.

7. The method of claim 1, the method further comprising: while displaying the first portion of the virtual environment using the first technique and while displaying the second portion of the virtual environment using the second technique, receiving one or more inputs modifying a positioning of the region relative to the three-dimensional environment; and in response to receiving the one or more inputs, ceasing display of at least a respective portion of the first portion of the virtual environment.

8. The method of claim 1, wherein: the second portion of the virtual environment corresponds to a projection of a viewpoint of a user of the electronic device relative to the region corresponding to the predefined three-dimensional region of the physical environment.

9. The method of claim 1, wherein displaying the second portion of the virtual environment comprises: generating a depth map between a viewpoint of a user of the electronic device and the region corresponding to the predefined three-dimensional region of the physical environment, and displaying one or more images based on respective virtual content corresponding to a plurality of depths included in the depth map.

10. The method of claim 1, the method further comprising: streaming information representative of the virtual environment from a computer system different from the electronic device, wherein respective content included in the first portion of the virtual environment is based on the information.

11. The method of claim 1, the method further comprising: while displaying the first portion of the virtual environment using the first technique and while displaying the second portion of the virtual environment using the second technique, receiving an indication from a computer system other than the electronic device changing which respective virtual content is included in the region of the user interface corresponding to the predefined three-dimensional region of the physical environment; and in response to receiving the indication: displaying a third portion of the virtual environment different from the first portion of the virtual environment in the region corresponding to the predefined three-dimensional region of the physical environment with the first technique; and displaying a fourth portion of the virtual environment different from the second portion of the virtual environment with the second technique.

12. The method of claim 1, wherein the first technique and the second technique are defined by a user of the electronic device prior to initiating display of the user interface for creating content.

13. The method of claim 1, the method further comprising: ​ communicating, to a computer system other than the electronic device, first information corresponding to a first viewpoint of a user of the electronic device relative to the first portion of the virtual environment when the first portion of the virtual environment is displayed.

14. The method of claim 1, wherein: the plurality of rendering techniques includes a third technique, and initiating display of the virtual environment includes: initiating display of a third portion of the virtual environment with the third technique of the plurality of rendering techniques when the first portion of the virtual environment is displayed using the first technique and when the second portion of the virtual environment is displayed using the second technique, the third portion of the virtual environment being different from the first portion of the virtual environment and the second portion of the virtual environment.

15. The method of claim 1, wherein: displaying the second portion of the virtual environment with the second technique includes displaying respective virtual content corresponding to one or more visual features included in a two-dimensional image, the one or more visual features in the two-dimensional image are arranged in a determined first spatial arrangement, and the respective virtual content is displayed within the three-dimensional environment in a second spatial arrangement corresponding to the determined first spatial arrangement.

16. The method of claim 1, wherein the virtual environment is shared with one or more other electronic devices different from the electronic device via a multi-user communication session.

17. The method of claim 1, the method further comprising: when a third portion of the virtual environment is presented in the region of the user interface corresponding to the predefined three-dimensional region of the physical environment, displaying the third portion of the virtual environment using the first technique and displaying a fourth portion of the virtual environment using the second technique.

18. An electronic device, comprising a memory and one or more processors coupled to the memory and configured to perform the method of any of claims 1-17.

19. A non-transitory computer-readable storage medium storing instructions that, when executed by an electronic device comprising a memory and one or more processors coupled to the memory, cause the electronic device to perform the method of any of claims 1-17.

20. A method, comprising: at an electronic device in communication with one or more input devices and one or more displays: means for initiating display of a user interface for creating content when a three- dimensional environment is displayed, the user interface including a region corresponding to a predefined three-dimensional region of a physical environment of the electronic device, wherein initiating display of the user interface includes displaying at least a portion of a virtual environment using a plurality of rendering techniques, wherein the plurality of rendering techniques includes a first technique and a second technique, the method further comprising: components for: when presenting a first portion of the virtual environment in the region of the user interface that corresponds to the predefined three-dimensional region of the physical environment, displaying the first portion of the virtual environment using the first technique and displaying a second portion of the virtual environment using the second technique.

Images