System and method for layout and presentation of creative workflows

Abstract

A system and method for layout and presentation of creative workflows is provided. Some examples of the present disclosure relate to systems and methods for displaying a three-dimensional model of a virtual three-dimensional environment. In some examples, the three-dimensional model includes a representation of the virtual object included in the environment, a representation of a viewpoint of a user of an electronic device in the environment, and a representation of a viewpoint of a second user of a different electronic device in the environment. In some examples, in response to receiving an input requesting display of the virtual three-dimensional environment (e.g., in full size), the electronic device displays the virtual three-dimensional environment from the viewpoint of the user of the electronic device indicated in the model.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 585,193 filed on September 25, 2023, U.S. Patent Application No. 18 / 892,227 filed on September 20, 2024, and U.S. Patent Application No. 18 / 892,242 filed on September 20, 2024, the contents of which are hereby incorporated by reference in their entirety for all purposes. Technical Field

[0003] The present invention generally relates to systems and methods for presenting a virtual three-dimensional environment, and more particularly to displaying a three-dimensional model of a virtual three-dimensional environment. Background Art

[0004] Some computer graphics environments provide two-dimensional environments and / or three-dimensional environments in which at least some of the objects presented for viewing by a user are virtual and generated by a computer. In some examples, the virtual three-dimensional environment may be based on one or more images of the computer's physical environment. In some examples, the virtual three-dimensional environment does not include an image of the computer's physical environment. Summary of the invention

[0005] The present invention generally relates to systems and methods for presenting a virtual three-dimensional environment, and more specifically to displaying a three-dimensional model of a virtual three-dimensional environment. In some examples, the three-dimensional model of the virtual three-dimensional environment includes representations of the virtual objects included in the environment, a representation of a viewpoint of a user of the electronic device in the environment, and a representation of a viewpoint of a second user of a different electronic device in the environment. In some examples, in response to receiving an input requesting display of the virtual three-dimensional environment (e.g., at full size), the electronic device displays the virtual three-dimensional environment from the viewpoint of the user of the electronic device indicated in the model.

[0006] A comprehensive description of these examples is provided in the drawings and detailed description, and it should be understood that this summary does not limit the scope of the present disclosure in any way. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] For a better understanding of the various examples described herein, reference should be made to the following detailed description and the following drawings. Throughout the drawings, like reference numerals generally refer to corresponding parts.

[0008] Figure 1 An electronic device presenting an extended reality environment according to some examples of the present disclosure is illustrated.

[0009] Figure 2 A block diagram illustrating an example architecture of a device according to some examples of the present disclosure.

[0010] Figure 3A An electronic device displaying a three-dimensional model of a virtual three-dimensional environment according to some examples of the present disclosure is illustrated.

[0011] Figure 3B An electronic device displaying a preview of a virtual three-dimensional environment according to some examples of the present disclosure is illustrated.

[0012] Figure 3C An electronic device for simultaneously displaying a virtual three-dimensional environment and a model according to some examples of the present disclosure is illustrated.

[0013] Figure 3D An electronic device that displays a virtual three-dimensional environment instead of a model according to some examples of the present disclosure is illustrated.

[0014] 4A to 4F Examples of electronic devices interacting with models according to some examples of the present disclosure are illustrated.

[0015] FIG. 5A to FIG. 5B Updating a viewpoint of a user in a virtual three-dimensional environment and a position of a representation of the user in a model of the virtual three-dimensional environment in response to movement of an electronic device in a physical environment according to some examples of the present disclosure is illustrated.

[0016] Fig. 6A An electronic device that simultaneously displays a two-dimensional image of a virtual three-dimensional environment and a model of the virtual three-dimensional environment according to some examples of the present disclosure is illustrated.

[0017] Figure 6B An electronic device 101 is illustrated displaying a three-dimensional image 618 captured by a virtual camera 614 positioned and oriented in a model 605 according to some examples of the present disclosure.

[0018] Figure 7 An electronic device that simultaneously displays multiple virtual images of a virtual three-dimensional environment and a view of the virtual three-dimensional environment corresponding to one of the images according to some examples of the present disclosure is illustrated.

[0019] Figure 8 is a flowchart of a method of displaying a model of a virtual three-dimensional environment according to some examples of the present disclosure. DETAILED DESCRIPTION

[0020] The present invention generally relates to systems and methods for presenting a virtual three-dimensional environment, and more specifically to displaying a three-dimensional model of a virtual three-dimensional environment. In some examples, the three-dimensional model of the virtual three-dimensional environment includes representations of virtual objects included in the environment, representations of a viewpoint of a user of an electronic device in the environment, and representations of a viewpoint of a second user of a different electronic device in the environment. In some examples, in response to receiving an input requesting display of the virtual three-dimensional environment (e.g., at full size), the electronic device displays the virtual three-dimensional environment from the viewpoint of the user of the electronic device indicated in the model.

[0021] In some examples, a three-dimensional object is displayed in a computer-generated three-dimensional environment at a particular orientation that controls one or more behaviors of the three-dimensional object (e.g., when the three-dimensional object moves within the three-dimensional environment). In some examples, the orientation in which the three-dimensional object is displayed in the three-dimensional environment is selected by a user of the electronic device or automatically selected by the electronic device. For example, when initiating presentation of the three-dimensional object in the three-dimensional environment, the user may select a particular orientation of the three-dimensional object, or the electronic device may automatically select the orientation of the three-dimensional object (e.g., based on the type of the three-dimensional object).

[0022] In some examples, a three-dimensional object may be displayed in a three-dimensional environment in a world-locked orientation, a body-locked orientation, a tilt-locked orientation, or a head-locked orientation, as described below. As used herein, an object displayed in a three-dimensional environment in a body-locked orientation has a distance and orientation offset relative to a portion of the user's body (e.g., the user's torso). Alternatively, in some examples, the body-locked object has a fixed distance from the user, and the orientation of the content does not reference any part of the user's body (e.g., may be displayed in the same basic direction relative to the user regardless of head and / or body movement). Additionally or alternatively, in some examples, the body-locked object may be configured to always maintain gravity or horizon (e.g., perpendicular to gravity) alignment, so that changes in the head and / or body in the rolling direction will not cause the body-locked object to move within the three-dimensional environment. Instead, translational movement in either configuration will cause the body-locked object to reposition within the three-dimensional environment to maintain the distance offset.

[0023] As used herein, an object displayed in a head-locked orientation in a three-dimensional environment has a distance and orientation offset relative to the user's head. In some examples, the head-locked object moves within the three-dimensional environment when the user's head moves (when the user's viewpoint changes).

[0024] As used herein, an object displayed in a world-locked orientation in a three-dimensional environment has no distance or orientation offset relative to the user.

[0025] As used herein, an object displayed in a three-dimensional environment with a tilt-locked orientation (referred to herein as a tilt-locked object) has a distance offset relative to a user, such as a portion of a user's body (e.g., a user's torso) or a user's head. In some examples, the tilt-locked object is displayed with a fixed orientation relative to the three-dimensional environment. In some examples, the tilt-locked object moves according to a polar (e.g., spherical) coordinate system centered on a pole passing through the user (e.g., the user's head). For example, the tilt-locked object is moved in the three-dimensional environment based on the movement of the user's head in a spherical space around the user's head (e.g., centered on the user's head). Thus, if the user tilts his head relative to gravity (e.g., up or down in a pitch direction), the tilt-locked object will follow the head tilt and move radially along the sphere, so that the tilt-locked object is repositioned within the three-dimensional environment to the same distance offset relative to the user as before the head tilt, while optionally maintaining the same orientation relative to the three-dimensional environment. In some examples, if the user moves his head in a rolling direction (e.g., clockwise or counterclockwise) relative to gravity, the tilt-locked object is not repositioned within the three-dimensional environment.

[0026] Figure 1 An electronic device 101 is illustrated that presents an extended reality (XR) environment (e.g., a computer-generated environment that optionally includes representations of physical and / or virtual objects) according to some examples of the present disclosure. In some examples, such as Figure 1 As shown, the electronic device 101 is a head mounted display or other head mounted device configured to be worn on the head of the user of the electronic device 101. Figure 2 An example of an electronic device 101 is described by using an architectural block diagram of FIG. Figure 1 As shown, electronic device 101 and table 106 are located in a physical environment. The physical environment may include physical features, such as physical surfaces (e.g., floors, walls) or physical objects (e.g., tables, lights, etc.). In some examples, electronic device 101 may be configured to use display 120 to detect and / or capture an image of the physical environment including table 106 (illustrated in the field of view of electronic device 101).

[0027] In some examples, such as Figure 1 As shown, the display 120 includes one or more internal image sensors 114a (eg, as described below with reference to FIG. 1 ) oriented toward the user's face. Figure 2100 ). In some examples, internal image sensor 114a is used for eye tracking (e.g., detecting the gaze of a user). Internal image sensor 114a is optionally arranged on the left and right portions of display 120 to enable eye tracking of the user's left and right eyes. In some examples, display 120 also includes external image sensors 114b and 114c facing outward from the user to detect and / or capture the physical environment of electronic device 101 and / or the movement of the user's hands or other body parts.

[0028] In some examples, display 120 has a field of view (e.g., a field of view captured by external image sensors 114b and 114c and / or visible to the user via display 120). Because display 120 is optionally part of a head mounted device, the field of view of display 120 is optionally the same as or similar to the field of view of the user's eyes. In some examples, display 120 is a transparent or semi-transparent display through which portions of the physical environment in the field of view of electronic device 101 pass. For example, the computer-generated environment includes an optical see-through or video pass-through portion of the physical environment in which electronic device 101 is located.

[0029] In some examples, in response to a trigger, the electronic device 101 may be configured to display in the XR environment Figure 1 , which does not exist in the physical environment but is displayed in XR on top of a real-world table 106 (or a representation thereof). Optionally, in response to detecting the plane of the table 106 in the physical environment 100, the virtual object 104 may be displayed on the surface of the table 106 in the XR environment displayed via the display 120 of the electronic device 101.

[0030] It should be understood that virtual object 104 is a representative virtual object and may include and render one or more different virtual objects (e.g., virtual objects with various dimensions, such as two-dimensional or other three-dimensional virtual objects) in a three-dimensional XR environment. For example, a virtual object may represent an application or user interface displayed in an XR environment. In some examples, a virtual object may represent content corresponding to an application and / or displayed via a user interface in an XR environment. In some examples, virtual object 104 is optionally configured to be interactive and responsive to user input (e.g., air gestures, such as air pinch gestures, air tap gestures, and / or air touch gestures), so that a user virtually touches, taps, moves, rotates, or otherwise interacts with virtual object 104.

[0031] In some examples, displaying an object in a three-dimensional environment may include interaction with one or more user interface objects in the three-dimensional environment. For example, initiation of display of an object in a three-dimensional environment may include interaction with one or more virtual options / indicators displayed in the three-dimensional environment. In some examples, when display of an object is initiated in a three-dimensional environment, an electronic device may track the user's gaze as an input for identifying one or more virtual options / indicators as targets for selection. For example, gaze may be used to identify one or more virtual options / indicators as targets for selection using another selection input. In some examples, a virtual option / indicator may be selected using hand tracking input detected via an input device that communicates with the electronic device. In some examples, an object displayed in a three-dimensional environment may be moved and / or reoriented in the three-dimensional environment based on movement input detected via an input device.

[0032] In the following discussion, an electronic device that communicates with a display generation component and one or more input devices is described. It should be understood that the electronic device optionally communicates with one or more other physical user interface devices, such as a touch-sensitive surface, a physical keyboard, a mouse, a joystick, a hand tracking device, an eye tracking device, a stylus, etc. In addition, as described above, it should be understood that the described electronic device, display, and touch-sensitive surface are optionally distributed between two or more devices. Therefore, as used in the present disclosure, information displayed on or by an electronic device is optionally used to describe information output by the electronic device to be displayed on an independent display device (touch-sensitive or non-touch-sensitive). Similarly, as used in the present disclosure, input received on an electronic device (e.g., touch input received on a touch-sensitive surface of an electronic device, or touch input received on the surface of a stylus) is optionally used to describe input received on an independent input device, and the electronic device receives input information from the independent input device.

[0033] The device typically supports a variety of applications, such as one or more of the following applications: a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, a television channel browsing application, and / or a digital video player application.

[0034] Figure 2A block diagram of an example architecture of an electronic device 201 according to some examples of the present disclosure is illustrated. In some examples, the electronic device 201 includes one or more electronic devices. For example, the electronic device 201 can be a portable device, an auxiliary device that communicates with another device, a head-mounted display, etc. In some examples, the electronic device 201 corresponds to the above reference Figure 1 An electronic device 101 is described.

[0035] like Figure 2 As illustrated, the electronic device 201 optionally includes various sensors, such as one or more hand tracking sensors 202, one or more position sensors 204, one or more image sensors 206 (optionally corresponding to Figure 1 114a and / or external image sensors 114b and 114c in the embodiment of the present invention), one or more touch-sensitive surfaces 209, one or more motion and / or orientation sensors 210, one or more eye tracking sensors 212, one or more microphones 213 or other audio sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors), one or more display generation components 214 (optionally corresponding to Figure 1 The display 120 in the electronic device 201, one or more speakers 216, one or more processors 218, one or more memories 220 and / or communication circuits 222. One or more communication buses 208 are optionally used for communication between the above components of the electronic device 201.

[0036] The communication circuit 222 optionally includes circuits for communicating with electronic devices, networks such as the Internet, intranets, wired networks and / or wireless networks, cellular networks, and wireless local area networks (LANs). The communication circuit 222 optionally includes circuits for using near field communication (NFC) and / or short-range communication such as A circuit for communicating.

[0037] Processor 218 includes one or more general purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, memory 220 is a non-transitory computer-readable storage medium (e.g., flash memory, random access memory, or other volatile or non-volatile memory or storage device) storing computer-readable instructions, which are configured to be executed by processor 218 to perform the techniques, processes, and / or methods described below. In some examples, memory 220 may include more than one non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium may be any medium (e.g., excluding signals) that can tangibly contain or store computer-executable instructions for use by or in combination with an instruction execution system, device, and apparatus. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium may include, but is not limited to, a magnetic storage device, an optical storage device, and / or a semiconductor storage device. Examples of such storage devices include magnetic disks, optical disks based on compact disc (CD), digital versatile disc (DVD) or Blu-ray technology, and persistent solid-state memory (such as flash memory, solid-state drives, etc.).

[0038] In some examples, display generation component 214 includes a single display (e.g., a liquid crystal display (LCD), an organic light emitting diode (OLED), or other type of display). In some examples, display generation component 214 includes multiple displays. In some examples, display generation component 214 may include a display with touch capabilities (e.g., a touch screen), a projector, a holographic projector, a retinal projector, a transparent or translucent display, etc. In some examples, electronic device 201 includes touch-sensitive surfaces 209 for receiving user inputs (such as tap inputs and swipe inputs or other gestures). In some examples, display generation component 214 and touch-sensitive surface 209 form a touch-sensitive display (e.g., a touch screen integrated with electronic device 201 or a touch screen that communicates with electronic device 201 outside electronic device 201).

[0039] The electronic device 201 optionally includes an image sensor 206. The image sensor 206 optionally includes one or more visible light image sensors (such as a charge coupled device (CCD) sensor) and / or a complementary metal oxide semiconductor (CMOS) sensor operable to obtain an image of a physical object from a real-world environment. The image sensor 206 also optionally includes one or more infrared (IR) sensors, such as a passive IR sensor or an active IR sensor, for detecting infrared light from the real-world environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real-world environment. The image sensor 206 also optionally includes one or more cameras configured to capture the movement of physical objects in the real-world environment. The image sensor 206 also optionally includes one or more depth sensors configured to detect the distance of the physical object from the electronic device 201. In some examples, information from the one or more depth sensors may allow the device to identify an object in the real-world environment and distinguish it from other objects in the real-world environment. In some examples, the one or more depth sensors may allow the device to determine the texture and / or topography of an object in the real-world environment.

[0040] In some examples, the electronic device 201 uses a combination of a CCD sensor, an event camera, and a depth sensor to detect the physical environment around the electronic device 201. In some examples, the image sensor 206 includes a first image sensor and a second image sensor. The first image sensor and the second image sensor work in tandem and are optionally configured to capture different information of physical objects in the real-world environment. In some examples, the first image sensor is a visible light image sensor and the second image sensor is a depth sensor. In some examples, the electronic device 201 uses the image sensor 206 to detect the positioning and orientation of the electronic device 201 and / or the display generation component 214 in the real-world environment. For example, the electronic device 201 uses the image sensor 206 to track the positioning and orientation of the display generation component 214 relative to one or more fixed objects in the real-world environment.

[0041] In some examples, the electronic device 201 includes a microphone 213 or other audio sensor. The electronic device 201 optionally uses the microphone 213 to detect sounds from the user and / or the user's real-world environment. In some examples, the microphone 213 includes a microphone array (multiple microphones) that optionally operate in conjunction to identify ambient noise or locate a sound source in the space of the real-world environment.

[0042] The electronic device 201 includes a location sensor 204 for detecting the location of the electronic device 201 and / or the display generating component 214. For example, the location sensor 204 may include a global positioning system (GPS) receiver that receives data from one or more satellites and allows the electronic device 201 to determine the absolute location of the device in the physical world.

[0043] The electronic device 201 includes an orientation sensor 210 for detecting the orientation and / or movement of the electronic device 201 and / or the display generation component 214. For example, the electronic device 201 uses the orientation sensor 210 to track changes in the position and / or orientation of the electronic device 201 and / or the display generation component 214, such as relative to physical objects in the real world environment. The orientation sensor 210 optionally includes one or more gyroscopes and / or one or more accelerometers.

[0044] In some examples, the electronic device 201 includes a hand tracking sensor 202 and / or an eye tracking sensor 212 and / or other body tracking sensors, such as a leg tracking sensor, a torso tracking sensor, and / or a head tracking sensor. The hand tracking sensor 202 is configured to track the position / location of one or more parts of the user's hand, and / or the movement of one or more parts of the user's hand relative to the extended reality environment, relative to the display generation component 214, and / or relative to another defined coordinate system. The eye tracking sensor 212 is configured to track the position and movement of the user's gaze (more generally, eyes, face, or head) relative to the real world or extended reality environment and / or relative to the display generation component 214. In some examples, the hand tracking sensor 202 and / or the eye tracking sensor 212 are implemented together with the display generation component 214. In some examples, the hand tracking sensor 202 and / or the eye tracking sensor 212 are implemented separately from the display generation component 214.

[0045] In some examples, the hand tracking sensor 202 may use an image sensor 206 (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that captures three-dimensional information from the real world including one or more hands (e.g., one or more hands of a human user). In some examples, the hands may be resolved with sufficient resolution to distinguish fingers and their corresponding positioning. In some examples, one or more image sensors 206 are positioned relative to the user to define a field of view and an interaction space of the image sensor 206, in which the finger / hand position, orientation, and / or movement captured by the image sensor is used as input (e.g., to distinguish from the user's resting hand or other hands of other people in the real-world environment). Tracking fingers / hands for input (e.g., gestures, touches, taps, etc.) may be advantageous because it does not require the user to touch, hold, or wear any type of beacon, sensor, or other marker.

[0046] In some examples, eye tracking sensor 212 includes at least one eye tracking camera (e.g., an infrared (IR) camera) and / or an illumination source (e.g., an IR light source such as an LED) that emits light toward the user's eyes. The eye tracking camera may be pointed toward the user's eyes to receive reflected IR light from the light source directly or indirectly from the eyes. In some examples, both eyes are tracked separately by corresponding eye tracking cameras and illumination sources, and focus / gaze may be determined by tracking both eyes. In some examples, one eye (e.g., the dominant eye) is tracked by one or more corresponding eye tracking cameras / illumination sources.

[0047] The electronic device 201 is not limited to Figure 2 Rather than components and configurations of the electronic device 201, it may include fewer components, other components, or additional components in a variety of configurations. In some examples, the electronic device 201 may be implemented between two electronic devices (e.g., as a system). In some such examples, each (or more) electronic device may each include one or more of the same components discussed above, such as various sensors, one or more display generating components, one or more speakers, one or more processors, one or more memories, and / or communication circuits. One or more people using the electronic device 201 are optionally referred to herein as one or more users of the device.

[0048] Figure 3A An electronic device 101 is illustrated that displays a three-dimensional model 305 of a virtual three-dimensional environment according to some examples of the present disclosure. In some examples, the electronic device 101 has the same Figure 1 The electronic device 101 described and / or the above reference Figure 2 The electronic device 201 is described with the same architecture.

[0049] exist Figure 3A In the example, the electronic device 101 displays a three-dimensional model 305 of a virtual three-dimensional environment. In some examples, the virtual three-dimensional environment is an immersive virtual reality (VR) environment accessible to the electronic device 101 and, in some examples, one or more additional electronic devices. The virtual three-dimensional environment optionally includes a virtual landscape including one or more virtual objects. For example, Figure 3A As shown, electronic device 101 displays model 305 and a view of physical environment 300 of electronic device 101 without displaying a virtual three-dimensional environment. In some examples, presenting a view of the physical environment includes presenting a view of a real object 314 (e.g., a houseplant). The electronic device optionally presents the view of the physical environment using perspective or video pass-through. In some examples, electronic device 101 displays model 305 simultaneously with a preview of the virtual three-dimensional environment, such as in Figure 3BIn some examples, the electronic device 101 displays the model 305 and the three-dimensional environment simultaneously, such as in Figure 3C middle.

[0050] In some examples, such as Figure 3A As shown, the model 305 includes representations 302a to 302d of virtual objects (e.g., buildings) included in the virtual three-dimensional environment. The representations 302a to 302d of the virtual objects are optionally displayed at a scaled-down size from the size of the corresponding virtual objects included in the virtual three-dimensional environment. In some examples, the spatial arrangement of the representations 302a to 302d of the virtual objects in the model 305 corresponds to the spatial arrangement of the corresponding virtual objects in the virtual three-dimensional environment.

[0051] In some examples, the model 305 also includes a representation 306a of the user of the electronic device 101 and representations 306b and 306c of other users of other electronic devices. In some examples, the electronic device 101 communicates with other electronic devices used by other users. For example, the electronic device is participating in a communication session that includes presenting one or more shared virtual objects. The spatial arrangement of the representations 306a to 306c relative to each other optionally corresponds to the spatial arrangement of the user of the electronic device 101 and other users of other electronic devices in the physical environment of the electronic device 101. When the electronic device displays a virtual three-dimensional environment, the spatial arrangement of the representations 306a to 306c relative to the representations 302a to 302d of virtual objects in the model optionally corresponds to the spatial arrangement of the user's virtual objects relative to the virtual three-dimensional environment. For example, when the electronic device 101 displays a virtual three-dimensional environment, such as in Figure 3C , the position and optional orientation of the representation 306a of the user of the electronic device 101 corresponds to the viewpoint of the user of the electronic device 101 in the virtual three-dimensional environment.

[0052] In some examples, representations 306a to 306c of the users are displayed within a representation 304 of a virtual stage of the virtual three-dimensional environment. In some examples, the virtual stage of the virtual three-dimensional environment is an area of ​​the three-dimensional environment that corresponds to a predefined area in the three-dimensional environment. In some examples, the size of the virtual stage corresponds to (e.g., is the same as) the size of the predefined area in the three-dimensional environment. In some examples, when the electronic device 101 displays the virtual three-dimensional environment, when the electronic device 101 is located within the predefined area of ​​the physical environment, the electronic device 101 presents a more immersive experience than when the electronic device 101 is located outside the predefined area of ​​the physical environment. For example, when the electronic device 101 is located within the predefined area of ​​the physical environment, the electronic device 101 displays portions of the virtual three-dimensional environment within and outside the virtual stage. In this example, when the electronic device 101 is located outside the predefined area of ​​the physical environment, the electronic device 101 displays portions of the virtual three-dimensional environment within the virtual stage and does not display portions of the virtual three-dimensional environment outside the virtual stage. As described in reference FIG. 4C to FIG. 4D As described, for example, the electronic device 101 can reposition the representation 304 of the virtual stage to change the portion of the virtual three-dimensional environment that corresponds to a predefined area of ​​the physical environment.

[0053] exist Figure 3A In the example shown, the electronic device 101 displaying the model 305 uses a display configured to display three-dimensional content (such as a head-mounted display) to display the model 305. In some examples, the users corresponding to the representations 306b and 306c optionally also use a three-dimensional display, such as a head-mounted display. In some examples, when the electronic device 101 displays the model 305 without displaying the virtual three-dimensional environment, the other electronic devices also display the model 305 without displaying the three-dimensional environment. In some examples, when the electronic device 101 displays the model 305 without displaying the virtual three-dimensional environment, one or more of the other electronic devices display a preview of the three-dimensional environment (such as in Figure 3B ), or with or without displaying the model 305 (such as in Figure 3C or Figure 3D The virtual three-dimensional environment is shown in Figure 2.

[0054] In some examples, an electronic device that is not in communication with a three-dimensional environment is able to display a representation of a three-dimensional model 305 in two dimensions. For example, an electronic device that uses a two-dimensional display to display content communicates with the electronic device 101 and the electronic devices corresponding to the representations 306b and 306c, and is able to access the model 305 and optionally the virtual three-dimensional environment. The electronic device using a two-dimensional display is optionally one of a computer, a smart phone, a tablet, a media player, or a set-top box that communicates with a two-dimensional display (e.g., a television screen). In some examples, such a device is able to view a two-dimensional representation of the model 305 and interact with the model in one or more of the ways described herein, such as zooming, panning, and / or rotating the model and / or updating the position of the virtual stage relative to the virtual three-dimensional environment by interacting with the representation 304 of the virtual stage included in the model 305. In some examples, when a device with a two-dimensional display updates the model 305, the electronic device 101 displays the model 305 updated according to the updates made by the device with the two-dimensional display. In some examples, a device with a two-dimensional display may cause one or more of the devices with a three-dimensional display to display a virtual three-dimensional environment. For example, in response to receiving a command to display a virtual three-dimensional environment from a device with a two-dimensional display, the electronic device 101 displays the virtual three-dimensional environment. In some examples, the device with a two-dimensional display may update the position of the virtual stage within the virtual three-dimensional environment by interacting with the representation 304 of the virtual stage included in the model 305. For example, in response to receiving a command to update the position of the virtual stage relative to the virtual three-dimensional environment from the device with a two-dimensional display, the electronic device 101 updates the position of the virtual stage relative to the virtual three-dimensional environment, including updating the position of the representation 304 of the virtual stage within the model 305 according to the updated position of the virtual stage relative to the virtual three-dimensional environment.

[0055] like Figure 3A As shown, in some examples, the electronic device 101 displays options 308a and 308b simultaneously with the model 305. In response to detecting the selection of option 308a, the electronic device 101 optionally displays a preview of the virtual three-dimensional environment, such as in Figure 3B In response to detecting the selection of option 308b, the electronic device 101 optionally displays a virtual three-dimensional environment, such as in Figure 3C In. Figure 3A In the embodiment, the electronic device 101 detects the selection of option 308a. For example, detecting the selection includes detecting the user's gaze 303a pointing to option 308a and simultaneously detecting the user performing a predefined gesture with hand 313a, such as moving the hand into Figure 3A The pinch shape shown, then move the fingers touching in the pinch shape away from each other.

[0056] Figure 3BAn electronic device 101 that displays a preview of a virtual three-dimensional environment according to some examples of the present disclosure is illustrated. In some examples, the electronic device 101 responds to receiving Figure 3A A preview of the virtual three-dimensional environment is displayed based on the input illustrated in FIG.

[0057] In some examples, when displaying a preview of the virtual three-dimensional environment 301, the electronic device 101 presents a view of the virtual three-dimensional environment 301 through the portal. For example, outside the portal, the electronic device 101 presents a view of the physical environment 300, which includes the electronic device 101 in Figure 3A In some examples, the preview of virtual three-dimensional environment 301 is semi-transparent, and portions of the physical environment are visible through the preview of three-dimensional environment 301. In some examples, the preview of virtual three-dimensional environment 301 is displayed at a reduced rendering resolution compared to the rendering resolution at which virtual three-dimensional environment 301 is displayed.

[0058] The electronic device 101 optionally displays a preview of the virtual three-dimensional environment 301 from a user's viewpoint corresponding to a position of the representation 306a in the model 305. For example, the model 305 includes the representation 306a in front of a representation 306a of a corresponding virtual object (such as, for example, a corresponding building included in the virtual three-dimensional environment), and the preview of the virtual three-dimensional environment 301 includes a view of a virtual object 312a corresponding to the representation 302a. In some examples, if the user's viewpoint has a different position, the position of the representation 306a and the viewpoint of the preview of the virtual three-dimensional environment 301 will be different and will correspond to each other.

[0059] In some examples, when electronic device 101 displays a preview of virtual 3D environment 301, other electronic devices that can access virtual 3D environment 301 simultaneously display previews of 3D environment 301. For example, electronic device 101 responds to receiving Figure 3A The input shown sends an instruction to display a preview of the three-dimensional environment 301 to other electronic devices.

[0060] In some examples, the other electronic devices display a preview of virtual three-dimensional environment 301 at the same location relative to the physical environment as electronic device 101 displays the preview of virtual three-dimensional environment 301. For example, the preview of virtual three-dimensional environment 301 is world-locked. In some examples, if the preview of virtual three-dimensional environment 301 is world-locked, the location of the preview of virtual three-dimensional environment 301 does not change in response to detecting movement of electronic device 101 (or one of the other electronic devices capable of accessing the virtual three-dimensional environment).

[0061] In some examples, the other electronic devices display a preview of the virtual three-dimensional environment 301 at a location relative to the physical environment that is different from the location at which the electronic device 101 displays the preview of the virtual three-dimensional environment 301. For example, the preview of the virtual three-dimensional environment 301 is world-locked. In some examples, if the preview of the virtual three-dimensional environment 301 is world-locked, the location of the preview of the virtual three-dimensional environment 301 does not change in response to detecting movement of the electronic device 101 (or one of the other electronic devices capable of accessing the virtual three-dimensional environment). As another example, the preview of the virtual three-dimensional environment 301 is body-locked relative to the corresponding electronic device displaying the corresponding preview of the virtual three-dimensional environment 301. In some examples, if the preview of the virtual three-dimensional environment 301 is body-locked, the location of the corresponding preview of the virtual three-dimensional environment 301 changes in response to detecting movement of the corresponding electronic device 101 displaying the corresponding preview of the virtual three-dimensional environment 301.

[0062] In some examples, other electronic devices that can access the virtual three-dimensional environment do not display a preview of the virtual three-dimensional environment unless and until they receive input requesting display of a preview of the virtual three-dimensional environment. Thus, in some examples, other electronic devices do not necessarily display a preview of the virtual three-dimensional environment 301 simply because the electronic device 101 displays a preview of the virtual three-dimensional environment 301. In some examples where the electronic device independently displays a preview of the virtual three-dimensional environment 301, the electronic device 101 displays the preview of the virtual three-dimensional environment 301 in a world-locked manner as described above. In some examples where the electronic device independently displays a preview of the virtual three-dimensional environment 301, the electronic device 101 displays the preview of the virtual three-dimensional environment 301 in a body-locked manner as described above.

[0063] like Figure 3B As shown, the electronic device 101 displays options 308c and 308b simultaneously with the preview of the virtual three-dimensional environment 301. In some examples, in response to detecting the selection of option 308c, the electronic device 101 stops displaying the preview of the virtual three-dimensional environment 301 and maintains displaying the model 305, such as in Figure 3A In. Figure 3B As shown, electronic device 101 detects a selection of option 308b, as described in more detail above. In some examples, electronic device 101 detects an air gesture pointing to option 308b, including a user's gaze 303b pointing to option 308b when the user performs a pinch gesture with hand 313b. In some examples, in response to detecting the selection of option 308b, electronic device 101 displays virtual three-dimensional environment 301, such as in Figure 3C middle.

[0064] Figure 3CThe electronic device 101 is illustrated to display a virtual three-dimensional environment 301 and a model 305 simultaneously according to some examples of the present disclosure. Figure 3C In the example, the electronic device 101 is displayed as Figure 3B The preview of the virtual three-dimensional environment 301 in the virtual three-dimensional environment 301 is displayed, and the portion of the virtual three-dimensional environment 301 not included in the preview is displayed. Figure 3B Additional parts of the preview of the virtual three-dimensional environment 301 in.

[0065] exist Figure 3C , the electronic device 101 displays the virtual three-dimensional environment 301 from the user's viewpoint corresponding to the position of the representation 306a included in the model 305, as described above with reference to Figure 3B In some examples, if the orientation and / or position of electronic device 101 were to change, electronic device 101 would present different portions of virtual three-dimensional environment 301 based on an updated viewpoint of the user updated according to the updated orientation and / or position of electronic device 101 .

[0066] In some examples, the display Figure 3C The three-dimensional environment 301 shown is similar to the display shown in FIG. Figure 3B The preview of the three-dimensional environment 301 shown differs in that more of the virtual three-dimensional environment 301 is displayed than when a preview of the virtual three-dimensional environment 301 is displayed. Additionally or alternatively, in some examples, the display Figure 3C The three-dimensional environment 301 shown is similar to the display shown in FIG. Figure 3B The preview of the three-dimensional environment 301 shown differs in that, unlike displaying a virtual three-dimensional environment 301 (such as in Figure 3C Compared to the situation in which the electronic device 101 simultaneously presents more parts of the physical environment with the preview of the three-dimensional environment 301 (such as in Figure 3B Optionally, when displaying the virtual three-dimensional environment 301, the electronic device 101 does not present parts of the physical environment. Additionally or alternatively, in some examples, displaying Figure 3C The three-dimensional environment 301 shown is similar to the display shown in FIG. Figure 3B The preview of the three-dimensional environment 301 shown is different in that the three-dimensional environment 301 is world-locked to the physical environment, but the preview of the three-dimensional environment 301 is optionally body-locked to the electronic device 101. Additionally or alternatively, in some examples, the display is Figure 3C The three-dimensional environment 301 shown is similar to the display shown in FIG. Figure 3B The preview of the three-dimensional environment 301 shown is different from that displayed by the electronic device 101 (such as in Figure 3B The electronic device 101 displays the virtual three-dimensional environment 301 (such as in Figure 3CAdditionally or alternatively, in some examples, the display Figure 3C The three-dimensional environment 301 shown is similar to the display shown in FIG. Figure 3B The preview of the three-dimensional environment 301 shown is different from that displayed by the electronic device 101 (such as in Figure 3B The electronic device 101 displays the virtual three-dimensional environment 301 (such as in Figure 3C middle).

[0067] In some examples, when electronic device 101 displays virtual 3D environment 301, other electronic devices (e.g., devices corresponding to representations 306b and 306c) that can access virtual 3D environment 301 also display virtual 3D environment 301. For example, in response to receiving an input requesting display of virtual 3D environment 301 (such as in Figure 3B ), the electronic device 101 optionally sends instructions to other electronic devices to display the three-dimensional environment 301 from their respective viewpoints. In some examples, the electronic devices display the virtual three-dimensional environment 301 independently of each other. For example, when the electronic device 101 displays the following Figure 3C When electronic device 101 displays virtual three-dimensional environment 301, one or more of the other electronic devices optionally do not display virtual three-dimensional environment 301. In some examples, when electronic device 101 displays virtual three-dimensional environment 301, one or more other electronic devices display a preview of the virtual three-dimensional environment or give up displaying virtual three-dimensional environment 301 and give up displaying a preview of virtual three-dimensional environment 301.

[0068] In some examples, model 305 is body-locked to the user. For example, regardless of whether the electronic device displays the model without displaying virtual three-dimensional environment 301 or a preview of virtual three-dimensional environment 301, displays the model simultaneously with virtual three-dimensional environment 301, or displays the model simultaneously with a preview of virtual three-dimensional environment 301, model 305 is body-locked to the user. In some examples, model 305 is world-locked. For example, regardless of whether the electronic device displays the model without displaying virtual three-dimensional environment 301 or a preview of virtual three-dimensional environment 301, displays the model simultaneously with virtual three-dimensional environment 301, or displays the model simultaneously with a preview of virtual three-dimensional environment 301, model 305 is world-locked.

[0069] like Figure 3C As shown, when displaying the virtual three-dimensional environment 301, the electronic device 101 displays a return option 308c and an option 309 to stop displaying the model 305. In some examples, in response to detecting a selection of the return option 308c, the electronic device 101 navigates back to the previous view, such as displaying a preview of the virtual three-dimensional environment 301 while maintaining the display of the model 305, such as Figure 3CIn some examples, in response to detecting a selection of option 309, electronic device 101 maintains display of virtual three-dimensional environment 301 and stops display of model 305. Figure 3C , the electronic device 101 detects a selection of option 309, including detecting a user's gaze 303c directed toward option 309 when the user performs a pinch gesture with their hand 313c. Figure 3C The electronic device 101 stops displaying the model 305 and maintains displaying the virtual three-dimensional environment 301, such as in Figure 3D middle.

[0070] Figure 3D The electronic device 101 is illustrated to display a virtual three-dimensional environment 301 without displaying a model 305 according to some examples of the present disclosure. In some examples, in addition to not displaying Figure 3C In addition to the model 305 shown in FIG. 1 , the electronic device 101 is similar to the one described above with reference to FIG. Figure 3C In some examples, in response to receiving input (e.g., voice input, input received through a hardware input device, or selection of a displayed user interface element), electronic device 101 restores display of model 305, such as in Figure 3C middle.

[0071] In some examples, when electronic device 101 responds to Figure 3C When the display of the model 305 is stopped in response to the input shown, other electronic devices capable of accessing the virtual three-dimensional environment 301 also stop displaying the model 305. For example, in response to receiving Figure 3C , the electronic device 101 sends a signal to the other electronic devices to stop displaying the model 305. In some examples, the electronic devices independently display the model 305. For example, even when the electronic device 101 does not display the model 305, the other electronic devices continue to display the model 305.

[0072] Although the above reference FIG. 3A to FIG. 3D Examples of descriptions include FIG. 3A to FIG. 3C 305, a preview of the virtual three-dimensional environment 301, and / or the virtual three-dimensional environment 301. Additionally or alternatively, in some examples, the electronic device 101 performs one or more actions described herein in response to different types of inputs, such as voice input, input using a hardware input device (e.g., a keyboard, a mouse, a stylus, or a touch screen), and / or gesture input that is not directed to a displayed element.

[0073] 4A to 4FAn example of an electronic device 101 interacting with a model 405 according to some examples of the present disclosure is illustrated. Figure 4A , the electronic device 101 displays the corresponding FIG. 3A to FIG. 3D A model 405 of a virtual three-dimensional environment of the described model 305. Figure 4A In some examples, electronic device 101 displays model 405 simultaneously with a view of physical environment 400, but in some examples, electronic device 101 performs one or more of the interactions with the model described herein, such as when presenting a view of physical environment 400, when displaying a preview of a virtual three-dimensional environment corresponding to model 405, or when displaying virtual three-dimensional environment 401 corresponding to the model.

[0074] Model 405 optionally includes representations 402a to 402d of virtual objects corresponding to representations 302a to 302d of virtual objects included in model 305. Additionally or alternatively, model 405 optionally includes representation 404 of a virtual stage and representations 406a to 406c of users capable of accessing a virtual three-dimensional environment. For example, representation 406a corresponds to a user of electronic device 101.

[0075] exist Figure 4A In FIG. 4 , the electronic device 101 displays a model 405 having a slider element 412 that controls the zoom level of the model 405. For example, a first end 414a of the model corresponds to a minimum size of the model (such as Figure 4A The size shown in FIG. 4 ), and the second end 414 b of the model corresponds to displaying the virtual three-dimensional environment 401 at full size. The slider optionally includes an indicator 416 that indicates the current zoom level and is interactive to change the zoom level of the model 405. In some examples, the electronic device 101 combines the slider 412 with the above reference FIG. 3A to FIG. 3D In some examples, the electronic device 101 displays the slider 412 without displaying the control elements described above. FIG. 3A to FIG. 3D For example, the electronic device 101 provides a method for switching between the display slider 412 and the display FIG. 3A to FIG. 3D A mechanism for navigating between one or more of the control elements shown, such as selectable navigation options or voice, gesture, or hardware input device input.

[0076] In some examples, Figure 4A Also included is a bounding box 418 that defines the volume in which the model 405 is displayed. The electronic device 101 optionally does not display the bounding box 418, and the bounding box 418 is shown only for illustration purposes. In some examples, when the electronic device 101 changes the size, orientation, and / or position of the model 405, the bounding box 418 remains Figure 4A405 is cut away at the boundary of the bounding box 418, such as at Figure 4B In some examples, when electronic device 101 changes the size, orientation, and / or position of model 405, bounding box 418 changes the size, orientation, and / or position according to the change of model 405 so that the model is not cut off, such as Figure 4C shown.

[0077] exist Figure 4A , the electronic device 101 receives input to move the pointer 416 of the slider 512 to the right, including detecting that the user pinches and moves their hand 413a to the right while the user's gaze 403a is directed to the slider 412. For example, this interaction with the slider 412 corresponds to a request to display the enlarged model 405. In response to Figure 4A In response to the input shown, or in response to another zoom input (e.g., voice input, a gesture not pointing to slider 412, and / or input received via a hardware input device), electronic device 101 increases the scale of model 405 to enlarge model 405, such as in Figure 4B or Figure 4C middle.

[0078] Figure 4B Is displayed with Figure 4A An example of an electronic device 101 that is magnified model 405 compared to the zoom amount in. For example, the electronic device 101 responds to receiving the above reference Figure 4A The scaled input described is displayed as Figure 4B Model 405 is shown. Figure 4B In the example shown, model 405 is cropped at the edge of bounding box 418. For example, changing the zoom level of model 405 does not change the size of model 405, or changes the size of model 405 up to a maximum size defined by bounding box 418 and does not increase the size of model 405 beyond the size of the bounding box. In some examples, electronic device 101 similarly crops model 405 when translating and / or rotating the model (e.g., without scaling) in response to receiving input requesting translation and / or rotation of the model.

[0079] Figure 4C is with Figure 4A For example, the electronic device 101 responds to receiving the above reference Figure 4A The scaled input described is displayed as Figure 4C Model 405 is shown. Figure 4CIn the example shown, the size of bounding box 418 is increased to accommodate the enlarged larger model 405. For example, changing the zoom level of model 405 changes the size of model 405 and bounding box 418 according to the zoom amount. For example, if the model is displayed at a larger size in response to the zoom input, then bounding box 418 will also be larger, or if the model is displayed at a smaller size in response to the zoom input, then bounding box 418 will also be smaller. In some examples, electronic device 101 similarly updates bounding box 418 to accommodate model 405 when translating and / or rotating the model (e.g., without zooming) in response to receiving input requesting translation and / or rotation of the model.

[0080] like Figure 4C As shown, the electronic device 101 receives input pointing to the representation 404 of the virtual stage. The input optionally includes the user's gaze 403c pointing to the representation 404 of the virtual stage and the user moving their hand 413c to the right while maintaining the pinched hand shape. In some examples, the input corresponds to a request to move the representation 404 of the virtual stage to the right. Figure 4D As shown, in response to Figure 4C According to the input shown, the electronic device 101 moves the representation 404 of the virtual stage in the virtual 3D environment and the representations 406a to 406c of the users to the right within the model 405. As described herein, the electronic device 101 updates the viewpoints of the users corresponding to the representations 406a to 406c according to the movement of the representations 406a to 406c within the model 405.

[0081] Figure 4D is an example of an electronic device displaying a model 405, wherein the positions of the representation 404 of the virtual stage and the representations 406a to 406c of the users are as described above with reference to Figure 4C The input described here moves. Figure 4D As shown, for example, the electronic device 101 Figure 4D The representation of the virtual stage in Figure 4C 406a to 406c within the model 405. In some examples, when the electronic device 101 moves the representation 404 of the virtual stage, the electronic device 101 maintains the spatial relationship of the representations 406a to 406c of the user relative to each other and relative to the representation 404 of the virtual stage. As described herein, in some examples, when the electronic device 101 moves the representations 406a to 406c of the user within the model 405, the electronic device 101 updates the viewpoint of the user in the virtual three-dimensional environment according to the updated positions of the representations 406a to 406c within the model 405.

[0082] exist Figure 4D, the electronic device 101 receives input directed to the slider element 412, the input including detecting the user's gaze 403d directed to the slider element 412 and / or the indication 416 of the slider element and the hand 413a moving to the right while in a pinched hand shape. In some examples, the input corresponds to a request to move the indicator 416 of the slider element 412 to the right to the maximum position of the slider element 412, thereby causing the electronic device 101 to display a virtual three-dimensional environment, such as Figure 4E shown.

[0083] Figure 4E In response to receiving Figure 4D An example of an electronic device 101 displaying a virtual three-dimensional environment 401 based on the input shown. In some examples, the electronic device 101 receives Figure 4D The virtual three-dimensional environment 401 is displayed from the user's viewpoint represented by the representation 406a in the model 405. For example, Figure 4E The view of the virtual three-dimensional environment 401 in the embodiment includes the image corresponding to Figure 4D Virtual objects 412a and 412b (eg, buildings) representing 402a and 402b in model 405 in FIG.

[0084] Figure 4F 4 is an example of an electronic device 101 that displays a zoom control element 430a and a pan control element 430c simultaneously with a model 405. Figure 4F The display of the zoom control element 430a and the pan control element 430c near the model 405 as shown makes these elements more accessible to the user and enhances the user experience. In some examples, the electronic device 101 displays the zoom control element 430a and the pan control element 430c in the three-dimensional environment 400 at the same height as the model 405, such as in the same horizontal plane as the base of the model 405. Additionally or alternatively, in some examples, the electronic device 101 displays the movement control element 430b simultaneously with the model 405. In some examples, the electronic device 101 receives input pointing to the control elements 430a, 430b, and / or 430c based on detecting the user's attention (e.g., including gaze) pointing to the corresponding control element while detecting the user performing a gesture associated with selection (such as a pinch gesture, an air tap gesture, and / or a movement of a hand in a pinch hand shape). In some examples, in response to detecting input selecting a respective control element 430a, 430b, and / or 430c, the electronic device 101 modifies the display of the model 405 in a manner associated with the selected control element in response to detecting further input.

[0085] For example, in response to detecting an input pointing to the zoom control element 430a, the electronic device 101 scales the model 405 in a manner similar to the above-described scaling model 405 in response to detecting further input. For example, the electronic device 101 detects user attention pointing to the zoom control element 430a, and the user makes a pinch hand shape. In response to detecting movement of the hand in the pinch hand shape while maintaining the pinch hand shape, the electronic device 101 optionally adjusts the zoom level of the model 405 according to the movement of the hand. For example, the first movement direction corresponds to enlarging the model 405, and the second movement direction (e.g., the opposite direction) corresponds to reducing the model 405. Additionally or alternatively, for example, the electronic device 101 adjusts the zoom level with a magnitude corresponding to the magnitude of the movement of the hand (such as the speed, distance, and / or duration of the movement). Additionally or alternatively, in some examples, when the user's attention is directed to the zoom control element 430a, in response to detecting a gesture performed by one hand of the user after making the pinch hand shape and while the user maintains the pinch hand shape with his other hand, the electronic device 101 adjusts the zoom level. Additionally or alternatively, in some examples, when the user's attention is directed to the zoom control element 430a, in response to detecting a gesture performed by the user's hand after the user makes the pinch gesture, the electronic device 101 adjusts the zoom level. In some examples, the zooming performed by the electronic device 101 in response to the input directed to the zoom control element 430a has one or more characteristics of other zooming operations described herein, such as cropping or not cropping the model 405 within the bounding box 418 and / or causing the rendering of the model 405 at other electronic devices communicating with the electronic device to be updated or not updated.

[0086] In some examples, in response to detecting input directed toward mobile control element 430b, electronic device 101 moves model 405 relative to three-dimensional environment 400. For example, electronic device 101 detects that a user makes a pinching hand shape while the user's attention is directed toward mobile control element 430b, and then moves the hand in the pinching hand shape. In this example, electronic device 101 moves model 405 and optional control elements 430a to 430b by an amount and direction corresponding to the amount and direction of movement of the hand in the pinching hand shape. For example, when moving model 405, electronic device 101 maintains the placement of control elements 430a to 403c relative to model 405.

[0087] In some examples, in response to detecting an input directed to the translation control element 430c, the electronic device 101 translates the model 405 in response to detecting further input. In some examples, translating the model changes which portion of the model 405 is displayed by the electronic device 101 in the bounding box 418 without changing the position of the bounding box 418 in the three-dimensional environment 400. For example, the electronic device 101 detects user attention directed to the translation control element 430c, and the user makes a pinch hand shape. In response to detecting movement of the hand in the pinch hand shape while maintaining the pinch hand shape, the electronic device 101 optionally translates the model 405 in a certain direction and by a certain amount according to the direction and amount of the movement of the hand. Additionally or alternatively, in some examples, when the user's attention is directed to the translation control element 430c, in response to detecting a gesture performed by one hand of the user after making the pinch hand shape and while the user maintains the pinch hand shape with his other hand, the electronic device 101 translates the model 405. Additionally or alternatively, in some examples, when the user's attention is directed to the translation control element 430c, in response to detecting a gesture performed by the user's hand after the user makes the pinch gesture, the electronic device 101 translates the model 405. In some examples, the translation performed by the electronic device 101 in response to the input directed to the translation control element 430c causes or does not cause the presentation of the model 405 at other electronic devices in communication with the electronic device to be updated or not updated, similar to the manner described above with respect to zooming.

[0088] As described above, in some examples, when electronic device 101 displays virtual 3D environment 401, other electronic devices also display the virtual 3D environment from their respective viewpoints. As described above, in some examples, when electronic device 101 displays virtual 3D environment 401, other electronic devices do not necessarily display the virtual 3D environment.

[0089] Likewise, in some examples, when the electronic device 101 updates the model 405, such as described above with reference to 4A to 4F The model 405 is translated, scaled, and / or rotated as described above, and other electronic devices update the model 405 in the same manner. In some examples, when the electronic device 101 updates the model 405, such as described above with reference to 4A to 4F As described for translating, scaling, and / or rotating model 405 , other electronic devices do not update model 405 in the same manner.

[0090] FIG. 5A to FIG. 5B The present invention illustrates updating the viewpoint of a user in a virtual three-dimensional environment 501 and the position of a representation 506a of the user in a model 505 of the virtual three-dimensional environment 501 in response to movement of the electronic device 101 in a physical environment according to some examples of the present disclosure. Figure 5AIn FIG. 5 , the electronic device 101 simultaneously displays a view of the virtual three-dimensional environment 501 and a model 505 of the virtual three-dimensional environment. For example, the view of the virtual three-dimensional environment 501 includes a view of a virtual object 512a (e.g., a building). Figure 5A In the example of , model 505 is similar to other models of the virtual three-dimensional environment described herein, including representations 502a-502d of virtual objects (e.g., buildings) of the environment displayed at locations corresponding to the viewpoint of the user in the virtual three-dimensional environment 501, a representation 504 of the virtual stage, and representations 506a-506c of users able to access the three-dimensional environment 501. In some examples, the electronic device 101 simultaneously displays an option 508c for navigating back to a previously displayed user interface and an option 509 for stopping displaying the model 505 with the model 505 and the three-dimensional environment 501 simultaneously.

[0091] like Figure 5A As shown, the electronic device 101 detects the movement of the electronic device 101 and / or the display 120 to the right. For example, the electronic device 101 includes the head mounted display 120, and the electronic device 101 detects the movement of the user when the user of the electronic device 101 is wearing the head mounted display. Figure 5B As shown, in response to detecting movement of the electronic device 101 and / or the display 120, the electronic device 101 updates the user's viewpoint in the virtual three-dimensional environment 501 and the position of the user's representation 506a in the model according to the movement of the electronic device 101 and / or the display.

[0092] Figure 5B The electronic device 101 is illustrated to display the updated model 505 and the virtual three-dimensional environment 501 simultaneously from the updated viewpoint of the user according to some examples of the present disclosure. Figure 5B In the virtual three-dimensional environment 501, the view is based on Figure 5A The object 512a (eg, a building) may be displaced by the movement of the electronic device 101 and / or the display 120 in the image. For example, because the user's viewpoint moves to the right, the object 512a (eg, a building) may appear to have shifted to the left. Figure 5B As shown, the electronic device 101 updates the position of the representation 506a of the user of the electronic device 101 to be based on, for example Figure 5A506a to the right. In some examples, moving representation 506a updates the spatial relationship of representations 506a to 506c so that the spatial representations of representations 506a to 506c correspond to the updated spatial relationship of the user in the physical environment. In some examples, if a different electronic device or a display of a different electronic device corresponding to representation 506b or 506c were to move in the physical environment, the electronic device 101 would update the position of representation 506b or 506c in the model 505 according to the movement.

[0093] Fig. 6A An electronic device 101 is illustrated that displays a two-dimensional image 608 of a virtual three-dimensional environment simultaneously with a model 605 of the virtual three-dimensional environment according to some examples of the present disclosure. In some examples, the model 605 is similar to other models of the virtual three-dimensional environment described herein. For example, the model 605 includes representations 602a to 602d of virtual objects (e.g., buildings) in the virtual three-dimensional environment. In some examples, the model 605 includes an indication of a virtual camera 604 in the three-dimensional environment, which illustrates a viewpoint of the three-dimensional environment from which the two-dimensional image 608 of the virtual three-dimensional environment can be taken. For example, the model 605 includes an indication that the virtual camera 604 faces the representation 602a of the corresponding virtual object, and the image 608 includes an image 612a of the virtual object. In some examples, the virtual camera 604 is not associated with the viewpoint of the user in the three-dimensional environment. In some examples, in response to an input for updating the position and / or orientation of the virtual camera 604, the electronic device 101 updates the virtual camera 604 in the model 605 and displays another image corresponding to the viewpoint indicated by the virtual camera 604 instead of the image 608. In some examples, the electronic device 101 is capable of storing one or more virtual images of a virtual three-dimensional environment and their associated viewpoints, as described below with reference to Figure 7 described.

[0094] Figure 6B An electronic device 101 is illustrated showing a three-dimensional image 618 captured by a virtual camera 614 positioned and oriented in a model 605 according to some examples of the present disclosure. Fig. 6A As described, the viewpoint of image 618 corresponds to the viewpoint of virtual camera element 614 representing the viewpoint in model 605 .

[0095] In some examples, the electronic device 101 uses the virtual camera 614 to generate a three-dimensional image of the virtual three-dimensional environment represented by the model 605. For example, the electronic device 101 displays the image 618 as an entrance into the virtual three-dimensional environment represented by the model 605. Optionally, one or more objects included in the image 618 extend beyond the boundaries of the image 618 to simulate one or more objects of the image 618 "pop-up" toward the viewer. For example, portions of objects that extend beyond the boundaries in some dimensions (such as height and / or width) are cropped from the image 618, but portions of objects that extend beyond the boundaries in other dimensions (such as depth) extend beyond the boundaries of the image 618. As another example, some types of objects (such as objects designated as part of a set or background of the image 618) are cropped to fit within the boundaries of the image 618, but other objects (such as objects designated as the subject of the image 618) are displayed as extending beyond the boundaries of the image 618.

[0096] In some examples, the electronic device 101 can use options 624a and 624b to switch between presenting the image captured by the virtual camera 605 as a three-dimensional image and a two-dimensional image. For example, in response to detecting a selection of option 624b while displaying the image as a three-dimensional image, as shown in FIG. Figure 6B As shown by the three-dimensional image 618, the electronic device 101 will stop displaying the three-dimensional image 618 and instead display a two-dimensional image (similar to Fig. 6A As another example, in response to displaying the image as a two-dimensional image (similar to Fig. 6A When the electronic device 101 detects the selection of option 624a while displaying the two-dimensional image 608 in FIG. 1 , the electronic device 101 stops displaying the two-dimensional image and displays a three-dimensional image (such as Figure 6B 3D image 618 shown). Figure 6B As shown, electronic device 101 indicates that image 618 is three-dimensional by displaying option 624a with indication 626, although in some examples other indications are possible, such as changing the size, color, line style, and / or translucency of option 624a when it is selected.

[0097] In some examples, the electronic device 101 uses the virtual camera 614 to capture a static image, as described above with reference to Fig. 6AThe electronic device 101 optionally updates the viewpoints of the virtual camera 614 as described below to capture images of the virtual three-dimensional environment corresponding to the model 605 from those updated viewpoints. In some examples, the electronic device 101 uses the virtual camera 614 to capture video images that include movement of the viewpoint of the virtual camera 614 and corresponding movement of the viewpoint of the video image. For example, the virtual camera 614 simulates the camera translating, rotating, or otherwise moving through the virtual three-dimensional environment represented by the model 605. Figure 6B As shown, in some examples, the virtual camera is a three-dimensional element that indicates a position and orientation corresponding to a viewpoint of image 618 .

[0098] In some examples, the electronic device 101 updates the viewpoint of the image 618 in response to receiving one or more inputs directed to the control elements 616a, 616b, and / or 617. For example, in response to detecting a selection of one of the control elements 616a, the electronic device 101 translates the virtual camera 614 relative to the model 605 and updates the viewpoint of the video image 618 accordingly, optionally without rotating the virtual camera 614. As another example, in response to detecting a selection of one of the control elements 616b, the electronic device 101 rotates the virtual camera 614 relative to the model 605 and updates the viewpoint of the video image 618 accordingly, optionally without translating the virtual camera 614. As another example, in response to detecting a selection and movement of the control element 617, the electronic device 101 rotates the virtual camera 614 to capture a portion of the virtual three-dimensional environment corresponding to the location of the model 605 including the control element 617. In some examples, capturing the movement of the camera includes capturing the speed and amount of movement of the virtual camera 614, and capturing the duration of time that the viewpoint of the virtual camera 614 is stationary. In some examples, the electronic device 101 presents a video image 618 with a moving viewpoint in real time while receiving input to control the viewpoint of the virtual camera 614. In some examples, the electronic device 101 records a sequence of inputs to move the virtual camera 614, and then captures the video image 618 after receiving the input. For example, the electronic device 101 reuses the sequence of movements of the virtual camera 614 to generate multiple video images, optionally with different starting viewpoints in the virtual three-dimensional environment represented by the model 605.

[0099] As another example, the electronic device 101 optionally moves the virtual camera 614 according to the movement of the real camera used to capture the real video of the physical environment. In some examples, the real video of the physical environment is a three-dimensional (e.g., stereo) video. The electronic device 101 optionally combines the video of the virtual three-dimensional environment represented by the model 605 with the real video of the physical environment. For example, the electronic device 101 generates a video including a set, background content, and / or one or more virtual objects captured by the virtual camera 614 in the virtual three-dimensional environment of the model 605, and a shot of one or more real objects included in the real video.

[0100] Figure 7 An electronic device 101 is illustrated that simultaneously displays multiple virtual images 702a to 702c of a virtual three-dimensional environment and a view 704 of the virtual three-dimensional environment 701 corresponding to one of the images 702a according to some examples of the present disclosure. In some examples, the images 702a to 702c are two-dimensional images, and the view 704 entering the virtual three-dimensional environment 701 presents a three-dimensional view of a portion of the virtual three-dimensional environment 701. In some examples, the images 702a to 702c are captured using a virtual camera (such as the virtual camera described above with reference to FIG. 6). In some examples, the electronic device 101 displays the virtual images 702a to 702c and the view 704 of the virtual three-dimensional environment 701 while presenting portions of the physical environment 700, such as Figure 7 In some examples, in response to detecting a selection of a different image (such as image 702b or image 702c), electronic device 101 displays a preview of virtual three-dimensional environment 701 from a viewpoint corresponding to the selected image, rather than displaying view 704 corresponding to image 702a, as shown. Figure 7 shown.

[0101] Figure 8 800 is a flowchart of a method 800 of displaying a model of a virtual three-dimensional environment according to some examples of the present disclosure. In some examples, instructions for executing method 800 are stored using a (e.g., non-transitory) computer-readable storage medium, and executing the instructions causes an electronic device (e.g., electronic device 101 or electronic device 201) to execute method 800.

[0102] At 802, in some examples, an electronic device displays a three-dimensional model of a virtual three-dimensional environment, the three-dimensional model comprising: (i) one or more representations of one or more virtual objects of the three-dimensional environment, (ii) a first representation of a viewpoint of a user of the electronic device in the three-dimensional environment, the first representation displayed at a first position of the model corresponding to the position of the viewpoint, and (iii) a second representation of a viewpoint of a second user of a second electronic device different from the electronic device in the three-dimensional environment, the second representation displayed at a second position of the model corresponding to the position of the second viewpoint, wherein the first representation, the second representation, and the three-dimensional model have a first spatial arrangement. At 804, in some examples, when displaying the three-dimensional model of the three-dimensional environment, the electronic device receives an input corresponding to a request to display the three-dimensional environment. At 806, in some examples, in response to receiving the input, the electronic device displays the virtual three-dimensional environment from the user's viewpoint via the display in a spatial arrangement corresponding to the first spatial arrangement.

[0103] Additionally or alternatively, in some examples, method 800 includes, when displaying the three-dimensional model: receiving a second input corresponding to a request to move the first representation and the second representation relative to the three-dimensional model; in response to receiving the second input, updating the three-dimensional model according to the second input so that the first representation and the second representation have a second spatial arrangement relative to the three-dimensional environment that is different from the first spatial arrangement; and receiving a third input corresponding to a request to display the three-dimensional environment from a second user's viewpoint; and in response to receiving the third input, displaying the virtual three-dimensional environment from the user's viewpoint via the display in a spatial arrangement corresponding to the second spatial arrangement. Additionally or alternatively, in some examples, based on determining that the input corresponding to the request to display the three-dimensional environment points to an option to preview the three-dimensional environment, displaying the three-dimensional environment includes displaying a partially rendered version of a portion of the three-dimensional environment. Additionally or alternatively, in some examples, based on determining that the input corresponding to the request to display the three-dimensional environment points to an option to fully display the three-dimensional environment, displaying the three-dimensional environment includes displaying a fully rendered version of the three-dimensional environment. Additionally or alternatively, in some examples, method 800 includes displaying, via a display, one or more two-dimensional representations of one or more saved views of the three-dimensional environment, wherein the one or more two-dimensional representations of the one or more saved views include one or more two-dimensional renderings of the three-dimensional environment from one or more viewpoints corresponding to the one or more saved views. Additionally or alternatively, in some examples, method 800 includes, while displaying the one or more two-dimensional representations of the one or more saved views: receiving, via one or more input devices, a second input that selects a corresponding two-dimensional representation of a corresponding saved view included in the one or more two-dimensional representations of the one or more saved views; and in response to receiving the second input, displaying a portion of the three-dimensional environment from the viewpoint of the corresponding saved view. Additionally or alternatively, in some examples, displaying the three-dimensional model includes displaying the three-dimensional model within a predefined three-dimensional volume, and method 800 also includes, while displaying the three-dimensional model in the predefined three-dimensional volume: receiving, via one or more input devices, a second input corresponding to a request to change a position and / or orientation and / or size of the three-dimensional model relative to the predefined three-dimensional volume; and in response to receiving the second input: updating the position and / or orientation and / or size of the three-dimensional model relative to the predefined three-dimensional volume according to the second input, including ceasing to display a portion of the three-dimensional model that extends beyond the predefined three-dimensional volume. Additionally or alternatively, in some examples, displaying the three-dimensional model includes displaying the three-dimensional model within a predefined three-dimensional volume, and the method further includes, while displaying the three-dimensional model in the predefined three-dimensional volume: receiving, via one or more input devices, a second input corresponding to a request to change a position and / or orientation and / or size of the three-dimensional model relative to the three-dimensional environment; and in response to receiving the second input: updating the position and / or orientation and / or size of the three-dimensional model relative to the three-dimensional environment based on the second input; and updating the predefined three-dimensional volume based on the updated position and / or orientation and / or size of the three-dimensional model.Additionally or alternatively, in some examples, method 800 includes, while displaying the three-dimensional model, displaying via the display a two-dimensional representation of a corresponding view of the three-dimensional environment, wherein the three-dimensional model includes a third representation of the corresponding view, the third representation having a position and orientation relative to the model corresponding to the position and orientation of the corresponding view relative to the three-dimensional environment. Additionally or alternatively, in some examples, method 800 includes, while displaying the three-dimensional model: displaying via the display a slider control element that controls the scale of the three-dimensional model; and based on determining that the current value of the slider control element is a minimum value, using the display to simultaneously display the three-dimensional model and a view of the physical environment of the electronic device. Additionally or alternatively, in some examples, method 800 includes, while displaying the three-dimensional model: in response to receiving a second input, displaying via the display the three-dimensional model without displaying the virtual three-dimensional environment at full size; and in response to receiving a third input, displaying via the display the three-dimensional model and simultaneously displaying the virtual three-dimensional environment at full size. Additionally or alternatively, in some examples, method 800 includes, while displaying the three-dimensional model, receiving a fourth input; and in response to receiving the fourth input: displaying the three-dimensional environment at full size via the display; and stopping display of the three-dimensional model. Additionally or alternatively, in some examples, method 800 includes when simultaneously displaying the three-dimensional model and simultaneously displaying the virtual three-dimensional environment at full size: prioritizing the rendering of the three-dimensional environment over the rendering of the three-dimensional model based on determining that one or more first criteria are satisfied; and prioritizing the rendering of the three-dimensional model over the rendering of the three-dimensional environment based on determining that one or more second criteria different from the one or more first criteria are satisfied. Additionally or alternatively, in some examples, a first spatial arrangement of the first representation, the second representation, and the three-dimensional model corresponds to a spatial arrangement of a user of the electronic device, a second user of the second electronic device, and a physical environment of the electronic device. Additionally or alternatively, in some examples, method 800 includes, while displaying the three-dimensional model, detecting a change from the spatial arrangement of the user of the electronic device, the second user of the second electronic device, and the physical environment of the electronic device to a second spatial arrangement; and in response to detecting a change from the spatial arrangement of the user of the electronic device, the second user of the second electronic device, and the physical environment of the electronic device to a second spatial arrangement: updating the model to include the first representation and the second representation in a spatial arrangement different from the first spatial arrangement and corresponding to the user of the electronic device, the second user of the second electronic device, and the physical environment of the electronic device.

[0104] Some examples of the present disclosure relate to a method, the method comprising: using the display at an electronic device in communication with the display to simultaneously display: a three-dimensional model of a virtual three-dimensional environment, the three-dimensional model comprising a representation of a viewpoint in the virtual three-dimensional environment; and content, the content comprising an image of the virtual three-dimensional environment from the viewpoint, wherein based on determining that the viewpoint in the virtual three-dimensional environment is a first viewpoint, the image is a first image from the first viewpoint; and based on determining that the viewpoint in the virtual three-dimensional environment is a second viewpoint different from the first viewpoint, the image is a second image from the second viewpoint. Additionally or alternatively, in some examples, the method further comprises: based on determining that the image is displayed as a two-dimensional image, using the display to display a selectable option, the selectable option causing the electronic device to display the image as a three-dimensional image when selected; and based on determining that the image is displayed as a three-dimensional image, using the display to display a selectable option, the selectable option causing the electronic device to display the image as a two-dimensional image when selected. Additionally or alternatively, in some examples, the content comprises video content, the video content comprising movement of the viewpoint in the virtual three-dimensional environment. Additionally or alternatively, in some examples, the method further includes: capturing video content, including: when displaying the video content: receiving one or more inputs to update a viewpoint in a three-dimensional environment via one or more input devices in communication with the electronic device; and in response to receiving the one or more inputs, updating the viewpoint in the three-dimensional environment according to the one or more inputs and updating the video according to the viewpoint. Additionally or alternatively, in some examples, the method further includes: before capturing the video content, receiving one or more inputs defining a sequence of movement of a viewpoint in a virtual three-dimensional environment via one or more input devices in communication with the electronic device; and after receiving the one or more inputs, capturing the video content, including updating the viewpoint in the three-dimensional environment according to the one or more inputs and updating the video according to the viewpoint. Additionally or alternatively, in some examples, the method further includes: using a display to display a plurality of control elements associated with the viewpoint in the virtual three-dimensional environment; and receiving one or more inputs directed to the plurality of control elements via one or more input devices in communication with the electronic device, wherein the movement of the viewpoint in the three-dimensional environment in the video content is based on the one or more inputs directed to the plurality of control elements. Additionally or alternatively, in some examples, the movement of the viewpoint in the virtual three-dimensional environment in the video content is based on the movement of a physical camera capturing real video footage. Additionally or alternatively, in some examples, the method further includes presenting a second video content using a display, the second video content simultaneously including a portion of the video content of the virtual three-dimensional environment and a portion of the real video footage. Additionally or alternatively, in some examples, the method further includes: playing the video content; and while playing the video content, displaying movement of a representation of a viewpoint in the virtual three-dimensional environment in the three-dimensional model in sync with playback of the video content.

[0105] Some examples of the present disclosure relate to an electronic device, the electronic device comprising: a memory; and one or more processors, the processors coupled to the memory and configured to execute a method, the method comprising: using a display in communication with the electronic device to simultaneously display a three-dimensional model of a virtual three-dimensional environment, the three-dimensional model comprising a representation of a viewpoint in the virtual three-dimensional environment; and content, the content comprising an image of the virtual three-dimensional environment from a viewpoint, wherein according to determining that the viewpoint in the virtual three-dimensional environment is a first viewpoint, the image is a first image from the first viewpoint; and according to determining that the viewpoint in the virtual three-dimensional environment is a second viewpoint different from the first viewpoint, the image is a second image from the second viewpoint. Additionally or alternatively, in some examples, the method further comprises: according to determining that the image is displayed as a two-dimensional image, using the display to display a selectable option, the selectable option when selected causes the electronic device to display the image as a three-dimensional image; and according to determining that the image is displayed as a three-dimensional image, using the display to display a selectable option, the selectable option when selected causes the electronic device to display the image as a two-dimensional image. Additionally or alternatively, in some examples, the content comprises video content, the video content comprising movement of the viewpoint in the virtual three-dimensional environment. Additionally or alternatively, in some examples, the method further includes: capturing video content, including: when displaying the video content: receiving one or more inputs to update a viewpoint in a three-dimensional environment via one or more input devices in communication with the electronic device; and in response to receiving the one or more inputs, updating the viewpoint in the three-dimensional environment according to the one or more inputs and updating the video according to the viewpoint. Additionally or alternatively, in some examples, the method further includes: before capturing the video content, receiving one or more inputs defining a sequence of movement of a viewpoint in a virtual three-dimensional environment via one or more input devices in communication with the electronic device; and after receiving the one or more inputs, capturing the video content, including updating the viewpoint in the three-dimensional environment according to the one or more inputs and updating the video according to the viewpoint. Additionally or alternatively, in some examples, the method further includes: using a display to display a plurality of control elements associated with the viewpoint in the virtual three-dimensional environment; and receiving one or more inputs directed to the plurality of control elements via one or more input devices in communication with the electronic device, wherein the movement of the viewpoint in the three-dimensional environment in the video content is based on the one or more inputs directed to the plurality of control elements. Additionally or alternatively, in some examples, the movement of the viewpoint in the virtual three-dimensional environment in the video content is based on the movement of a physical camera capturing real video footage. Additionally or alternatively, in some examples, the method further includes presenting a second video content using a display, the second video content simultaneously including a portion of the video content of the virtual three-dimensional environment and a portion of the real video footage. Additionally or alternatively, in some examples, the method further includes: playing the video content; and while playing the video content, displaying movement of a representation of a viewpoint in the virtual three-dimensional environment in the three-dimensional model in sync with playback of the video content.

[0106] Some examples of the present disclosure relate to a non-transitory computer-readable storage medium storing instructions, which, when executed by an electronic device including a memory and one or more processors coupled to the memory, causes the electronic device to perform a method including the following items: using a display in communication with the electronic device to simultaneously display: a three-dimensional model of a virtual three-dimensional environment, the three-dimensional model including a representation of a viewpoint in the virtual three-dimensional environment; and content, the content including an image of the virtual three-dimensional environment from the viewpoint, wherein based on determining that the viewpoint in the virtual three-dimensional environment is a first viewpoint, the image is a first image from the first viewpoint; and based on determining that the viewpoint in the virtual three-dimensional environment is a second viewpoint different from the first viewpoint, the image is a second image from the second viewpoint. Additionally or alternatively, in some examples, the method further includes: based on determining that the image is displayed as a two-dimensional image, using the display to display a selectable option, the selectable option when selected causes the electronic device to display the image as a three-dimensional image; and based on determining that the image is displayed as a three-dimensional image, using the display to display a selectable option, the selectable option when selected causes the electronic device to display the image as a two-dimensional image. Additionally or alternatively, in some examples, the content includes video content, the video content including movement of a viewpoint in a virtual three-dimensional environment. Additionally or alternatively, in some examples, the method further includes: capturing video content, including: when displaying the video content, receiving one or more inputs to update the viewpoint in the three-dimensional environment via one or more input devices in communication with the electronic device; and in response to receiving the one or more inputs, updating the viewpoint in the three-dimensional environment according to the one or more inputs and updating the video according to the viewpoint. Additionally or alternatively, in some examples, the method further includes: before capturing the video content, receiving one or more inputs defining a sequence of movement of the viewpoint in the virtual three-dimensional environment via one or more input devices in communication with the electronic device; and after receiving the one or more inputs, capturing the video content, including updating the viewpoint in the three-dimensional environment according to the one or more inputs and updating the video according to the viewpoint. Additionally or alternatively, in some examples, the method further includes: using a display to display a plurality of control elements associated with the viewpoint in the virtual three-dimensional environment; and receiving one or more inputs directed to the plurality of control elements via one or more input devices in communication with the electronic device, wherein the movement of the viewpoint in the three-dimensional environment in the video content is based on the one or more inputs directed to the plurality of control elements. Additionally or alternatively, in some examples, movement of a viewpoint in a virtual three-dimensional environment in the video content is based on movement of a physical camera that captures the real video footage. Additionally or alternatively, in some examples, the method further includes presenting a second video content using a display, the second video content including both a portion of the video content of the virtual three-dimensional environment and a portion of the real video footage.Additionally or alternatively, in some examples, the method further comprises: playing the video content; and while playing the video content, displaying movement of a representation of a viewpoint in the virtual three-dimensional environment in the three-dimensional model in synchronization with playback of the video content.

[0107] For the purpose of explanation, the foregoing description is described by reference to specific examples. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Based on the above teachings, many modifications and variations are possible. The examples are selected and described in order to best illustrate the principles of the present disclosure and its practical application, so as to enable other technical personnel in the field to best use the present disclosure and the various described examples with various modifications suitable for the specific purpose contemplated.

Claims

1. A method, comprising: At an electronic device in communication with a display and one or more input devices: A three-dimensional model of a virtual three-dimensional environment is displayed via a display, wherein the three-dimensional model includes: (i) one or more representations of one or more virtual objects of the three-dimensional environment, (ii) a first representation of a user of the electronic device displayed at a first location of the three-dimensional model, and (iii) a second representation of a second user of a second electronic device different from the electronic device in the three-dimensional environment displayed at a second location of the three-dimensional model, wherein the first representation, the second representation, and the three-dimensional model have a first spatial arrangement; while displaying the three-dimensional model of the three-dimensional environment, receiving input corresponding to a request to display the three-dimensional environment from the user's viewpoint; and In response to receiving the input: The virtual three-dimensional environment is displayed via the display from the viewpoint of the user in a spatial arrangement corresponding to the first spatial arrangement.

2. The method according to claim 1, further comprising: When displaying the 3D model: receiving a second input corresponding to a request to move the first representation and the second representation relative to the three-dimensional model; In response to receiving the second input, updating the three-dimensional model according to the second input so that the first representation and the second representation have a second spatial arrangement relative to the three-dimensional environment that is different from the first spatial arrangement; as well as receiving a third input corresponding to a request to display the three-dimensional environment from the viewpoint of the second user; as well as In response to receiving the third input, The virtual three-dimensional environment is displayed via the display from the viewpoint of the user in a spatial arrangement corresponding to the second spatial arrangement.

3. The method according to claim 1, wherein: Based on determining that the input corresponding to the request to display the three-dimensional environment points to an option to preview the three-dimensional environment, displaying the three-dimensional environment includes displaying a partially rendered version of a portion of the three-dimensional environment.

4. The method according to claim 1, wherein: Based on determining that the input corresponding to the request to display the three-dimensional environment points to an option to fully display the three-dimensional environment, displaying the three-dimensional environment includes displaying a fully rendered version of the three-dimensional environment.

5. The method according to claim 1, further comprising: One or more two-dimensional representations of the one or more saved views of the three-dimensional environment are displayed via the display, wherein the one or more two-dimensional representations of the one or more saved views include one or more two-dimensional renderings of the three-dimensional environment from one or more viewpoints corresponding to the one or more saved views.

6. The method of claim 5, further comprising, while displaying the one or more two-dimensional representations of the one or more saved views: receiving, via the one or more input devices, a second input that selects a respective two-dimensional representation of a respective saved view included in the one or more two-dimensional representations of the one or more saved views; and In response to receiving the second input, a portion of the three-dimensional environment is displayed from a viewpoint of the corresponding saved view.

7. The method of claim 1 , wherein displaying the three-dimensional model comprises displaying the three-dimensional model within a predefined three-dimensional volume, and the method further comprises: When displaying the three-dimensional model in the predefined three-dimensional volume: receiving, via the one or more input devices, a second input corresponding to a request to change a position and / or orientation and / or size of the three-dimensional model relative to the predefined three-dimensional volume; as well as In response to receiving the second input: Updating the position and / or the orientation and / or the size of the three-dimensional model relative to the predefined three-dimensional volume according to the second input includes ceasing display of a portion of the three-dimensional model extending beyond the predefined three-dimensional volume.

8. The method of claim 1 , wherein displaying the three-dimensional model comprises displaying the three-dimensional model within a predefined three-dimensional volume, and the method further comprises: When displaying the three-dimensional model in the predefined three-dimensional volume: receiving, via the one or more input devices, a second input corresponding to a request to change a position and / or orientation and / or size of the three-dimensional model relative to the three-dimensional environment; as well as In response to receiving the second input: updating the position and / or the orientation and / or the size of the three-dimensional model relative to the three-dimensional environment according to the second input; as well as The predefined three-dimensional volume is updated according to updating the position and / or the orientation and / or the size of the three-dimensional model.

9. The method according to claim 1, further comprising: While displaying the three-dimensional model, a two-dimensional representation of a corresponding view of the three-dimensional environment is displayed via the display, wherein the three-dimensional model includes a third representation of the corresponding view having a position and orientation relative to the model corresponding to the position and orientation of the corresponding view relative to the three-dimensional environment.

10. The method according to claim 1, further comprising: When displaying the 3D model: displaying, via the display, a slider control element for controlling a scale of the three-dimensional model; as well as Based on determining that the current value of the slider control element is a minimum value, using the display to simultaneously display the three-dimensional model and a view of a physical environment of the electronic device.

11. The method according to claim 1, further comprising: When displaying the 3D model: In response to receiving a second input, displaying the three-dimensional model via the display without displaying the virtual three-dimensional environment in full size; as well as In response to receiving a third input, the three-dimensional model is displayed via the display and the virtual three-dimensional environment is simultaneously displayed at the full size.

12. The method according to claim 11, further comprising: While displaying the three-dimensional model, receiving a fourth input; as well as In response to receiving the fourth input: displaying the three-dimensional environment at the full size via the display; as well as Stop displaying the three-dimensional model.

13. The method according to claim 11, further comprising: When the three-dimensional model is displayed simultaneously and the virtual three-dimensional environment is displayed in full size simultaneously: Prioritizing rendering of the three-dimensional environment over rendering of the three-dimensional model based on determining that one or more first criteria are satisfied; and Rendering the three-dimensional model is prioritized over rendering the three-dimensional environment based on determining that one or more second criteria different from the one or more first criteria are satisfied.

14. The method of claim 1, wherein the first spatial arrangement of the first representation, the second representation, and the three-dimensional model corresponds to a spatial arrangement of the user of the electronic device, the second user of the second electronic device, and a physical environment of the electronic device.

15. The method according to claim 14, further comprising: while displaying the three-dimensional model, detecting a change from the spatial arrangement of the user of the electronic device, the second user of the second electronic device, and the physical environment of the electronic device to a second spatial arrangement; as well as In response to detecting a change of the spatial arrangement of the user of the electronic device, the second user of the second electronic device, and the physical environment of the electronic device to the second spatial arrangement: The model is updated to include the first representation and the second representation of a second spatial arrangement in a spatial arrangement different from the first spatial arrangement and corresponding to the user of the electronic device, the second user of the second electronic device, and the physical environment of the electronic device.

16. The method according to claim 1, further comprising: A representation of at least a portion of a physical environment of the electronic device is presented while displaying the three-dimensional model of the virtual three-dimensional environment. 17 . An electronic device comprising a memory and one or more processors, the processors being coupled to the memory and configured to execute the method according to claim 1 .

18. A non-transitory computer-readable storage medium storing instructions which, when executed by an electronic device comprising a memory and one or more processors coupled to the memory, cause the electronic device to perform the method according to any one of claims 1 to 16.

19. A method comprising: At the electronics that communicate with the display: The display is used to simultaneously display: a three-dimensional model of a virtual three-dimensional environment, the three-dimensional model comprising a representation of a viewpoint in the virtual three-dimensional environment; and content, the content comprising an image of the virtual three-dimensional environment from the viewpoint, wherein According to determining that the viewpoint in the virtual three-dimensional environment is a first viewpoint, the image is a first image from the first viewpoint; and According to determining that the viewpoint in the virtual three-dimensional environment is a second viewpoint different from the first viewpoint, the image is a second image from the second viewpoint.

20. The method according to claim 19, further comprising: based on determining that the image is displayed as a two-dimensional image, displaying a selectable option using the display, the selectable option causing the electronic device to display the image as a three-dimensional image when selected; as well as Based on determining that the image is displayed as a three-dimensional image, a selectable option is displayed using the display, the selectable option causing the electronic device to display the image as the two-dimensional image when selected.

21. The method of claim 19, wherein the content comprises video content including movement of the viewpoint in the virtual three-dimensional environment.

22. The method according to claim 21, further comprising: displaying, using the display, a plurality of control elements associated with the viewpoint in the virtual three-dimensional environment; as well as One or more inputs directed to the plurality of control elements are received via one or more input devices in communication with the electronic device, wherein the movement of the viewpoint in the three-dimensional environment in the video content is based on the one or more inputs directed to the plurality of control elements.

23. The method of claim 21, wherein the movement of the viewpoint in the virtual three-dimensional environment in the video content is based on movement of a physical camera capturing real video footage.

24. An electronic device comprising a memory and one or more processors, the processors being coupled to the memory and configured to perform the method according to any one of claims 19 to 23.

25. A non-transitory computer-readable storage medium storing instructions which, when executed by an electronic device comprising a memory and one or more processors coupled to the memory, cause the electronic device to perform the method according to any one of claims 19 to 23.

Images