Device, method and graphical user interface for three-dimensional preview of objects

Abstract

The present disclosure relates to devices, methods, and graphical user interfaces for three-dimensional previews of objects. A three-dimensional preview of content can be generated and presented at an electronic device in a three-dimensional environment. This three-dimensional preview of content can be presented concurrently with a two-dimensional representation of the content in the content generation environment that is presented in the three-dimensional environment. When the three-dimensional preview of content is presented in the three-dimensional environment, one or more affordances can be provided for interacting with one or more computer-generated virtual objects of the three-dimensional preview. The one or more affordances can be displayed with the three-dimensional preview of content in the three-dimensional environment. The three-dimensional preview of content can be presented on a three-dimensional tray, and the one or more affordances can be presented outside of the tray perimeter and / or in a control bar or other control grouping along the tray perimeter.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 226,724, filed July 28, 2021, and U.S. Patent Application No. 17 / 812,965, filed July 15, 2022, the contents of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] This invention relates generally to computer graphics editors, and more particularly to devices, methods, and user interfaces for three-dimensional previewing of computer graphics objects. Background Technology

[0004] Some computer graphics environments provide two-dimensional and / or three-dimensional environments in which at least some objects displayed for user observation are virtual and computer-generated. For example, in some use cases, users can create or modify extended reality (XR) environments, such as by editing, generating, or otherwise manipulating XR virtual objects using a content generation environment, such as a graphical editor or graphical editing interface running on a content creation application. In some implementations, the creation or modification of an XR environment, including content items (e.g., two-dimensional and / or three-dimensional objects) within the XR environment, may include generating previews of the content items at various intermediate stages of the content creation process and presenting these previews to the user. However, such previews of content items generated in a two-dimensional manner and presented to the user are limited by the two-dimensional display and graphics processing characteristics of the device running the content creation application. Therefore, there is a need for editors that allow intuitive editing of computer-generated virtual objects presented in a three-dimensional manner. Summary of the Invention

[0005] Some embodiments of this disclosure relate to a 3D preview of content (e.g., XR content, also referred to herein as an XR content item) generated and rendered in a 3D environment (e.g., a computer-generated environment) on an electronic device. In some embodiments, the 3D preview of the content is presented simultaneously with a 2D representation of the content in a content-generating environment (e.g., a content creation application) rendered in the 3D environment. In some embodiments, when a 3D preview of the content is rendered in a 3D environment, one or more display representations are provided for interacting with one or more computer-generated virtual objects of the 3D preview. In some embodiments, one or more display representations may be displayed together with the 3D preview of the content in the 3D environment (e.g., displayed below, in front of, above, near, etc., the one or more virtual objects of the 3D preview). In some embodiments, the 3D preview of the content is presented on a 3D tray (e.g., a user interface element), and one or more display representations are presented outside the tray perimeter and / or in a control bar or other control grouping along the tray perimeter. In some embodiments, as described herein, a group of some or all of the display representations is referred to as a manipulator or object manipulator.

[0006] In some implementations, a 3D preview of the content can be configured to operate in at least two modes. A first mode (e.g., playback mode) can simulate the runtime of the content, where one or more actions associated with the content (e.g., animation, audio clipping, etc.) can be performed. For example, one or more virtual objects in the 3D preview can be animated to move, and one or more virtual objects can perform additional animations or other behaviors in response to input. A second mode (e.g., editing mode) can provide a 3D preview of the content to allow the user to interact with the content for editing. For example, the user can select elements in the 3D content and can select corresponding elements in a 2D representation of the content generation environment.

[0007] In some embodiments, the object manipulator may include a first power representation selectable to operate the electronic device in a first mode. In some embodiments, the object manipulator may include a second power representation selectable to operate the electronic device in a second mode different from the first mode. In some embodiments, the first and second power representations may be a single power representation for switching modes (e.g., a play / pause button). In some embodiments, the object manipulator may include a third power representation selectable to scale the size of one or more virtual objects in the preview. In some embodiments, the object manipulator may include a fourth power representation selectable to execute executable code associated with playback of the preview content, such that one or more actions (e.g., animation, audio clipping, etc.) are incrementally performed by one or more virtual objects with each selection of the fourth power representation. In some embodiments, the object manipulator may also include a fifth power representation selectable to operate the electronic device in a third mode different from the first and second modes. In the third mode, a full-size representation of one or more virtual objects in the 3D preview is displayed within a 3D environment.

[0008] As described herein, some embodiments of this disclosure relate to user interaction with and / or manipulation of a 3D preview of content displayed on an electronic device (e.g., an XR content item). In some embodiments, a 2D representation of an XR content item generated in a content creation application displayed on a first electronic device can be displayed simultaneously with a 3D preview of the XR content item on a second electronic device. In some embodiments, user interaction with a 3D preview of an XR content item received at the second electronic device (e.g., user input, such as touch, tap, movement, reorientation, etc.) can update the display of the 3D preview of the XR content item based on the input. In some embodiments, user input received at the second electronic device is transmitted to the first electronic device in real time, such that the display of the 2D representation of the XR content item and the 3D preview of the XR content item are optionally manipulated simultaneously or nearly simultaneously (e.g., within less than 50 ms of each other).

[0009] Manipulating a 3D preview of content in a 3D environment can include changing the appearance of one or more virtual objects in the 3D preview. In some embodiments, manipulation of the 3D preview is optionally determined by the operating mode of the electronic device presenting the 3D environment. In some embodiments, a computer-generated object manipulator is optionally presented in the 3D environment along with an interactive tray, wherein the interactive tray optionally contains the 3D preview. The interactive tray can be manipulated such that user interaction with the tray can change the appearance of the interactive tray within the 3D environment without necessarily changing the appearance of one or more virtual objects in the 3D preview contained on the tray. In some embodiments, the appearance of the interactive tray can change in response to manipulation of one or more virtual objects in the 3D preview contained on the tray. In some embodiments, a change in the viewpoint associated with the electronic device can change the view of the 3D preview visible to the user. In some embodiments, a change in the viewpoint associated with the electronic device can change the orientation and / or position of the object manipulator within the 3D environment, such that the object manipulator optionally continues to face the user. In some embodiments, a 2D representation of one or more virtual objects generated in a content creation application displayed on a first electronic device can be displayed simultaneously with a 3D preview of one or more virtual objects on a second electronic device. Changes to the viewpoint associated with the second electronic device are optionally independent of the viewpoint of the first electronic device, such that changes to the view of the 3D preview optionally do not alter the view of the 2D representation of one or more virtual objects.

[0010] The accompanying drawings and detailed descriptions provide a full description of these embodiments. It should be understood that the content of this invention does not limit the scope of this disclosure in any way. Attached Figure Description

[0011] To better understand the various embodiments described, reference should be made to the following detailed description in conjunction with the accompanying drawings, in which similar reference numerals often indicate corresponding parts throughout the drawings.

[0012] Figure 1 An electronic device is shown that displays an extended reality environment (e.g., a computer-generated environment) according to an embodiment of the present disclosure.

[0013] Figure 2 A block diagram of an exemplary architecture of a system or device according to an embodiment of this disclosure is shown.

[0014] Figure 3A The present disclosure illustrates a content editing application that includes an editing environment graphical user interface and representative content, according to some embodiments thereof.

[0015] Figure 3B An XR environment is shown to be presented to a user using a second electronic device according to an embodiment of this disclosure.

[0016] Figures 4A to 4S Exemplary user interfaces and / or user interactions with one or more objects that provide a 3D preview of content within a 3D environment are shown according to embodiments of this disclosure.

[0017] Figures 5A to 5B A flowchart illustrating a process for manipulating virtual objects according to an embodiment of this disclosure is shown. Detailed Implementation

[0018] A physical environment refers to the physical world that people can sense and / or interact with without the aid of electronic devices. A physical environment can include physical features such as physical surfaces or physical objects. For example, a physical environment corresponds to a physical park that includes physical trees, physical buildings, and physical people. People can directly sense and / or interact with a physical environment through senses such as sight, touch, hearing, taste, and smell. Conversely, an extended reality (XR) environment refers to a fully or partially simulated environment that people sense and / or interact with via electronic devices. For example, an XR environment can include augmented reality (AR) content, mixed reality (MR) content, virtual reality (VR) content, etc. In the case of an XR system, a subset of a person's physical motion or a representation thereof is tracked, and in response, one or more features of one or more virtual objects simulated in the XR system are adjusted in a manner consistent with at least one physical law. For example, an XR system can detect head movement and, in response, adjust the graphical content and sound field presented to the person in a manner similar to how such views and sounds change in a physical environment. For example, an XR system can detect movement of an electronic device (e.g., a mobile phone, tablet, laptop, etc.) presenting the XR environment, and in response, adjust the graphical content and sound field presented to the user in a manner similar to how such views and sounds would change in a physical environment. Similarly, an XR system can detect interaction with one or more objects in the XR environment (e.g., (virtual) touch, tap, pinch, etc.), and in response, adjust and / or update the graphical content presented to the user in a manner similar to how such objects or views of such objects would change in a physical environment. In some implementations, the XR system may adjust the characteristics of the graphical content in the XR environment in response to representations of physical motion (e.g., voice commands).

[0019] Many different types of electronic systems enable people to sense and / or interact with a variety of XR environments. Examples include head-mounted systems, projection-based systems (including hologram-based systems), head-up displays (HUDs), head-mounted displays (HMDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays formed as lenses designed to be placed on a person's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without haptic feedback), smartphones, tablets, and desktop / laptop computers. Head-mounted systems may have an integrated opaque display and one or more speakers. Alternatively, head-mounted systems may be configured to receive external opaque displays (e.g., smartphones). Head-mounted systems may incorporate one or more imaging sensors for capturing images or video of the physical environment, and / or one or more microphones for capturing audio of the physical environment. Head-mounted systems may have transparent or semi-transparent displays instead of opaque displays. Transparent or semi-transparent displays may have a medium through which light representing the image is directed to the person's eyes. The display can utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser scanning light source, or any combination of these technologies. The medium can be an optical waveguide, holographic medium, optical combiner, optical reflector, or any combination thereof. In some implementations, transparent or translucent displays can be configured to selectively become opaque. Projection-based systems can employ retinal projection technology, which projects graphic images onto the human retina. Projection systems can also be configured to project virtual objects onto a physical environment, such as as holograms or on a physical surface.

[0020] In some implementations, XR content can be presented to the user via an XR data file (data file) (including scripts, executable code, etc.), which includes data representing the XR content and / or data describing how the XR content should be presented. In some implementations, the XR file includes data representing one or more XR scenes and one or more triggers for presenting the one or more XR scenes. For example, an XR scene can be anchored to a horizontal, flat surface such that the XR scene can be presented when the horizontal, flat surface is detected (e.g., within the field of view of one or more cameras). The XR file may also include data about one or more virtual objects associated with the XR scene, and / or triggers and actions associated with the XR virtual objects.

[0021] Typically, to simplify the generation of XR files and / or the editing of computer-generated graphics, content creation applications that include a content generation environment (e.g., an editing environment graphical user interface (GUI)) can be used. In some embodiments, the content generation environment itself is an XR environment (e.g., a two-dimensional and / or three-dimensional environment). For example, the content generation environment may include one or more virtual objects and one or more representations of real-world objects. In some embodiments, virtual objects are overlaid on the physical environment or its representation. In some embodiments, the physical environment is captured via one or more cameras of an electronic device and actively displayed in the XR environment (e.g., via a display generation component). In some embodiments, the physical environment is provided (e.g., passively) by an electronic device, for example, if the display generation component includes a translucent or transparent element through which the user can see the physical environment.

[0022] In such a content-generating environment, users can create virtual objects from scratch (including the appearance of the virtual objects, their behavior / actions, and / or the triggering of their behavior / actions). Additionally or alternatively, virtual objects can be created by other content creators and imported into the content-generating environment, where they can be placed within an XR environment or scene. In some implementations, virtual objects generated in or throughout the content-generating environment (e.g., via generating XR files and importing or opening them in a content creation application or XR viewer application) can be exported to other environments or XR scenes.

[0023] Some embodiments of this disclosure relate to a 3D preview of content (e.g., XR content, also referred to herein as an XR content item) generated and rendered in a 3D environment (e.g., a computer-generated environment) on an electronic device. In some embodiments, the 3D preview of the content is presented simultaneously with a 2D representation of the content in a content-generating environment (e.g., a content creation application) rendered in the 3D environment. In some embodiments, when a 3D preview of the content is rendered in a 3D environment, one or more display representations are provided for interacting with one or more computer-generated virtual objects of the 3D preview. In some embodiments, one or more display representations may be displayed together with the 3D preview of the content in the 3D environment (e.g., displayed below, in front of, above, near, etc., the one or more virtual objects of the 3D preview). In some embodiments, the 3D preview of the content is presented on a 3D tray (e.g., a user interface element), and one or more display representations are presented outside the tray perimeter and / or in a control bar or other control grouping along the tray perimeter. In some embodiments, as described herein, a group of some or all of the display representations is referred to as a manipulator or object manipulator.

[0024] In some implementations, a 3D preview of the content can be configured to operate in at least two modes. A first mode (e.g., playback mode) can simulate the runtime of the content, where one or more actions associated with the content (e.g., animation, audio clipping, etc.) can be performed. For example, one or more virtual objects in the 3D preview can be animated to move, and one or more virtual objects can perform additional animations or other behaviors in response to input. A second mode (e.g., editing mode) can provide a 3D preview of the content to allow the user to interact with the content for editing. For example, the user can select elements in the 3D content and can select corresponding elements in a 2D representation of the content generation environment.

[0025] In some embodiments, the object manipulator may include a first power representation selectable to enable the 3D preview to operate in a first mode. In some embodiments, the object manipulator may include a second power representation selectable to enable the 3D preview to operate in a second mode different from the first mode. In some embodiments, the first and second power representations may be a single power representation for switching modes (e.g., a play / pause button). In some embodiments, the object manipulator may include a third power representation selectable to scale the size of one or more virtual objects in the preview. In some embodiments, the object manipulator may include a fourth power representation selectable to execute executable code associated with playback of the preview content, such that one or more actions (e.g., animation, audio clipping, etc.) are incrementally performed by one or more virtual objects with each selection of the fourth power representation. In some embodiments, the object manipulator may also include a fifth power representation selectable to enable the 3D preview to operate in a third mode different from the first and second modes. In the third mode, a full-size representation of one or more virtual objects in the 3D preview is displayed within the 3D environment.

[0026] As described herein, some embodiments of this disclosure relate to user interaction with and / or manipulation of a 3D preview of content displayed on an electronic device (e.g., an XR content item). In some embodiments, a 2D representation of an XR content item generated in a content creation application displayed on a first electronic device can be displayed simultaneously with a 3D preview of the XR content item on a second electronic device. In some embodiments, user interaction with a 3D preview of an XR content item received at the second electronic device (e.g., user input, such as touch, tap, movement, reorientation, etc.) can update the display of the 3D preview of the XR content item based on the input. In some embodiments, user input received at the second electronic device is transmitted to the first electronic device in real time, such that the display of the 2D representation of the XR content item and the 3D preview of the XR content item are optionally manipulated simultaneously or nearly simultaneously (e.g., within less than 50 ms of each other).

[0027] Manipulating a 3D preview of content in a 3D environment can include changing the appearance of one or more virtual objects in the 3D preview. In some embodiments, manipulation of the 3D preview is optionally determined by the operating mode of the electronic device presenting the 3D environment. In some embodiments, a computer-generated object manipulator is optionally presented in the 3D environment along with an interactive tray, wherein the interactive tray optionally contains the 3D preview. The interactive tray can be manipulated such that user interaction with the tray can change the appearance of the interactive tray within the 3D environment without necessarily changing the appearance of one or more virtual objects in the 3D preview contained on the tray. In some embodiments, the appearance of the interactive tray can change in response to manipulation of one or more virtual objects in the 3D preview contained on the tray. In some embodiments, a change in the viewpoint associated with the electronic device can change the view of the 3D preview visible to the user. In some embodiments, a change in the viewpoint associated with the electronic device can change the orientation and / or position of the object manipulator within the 3D environment, such that the object manipulator optionally continues to face the user. In some embodiments, a 2D representation of one or more virtual objects generated in a content creation application displayed on a first electronic device can be displayed simultaneously with a 3D preview of one or more virtual objects on a second electronic device. Changes to the viewpoint associated with the second electronic device are optionally independent of the viewpoint of the first electronic device, such that changes to the view of the 3D preview do not optionally alter the view of the 2D representation of one or more virtual objects.

[0028] Figure 1 An electronic device 100 displaying an XR environment (e.g., a computer-generated environment) according to an embodiment of this disclosure is shown. In some embodiments, the electronic device 100 is a handheld or mobile device, such as a tablet computer, laptop computer, smartphone, or head-mounted display. Reference is made below. Figure 2 An example of describing device 100 is a structural block diagram. Figure 1 As shown, electronic device 100 and desktop 120 are located in physical environment 110. In some embodiments, electronic device 100 may be configured to capture an area of ​​physical environment 110 including desktop 120 and plant 156 (shown in the field of view of electronic device 100). In some embodiments, in response to a trigger, electronic device 100 may be configured to be in a computer-generated environment (e.g., by...). Figure 1 A virtual object 130 is displayed in the computer-generated environment (shown as a chair and table representation). This virtual object does not exist in the physical environment 110, but is displayed on top of a computer-generated representation 120' positioned (e.g., anchored to) on top of a real-world desktop 120. For example, the virtual object 130 may be displayed on the surface of desktop 120' in the computer-generated environment displayed via device 100 in response to detecting a planar surface of desktop 120 in the physical environment 110. Figure 1 As illustrated in the examples, the computer-generated environment may include representations of additional real-world objects, such as representation 156' of real-world plant 156. It should be understood that virtual object 130 is a representative virtual object and may include and render one or more different virtual objects (e.g., virtual objects with various dimensions, such as two-dimensional or three-dimensional virtual objects) in a three-dimensional computer-generated environment. For example, a virtual object may represent an application or user interface displayed in the computer-generated environment. In some embodiments, the application or user interface may include the display of content items of a content application (e.g., photos, videos, etc.). In some embodiments, virtual object 130 is optionally configured to be interactive and responsive to user input, allowing the user to virtually touch, tap, move, rotate, or otherwise interact with the virtual object. Furthermore, it should be understood that the 3D environment (or 3D virtual object) described herein may be a representation of a 3D environment (or three-dimensional virtual object) projected or rendered at an electronic device.

[0029] In the following discussion, an electronic device communicating with a display generating component and one or more input devices is described. It should be understood that the electronic device optionally communicates with one or more other physical user interface devices, such as a touch-sensitive surface, physical keyboard, mouse, joystick, hand-tracking device, eye-tracking device, stylus, etc. Furthermore, as mentioned above, it should be understood that the described electronic device, display, and touch-sensitive surface are optionally distributed among two or more devices. Therefore, as used in this disclosure, information on or displayed by the electronic device is optionally used to describe information output by the electronic device for display on a separate display device (touch-sensitive or non-touch-sensitive). Similarly, as used in this disclosure, input received on the electronic device (e.g., touch input received on a touch-sensitive surface of the electronic device, or touch input received on the surface of a stylus) is optionally used to describe input received on a separate input device from which the electronic device receives input information.

[0030] The device typically supports a variety of applications, such as one or more of the following: drawing applications, rendering applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, TV channel browsing applications, and / or digital video player applications. Additionally, the device may support applications for generating or editing computer-generated graphics and / or XR environment content (e.g., applications with content generation environments). Furthermore, the device may support 3D graphics rendering applications for generating XR content and / or XR environments and rendering them in a 3D manner.

[0031] Figure 2 A block diagram illustrating an exemplary architecture of a system or device 250 according to an embodiment of this disclosure is shown. In some embodiments, device 250 is a mobile device, such as a mobile phone (e.g., a smartphone), a tablet computer, a laptop computer, a desktop computer, a head-mounted display, an assistive device for communicating with another device, etc. Device 250 optionally includes various sensors (e.g., one or more hand tracking sensors, one or more position sensors, one or more image sensors, one or more touch-sensitive surfaces, one or more motion and / or orientation sensors, one or more eye-tracking sensors, one or more microphones or other audio sensors, etc.), one or more display generation components, one or more speakers, one or more processors, one or more memories, and / or communication circuitry. One or more communication buses are optionally used for communication between the aforementioned components of device 250.

[0032] In some implementation schemes, such as Figure 2 As shown, system / device 250 can be divided among multiple devices. For example, first device 260 optionally includes processor 218A, one or more memories 220A, communication circuitry 222A, and display generation components 214A that optionally communicate via communication bus 208A. Second device 270 (e.g., corresponding to device 200) optionally includes various sensors (e.g., one or more hand tracking sensors 202, one or more position sensors 204, one or more image sensors 206, one or more touch-sensitive surfaces 208, one or more motion and / or orientation sensors 210, one or more eye-tracking sensors 212, one or more microphones 213 or other audio sensors, etc.), one or more display generation components 214B, one or more speakers 216, one or more processors 218B, one or more memories 220B, and / or communication circuitry 222B. One or more communication buses 208B are optionally used for communication between the aforementioned components of device 270. The first device 260 and the second device 270 may optionally communicate via a wired or wireless connection between the two devices (e.g., via communication circuits 222A to 222B).

[0033] Communication circuits 222A and 222B optionally include circuitry for communicating with electronic devices and networks such as the Internet, intranets, wired and / or wireless networks, cellular networks, and wireless local area networks (LANs). Communication circuits 222A and 222B optionally include circuitry for using near-field communication (NFC) and / or short-range communication such as... The circuit used for communication.

[0034] Processors 218A and 218B include one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some embodiments, memories 220A and 220B are non-transitory computer-readable storage media (e.g., flash memory, random access memory, or other volatile or non-volatile memory or storage devices) storing computer-readable instructions configured to be executed by processors 218A and 218B to perform the techniques, processes, and / or methods described below. In some embodiments, memories 220A and 220B may include more than one non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium can be any medium (e.g., excluding signals) that can tangibly contain or store computer-executable instructions for use by or in connection with instruction execution systems, apparatuses, and devices. In some embodiments, the storage medium is a transient computer-readable storage medium. In some embodiments, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include hard disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state storage such as flash memory and solid-state drives.

[0035] In some embodiments, display generating components 214A, 214B include a single display (e.g., a liquid crystal display (LCD), an organic light-emitting diode (OLED), or other types of display). In some embodiments, display generating components 214A, 214B include multiple displays. In some embodiments, display generating components 214A, 214B may include a display with touch capability (e.g., a touchscreen), a projector, a holographic projector, a retinal projector, etc. In some embodiments, device 270 includes a touch-sensitive surface 208 for receiving user input such as tap input and swipe input or other gestures. In some embodiments, display generating components 214B and touch-sensitive surface 208 form a touch-sensitive display (e.g., a touchscreen integrated with device 270 or a touchscreen external to device 270 that communicates with device 270).

[0036] Device 270 optionally includes an image sensor 206. Image sensor 206 optionally includes one or more visible light image sensors (such as charge-coupled device (CCD) sensors) and / or complementary metal-oxide-semiconductor (CMOS) sensors operable to obtain images of physical objects from the real-world environment. Image sensor 206 also optionally includes one or more infrared (IR) sensors, such as passive or active IR sensors, for detecting infrared light from the real-world environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real-world environment. Image sensor 206 also optionally includes one or more cameras configured to capture movement of physical objects in the real-world environment. Image sensor 206 also optionally includes one or more depth sensors configured to detect the distance between the physical object and device 270. In some embodiments, information from one or more depth sensors may allow the device to identify objects in the real-world environment and distinguish them from other objects in the real-world environment. In some embodiments, one or more depth sensors may allow the device to determine the texture and / or shape of objects in the real-world environment.

[0037] In some embodiments, device 270 uses a combination of a CCD sensor, an event camera, and a depth sensor to detect the physical environment surrounding device 270. In some embodiments, image sensor 206 includes a first image sensor and a second image sensor. The first and second image sensors work cooperatively and are optionally configured to capture different information about physical objects in the real-world environment. In some embodiments, the first image sensor is a visible light image sensor, and the second image sensor is a depth sensor. In some embodiments, device 270 uses image sensor 206 to detect the position and orientation of device 270 and / or display generation component 214 in the real-world environment. For example, device 270 uses image sensor 206 to track the position and orientation of display generation component 214B relative to one or more stationary objects in the real-world environment.

[0038] In some embodiments, device 270 includes microphone 213 or other audio sensors. Device 270 uses microphone 213 to detect sound from the user and / or the user's real-world environment. In some embodiments, microphone 213 includes an array of microphones (multiple microphones) that optionally operate in cooperation to identify ambient noise or locate sound sources in the space of the real-world environment.

[0039] Device 270 includes a position sensor 204 for detecting the position of device 270 and / or displaying the position of generating component 214B. For example, position sensor 204 may include a GPS receiver that receives data from one or more satellites and allows device 270 to determine the absolute position of the device in the physical world.

[0040] Device 270 includes an orientation sensor 210 for detecting the orientation and / or movement of the device 270 and / or display generating component 214B. For example, device 270 uses the orientation sensor 210 to track changes in the position and / or orientation of the device 270 and / or display generating component 214B, such as changes relative to a physical object in a real-world environment. The orientation sensor 210 optionally includes one or more gyroscopes and / or one or more accelerometers.

[0041] In some embodiments, device 270 includes a hand-tracking sensor 202 and / or an eye-tracking sensor 212. The hand-tracking sensor 202 is configured to track the position / location of one or more portions of a user's hand, and / or the movement of one or more portions of the user's hand relative to the extended reality environment, relative to the display generation component 214B, and / or relative to another defined coordinate system. The eye-tracking sensor 212 is configured to track the position and movement of the user's gaze (more generally, the eyes, face, or head) relative to the real world or extended reality environment and / or relative to the display generation component 214B. In some embodiments, the hand-tracking sensor 202 and / or the eye-tracking sensor 212 are implemented together with the display generation component 214B. In some embodiments, the hand-tracking sensor 202 and / or the eye-tracking sensor 212 are implemented separately from the display generation component 214B.

[0042] In some embodiments, the hand-tracking sensor 202 may use an image sensor 206 (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that captures three-dimensional information from the real world, including one or more hands (e.g., one or more hands of a human user). In some embodiments, the hand can be resolved with sufficient resolution to distinguish the fingers and their corresponding positions. In some embodiments, one or more image sensors 206 are positioned relative to the user to define the field of view and interaction space of the image sensors 206, in which the finger / hand positions, orientations, and / or movements captured by the image sensors are used as input (e.g., to distinguish them from the user's resting hand or other hands of other people in the real-world environment). Tracking the fingers / hands used for input (e.g., gestures, touches, taps, etc.) may be advantageous because it does not require the user to touch, hold, or wear any type of beacon, sensor, or other marker.

[0043] In some embodiments, the eye-tracking sensor 212 includes at least one eye-tracking camera (e.g., an infrared (IR) camera) and / or an illumination source (e.g., an IR light source, such as an LED) that emits light toward the user's eyes. The eye-tracking camera may be pointed at the user's eyes to receive reflected IR light from the light source directly or indirectly from the eyes. In some embodiments, both eyes are tracked separately by the respective eye-tracking camera and illumination source, and focus / gaze can be determined by tracking both eyes. In some embodiments, one eye (e.g., the dominant eye) is tracked by the respective eye-tracking camera / illumination source.

[0044] Device 270 and system 250 are not limited to Figure 2 The components and configurations may include fewer components, alternative components, or additional components in a variety of configurations. In some embodiments, system 250 may be implemented in a single device. A person using system 250 is optionally referred to herein as a user of the device. Concern now is the exemplary simultaneous display of a two-dimensional representation of a content item and a corresponding three-dimensional preview of the content item. As described below, a two-dimensional representation of the content item may be displayed on a first electronic device (e.g., via a content creation application), and a three-dimensional preview of the content item may be simultaneously displayed on a second electronic device (e.g., via a three-dimensional graphics rendering application). In some embodiments, the process of generating a three-dimensional preview of the content item, as described below, may be performed by processors 218A, 218B of devices 260 and 270.

[0045] Figure 3A A content creation application 362, including an editing environment GUI and representative content 364, is shown according to some embodiments of the present disclosure. The content creation application 362, including the editing environment GUI, may be displayed on an electronic device 360 ​​(e.g., similar to device 100 or device 260), including but not limited to portable or non-portable computing devices such as tablet computing devices, laptop computing devices, or desktop computing devices. Figure 3A A real-world environment (e.g., a room) 352 is shown, which includes shelves 356 and plants 354 located in the rear of the real-world environment behind an electronic device 360. As an example, a content creation application 362 can display a two-dimensional representation of a 3D computer environment defined by the X, Y, and Z axes and including content 364. Figure 3A In the example, content 364 is a chair and a table, but it should be understood that the chair and table are merely representative, and one or more different virtual objects (e.g., one-dimensional (1D), 2D, or 3D objects) can be imported or selected from the content library (which includes multiple shapes, objects, symbols, text, numbers, etc.) and included in the 3D environment.

[0046] exist Figure 3A In the example, content items (e.g., XR content items, such as...) are created in a content creation application 362 running on an electronic device 360. Figure 1 The virtual object 130 in the image can be previewed in three dimensions via a three-dimensional graphics rendering application running on a second electronic device, as shown in the following reference. Figure 3B Discussed in more detail, the 3D graphics rendering application optionally communicates with the electronic device 360. In some embodiments, the content creation application may optionally, in response to a user's request to preview the content item 364 in a 3D manner, transmit the 3D graphics data corresponding to the content item to the 3D graphics rendering application, which can generate and (e.g., in an XR environment) render a 3D preview of the content item.

[0047] In some implementation schemes, such as Figure 3A As shown, a user can work within a content creation application 362 via an electronic device 360 ​​to design and / or modify content 364. The content creation application optionally communicates with an integrated design environment (IDE). The content creation application 362 and / or the IDE (not shown) optionally utilize graphical data files (e.g., including scripts, executable code, etc.) that describe content items (e.g., defining the appearance, actions, responsiveness, etc. of content 364) targeted at a three-dimensional operating system (e.g., designed for a three-dimensional graphics environment). In some embodiments, the data file describing the content items may be uploaded to and / or initiated by the content creation application 362, such that a two-dimensional representation of the content items can be displayed on the electronic device 360 ​​(e.g., via...). Figure 2 The display generation component 214A in the first electronic device displays the two-dimensional content. It should be understood that the two-dimensional representation is a function of the two-dimensional display of the first electronic device, but the two-dimensional representation can represent three-dimensional content.

[0048] In some implementations, a two-dimensional representation of the content item may be displayed within the display GUI of the content creation application 362 (e.g., within or near the editing environment GUI). In some implementations, graphical data files uploaded to the content creation application may be stored on the electronic device 360 ​​(e.g., stored in memory 220A or downloaded and accessed from a web-based storage device).

[0049] In some implementations, the data file can be edited while running on the content creation application 362. In such implementations, scripts, executable code, etc., can be displayed within the editing environment's GUI, allowing the user to directly edit portions of the scripts, executable code, etc., at the electronic device 360. Where applicable, editing of the graphical data file can update the appearance of the two-dimensional representation of the content item 364 displayed on the electronic device 360. As described herein, in some implementations, editing of the data file can be performed within an IDE that communicates with the content creation application 362.

[0050] As mentioned above, a preview of the content item is generated in two dimensions and presented to the user, such as... Figure 3A As illustrated in the example, such a preview can be limited by the two-dimensional display of the device 360 ​​running the content creation application 362. While some aspects of content items created in the content creation application 362 can be captured in two dimensions (e.g., color, two-dimensional dimensions such as height and width, planar views, etc.), others cannot. In particular, for example, because content items created in the content creation application 362 are explicitly designed to be displayed in a three-dimensional environment, a two-dimensional preview may not provide the user with complete information about the three-dimensional appearance and features that might actually be useful during the design phase. Alternative views (e.g., side and rear views), surface textures, lighting effects, etc., may not be visible or captureable within a two-dimensional preview. Furthermore, to view alternative views of the content, for example, the user may need to generate a new preview for each alternative view, which increases the time and effort required for the workflow of designing, previewing, and modifying content items, and thus increases complexity. Therefore, as described below, providing an interactive preview of the three-dimensional content during the design phase of the digital content creation process can be particularly useful.

[0051] In some implementations, a user may request a three-dimensional preview of a two-dimensional representation of a content item (e.g., the chair and table in content 364). As an example, the content creation application 362 may display a menu 370 including (e.g., displayed in an editing environment GUI, a display GUI, or in or in another GUI within the content creation application or communicating with the content creation application) one or more optional graphical user interface elements, which generates a request when selected. Additionally or alternatively, in some implementations, the request may be entered using one or more input devices 366 communicating with the electronic device 360, such as by pressing one or more keys on a keyboard. Figure 3AAs shown in the example, a user can select "Preview" from menu 370, as indicated by selection 350, to request a three-dimensional preview of content 364. In some embodiments, in response to receiving a preview request, the electronic device can initiate a data transmission of three-dimensional graphic data defining the content item (e.g., the table and chair of content 364), wherein the three-dimensional graphic data is optionally transmitted to a second electronic device. The electronic device 360 ​​can communicate with the second electronic device via any suitable communication method, such as via wired or cable transmission (e.g., Universal Serial Bus), wireless transmission (e.g., Wi-Fi or...). In some implementations, the 3D graphics data may be received by a 3D graphics rendering application running on a second electronic device, wherein the 3D graphics rendering application is configured to generate and render a 3D preview of content items defined by the 3D graphics data, as described below.

[0052] Figure 3B The use of a second electronic device (e.g., corresponding to an embodiment of the present disclosure) is illustrated in the embodiment of the present disclosure. Figure 2 An XR environment 368 is presented to the user by an electronic device 270. For example, when viewed from the perspective of a second electronic device (e.g., a viewpoint associated with the second electronic device, which may be, for example, a head-mounted display), the 3D computer-generated environment 368 can be defined by the X, Y, and Z axes. In some embodiments, the second electronic device can capture portions of the real-world environment (e.g., via an image sensor). Figure 3B As shown, the three-dimensional computer-generated environment 368 may include a first electronic device 360 ​​that presents and displays... Figure 3A A two-dimensional representation of the content items (or a representation of the content items 364' displayed in the first electronic device 360' and / or content application 362'). Additionally or alternatively, the three-dimensional environment 368 includes presenting one or more input devices 366 (or displaying a representation of one or more input devices). Although in Figures 3A to 3B Not shown, but the first electronic device and one or more input devices may be placed on a table that can be presented in the environment (or a representation of the table may be displayed in the environment). Additionally or alternatively, the three-dimensional environment 368 includes portions that present a representation of the real-world environment, including shelves 356 and plants 354 or shelves 356' and plants 354'.

[0053] Additional or alternative locations, the 3D environment 368 optional locations include presentation and Figure 3AThe content 364 shown corresponds to a 3D preview 334 of one or more virtual objects 332. It should be understood that the virtual objects shown are merely representative, and one or more different virtual objects can be imported or designed within a content creation application and included in the 3D environment 368. In some embodiments, the 3D preview 334 and the content creation application are displayed simultaneously on the first electronic device. This allows designers to create content using familiar editing tools and enhance the design process with a real-time 3D preview. Additionally, the 3D preview can provide designers with additional ways to interact with the content in the 3D preview and / or with the content in the content creation application on the first electronic device.

[0054] For example, in some embodiments, the second electronic device optionally includes a hand-tracking sensor (e.g., corresponding to hand-tracking sensor 202) and an eye-tracking sensor (e.g., corresponding to eye-tracking sensor 212), which allows a user to interact with and manipulate one or more virtual objects within a three-dimensional environment. As an example, the eye-tracking sensor can track the gaze associated with one or both of the user's eyes to determine the viewpoint associated with the second electronic device, and thus determine the orientation of the viewpoint within the three-dimensional environment. For example, the hand-tracking sensor can track the movement of one or more of the user's fingers to associate the corresponding finger movements (e.g., touch / tap, pinch, drag, etc.) with one or more interactions with one or more elements in the three-dimensional environment. The user can provide input corresponding to the selection and / or manipulation of one or more elements within the three-dimensional environment via the corresponding finger movements.

[0055] like Figure 3BAs shown, the three-dimensional environment optionally includes an interactive tray (“tray”) 336 on which a three-dimensional representation 332 of content is presented within the three-dimensional environment 368. Users can interact with and / or manipulate the tray and / or its content. For example, interaction with a three-dimensional preview 334 of the content can cause the content to be repositioned in two or three dimensions within the three-dimensional environment 368 (e.g., moved in the plane of the tray and / or moved above or below the tray) and / or reoriented (e.g., rotated). As shown, the three-dimensional environment may also include an interactive toolbar (“toolbar”, “object manipulator”) 338 associated with the tray 336 and including one or more user interface elements (enabled representations) 340 capable of receiving user input. In some embodiments, some or all enabled representations may be selectable to control the appearance and / or one or more actions of one or more virtual objects 332 of the three-dimensional preview 334. As discussed in detail below, a user can interact with one or more of the power indicators 340 to activate one or more modes of the device (e.g., a first mode, a second mode, or a third mode). This allows the user to view animations of one or more virtual objects, select virtual objects from one or more virtual objects for editing, and / or zoom and project full-size renderings of one or more virtual objects within the 3D environment, etc. It should be understood that in some embodiments, the interactive tray 336 and / or toolbar 338 (and the associated power indicators 340) are optionally not displayed within the 3D environment 368. For example, in some embodiments, the 3D preview 334 may include one or more virtual objects 332 without including the interactive tray 336 and toolbar 338. Additionally or alternatively, in some embodiments, for example, power indicators or menus (not shown) may be presented within the 3D environment 368 to control whether the tray 336 and / or toolbar 338 are presented within the 3D environment 368.

[0056] As discussed herein, a 3D preview of content displayed simultaneously with a 2D representation of the content in a content creation application can provide the user with useful visual feedback about the appearance of the 3D content, which would otherwise be provided without the 2D representation. In some implementations, editing or modifying a data file running in a content creation application can produce a corresponding change in the appearance of the 3D preview displayed on a second electronic device. As an example, a user may wish to edit or modify one or more features of a content item and view a new 3D preview of the content item based on the edit or modification. For example, a user may rewrite portions of a data file's script, executable code, etc., via one or more input devices (e.g., via a keyboard) that communicate with a first electronic device, while a 2D representation of the content item is displayed on the first electronic device and a 3D preview of the content item is simultaneously displayed on a second electronic device. The user can eventually complete the edit or modification (e.g., by saving changes to the data file) and can request a new preview representing the content item of the data file. Additionally or alternatively, a new preview can be automatically requested once the user has completed the edit or modification. New (e.g., newly updated) data can be transferred from the content creation application to the 3D rendering application in the manner described above, and the 3D preview of the content item currently displayed on the second electronic device can be updated according to the corresponding change in the 2D representation of the content item displayed on the first electronic device, so that the 3D preview of the content item has an updated appearance.

[0057] As described above, a 3D preview 334 of the content item can be displayed on a second electronic device, while a 2D representation 364' of the content item is simultaneously displayed on a first electronic device 360'. In some embodiments, the 2D representation and 3D preview of the content can be provided at a single electronic device (e.g., a laptop computer, desktop computer, mobile device, etc.) instead of being provided at two separate electronic devices. For example, a 3D graphics rendering application can be provided within or at least partially as a simulator configured to display a computer-generated environment in a 3D manner. In some such embodiments, in response to receiving a request to display the content item in a 3D manner, the 3D graphics rendering application can generate a preview of the content in the computer-generated environment of the simulator (e.g., in a different window on the monitor of the electronic device) and render it in a 3D manner. In some such embodiments, for example, the 2D representation of the content can be displayed within the display GUI of a content creation application, while the 3D preview of the content is simultaneously displayed within the computer-generated environment of the simulator. Additionally or alternatively, in some embodiments, the content creation application can be directly communicatively linked to the 3D graphics rendering application. In some of these implementations, all or part of the scripts, executable code, etc. of the data file at 310 can be directly transmitted to the 3D graphics rendering application at 318.

[0058] In some implementations, the graphics data transmitted along the communication channel between the first electronic device 360 ​​and the second electronic device can be synchronized. In such implementations, for example, the communication channel between the content creation application 362 and the 3D rendering application (not shown) can be bidirectional, thereby allowing data to be selectively transmitted between them in either direction, as indicated by the operating mode of the second electronic device (e.g., a head-mounted display).

[0059] In some implementations, as used herein, a first operating mode of the second electronic device (“first operating mode”, “first mode”) may refer to a playback mode (“playback mode”, “live mode”), during which animations (including audio effects and / or lighting effects) of one or more virtual objects within a three-dimensional environment can be presented to the user. In some implementations, as used herein, a second operating mode (“second operating mode”, “second mode”) may refer to a selection mode (“selection mode”, “edit mode”), during which the user can select a corresponding virtual object of one or more virtual objects at the second electronic device to allow the user to edit / modify the appearance, animation, etc., of the corresponding virtual object at the first electronic device. In some implementations, user input from the second electronic device is used, and some editing is also possible. In some implementations, as used herein, a third operating mode (“third operating mode”, “third mode”) may refer to an immersive mode, during which one or more virtual objects are displayed at full size on occluded portions of the real-world environment within the three-dimensional environment. As described below, when the second electronic device is operated in one of the operating modes, the user can interact with a three-dimensional preview of the content item to improve the content creation of the XR content item.

[0060] Figures 4A to 4S Exemplary user interfaces and / or user interactions with one or more objects of a 3D preview 434 of content 432 within a 3D environment 468 are illustrated according to embodiments of this disclosure. The exemplary user interactions continue below. Figures 3A to 3B The examples shown and discussed above illustrate this. Therefore, the 3D environment generated and rendered at the second electronic device may include a 3D preview of the content and a representation of captured portions of the real-world environment, such as... Figure 3BAs shown above, a 3D preview of a content item can provide users with an improved perspective on how the content item looks and appears in 3D. It can be advantageous for users to view the 3D preview at various sizes and orientations from various angles and perspectives, as well as to test and watch the animations and related actions associated with the content item in the 3D preview. Various methods for interacting with and manipulating the 3D preview of a content item are provided below, thereby offering an enhanced and improved user experience for viewing content items in 3D.

[0061] Figure 4A An exemplary user interaction with a 3D preview 434 in an activated playback mode, according to an embodiment of this disclosure, is illustrated. As described above, the 3D preview displayed by the second electronic device can be configured to be in one or more modes, including Figure 4A The example shows the operation under the playback mode (e.g., the first operation mode). Figure 4A As shown in the example, a 3D preview 434 of one or more virtual objects 432 may be displayed within a 3D environment 468, which optionally includes representations of captured portions of the real world (e.g., shelves 456' and plants 454'). In some embodiments, a user may select a first power indicator 442 of the toolbar 438 (“first power indicator”, “playback mode power indicator”, “live mode power indicator”), as shown in selection 450A. The selection of the first power indicator 442 may cause one or more virtual objects on top of the tray 436 to perform any associated behavior. For example, a first virtual object (e.g., a chair) 435 of one or more virtual objects 432 may be configured to perform an action (e.g., animation, such as a rotation / turning motion, as shown by arrow 437), and this action may be performed in live mode (e.g., the action is designed to be triggered without further user input). In some embodiments, scripts, executable code, etc., defining one or more virtual objects 432 may include instructions for animate one or more virtual objects within a specific 3D environment (e.g., a computer game, mobile application interface, television program, movie, etc.). In some implementations, when the device is operating in playback mode, one or more virtual objects may perform alternative or additional actions, such as audio manipulation, lighting manipulation, object manipulation (e.g., object scaling), movement, and so on.

[0062] In playback mode, users can easily and clearly observe animations programmed and defined within a content creation application (e.g., 362) operating on a first electronic device (e.g., 360). This allows users to edit or modify the animation of the first virtual object 435 via the first electronic device (e.g., by editing corresponding portions of scripts or code in the content creation application and / or IDE), and to view the updated animation of the first virtual object 435 in real time (e.g., within a threshold amount of time (e.g., 50 ms)). Furthermore, as discussed herein, users can view the animation from different perspectives (e.g., by walking around the 3D preview or viewing the 3D preview from above), which might be difficult or impossible in a 2D view (e.g., on a 2D screen of the first electronic device). Therefore, an advantage of the disclosed method is that animations and related actions associated with one or more virtual objects can be observed in a 3D preview, allowing users to easily and selectively modify the animations and related actions as needed. In some embodiments, the animations of one or more virtual objects 432 presented at the second electronic device can be simultaneously displayed in a 2D manner on the first electronic device. For example, when the three-dimensional chair 435 presented to the user of the second electronic device rotates on top of the tray 436, the corresponding two-dimensional representation of the chair displayed on the first electronic device (e.g., as shown in the image) Figure 3A (As shown) can also be rotated, allowing the user to clearly identify the corresponding representation on the first electronic device for editing as needed.

[0063] Figure 4B An exemplary user interaction with a 3D preview 434 is shown when a second electronic device is operating in playback mode, according to an embodiment of this disclosure, to select a virtual object. (Continued) Figure 4A For example, when a playback mode is active (e.g., when animations and related actions are actively presented), a user can interact with one or more virtual objects 432 to observe responses associated with user input. For instance, a user can provide input to select a second virtual object 433 from one or more virtual objects 432. In some examples, the selection input can be an indirect interaction. For example, a user can gaze at the second virtual object and select it by keeping their gaze on it for a threshold time period, or alternatively by aiming their gaze at the object and performing a selection via another input such as touch input (e.g., pinching two fingers, pressing a button, performing a gesture), using a voice command, etc., as shown in selection 450B. In some embodiments, the selection input can be a direct interaction of the user directly touching an object in the three-dimensional environment, as shown in selection 450B. In some embodiments, such as... Figure 4BAs shown in the example, an action corresponding to table 433 occurs (e.g., in response to receiving a selection 450B on table 433). This action can be defined by one or more instructions encoded for table 433. For example, table 433 can be encoded (via content creation application 362 or IDE) to perform a rotation / turning action triggered by the selection input, as indicated by arrow 439. Alternatively, in some embodiments, alternative actions associated with the table can be encoded to be performed in response to selection 450B (e.g., emitting audio, performing an alternative movement, etc.). It should be understood that additional or alternative actions can be performed by one or more virtual objects, or associated with an object in response to the same input, additional input, or alternative input. For example, a double-click input (e.g., by one or more fingers) can move table 433 to a corresponding position on tray 436 different from its initial position before the double-click input.

[0064] In some implementations, animations of one or more virtual objects 432, or other actions associated with virtual objects presented at the second electronic device, can be simultaneously displayed in a two-dimensional manner on the first electronic device. For example, when table 433 rotates on top of tray 436 at the second electronic device in response to selection 450, a corresponding two-dimensional representation of the table displayed on the first electronic device (e.g., as shown in the image) Figure 3A (As shown) can also be rotated, allowing the user to clearly identify the corresponding representation on the first electronic device for editing as needed.

[0065] Figure 4C An exemplary user interaction with a 3D preview 434 in stop-play mode, according to an embodiment of this disclosure, is shown. Continue Figures 4A to 4B For example, when the playback mode is active (i.e., when animations and / or other actions associated with one or more virtual objects 432 are actively rendered), the user can interact with the first enabled representation 442 to stop the playback mode from operating. For example, as Figure 4C As shown in the example, the user can select the first power indicator 442 of toolbar 438 a second time, as shown in selection 450C, to stop the first virtual object 435 and / or the second virtual object 433 from rotating / turning (as indicated by the omission of arrows 437 and 439). In this way, the user effectively stops operation in playback mode. In some embodiments, stopping playback mode results in a transition to a second mode (e.g., selection mode / edit mode). In some embodiments, stopping playback mode pauses the playback of the animation / action, but playback can be resumed when the user later selects the same or a different power indicator.

[0066] Figure 4DAn exemplary user interaction is shown, illustrating the selection of a virtual object and operation of a 3D preview 434 in selection / edit mode according to an embodiment of this disclosure. As described above, and continuing... Figure 4C For example, in some implementations, pausing playback causes a transition from playback mode to selection mode. In selection mode, individual items of content 432 can be selected to quickly edit those items. The selection input can be related to the above regarding... Figure 4B The described interactions are the same or similar, whether direct or indirect.

[0067] In some implementation schemes, such as Figure 4D As shown in the example, when the second electronic device operates in selection mode, the user can select a first virtual object 435, as shown in selection 450D. In some examples, the appearance of the selected virtual object changes. For example, the first virtual object 435 may be highlighted in response to selection, as shown in highlight 476 around object 435. It should be understood that additional or alternative indications may be provided to indicate the selection of one of the virtual objects 432 (e.g., glow effects, animation of arrows pointing to virtual objects, etc.). Figure 4D As shown, in some embodiments, the selection of a first virtual object 435 when in selection mode optionally causes the selection and change of the appearance of the corresponding two-dimensional representation 435' of the virtual object (chair), as highlighted in highlight 475. In some embodiments, the change in the appearance of the selected object 435 and object 435' may differ between the 3D preview and the content creation application (e.g., different highlightings with shadows, different effects, etc.). Additionally or alternatively, in some embodiments, the change in the appearance of the corresponding portion of the script or executable code defining chair 435' in the content creation application 462' (and / or in the IDE) may occur simultaneously with the change in the appearance of chair 435 and / or chair 435', as highlighted in highlight 477. As illustrated by example, the code may be rendered within a script editor window / UI 458' (or a representation of a script editor window) that may be rendered and / or updated in response to the user selecting chair 435 in the 3D preview 434 (and / or within the content creation application).

[0068] As previously mentioned in this disclosure, the user can actively edit the content 464' within the content creation application 462' at the first electronic device 460' using feedback provided by the 3D preview 434. Figure 4DAs shown in the example, when a portion of the code within the script editor 458' is highlighted, the user can select "Edit" from menu 470', as shown in selection 452. The user can then edit the highlighted portion of the code shown at 477 to change the position of the chair 435 relative to the table 433 in the 3D environment 468, for example, as discussed in more detail below. In some implementations, the user can use an input device to perform editing on the characteristics of a selected object in other user interfaces within the script editor 458' and / or the content creation application 462'. For example, the user can edit the highlighted portion of the code or modify other editable user interface fields via one or more input devices 466' (e.g., by clicking or double-clicking the code within the script editor window 458' with a mouse and modifying the code with a keyboard). Additionally or alternatively, the user can click and drag the arrow keys on the keyboard to change the position of the selected object.

[0069] Figure 4E An exemplary user interaction with a 3D preview 434, shown in selection mode, is illustrated according to an embodiment of this disclosure. (Continued) Figure 4D For example, in some implementations, a user can edit content creation application 464' within a first electronic device 460'. As described above, a user can reposition a chair 435 relative to a table 433 in a three-dimensional environment 468, such as... Figure 4E As shown in the example. After editing, the user can optionally complete the editing by selecting "Save" from menu 470', as shown in selection 450E. It should be understood that the user can complete / save changes made to the code defining one or more virtual objects in other ways, for example, by inputting a combination of keystrokes from a keyboard communicating with the first electronic device. In some implementations, the editing is reflected in real time without being saved.

[0070] In some implementations, as shown, the 3D preview 434 can be updated in response to changes made to content 464' in content creation application 462' at the first electronic device 460'. In some implementations, the 3D preview 434 can be updated in real time (e.g., within a threshold amount of time) without requiring the user to request a new (e.g., a second) preview. Figure 4EAs shown, chair 435 is repositioned to the right of table 433 while maintaining its original orientation (e.g., continuing to face outwards). Since the only change made to chair 435 involves its position on tray 436, according to this example, selecting the first power indicator (e.g., 442) optionally activates the playback mode and causes the chair to rotate / turn in the same manner as previously shown and described. Therefore, as described above, one advantage of the disclosed method is that the user can easily edit or modify virtual objects by selecting them in three-dimensional space at the second electronic device and directly editing the relevant portions of the code defining the virtual object at the first electronic device, thus allowing for an interactive and concise content creation and editing process.

[0071] Figure 4F An exemplary user interaction with a 3D preview 434 of expanding the size of one or more virtual objects according to an embodiment of this disclosure is illustrated. In some embodiments, a user can increase or decrease the size of some or all of the one or more virtual objects 432 within a 3D environment 468 to, for example, view additional or alternative features of the one or more virtual objects (e.g., surface texture, color, etc.). In some embodiments, a user can increase or decrease the size of one or more virtual objects via eye gaze and / or by using hand / finger manipulation.

[0072] In some implementation schemes, such as Figure 4F As shown in the example, the user's gaze is directed towards the 3D preview (e.g., looking at one or more virtual objects 432 on top of tray 436), and the user performs hand gestures to expand or shrink one or more virtual objects. Hand gestures may include pinching two fingers of a first hand, pinching two fingers of a second hand, and moving both hands closer or further apart while maintaining the pinch. The direction of the movement determines whether the object's size is increased or decreased. In some examples, movement of the separated hands (increasing the distance between the pinched finger pairs, as shown by arrows 437 and 439) increases the size of one or more objects. For example, as... Figure 4F As shown in the example, the sizes of chair 435 and table 433 are relative to Figure 4C There is an increase. In some examples, the movement of the hands together (reducing the distance between the pinching finger pairs) reduces the size of one or more objects. In some implementations, the amount of change in the size of one or more objects is a function (linear or non-linear) of the amount of change in the distance between the two hands. In some implementations, to expand or shrink the size of one or more virtual objects 432, a pinch of two fingers is initiated on opposite sides of the tray 436, as shown in pinch inputs 450F and 452F. In some implementations, the pinching of the fingers of both hands must occur within a threshold time period.

[0073] Additionally or alternatively, in some embodiments, the second electronic device may change the size of one or more virtual objects 432 in the three-dimensional environment 468 in response to user input pointing to a representation of a pivot point displayed together with the tray 436. For example, the second electronic device may display a pivot point (e.g., a graphic point, pin, marker, etc.) on the tray 436 in response to detecting a pinch selection and hold it for a threshold time period (e.g., 1 second, 2 seconds, 3 seconds, 4 seconds, etc.) while the user's gaze is directed toward the tray 436 (e.g., and the user's hand does not move, or has less than a threshold movement, until the threshold time period is met). In some embodiments, the pivot point may be displayed at a predefined location on the tray 436. For example, the pivot point is displayed at the location of the user's gaze on the tray 436. In some embodiments, the pivot point is displayed at a predetermined location on the tray 436 that is closest to (e.g., within a threshold distance such as 0.5 cm, 1 cm, 1.5 cm, 2 cm, 3 cm, etc.) the location of the user's gaze on the surface of the tray 436. In some embodiments, the pivot point is displayed at the center point of tray 436, regardless of the user's gaze position on tray 436. In some embodiments, the pivot point is displayed at a corner of tray 436, regardless of the user's gaze position on tray 436. In some embodiments, the pivot point is displayed at other locations on tray 436, regardless of the gaze position (e.g., at locations adjacent to and / or between virtual objects 432). In some embodiments, the pivot point is displayed at the center point or at the corresponding corner point closest to the gaze position. In some embodiments, the second electronic device does not display a representation of the pivot point and can change the size of one or more virtual objects 432 in the three-dimensional environment relative to the pivot point (e.g., the pivot point is optionally not actually displayed on tray 436 in the three-dimensional environment 468).

[0074] In some implementations, the second electronic device may scale one or more virtual objects 432 on top of the tray 436 in response to movement of the user's hand relative to a pivot point on the tray 436, while maintaining the pinch selection and hold discussed above. For example, when the second electronic device detects a pinch selection and holds the pivot point displayed on the tray 436, the tray 436 may increase in size in response to movement of the user's hand in a first direction (e.g., toward and / or away from the pivot point on the tray 436), such as... Figure 4LSimilar to that shown in the diagram. Additionally, in some embodiments, the size of the tray 436 may decrease in response to movement of the user's hand in a second direction opposite to the first direction (e.g., away from and / or toward a pivot point on the tray 436). In some embodiments, the second electronic device may scale one or more virtual objects 432 on top of the tray 436 in response to movement of the user's hand relative to a position in space (e.g., in the three-dimensional environment 468) where a pinch is established and a selection and holding (as described above) is made. For example, the movement of the user's hand is relative to a position in space where a selected pinch-hold threshold time period is detected (e.g., and no movement is detected or there is less than a threshold movement of the user's hand until the holding threshold time period is met).

[0075] In some embodiments, instead of scaling the tray 436, input can be used to scale virtual objects 432 displayed on the tray. In some embodiments, in response to detecting movement of a user's pinching hand relative to a pivot point along a first direction (e.g., toward a pivot point on the tray 436, away from the user's body), the second electronic device can shrink one or more virtual objects 432 on the tray 436 (e.g., reduce their size). In some embodiments, in response to detecting movement of a user's pinching hand along a second direction opposite to the first direction (e.g., away from a pivot point on the tray 436, toward the user's body), the second electronic device can enlarge one or more virtual objects 432 on the tray 436 (e.g., increase their size), such as... Figure 4F Similar to the illustration. In some embodiments, the amount by which one or more virtual objects 432 are scaled on the tray 436 (e.g., a scaling factor) is proportional to the amount of movement of the hand relative to a pivot point in the three-dimensional environment 468. Thus, as described above, providing a graphical pivot point on the tray 436 as a reference point by which one or more virtual objects 432 can be scaled in response to user input simplifies the gestures required to scale one or more virtual objects 432 in the three-dimensional environment 468 (e.g., by reducing gestures from two-handed gestures to one-handed gestures). Furthermore, providing a graphical pivot point on the tray 436 as a reference point simplifies the user interface objects provided for scaling one or more virtual objects 432, since the graphical pivot point is only optionally displayed in response to the detection of pinch-to-select and hold input, while maintaining the pinch-to-select and hold input as described above.

[0076] In some implementations, in response to detecting the release of a pinch selection (e.g., causing the user to no longer grip the pivot point), the second electronic device no longer displays the pivot point on tray 436 (e.g., stops displaying corner gripper 458). Subsequent movement after the release of the pinch does not scale virtual objects 432 and / or tray 436. For example, movement of the user's hand toward and / or away from the pivot point will not cause the second electronic device to scale one or more virtual objects 432 on top of tray 436 in the 3D environment 468. In some implementations, movement of the user's hand after a pinch selection (e.g., above a threshold movement before the threshold time period for establishing and holding the pinch selection) will not cause the pivot point to be displayed on tray 436. In some implementations, a pinch selection after a hand movement (e.g., without establishing a hold before the movement) can cause the second electronic device to adjust according to the hand movement (e.g., as...). Figure 4I (Similarly shown in the diagram) the virtual object 432 on the tray 436 is moved (translated) in the three-dimensional environment 468, rather than scaling one or more virtual objects 432.

[0077] like Figure 4F As illustrated in the examples, in some implementations, changing the size of virtual objects 433 and 435 does not change the size of the tray 436 or toolbar 438. Thus, scaling one or more virtual objects 432 does not cause scaling of other elements within the 3D environment 468, which could interfere with the visibility of the 3D preview 434 (e.g., if the tray shrinks) or other content in the 3D environment (e.g., other applications or due to occlusion between the toolbar 438 and the tray 436 or content items) and / or the ability to interact with controls (e.g., this may be difficult to use when the size and / or position of controls change). It should be noted that in some implementations, increasing or decreasing the size of one or more virtual objects 432 presented at the second electronic device is not received or interpreted as input corresponding to a request to edit or modify the actual size of the corresponding content at the first electronic device. For example, while the size of virtual objects 433 and 435 can be increased or decreased in the 3D preview 434, the corresponding two-dimensional representation displayed on the first electronic device (e.g., ...) remains unchanged. Figure 4E The 464' in the two-dimensional representation remains unchanged (e.g., the size of the two-dimensional representation does not increase or decrease). However, in some implementations, the size of the corresponding two-dimensional representation can be changed in size, but the amount of change can be reduced (e.g., the size ratio between the content creation application and the 3D preview in the 3D environment). In some implementations, users can (e.g., by editing the code section in the IDE that defines the size parameter) increase or decrease the actual size of content items in the content creation application, and the corresponding change in size can be reflected in the 3D preview.

[0078] In some implementations, hand gestures that do not initially select or target a specific object (or group of objects) cause size changes to be applied to all content in the 3D preview. In some examples, the same hand gestures cause size changes (e.g., tapping, pinching, and releasing with one hand while gazing at a target, or by keeping the gaze on an object for more than a threshold time period) to be applied only to the selected object or target object or group of objects.

[0079] Figure 4G An exemplary user interaction with resetting a 3D preview 434 of one or more virtual objects according to an embodiment of this disclosure is illustrated. As described above, in some embodiments, the user can change the size of one or more virtual objects 432 displayed in the 3D preview 434 within the 3D environment 468. Continuing... Figure 4F For example, in some implementations, a user can reset the appearance, form, orientation, etc. of one or more virtual objects 432, such that the first virtual object 433 and the second virtual object 435 are optionally restored to their original size and displayed at their original size (e.g., restored to their previous size). Figure 4C (The size shown in the image). Figure 4G As shown, the user can select to reset the power level 440G, as shown by selecting 450G, to restore the first virtual object 433 and the second virtual object 435 to their initial sizes. As shown, in response to the selection of resetting the power level 440G, the first virtual object 433 and the second virtual object 435 can be shrunk to reduce their sizes from... Figure 4F The enlarged size is reduced to its original size. In some implementations, the reset causes all content items to be displayed within the boundaries of tray 436.

[0080] In some embodiments, the selection of the reset power display 440G optionally resets some or all of the previous manipulations of the virtual objects (e.g., reorientation / change of view of one or more virtual objects) to their respective initial states (e.g., corresponding appearance, form, size, etc.), and not just size manipulations. In some embodiments, the selection of the reset power display 440G optionally resets some or all of the previous manipulations of the 3D preview (e.g., including the manipulation of the tray described herein). Additionally or alternatively, in some embodiments, in response to the selection of the reset power display 440G within the 3D environment 468, manipulations of content items performed on the first electronic device (e.g., changes in the position of one or more content items, changes in the orientation / view of one or more content items, etc.) can also be reset (e.g., simultaneously). In some embodiments, the reset of one or more virtual objects 432 within the 3D preview 434 can be animated, for example, making Figure 4GThe size of each of the first virtual object 433 and the second virtual object 435 is changed to Figure 4F Their corresponding original dimensions are animated and displayed on a second electronic device for user viewing. In some embodiments, although some manipulations of the content items can be reset, other manipulations of the tray 436 (e.g., changes to the size, height, position, etc. of the tray 436) may not be reset in response to the selection of the reset enable display 440G.

[0081] Figure 4H An exemplary user interaction with a 3D preview 434 of an animation corresponding to one or more virtual objects, according to an embodiment of this disclosure, is illustrated. In some embodiments, the user may incrementally view one or more animations and / or related actions associated with one or more virtual objects, rather than viewing the animations or related actions in a sequence and timing corresponding to these actions (e.g., during playback mode). In some embodiments, such as Figure 4H As shown in the example, the user can select a third power representation (“third power representation”, “dragner power representation”) 444 from toolbar 438, as shown in selection 450H. In response to receiving a selection of dragner power representation 444, the first virtual object 437 can advance from the first part performing the first action to the second part performing the first action or perform the second action. For example, as per the example regarding Figure 4B The chair 433 can be configured to rotate or turn. In some embodiments, a dragger indication can advance the chair 433 from a first position in the animation to a different position in the animation, as indicated by dashed arrow 437'. For example, the dragger can advance the playback in playback mode by a predefined amount (e.g., 500ms, 1s, etc.) and / or advance to a time associated with another event (e.g., the next triggering action). A subsequent selection of a second indication can advance the playback to jump to another part of the animation (e.g., a different position within the animation of the chair 433 rotating) or jump to a subsequent animation or action of a virtual object within a virtual object.

[0082] In some implementations, the aforementioned behavior is triggered by tapping and releasing the dragger power indicator 444. In some implementations, the dragger power indicator may have other input modes to provide different dragging behaviors. For example, a user can continuously select a second power indicator 444 (e.g., by pressing and holding button 444 or by holding a pinch, etc.) to continuously drag through the animation associated with the first virtual object 433, which is equivalent to quickly replaying the object's actions / animations in the 3D preview. For example, the chair 433 can rotate at a faster rate than the rotational animation during normal playback when performing continuous dragging. The ability to drag through the animation by selecting and / or pressing the dragger power indicator 444 allows the user to control the speed at which the animation is executed step by step, providing designers with greater flexibility to debug and analyze various aspects of the animation, such as transitions between different parts of the animation, interactions between virtual objects, and so on.

[0083] It should be understood that in some implementations, any and / or each virtual object can perform one or more actions in response to selecting and / or successively selecting the dragger power representation 444, not just the first virtual object 433, as discussed in the examples above. In some implementations, dragging an action associated with one or more virtual objects of the 3D preview 434 can also cause the corresponding 2D representation of a content item (e.g., 464') displayed on the first electronic device (e.g., 460') to move simultaneously according to the animated drag. For example, when the chair 433 incrementally rotates / turns in response to the selection of the power representation 444 (e.g., ... Figure 4H As shown, the two-dimensional representation of the chair displayed on the first electronic device optionally rotates / turns simultaneously within the content creation application (e.g., 462'). In some embodiments, dragging is reflected in the preview but not in the content creation application. In some embodiments, the dragger indication 444 may alternatively be provided as a drag bar (not shown) to allow the user to selectively navigate to a specific time marker or selectively view certain time intervals within the duration of the corresponding animation.

[0084] In some implementations, user input interacting with the 3D preview 434 can be represented in the 3D environment to aid user interaction. For example, re-reference. Figure 4D and Figure 4FSelecting input 450D or 450F can respectively represent input from a finger or finger pinch, which can be presented using a second electronic device (e.g., displayed as a circle, similar to 450 and 452). In some embodiments, a finger / hand or its representation can be presented to the user instead of a circle. This allows the user to visually track finger movement and more accurately select and / or interact with various elements of the 3D preview 434. Additionally, in some embodiments, the user's gaze can be represented within the 3D environment (e.g., a viewpoint associated with the second electronic device) to allow the user to accurately and precisely focus on an element within the 3D preview 434. In some embodiments, the gaze can be represented in a manner similar to a cursor (e.g., a dot or other shape). In some embodiments, the user's gaze can be indicated by changing the appearance of the currently focused virtual object or display (e.g., brightening, highlighting, magnifying, etc., the object or display currently focused on by the user's eye). In such embodiments, the representation of a finger / hand and / or gaze can be displayed in some operating modes and hidden in others. In some such implementations, additional power indication may be provided, which toggles whether to display an indication of the position of the user's fingertips and / or the user's gaze within the three-dimensional environment 468.

[0085] Figures 4I to 4L Various exemplary user interactions with and manipulations of the interactive tray 436 according to embodiments of this disclosure are illustrated. As described above, a user can interact with and manipulate one or more virtual objects in virtual objects 432 displayed within a 3D preview 434 to observe and receive feedback from various perspectives, sizes, and / or positions of the 3D content items. Focus is now on various exemplary interactions with and manipulations of the tray 436 to provide the user with additional perspectives regarding the appearance, form, and associated animations and / or actions of the 3D content items.

[0086] Figure 4I An exemplary user interaction with the tray 436 is illustrated, according to an embodiment of the present disclosure, to reposition the tray 436 (and thereby reposition the 3D preview) within a 3D environment 468. In some embodiments, the tray 436 can be repositioned within the 3D environment 468 using direct or indirect manipulation represented by a handle. For example, a user might wish to move a 3D preview of a content item to a different location in the environment (e.g., to a different location on a table, to a different surface within the user's real-world environment, etc.). In some embodiments, the tray 436 may include a handle 430, optionally positioned below the toolbar 430, such as... Figure 4IAs shown in the example. It should be understood that the handle 430 may alternatively be located in other areas or regions. For example, above the toolbar 438, along different edges of the tray 436, perpendicular to the side of the tray 436, on the top of the tray 436, etc. In some embodiments, the edge of the toolbar 438 may be represented as a handle indication (e.g., selecting the toolbar 438, rather than an indication within the toolbar 438).

[0087] In some implementations, as an example, a user may provide a pinch input at the handle 430, or while visually aiming at the handle, as shown in pinch 450I, corresponding to selection / clamping of the handle 430. Movement while maintaining selection (e.g., while maintaining a pinch) can cause movement of the 3D preview 434. In some implementations, the appearance of the handle 430 may change when selected. For example, in response to an input corresponding to selection / clamping of the handle 430, the handle 430 may become highlighted, as shown in highlight 476, indicating to the user that the tray can be moved. Figure 4I As shown in the example, when clamping / holding the handle 430, the user can reposition the tray 436 within the three-dimensional environment 468 to the right (e.g., relative to the right side). Figure 4C The positions shown are as indicated by dashed arrow 437 and by their positions relative to real-world plants 454' and shelves 456'. As shown, in some embodiments, one or more virtual objects 432 and toolbar 438 move along with tray 436 upon input to move handle 430 to the right. Thus, the user can move tray 436 and the remaining previewed components within the 3D environment 468 by selecting and moving handle 430. In some embodiments, after the user has finished moving tray 436 and after releasing the pinch input 450I (e.g., stopping selection of handle 430), handle 430 may return to its original appearance (e.g., stop being highlighted) before selection. In some embodiments, handle 430 may stop being displayed within the 3D environment after releasing pinch input 450I.

[0088] Figure 4J An exemplary user interaction with the tray 436 is illustrated within a three-dimensional environment 468, according to an embodiment of this disclosure. In some embodiments, the user can rotate the tray 436 to change the viewpoint of one or more virtual objects 432. Figure 4JAs illustrated in the examples, in some embodiments, tray 436 may include user interface elements, such as corner grippers 458, positioned at the corners of tray 436. In some embodiments, corner grippers 458 are optionally integrated with tray 436 and are always displayed with the tray. For example, the tray outline may appear to have multiple segments, including one or more corner segments connected by one or more linear segments, rather than representing the tray in a continuous shape. In some embodiments, the tray is represented in a continuous shape, but the corner segments may alter their appearance to depict the functionality of the corner grippers. For example, in some embodiments, corner grippers 458 are optionally displayed in response to user input that selects or grips a corner of tray 436 (e.g., based on direct and / or indirect aiming / selection using eye gaze and / or fingers). For example, when corner grippers 458 are aimed and / or selected, the corner segments may be highlighted, changed color, and / or have different effects applied. Although Figure 4J A corner gripper 458 is shown, but it should be understood that in some embodiments, corner grippers may optionally be available at more than one or at each corner of the pallet 436. Including multiple corner grippers can provide the user with more options for rotating the pallet, which can be useful because the relative orientation of the pallet and the user can change based on the movement of the pallet or the movement of the user in a real-world environment.

[0089] As described above, the user can rotate tray 436 to view content from an alternative perspective, including the first virtual object 433 and the second virtual object 435. In some implementations, such as Figure 4J As shown in the pinch selection 450J, the user can directly select / pinch the corner clamp 458, or the user can indirectly select / pinch the corner clamp 458 by visually aiming at it and using the pinch selection 450J. Alternatively, as described above, the corner clamp 458 can be displayed after receiving the pinch selection 450J and / or its appearance can be changed in response to aiming and / or selection. For example, in response to the selection of the corner clamp 458, the corner clamp 458 can become highlighted, as shown in 476, indicating to the user that the tray 436 can be rotated. Figure 4J As shown in the example, when gripping / holding the corner gripper 458, the user can rotate the tray 436 within the three-dimensional environment 468 by moving their hand / finger relative to the initial position, as indicated by arrow 437. In response to input to rotate the tray 436 within the three-dimensional environment 468, one or more virtual objects 432 may also be optionally rotated, such as... Figure 4JAs shown, the input to rotate the tray 436 causes the first virtual object 433 to be displayed to the right of the second virtual object 435 (e.g., showing a rotation of approximately 180 degrees). As illustrated, in some embodiments, the toolbar 438 remains in its initial position despite inputting to rotate the tray 436. In some such embodiments, the toolbar 438 may optionally be configured to face the user, regardless of changes to the tray 436 or one or more virtual objects 432, ensuring that the user always has access to the display 440. In some embodiments, after the user has finished rotating the tray 436 and after releasing the pinch input 450 (i.e., the user stops selecting the corner gripper 458), the corner gripper 458 may stop being highlighted and / or may stop being displayed in the 3D environment (e.g., the corner of the tray returns to its original appearance).

[0090] Additionally or alternatively, in some embodiments, the second electronic device may rotate the tray 436 within a three-dimensional environment 468 in response to user input pointing to a pivot point displayed together with the tray 436. For example, as referenced above. Figure 4F Similarly, when the user's gaze is directed at the tray 436 in the three-dimensional environment 468, the second electronic device, in response to detecting a pinch selection, displays a pivot point at a predefined location on the tray 436 and maintains it for a threshold time period (e.g., until the threshold time period is met, if no movement of the user's hand is detected or if less than a threshold movement of the user's hand is detected). For example, the pivot point may be displayed at the location of the gaze on the tray 436, a predefined location near the location of the gaze on the tray 436, the center point on the tray 436, and / or other locations on the tray 436, as similarly described above. In some embodiments, when the second electronic device detects and maintains a pinch selection when the user's gaze is directed at the tray 436, the pivot point may be displayed along with the tray 436, in addition to or as an alternative to the corner gripper 458.

[0091] In some implementations, when the pivot point (e.g., similar to...) Figure 4J When the clamping element 458 is displayed together with the tray 436, the second electronic device can manipulate the tray 436 in response to user input. For example, when the pivot point is displayed on the tray 436 (e.g., at the center of the tray 436, at a position based on the user's gaze, etc., as described above), the second electronic device rotates the tray 436 (e.g., and optionally rotates one or more virtual objects 432) in response to lateral movement of the user's hand (e.g., to the left or right) while the user is holding the pinch. Figure 4JSimilar to the illustration. In some embodiments, if the second electronic device detects a rightward movement of the user's hand while the hand is holding a pinch, the second electronic device can rotate the tray 436 counterclockwise around a pivot point in the three-dimensional environment 468 (e.g., the pivot point acts as an axis of rotation (perpendicular to the vertical axis of rotation of the tray)). For example, if the pivot point is located at the center of the tray 436, the second electronic device rotates the tray 436 around the center of the tray 436 (and optionally rotates one or more virtual objects 432), and if the pivot point is located at a corner of the tray 436, the second electronic device rotates the tray 436 around the corner of the tray 436 (and optionally rotates one or more virtual objects 432). Additionally, in some embodiments, if the second electronic device detects a leftward movement of the user's hand while the hand is holding a pinch, the second electronic device can rotate the tray 436 clockwise around a pivot point in the three-dimensional environment 468, such as... Figure 4J Similarly illustrated in [the diagram]. In some embodiments, the amount by which the tray 436 (and therefore one or more virtual objects 432) rotates about a pivot point in the three-dimensional environment 468 (e.g., in degrees) is proportional to the amount of lateral movement of the user's hand relative to the pivot point. In some embodiments, the direction of rotation of the tray 436 (and therefore one or more virtual objects 432) about the pivot point in the three-dimensional environment 468 (e.g., clockwise or counterclockwise) is based on the direction of lateral movement of the user's hand relative to the pivot point. Thus, as described above, providing a graphical pivot point on the tray 436 as a reference point by which one or more virtual objects 432 can be rotated in response to user input simplifies the gestures required to rotate one or more virtual objects 432 in the three-dimensional environment 468 (e.g., by reducing gestures from two-handed gestures to one-handed gestures). Additionally, providing a graphical pivot point on the tray 436 as a reference point simplifies the user interface objects provided for rotating one or more virtual objects 432, since the graphical pivot point is only optionally displayed in response to the detection of a pinch selection and hold input as described above.

[0092] In some embodiments, as described above, in response to detecting the release of the pinch selection (e.g., causing the user to no longer grip the pivot point), the second electronic device no longer displays the pivot point on tray 436 (e.g., stops displaying the corner gripper 458). For example, lateral movement of the user's hand will not cause the second electronic device to rotate tray 436 within the three-dimensional environment 468. In some embodiments, movement of the user's hand after a pinch selection (e.g., not held) may cause the second electronic device to not display the pivot point on tray 436 in the three-dimensional environment 468. In some embodiments, a pinch selection following a hand movement may cause the second electronic device to adjust according to the hand movement (e.g., as...). Figure 4I(Similarly shown in the diagram) the tray 436 is moved in the three-dimensional environment 468, rather than rotated.

[0093] In some implementations, the second electronic device may change the height of one or more virtual objects 432 on the tray 436 in response to detecting longitudinal movement of a user's hand (e.g., relative to a pivot point displayed on the tray 436) in space. For example, as Figure 4K Similarly, as illustrated below, if the second electronic device detects that the user's hand has moved upward in space relative to a pivot point on tray 436, the second electronic device can raise the plane displaying one or more virtual objects 432 on top of tray 436 (e.g., and thus raise one or more virtual objects 432 in the three-dimensional environment 468). Similarly, if the second electronic device detects that the user's hand has moved downward in space relative to a pivot point on tray 436, the second electronic device can lower the plane displaying one or more virtual objects 432 on top of tray 436 (e.g., and thus lower one or more virtual objects 432 in the three-dimensional environment 468), as described similarly below.

[0094] As mentioned herein, in some embodiments, some or all changes to the content in the 3D preview are reflected in the 2D representation in the content creation application, and some or all changes to the 2D representation in the content creation application are reflected in the 3D preview. In some embodiments, it may be desirable to separate some changes. For example, the camera view of the content in the 3D preview presented using a second electronic device may be separate from the camera view of the 2D representation of the content on the first electronic device. For example, as mentioned herein, a user may change their position in the physical environment while wearing the second electronic device. For example, a user may “walk around” the 3D preview to view the content from an alternative viewpoint and / or viewing angle. In such examples, although the viewpoint / view of the 3D preview changes for the user of the second electronic device, the viewpoint / view of the 2D representation of the content does not change. Similarly, changing the viewpoint / viewing angle of the 2D representation of the content in the content application does not change the viewpoint / view of the 3D preview. In a similar manner, in some embodiments, rotating the tray 436 in the 3D environment 46 does not change the viewpoint / view of the 2D representation of the content in the content creation application. Separating the perspective / view between the content creation application and the 3D preview can advantageously provide an alternative perspective for easily and quickly viewing 3D content items using a second electronic device without interfering with editing using the first electronic device.

[0095] It should be understood that in some implementations, the camera perspective / view can be partially or fully coupled, such that a change in the perspective / view at one device can change the perspective / view at the other device. For example, in collaborative mode, a first user may use a first electronic device (e.g., a desktop or laptop computer), and a second user may use a second electronic device (e.g., a head-mounted display). In some implementations, the view of the content on the first device may follow the camera perspective of the user on the second electronic device, allowing both users to view the same content from the same perspective. In some implementations, picture-in-picture rendering may be used to display the content perspective of another second user, but the main view of the content in the content creation application on the first electronic device may come from a perspective / view separate from the perspective of the content at the second electronic device.

[0096] Figure 4K An exemplary user interaction with the tray is illustrated, according to an embodiment of this disclosure, where previewed content is elevated above the tray's initial position within a three-dimensional environment 468. In some embodiments, the plane over which the content is presented may be elevated or lowered within the three-dimensional environment 468, such that content including one or more virtual objects 432 may also optionally be elevated or lowered. Continuing Figure 4J For example, a user might expect to lift the tray to receive feedback on the appearance, form, animation, and related actions of the bottom portion of one or more virtual objects 432, and / or experience the placement of content offset relative to an anchor point. In some embodiments, user interface elements represented by pods 445 may be provided on or near the sides or corners of the tray in the 3D environment 468. In some embodiments, pod 445 may be an initial representation of a third ball 494 prior to any offset. In some embodiments, pod 445 may have an alternative appearance (e.g., solid lines, dashed lines, dotted lines, outlined, wireframed, or with different shadows). Figure 4K As shown in the example, a user can grip or select pod 445 using direct or indirect selection (e.g., by providing direct pinch input at pod 445, as shown by pinch input 450K, or by using eye contact to aim an indirect pinch for selection) and movement, while maintaining the selection as an input that raises or lowers the content relative to the tray's initial position within the three-dimensional environment 468. In some embodiments, pod 445 may become highlighted (not shown) in response to receiving input to select / grip pod 445.

[0097] like Figure 4K As shown, the user can raise the pod 445 to create a new plane (or offset tray) 436B, which is offset within the three-dimensional environment 468 relative to the plane of the tray 436A at its initial position. The movement of the pod 445 is indicated by the dashed arrow 437. Figure 4K As shown, the bias tray 436B and content are moved closer to the representation of the real-world plant 454' and shelf 456'. In some embodiments, the appearance of the bias tray 436B is the same as that of the tray 436A. In some embodiments, the bias tray 436B may have a different appearance than the tray 436A. For example, the bias tray 436A may include different fills, outlines, shadows, line types, or other effects. As shown, the first virtual object 433 and the second virtual object 435 can be raised within the 3D environment 468 using the bias tray 436B, allowing the user to view the virtual objects 433 and 435 from a bottom or near-bottom perspective. This is advantageous for easily and quickly analyzing the appearance, form, color, shadows, etc., of the bottom portion of one or more virtual objects, or for viewing the appearance of objects that appear with an offset relative to the anchor point. As shown, the toolbar 438 can remain in its initial position attached to the tray 436 (at the tray's initial position), which is advantageous for maintaining the continuity of control positions. In some implementations, some or all of the toolbar 438 may be copied or moved entirely to a location near the bias tray 436B.

[0098] In some implementations, one or more user interface elements (e.g., balls) may be used to provide the user with context regarding the relative position of content within a 3D preview environment. Each of the one or more balls may provide a relative position along one dimension. For example, a first ball 474, a second ball 484, and a third ball 494 may be used to provide relative positions along the x-axis, y-axis, and z-axis, respectively. In some implementations, the first ball may appear along a first edge of the tray (e.g., corresponding to the x-axis), the second ball may appear along a second edge of the tray (e.g., corresponding to the y-axis), and the third ball may appear at or within a threshold distance from a corner of the tray (e.g., corresponding to the z-axis). In some implementations, the first ball 474 and / or the second ball 484 may appear at the midpoint between the corners of the tray 436 (or within a threshold distance from the midpoint) along the orthogonal side of the tray 436.

[0099] In some implementations, each ball is visible regardless of the tray's orientation or interaction with the tray. In some implementations, some balls may be visible or hidden, or have different appearances, depending on the tray's orientation and / or interaction with the tray. For example, in some implementations, the first ball 474 may be visible and the second ball 484 may be hidden when the corresponding first axis of the plane faces the user (or within a user-facing threshold), while the first ball 474 may be hidden and the second ball may be visible when the corresponding second axis of the plane faces the user (or within a user-facing threshold). In some examples, both the first and second balls are visible when the tray 436 is viewed after a rotation threshold amount. For example, when the tray 436 is viewed after a 45-degree rotation (facing a corner of the tray), in some implementations, both the first and second balls are visible. In some implementations, the first and second balls are visible when referenced... Figure 4J The rotation of the described disk is visible. In some embodiments, the appearance of the third ball 494 may differ when the disk is in its initial position from its appearance when the disk has deviated from its initial position. For example, as referenced... Figure 4K The described content items can be enhanced in the preview. In some examples, the third ball 494 may appear smaller and / or contain no content or different content before the offset changes, compared to its appearance during and / or after the height offset. Additionally or alternatively, the position of the third ball 494 may change due to the height offset. For example, the third ball 494 may appear at the midpoint between the tray's initial position and the offset position (or within a threshold distance of the midpoint).

[0100] The third ball 494 may include a textual indication of the displacement of the content within tray 436 relative to its initial placement along the z-axis. The textual indications within one or more balls may change as the user moves the content within the tray plane, such as regarding... Figures 4M to 4PThe description is more detailed, and / or can be altered by raising the tray, as described above. For example, the first ball may include a label “x.xx” indicating x-axis displacement, the second ball may include a label “y.yy” indicating y-axis displacement, and the third ball 494 may include a label “zz” indicating z-axis displacement relative to the starting position within the three-dimensional environment. Labels can advantageously provide designers with environmental awareness during the design process, particularly when the preview content is moved. In some embodiments, balls 474, 484, and 494 may express displacement in any unit of measurement, including unitless coordinates. In some embodiments, the balls may have different appearances depending on their relative positions within the environment. For example, the balls may have different shape styles (e.g., solid lines, dashed lines, dotted lines, colors, etc.), different fills (e.g., colors, patterns, etc.), and / or different effects (e.g., shadows, glows, etc.). For example, the three balls may be represented by different colors to distinguish them, regardless of their orientation. In some implementations, the outline, fill, or effect of the ball may differ from that when the ball is not occluded by a virtual object. Additionally or alternatively, the outline, fill, or effect may differ from that when the ball is occluded behind a virtual object compared to when the ball is occluded within the volume of the virtual object. In some implementations, the position and / or appearance of the ball, as well as information displayed within the ball, may be changed in a ball properties pane, optionally represented as an overlay map in the 3D environment 468, or, for example, displayed in a window within a content creation application (e.g., 462') (e.g., optionally when the ball is selected).

[0101] In some implementations, the position and / or appearance and / or information displayed by the third ball 494 can be changed in a ball properties pane, which may be represented as an overlay map in a 3D environment 468 or displayed in a window in a content creation application (e.g., 462'). In some implementations, the ball 494 is integrated with the pod 445 rather than being displayed as a separate user interface element.

[0102] Figure 4LAn exemplary user interaction with the tray 436, according to an embodiment of this disclosure, is illustrated, involving the expansion of the size of the tray 436 within a three-dimensional environment 468. As mentioned herein, one or more virtual objects 432 are optionally displayed at the top and center of the tray 436 (e.g., initially and / or in response to a view reset). In other words, the display of one or more virtual objects 432, and the movement or expansion of one or more virtual objects, may initially be limited by the size of the tray 436 in the three-dimensional environment. In some embodiments, the tray 436 may be resizable (e.g., its area may increase or decrease) such that the size of the tray 436 within the three-dimensional environment 468 is adjusted. In some embodiments, resizing the tray 436 changes the size of the tray while maintaining the size of one or more virtual objects 432.

[0103] As an example, a user might expect to expand the size of the tray 436 to provide additional space within the environment for interacting with and / or viewing one or more virtual objects 432 within the 3D environment 468. In some implementations, such as Figure 4L As illustrated in the example, a user can select or grip two (e.g., opposite) corners 424A and 424B of tray 436 by providing finger pinch input at two corners 424A and 424B, as shown by pinch 450 and pinch 452. In response to receiving pinch input 450 and pinch input 452, corners 424A and 424B optionally become highlighted, as shown by highlight 467 and highlight 469 respectively, thereby instructing the user to move the opposite sides of the tray in opposite directions to expand tray 436. As shown, when corners 424A and 424B are gripped, the user can expand the size of tray 436 by separating both hands and moving tray 436 in opposite directions, as shown by dashed arrows 437 and 439. The amount of change in size is a function of the change in distance between the two hands while maintaining selection (e.g., linear or non-linear). In some implementations, as the tray 436 expands within the 3D environment 468, one or more virtual objects 432 optionally maintain their size and remain centered on top of the tray 436. Additionally, in some implementations, the toolbar 438 maintains its relative position to the user-facing edge within the 3D environment 468. In some implementations, the center of the tray remains the same before and after a change in tray size. In some implementations, the size increases uniformly along each dimension. In some implementations, the size increases based on the direction of input movement. For example, x-axis movement causes a change in the x-axis dimension, y-axis movement causes a change in the y-axis dimension, and diagonal movement causes changes in both the x-axis and y-axis dimensions.

[0104] Now focus on various examples showing the movement of one or more virtual objects 432 within the three-dimensional environment 468 on top of the tray 436. Figures 4M to 4P Various exemplary user interactions with one or more virtual objects 432 are shown, which involve moving one or more virtual objects 432 along the surface of a tray 436 according to an embodiment of the present disclosure. Figure 4M An exemplary user interaction with the tray 436 is shown, illustrating the movement of one or more virtual objects 432 on the initiating tray 436 according to an embodiment of the present disclosure.

[0105] In some embodiments, tray 436 may include a plurality of grid lines 431 disposed across the surface of tray 436, such as Figure 4M As shown in the example. It can be along, as... Figure 4M The grid lines are shown along one axis (e.g., the axis the user is facing) or along two axes. In some embodiments, the multiple grid lines 431 may also be optional user interface elements, such that one or more virtual objects 432 on the top of the tray 436 can be moved, for example, based on the movement of the multiple grid lines 431. As an example, the user might expect to move one or more virtual objects 432 to the left across the surface of the tray 436. Figure 4M As shown in the example, a user can gaze at and select / grip a first grid line 431A using a pinch input, such as pinch 450M, to initiate movement of one or more virtual objects 432 across the tray 436. In some embodiments, in response to the selection of the first grid line 431A, a portion (or all) of the first grid line 431A optionally becomes highlighted, which optionally corresponds to the portion receiving the pinch input, as shown in highlight 437. When a portion of the first grid line 431A is selected / gripped, in some embodiments, a user can laterally move multiple grid lines 431 across the surface of the tray 436, which optionally causes corresponding movement of the first virtual object 433 and the second virtual object 435 across the top surface according to the movement of the multiple grid lines 431, as discussed in more detail below. In some examples, similar user input (e.g., movement on pinch) can be used, but by aiming at or selecting a plane of the tray (e.g., optionally, an area without content). For example, even in the absence of grid lines, viewing and / or pinching the flat area of ​​tray 436 can cause the content to move.

[0106] Figure 4NAn exemplary user interaction with the tray 436 is illustrated, involving the movement of one or more virtual objects 432 across the surface of the tray 436 according to an embodiment of this disclosure. As described above, as an example, a user can laterally move a first grid line (e.g., 431A) of a plurality of grid lines 431 disposed across the top surface of the tray 436 to simultaneously move one or more virtual objects 432 across the surface of the tray 436 (or alternatively, select and move the surface of the tray 436 to move one or more virtual objects). In some embodiments, continuing... Figure 4M For example, a user can continuously move the first grid line across the top of the tray 436 as shown by the dashed arrow 437, making it appear as if the first grid line is being dragged off the surface of the tray 436. Figure 4N As shown in the example, dragging the first grid line away from the surface of tray 436 can also drag the second virtual object 435 away from the top surface of tray 436, as indicated by a change in the appearance of the second virtual object (e.g., shadow 471). It should be understood that when the second virtual object 435... Figure 4N When the virtual object 435 is removed from the tray 436, it is optionally no longer visible to the user. Alternatively, in some embodiments, the appearance of the second virtual object 435 is optionally altered to indicate to the user that the virtual object has been moved away from the tray 436 (e.g., presented with a different colored shadow, dashed line, slightly transparent, etc.), such that the second virtual object 435 remains at least partially visible.

[0107] like Figure 4N As shown in the examples, in some implementations, the tray 436 and / or toolbar 438 remain stationary within the three-dimensional environment 468 when a user moves the contents of the tray across the surface of the tray 436.

[0108] Figure 4O An exemplary user interaction with tray 436 is illustrated, according to an embodiment of this disclosure, involving moving one or more virtual objects 432 away from the top of tray 436. As described above, the user can continue to move one or more virtual objects 432 laterally across the surface of tray 436, as described above. In some embodiments, as described below, tray 436 may be configured to provide the user with a visual indication that one or more virtual objects 432 have been moved away from tray 436 within the three-dimensional environment 468.

[0109] continue Figure 4N For example, the user can continue to move the first grid line to the left (e.g., 431A), causing the first virtual object 433 and the second virtual object 435 to be moved away from the top of the tray 436 and optionally no longer visible to the user, such as... Figure 4OThese are represented by shadows 473 and 471, respectively. As shown, in some embodiments, a portion of the boundary of tray 436 may change its appearance (e.g., become highlighted, glowing, brightening, etc.), as represented by glow 441, wherein the portion of the boundary that changes its appearance optionally corresponds to the position of one or more virtual objects 432 outside the boundary of tray 436 (e.g., when moved away from tray 436 or after being moved away from tray). For example, as... Figure 4O As shown, the contents outside the tray are positioned along the y-axis in the lower half, thus altering the appearance of the lower half of the tray boundary. In some embodiments, the length of the boundary that alters the appearance can be calculated based on a line from the contents orthogonal to the boundary, indicating the extent of the contents outside the tray along the axis of the corresponding boundary.

[0110] Additionally or alternatively, in some embodiments, variations in appearance may be displayed above the boundary in similar or different ways. In some embodiments, the area above the boundary may provide an indication of the contents outside the tray. For example, Figure 4O A plurality of vertical indicators 443 may be presented along the boundary of tray 436 in a highlighted portion 441, wherein the plurality of vertical indicators 443 may also have different appearances (e.g., highlighted, illuminated, brightened, etc.). In some embodiments, instead of vertical indicators, the entire area may display changes in appearance indicating the presence of content outside the tray. In some embodiments, the height of the vertical indicator or other effect projection above the boundary indicating the presence of content outside the tray may have a height characteristic corresponding to the height of the content outside the screen. In some embodiments, one vertical indicator may be presented for each virtual object outside the tray. In some embodiments, multiple (e.g., two, three, four, etc.) vertical indicators may be presented for each virtual object outside the tray, and optionally more vertical traces may be presented for wider objects. In some embodiments, the position of one or more corresponding vertical indicators (e.g., along the y-axis within the 3D environment 468) may be optionally positioned along the y-axis at a location corresponding to the center of the corresponding virtual object (e.g., the line from the center of the object to the vertical indicator is orthogonal to the boundary).

[0111] like Figure 4O As shown in the example, the highlighted portion of boundary 441 and / or multiple vertical indicators 443 can individually or in combination inform the user of the presence and / or approximate location of one or more virtual objects outside the tray presented on tray 436, which is advantageous when the user unintentionally or intentionally moves one or more virtual objects out of the tray.

[0112] As described herein, in some embodiments, a highlighted portion of boundary 441 and a plurality of vertical indicators 443 may be rendered in response to any movement or manipulation that causes one or more virtual objects to be moved or at least partially moved away from tray 436. In some embodiments, a portion of the boundary of tray 436 may become highlighted, and the plurality of vertical indicators 443 may be rendered in response to an increase in the size of one or more virtual objects, at least a portion of which crosses the boundary of tray 436. In some embodiments, in response to a decrease in the tray size that causes at least a portion of one or more virtual objects to cross the boundary of tray 436, a portion of the boundary of tray 436 may become highlighted, and the plurality of vertical indicators 443 may be rendered.

[0113] Figure 4P An exemplary user interaction for moving one or more virtual objects from outside the tray back onto the tray, according to an embodiment of this disclosure, is illustrated. As described above, one or more virtual objects 432 can move across the surface of the tray 436, including moving away from the tray 436, such that the one or more virtual objects optionally become invisible. Continuing Figure 4O For example, one or more virtual objects can be moved from outside the tray 436 back onto the tray 436 (e.g., by selecting and moving one of a plurality of grid lines 431 or by selecting and moving a surface of the tray). For example, as... Figure 4P As shown, the user can view and select a corresponding grid line 431B (or generally on the surface) of multiple grid lines 431, as indicated by pinch input 450P, and while selecting / holding the corresponding grid line 431B (or generally on the surface), move the corresponding grid line 431B to the right (and generally on the surface), as indicated by dashed arrow 439, to cause one or more virtual objects 432 to traverse from outside the tray 436 back onto the tray 436. Figure 4P As shown in the example, in some implementations, once the first virtual object 433 and the second virtual object 435 have been completely moved back onto the tray 436, the portion of the boundary of the tray 436 (e.g., Figure 4O 441) can be stopped from being highlighted and / or multiple vertical indicators (e.g., 443) can be stopped from being displayed.

[0114] Additionally or alternatively, the user can reset the power display by selecting from toolbar 438 (e.g., Figure 4GThe positioning of one or more virtual objects on the top of the tray 436 is restored using a display (440) in the image. It should be understood that while the above description relates to the leftward and rightward movement of one or more virtual objects 432 across the top of the tray 436, multiple grid lines 431 can also be used to traverse one or more virtual objects forward and backward along the top of the tray 436, where similar changes in appearance can be obtained for boundaries and / or above the boundaries where the content moves away from the tray. Additionally or alternatively, the content can be moved outside the boundaries of the preview in the 3D environment (e.g., the content extends beyond the upper or lower boundaries). In some such embodiments, an indication of content outside the tray can be provided by changing the appearance of the preview (e.g., providing lighting effects near the top of the preview when content exists outside the upper boundary of the preview on the z-axis).

[0115] Figure 4Q This illustration depicts an exemplary user interaction with one or more virtual objects 432, rendered at full size, according to an embodiment of this disclosure. As discussed herein, one or more virtual objects 432 may initially be presented in a 3D preview at a size permissible for display within the boundaries of a tray 436. The size of the virtual objects may be manually increased or decreased (e.g., by pinching and expanding / contracting the index finger and thumb, or within a content creation application) to allow the user to view one or more virtual objects 432 at various sizes based on user input. In some embodiments, one or more virtual objects may be scaled to full size (e.g., the ratio of the displayed size of the virtual object within the preview to the actual size of the virtual object in a specific implementation within the 3D environment is 1:1).

[0116] like Figure 4QAs shown in the example, a user can select a fourth power representation (“fourth power representation”, “scaled power representation”, “full-size power representation”) 446 from toolbar 438, as shown in selection 450 (e.g., directly or indirectly activating the power representation). In some embodiments, in response to the selection of the fourth power representation 446, one or more virtual objects 432 can be scaled on tray 436 such that the size of the first virtual object 433 and the size of the second virtual object 435 are each presented at full size. For example, if a chair is encoded in a content creation application (e.g., 462) within a first electronic device (e.g., 460) to have a height of one meter in a specific implementation (e.g., corresponding to a one-meter-high chair in the real world), then the virtual object 433 representing the chair is optionally displayed on tray 436 at a height of one meter within the three-dimensional environment 468. The selection of the fourth energy representation 446 allows the user to view one or more virtual objects 432 at full size, not only magnifying the first virtual object 433 and the second virtual object 435 to observe surface details such as color and texture, but also advantageously analyzing the size of the first virtual object 433 and the second virtual object 435 relative to the size of real-world objects in the corresponding real-world environment.

[0117] Figure 4R An exemplary user interaction with a 3D preview 434 in immersive mode, according to an embodiment of this disclosure, is illustrated. In some embodiments, an immersive mode (“immersive mode”, “third mode”, “third operating mode”) can be activated such that one or more virtual objects 432 are optionally displayed at full size within a 3D environment 468, unconstrained by the tray 436. As an example, a user may expect to view one or more virtual objects 432 at full size relative to a real-world environment (e.g., in the user’s office) or relative to a virtual environment. As described below, in immersive mode, a second electronic device may generate a full-size representation in the 3D environment, optionally partially or completely occluding the background of the 3D environment 468.

[0118] like Figure 4R As shown in the example, the user can select a fifth energy representation (“fifth energy representation”, “immersive mode energy representation”) 448 from toolbar 438, as shown in selection 450. In some embodiments, in response to the selection of the fifth energy representation 448, a full-size representation of the virtual object 432 can be generated and rendered within the 3D environment 468. In some embodiments, when immersive mode is activated, portions of the 3D environment 468 can be selectively occluded (e.g., darkened, faded, etc.). For example, as... Figure 4RAs shown, the representations of real-world plants 454' and shelves 456' can be occluded (i.e., stopped from display) within the three-dimensional environment 468. The representations of the first electronic device 460', one or more input devices 466', content creation application 462', and / or content item 464' can continue to be displayed at the second electronic device without being occluded.

[0119] In some implementations, a full-size representation of the virtual objects may be presented within a portion of the 3D environment 468, wherein representations of real-world plants 454' and shelves 456' are presented (at least partially). In some implementations, as shown, one or more virtual objects 432 may be simultaneously displayed on a tray 436 within the 3D preview 434 while a full-size representation of a first virtual object 433" and a full-size representation of a second virtual object 435" are displayed (e.g., an immersive representation of content is simultaneously displayed after or near the 3D preview 434). Thus, when the second electronic device operates in immersive mode, the user can simultaneously view the full-size representations of the first virtual object 433" and the second virtual object 435", one or more virtual objects 432 within the 3D preview 434, the representations of the first electronic device 460' (including the content creation application 462' and content items 464'), and one or more input devices 466'. Simultaneously displaying an immersive representation and 3D preview of the content 434 provides the user with the ability to interact with and manipulate one or more virtual objects 432 and / or the tray 436 in any of the aforementioned ways (e.g., resizing, moving, etc.), while maintaining the display of full-size representations 433” and 435” (and the representation of the first electronic device 460’, including the content creation application 462’ and the content item 464’), and including the input device 466’.

[0120] Additionally or alternatively, in some embodiments, the representation of the first electronic device 460' (including the content creation application 462' and content item 464') and / or the 3D preview 434 (including one or more virtual objects 432 and tray 436) may be occluded to provide a clearer, more focused view of the full-size representation of the first virtual object 433" and the second virtual object 435" in the 3D environment 468 (e.g., thereby occluding more 3D space to provide a full-size representation of the content). In some embodiments, the user may select a fourth display representation 448 a second time to exit immersive mode and stop displaying the full-size representation of the first virtual object 433" and the second virtual object 435" and restore the display of the occluded portions of the real-world environment (e.g., plants 454' and shelves 456') and / or other portions of the 3D environment 468. Additionally or alternatively, in some embodiments, the user may choose to reset the display representation (e.g., Figure 4G(440) exits immersive mode and restores the original display of one or more virtual objects 432 and the original representation of occluded parts of the 3D environment (e.g., plants 454', electronic devices 460', etc.).

[0121] Figure 4S An exemplary user interaction with a 3D preview 434 is illustrated using a second electronic device to change the viewpoint and preview of a 3D environment according to an embodiment of this disclosure. As discussed herein, in some embodiments, a change in the viewpoint associated with the second electronic device may be received at the second electronic device as input corresponding to the interaction with the 3D preview 434, as discussed below.

[0122] As an example, a user might expect to view one or more virtual objects 432 from an alternative perspective. The user can change the viewpoint associated with the second electronic device (e.g., the perspective of the three-dimensional environment 468) by moving the second electronic device relative to the three-dimensional environment 468 (e.g., when wearing the second electronic device), rather than as referenced... Figure 4J The described rotating tray is used to change the viewing angle or perspective. For example, Figure 4S An updated view of the 3D environment 468 is shown corresponding to a user wearing the second electronic device and moving to the left (and / or turning their head to the right) to a corresponding position within the real-world environment and viewing the 3D environment 468 from that position. In some embodiments, the representations of the preview 434 (including the tray 436 and one or more virtual objects 432), plants 454' and shelves 456', and the representations of the first electronic device 460' (including one or more input devices 466') and the content creation application 462' (including the representation of content items 464') can remain unchanged (e.g., fixed) within the 3D environment 468. Therefore, due to the new position of the second electronic device relative to the 3D environment, a new view of the 3D environment 468 is presented to the user. Figure 4S The example illustrates a new perspective. From this new perspective, the user views the preview (e.g., tray 436 and one or more virtual objects 432) from different angles, the representation of plants 454' and shelves 456' from different angles, and the representation of the first electronic device 460' (including content creation application 462') and input device from different angles. For example, a front view of one or more virtual objects 432 can be viewed from the user's new perspective at the corresponding location.

[0123] As described above, a user can move around the 3D preview 434 to view the first virtual object 433 and the second virtual object 435 from an alternative perspective. In some implementations, as shown, in response to a change in the viewpoint associated with the second electronic device, the position of the toolbar 438 can be changed within the 3D environment 468 to follow the user's viewpoint. For example, as shown, the toolbar 438 can be moved to face the user, while the corresponding position and / or orientation of each of the remaining objects within the 3D environment 468 can appear as an angle because their positions remain unchanged. Making the toolbar 438 follow the user allows the display representation 440 to be viewed by the user and easily accessible (e.g., selectable). Thus, when the user changes position within the 3D environment 468 to view one or more virtual objects 432 from an alternative perspective, easy access to the toolbar 438 can be advantageously provided to the user, and therefore easy access to each of the modes and / or functionalities associated with the display representation 440, thereby providing an efficient and improved workflow.

[0124] It should be understood that the embodiments shown and described herein are merely exemplary, and additional and / or alternative elements may be provided in a three-dimensional environment and / or in a three-dimensional preview. It should be understood that the appearance, shape, form, and size of each of the various user interface elements shown and described herein are exemplary, and alternative appearances, shapes, forms, and / or sizes may be provided. For example, the interactive tray 436 may be provided in alternative shapes (such as circles, triangles, ovals, etc.) other than rectangles. In some embodiments, for example, each power indicator (e.g., 340) may be provided with a unique shape and appearance to visually convey the functionality of the corresponding power indicator to the user (e.g., a "Play" button for activating playback mode and a "Pause" button for pausing / stopping playback mode). Additionally, for example, each power indicator may be provided with text (e.g., positioned within, above, or below the power indicator) to visually convey the functionality of the corresponding power indicator to the user (e.g., a "Play" label positioned below a first power indicator and a "1:1" label positioned below a fourth power indicator). Additionally, in some embodiments, the power indicator can be selected by sound via a user voice command (e.g., a "activate playback mode" voice command). Furthermore, in some embodiments, the power indicator may be provided entirely in the front portion of the tray or housed in a vertical toolbar located on one side of the tray 436, rather than in a horizontal toolbar below the tray, as shown and described.

[0125] In some implementations, simultaneously displaying content items in a two-dimensional manner on a first electronic device and in a three-dimensional manner using a second electronic device can provide a multi-user experience. For example, according to some implementations described herein, a first user can operate a first electronic device (e.g., a desktop computer) including a content creation application, and a second user can operate a second electronic device (e.g., a head-mounted display) including at least a 3D graphics rendering application. As described above, the content creation application can include content items comprising one or more virtual objects (e.g., 433 and 435) displayed in a two-dimensional representation on the first electronic device. The 3D graphics rendering application can be configured to display a preview of the content item in a three-dimensional manner on the second electronic device within a three-dimensional environment (e.g., 468). Thus, the first user can view a two-dimensional representation of the content item on the first electronic device, and the second user can simultaneously view a three-dimensional preview of the content item on the second electronic device (and view a two-dimensional representation of the content item on the first electronic device). The first and second users can then collaboratively view, select, modify, and update the content item by operating the first and / or second electronic devices separately. As described above, editing or modifying the content item on the first electronic device can update the corresponding three-dimensional representation of the content item at the second electronic device based on the editing or modification. Therefore, while the first user edits or modifies the code associated with the corresponding two-dimensional content item on the first electronic device to collaboratively modify and ultimately finalize the appearance and form of the content item, the second user can actively interact with and manipulate one or more virtual objects in the three-dimensional preview. Thus, another advantage is that simultaneously displaying content items in two dimensions on the first electronic device and simultaneously displaying them in three dimensions on the second electronic device can provide efficient and more convenient user collaboration between the first user operating the first electronic device and the second user operating the second electronic device.

[0126] Figures 5A to 5BA flowchart illustrating a process 500 for interacting with a 3D preview according to an embodiment of the present disclosure is shown. For example, the interaction may include selection input of a virtual object presented in the 3D preview. Process 500 begins at a first electronic device (e.g., a head-mounted display) where a 3D environment including the 3D preview is displayed. The 3D preview may include a first object and one or more user interface elements, as shown at 502. In some embodiments, the first electronic device may communicate with display generation components (e.g., a display) and one or more input devices (e.g., hand-tracking sensors, eye-tracking sensors, image sensors, etc.). In some embodiments, at 504, while displaying the 3D environment including the first object and one or more user interface elements, a first input corresponding to the selection of the first object may be received via one or more input devices. The first input optionally corresponds to selection input (e.g., touch or tap input) received by one or more hand-tracking sensors. In some embodiments, the first input may be received from the execution of code in a content creation application.

[0127] In some implementations, the 3D preview optionally operates according to a mode. In some implementations, the first mode optionally refers to a live animation mode (playback mode), the second mode optionally corresponds to a selection mode (edit mode), and the third mode optionally corresponds to an immersive mode. For example... Figure 5A As shown, in some embodiments, at 506, based on the determined 3D preview operating in a first mode, the first object performs a first action according to a selection input (e.g., assuming the first object is configured to perform a first action in response to a selection input). In some embodiments, the first action is performed in response to receiving a first input corresponding to a selection of the first object. The first action optionally corresponds to an animation, which may include movement of the first object, audio emission, lighting changes, etc.

[0128] In some implementations, at point 508, based on the determination that the 3D preview is operating in a second mode different from the first mode, the appearance of the first object is changed (e.g., highlighted) in the 3D environment. Changing the appearance of the first object in the 3D environment (e.g., highlighting) optionally enables editing of the first object (e.g., editing the appearance, form, corresponding position, etc. of the first object within the 3D environment).

[0129] Continuing with the flowchart, as follows: Figure 5BAs shown, in some embodiments, at 520, the first electronic device can communicate with the second electronic device (e.g., a desktop computer, laptop computer, tablet computer, mobile device, etc.). In some embodiments, the second electronic device displays a content creation application that includes a two-dimensional representation of the first object. The content creation application and the two-dimensional representation of the first object can be displayed on the second electronic device, while a three-dimensional environment including the first object with one or more user interface elements is optionally displayed simultaneously on the first electronic device (e.g., a head-mounted display). At 512, based on determining that the three-dimensional preview is operating in a first mode, and based on determining that the first object is configured to perform a first action, the second electronic device simultaneously displays a first animation (e.g., rotation / rotational motion) of the two-dimensional representation of the first object on the second electronic device. Therefore, in some embodiments, in response to receiving a first input, the same animation can be performed simultaneously by the first object in the three-dimensional environment and the two-dimensional representation of the first object.

[0130] At 514, in some embodiments, based on the determination that the 3D preview is operating in a second mode, the appearance of the 2D representation of the first object displayed at the second electronic device changes to indicate the selection of the first object. In some embodiments, in response to the first input, the first object in the 3D environment and / or the 2D representation of the first object is highlighted. In some embodiments, for example, the 3D representation and the 2D representation of the first object may change their appearance simultaneously (e.g., highlighted) in response to receiving the first input.

[0131] It should be understood that process 500 is an example, and more, fewer, or different operations may be performed in the same or different order. Furthermore, the operations in process 500 described above may optionally be performed by running an information processing device such as a general-purpose processor (e.g., as per [reference to...]). Figure 2 One or more functional modules in the aforementioned (or dedicated chip) and / or by Figure 2 Other components are used to implement it.

[0132] Therefore, based on the foregoing, some embodiments of this disclosure relate to a method for presenting and / or manipulating a three-dimensional preview of content. The method may include: at a first electronic device communicating with a display and one or more input devices: displaying a three-dimensional environment including the three-dimensional preview via a display generation component. The three-dimensional preview may include: a first object; and one or more user interface elements. The method may further include: while displaying the three-dimensional preview including the first object and one or more user interface elements, receiving a first input via one or more input devices corresponding to a selection of the first object; based on a determination that the three-dimensional preview is operating in a first mode, causing the first object to perform a first action based on a determination that the first object is configured to perform a first action according to the selection; and based on a determination that the three-dimensional preview is operating in a second mode different from the first mode, changing the appearance of the first object in the three-dimensional preview to indicate the selection of the first object.

[0133] Additionally or alternatively, in some embodiments, the first electronic device may further communicate with the second electronic device; and the second electronic device may be configured to display a content creation application including a two-dimensional representation of the first object. Additionally or alternatively, in some embodiments, the second electronic device may be a laptop computer, a desktop computer, or a tablet computer.

[0134] Additionally or alternatively, in some embodiments, causing the first object to perform the first action may include: presenting a first animation of the first object in a three-dimensional environment; and, based on the determination that the three-dimensional preview is operating in a first mode, simultaneously displaying a two-dimensional representation of the first object at a second electronic device.

[0135] Additionally or alternatively, in some embodiments, the method may further include: altering the appearance of a two-dimensional representation of a first object displayed at a second electronic device, based on determining that the three-dimensional preview is operating in a second mode, to indicate the selection of the first object.

[0136] Additionally or alternatively, in some embodiments, changing the appearance of a first object in a three-dimensional environment may include highlighting the first object in the three-dimensional environment; and changing the appearance of a two-dimensional representation of the first object displayed at a second electronic device may include highlighting the two-dimensional representation of the first object displayed at the second electronic device.

[0137] Additionally or alternatively, in some embodiments, the method may further include: receiving a second input at the second electronic device while simultaneously highlighting a first object in a three-dimensional environment and a two-dimensional representation of the first object displayed at the second electronic device; and updating features of the two-dimensional representation of the first object displayed at the second electronic device and updating features of the first object in the three-dimensional environment displayed at the first electronic device based on the second input.

[0138] Additionally or alternatively, in some embodiments, the first electronic device may be a head-mounted display device.

[0139] Additionally or alternatively, in some embodiments, the 3D preview may include a tray, and the first object may be set on a surface on the tray.

[0140] Additionally or alternatively, in some implementations, one or more user interface elements may be displayed within a toolbar associated with the tray.

[0141] Additionally or alternatively, in some embodiments, the method may further include capturing at least a portion of a real-world environment, including a second electronic device and a content creation application displayed on the second electronic device, via one or more input devices. The three-dimensional environment may further include: a representation of the captured portion of the real-world environment, including: a representation of the second electronic device; and a representation of the content creation application.

[0142] Additionally or alternatively, in some embodiments, one or more user interface elements may include: a first user interface element that can be selected to enable the 3D preview to operate in a first mode; a second user interface element that can be selected to enable the 3D preview to stop operating in the first mode and operate in a second mode; a third user interface element that can be selected to scale the first object from a first size to a second size larger than the first size; a fourth user interface element that can be selected to enable the first object to be dragged through a first action; and / or a fifth user interface element that can be selected to cause the display of a full-size representation of the first object.

[0143] Additionally or alternatively, in some embodiments, displaying a full-size representation may include occluding at least some portions of the three-dimensional environment and displaying a full-size representation of the first object on the occluded portions of the three-dimensional environment.

[0144] Additionally or alternatively, in some embodiments, displaying a full-size representation may include simultaneously displaying in a three-dimensional environment: a representation of a three-dimensional preview; and a representation of a second electronic device, including a representation of a content creation application and a two-dimensional representation of the first object.

[0145] Additionally or alternatively, in some embodiments, the method may further include: receiving a second input via one or more input devices; moving the 3D preview within a 3D environment according to the second input, based on a request to determine that the second input corresponds to a request to move the 3D preview; changing the size of the tray from a first size to a second size different from the first size according to the second input, based on a request to determine that the second input corresponds to a request to change the size of the tray; rotating the tray and a first object disposed on a surface of the tray within a 3D environment according to the second input; and moving the first object from a first height to a second height, offset from an initial position of the tray within a 3D environment, based on a request to determine that the second input corresponds to a request to lift the tray.

[0146] Additionally or alternatively, in some embodiments, determining that the second input corresponds to a request to rotate the tray is based on determining that the second input includes: an interactive input provided by a predetermined portion of the user of the first electronic device for a duration of at least a threshold amount when the user's gaze is directed at the tray, wherein, in response to detecting the interactive input, a representation of a pivot point is displayed at a corresponding location on the tray; and, while maintaining the interactive input, movement of the predetermined portion of the user of the first electronic device in a corresponding direction relative to the representation of the pivot point on the tray. In some embodiments, rotating the tray and a first object disposed on the tray surface in a three-dimensional environment based on the second input includes rotating the tray and the first object disposed on the tray surface about an axis passing through the pivot point.

[0147] Additionally or alternatively, in some embodiments, the method may also include stopping the display of a representation of a pivot point on the tray in response to the release of interactive input provided by a pre-determined portion of the user.

[0148] Additionally or alternatively, in some embodiments, the method may further include: receiving a second input via one or more input devices corresponding to a request to manipulate a first object in a three-dimensional environment; changing the size of the first object from a first size to a second size different from the first size based on the third input, based on the third input, if the third input corresponds to a request to change the size of the first object; and moving the first object on the tray from a first position to a second position different from the first position based on the third input, if the third input corresponds to a request to reposition the first object on the tray.

[0149] Additionally or alternatively, in some embodiments, determining that the second input corresponds to a request to change the size of the first object is based on determining that the second input includes an interactive input provided by a predetermined portion of the user of the first electronic device for at least a threshold amount of time when the user's gaze is directed towards the tray, wherein, in response to detecting the interactive input, a representation of a pivot point is displayed at a corresponding location on the tray. In some embodiments, the predetermined portion of the user of the first electronic device moves in a corresponding direction relative to the representation of the pivot point on the tray. In some embodiments, changing the size of the tray from a first size to a second size different from the first size based on the second input is based on the amount of movement of the predetermined portion of the user of the first electronic device.

[0150] Additionally or alternatively, in some embodiments, the method may further include, based on determining that the second input corresponds to a request to reposition the first object outside the boundaries of the tray: stopping the display of the first object; and changing the appearance of the 3D preview. Changing the appearance of the 3D preview may include: highlighting a portion of the boundary of the tray corresponding to a second location within the 3D environment; and / or displaying one or more vertical indicators above the boundary of the tray corresponding to the second location within the environment.

[0151] Some embodiments of this disclosure relate to an electronic device. The electronic device may include: one or more processors; a memory; and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described above.

[0152] Some embodiments of this disclosure relate to non-transitory computer-readable storage media. A non-transitory computer-readable storage medium is provided that can store one or more programs, the one or more programs including instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform any of the methods described above.

[0153] Some embodiments of this disclosure relate to an electronic device. The electronic device may include: one or more processors; a memory; and means for performing any of the methods described above.

[0154] Some embodiments of this disclosure relate to information processing apparatus used in electronic devices. The information processing apparatus may include means for performing any of the methods described above.

[0155] For purposes of explanation, the foregoing description has been given by reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible based on the teachings above. The embodiments were chosen and described to best elucidate the principles of the invention and its practical application, thereby enabling others skilled in the art to best utilize the invention with various modifications suitable for the contemplated particular purpose, as well as the various described embodiments.

Claims

1. A method comprising: At the first electronic device that communicates with the display and one or more input devices: The display shows a three-dimensional environment including a three-dimensional preview, wherein the three-dimensional preview includes: The first object; and One or more user interface elements; When displaying the 3D preview including the first object and the one or more user interface elements, a first input corresponding to the selection of the first object is received via the one or more input devices; Based on the determination that the 3D preview is operating in a first mode, the first object is caused to perform a first action based on a determination that the first object is configured to perform a first action according to the selection, wherein the first action includes movement of the first object in the 3D preview; and Based on the determination that the 3D preview is operating in a second mode different from the first mode, the appearance of the first object in the 3D preview is changed to indicate the selection of the first object, wherein: The first electronic device further communicates with the second electronic device; The second electronic device is configured to display a content creation application that includes a two-dimensional representation of the first object; and The 3D preview includes a tray, and the first object is disposed on a surface on the tray.

2. The method according to claim 1, wherein: The second electronic device is a laptop computer, desktop computer, or tablet computer; and The first electronic device is a head-mounted display device.

3. The method of claim 1, wherein causing the first object to perform the first action comprises: The first animation that renders the first object in the three-dimensional environment includes the movement of the first object; as well as Based on the determination that the 3D preview is operating in the first mode, and based on the determination that the first object is configured to perform the first action, the first animation of the 2D representation of the first object is simultaneously displayed at the second electronic device.

4. The method according to claim 1, further comprising: Based on the determination that the 3D preview is operating in the second mode, the appearance of the 2D representation of the first object displayed on the second electronic device is changed to indicate the selection of the first object.

5. The method according to claim 4, wherein: Changing the appearance of the first object in the three-dimensional environment includes highlighting the first object in the three-dimensional environment; and Changing the appearance of the two-dimensional representation of the first object displayed on the second electronic device includes highlighting the two-dimensional representation of the first object displayed on the second electronic device.

6. The method according to claim 5, further comprising: While simultaneously highlighting the first object in the three-dimensional environment and the two-dimensional representation of the first object displayed at the second electronic device, a second input is received at the second electronic device; as well as Based on the second input, the features of the two-dimensional representation of the first object displayed at the second electronic device are updated, and the features of the first object in the three-dimensional environment displayed at the first electronic device are also updated.

7. The method of claim 1, wherein one or more user interface elements are displayed in a toolbar associated with the tray.

8. The method according to claim 1, further comprising: Capturing at least multiple portions of a real-world environment via the one or more input devices, the real-world environment including the second electronic device and the content creation application displayed on the second electronic device; The three-dimensional environment further includes a representation of the captured portion of the real-world environment, the representation including: The representation of the second electronic device; as well as The content creates a representation of the application.

9. The method of claim 8, wherein the one or more user interface elements comprise: A first user interface element, which can be selected to enable the 3D preview to operate in the first mode; A second user interface element, which can be selected to stop the 3D preview from operating in the first mode and instead operate in the second mode; A third user interface element, which can be selected to scale the first object from a first size to a second size larger than the first size; A fourth user interface element, which can be selected to cause the first object to be dragged by the first action; and / or A fifth user interface element, which can be selected to cause a full-size representation of the first object to be displayed.

10. The method of claim 9, wherein displaying the full-size representation comprises: Occlude at least a plurality of portions of the three-dimensional environment and display the full-size representation of the first object on the occluded portions of the three-dimensional environment.

11. The method of claim 10, wherein displaying the full-size representation comprises: Simultaneously displayed in the three-dimensional environment: The representation of the 3D preview; as well as The representation of the second electronic device includes the representation of the content creation application and the two-dimensional representation of the first object.

12. The method according to claim 1, further comprising: Receive a second input via the one or more input devices; Based on the determination that the second input corresponds to a request to move the 3D preview, the 3D preview is moved within the 3D environment based on the second input; Based on the determination that the second input corresponds to a request to change the size of the tray, the size of the tray is changed from a first size to a second size different from the first size based on the second input; Based on the determination that the second input corresponds to a request to rotate the tray, the tray and the first object disposed on the surface of the tray are rotated within the three-dimensional environment according to the second input; as well as Based on the determination that the second input corresponds to a request to lift the tray, the first object is moved from a first height to a second height, offset from the initial position of the tray in the three-dimensional environment, according to the second input.

13. The method according to claim 12, wherein: The determination that the second input corresponds to the request to rotate the tray is based on the determination of the second input including: Interactive input provided by a predetermined portion of the user when the user's gaze is directed at the tray of the first electronic device for at least a threshold time, wherein, in response to detecting the interactive input, a representation of a pivot point is displayed at a corresponding location on the tray; and While maintaining the interactive input, the user's predetermined portion of the first electronic device moves in a corresponding direction relative to the representation of the pivot point on the tray; and The second input includes rotating the tray and the first object disposed on the surface of the tray about an axis passing through the pivot point, within the three-dimensional environment.

14. The method of claim 13, further comprising: The display of the pivot point on the tray is stopped in response to the release of the interactive input provided by the user's predetermined portion.

15. The method according to claim 1, further comprising: Receive a second input corresponding to a request to manipulate the first object in the three-dimensional environment via the one or more input devices; Based on the determination that the second input corresponds to a request to change the size of the first object, the size of the first object is changed from a first size to a second size different from the first size based on the second input; as well as Based on the determination that the second input corresponds to a request to reposition the first object on the tray, the first object is moved from a first position to a second position on the tray, different from the first position.

16. The method of claim 15, wherein: The determination that the second input corresponds to the request to change the size of the first object is based on determining the second input including: An interactive input provided by a predetermined portion of the user while the user's gaze is directed at the tray of the first electronic device for at least a threshold time, wherein, in response to detecting the interactive input, a representation of a pivot point is displayed at a corresponding position on the tray; The movement of the predetermined portion of the user of the first electronic device relative to the pivot point on the tray in the corresponding direction; and The change of the size of the tray from the first size to a second size, different from the first size, based on the second input, is the amount of movement of the predetermined portion of the user's device on the first electronic device.

17. The method of claim 15, further comprising: Based on the determination that the second input corresponds to a request to reposition the first object outside the boundaries of the tray: Stop displaying the first object; as well as Changing the appearance of the 3D preview, wherein changing the appearance of the 3D preview includes: Highlight a portion of the tray's boundary corresponding to the second location within the three-dimensional environment; and / or One or more vertical indicators are displayed above the boundary of the tray corresponding to the second position within the environment.

18. An electronic device comprising: One or more processors; Memory; as well as One or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing a method comprising the following steps: A three-dimensional environment, including a three-dimensional preview, is displayed on a monitor, wherein the three-dimensional preview includes: The first object; and One or more user interface elements; When displaying the 3D preview including the first object and the one or more user interface elements, a first input corresponding to the selection of the first object is received via one or more input devices; Based on the determination that the 3D preview is operating in a first mode, the first object is caused to perform a first action based on a determination that the first object is configured to perform a first action according to the selection, wherein the first action includes movement of the first object in the 3D preview; and Based on the determination that the 3D preview is operating in a second mode different from the first mode, the appearance of the first object in the 3D preview is changed to indicate the selection of the first object, wherein: The electronic device further communicates with the second electronic device; The second electronic device is configured to display a content creation application that includes a two-dimensional representation of the first object; and The 3D preview includes a tray, and the first object is disposed on a surface on the tray.

19. The electronic device of claim 18, wherein causing the first object to perform the first action comprises: The first animation that renders the first object in the three-dimensional environment includes the movement of the first object; as well as Based on the determination that the 3D preview is operating in the first mode, and based on the determination that the first object is configured to perform the first action, the first animation of the 2D representation of the first object is simultaneously displayed at the second electronic device.

20. The electronic device of claim 18, wherein the method further comprises: Based on the determination that the 3D preview is operating in the second mode, the appearance of the 2D representation of the first object displayed on the second electronic device is changed to indicate the selection of the first object.

21. The electronic device according to claim 20, wherein: Changing the appearance of the first object in the three-dimensional environment includes highlighting the first object in the three-dimensional environment; and Changing the appearance of the two-dimensional representation of the first object displayed on the second electronic device includes highlighting the two-dimensional representation of the first object displayed on the second electronic device.

22. The electronic device of claim 21, wherein the method further comprises: While simultaneously highlighting the first object in the three-dimensional environment and the two-dimensional representation of the first object displayed at the second electronic device, a second input is received at the second electronic device; as well as Based on the second input, the features of the two-dimensional representation of the first object displayed at the second electronic device are updated, and the features of the first object in the three-dimensional environment displayed at the electronic device are also updated.

23. A non-transitory computer-readable storage medium storing one or more programs, said one or more programs including instructions that, when executed by one or more processors of a first electronic device, cause the first electronic device to perform a method comprising the following steps: A three-dimensional environment, including a three-dimensional preview, is displayed on a monitor, wherein the three-dimensional preview includes: First object; as well as One or more user interface elements; When displaying the 3D preview including the first object and the one or more user interface elements, a first input corresponding to the selection of the first object is received via one or more input devices; Based on the determination that the 3D preview is operating in a first mode, the first object is made to perform the first action based on the determination that the first object is configured to perform a first action according to the selection, wherein the first action includes movement of the first object in the 3D preview; as well as Based on the determination that the 3D preview is operating in a second mode different from the first mode, the appearance of the first object in the 3D preview is changed to indicate the selection of the first object, wherein: The first electronic device further communicates with the second electronic device; The second electronic device is configured to display a content creation application that includes a two-dimensional representation of the first object; and The 3D preview includes a tray, and the first object is disposed on a surface on the tray.

24. The non-transitory computer-readable storage medium of claim 23, wherein the method further comprises: Capturing at least multiple portions of a real-world environment via the one or more input devices, the real-world environment including the second electronic device and the content creation application displayed on the second electronic device; The three-dimensional environment further includes a representation of the captured portion of the real-world environment, the representation including: The representation of the second electronic device; as well as The content creates a representation of the application.

25. The non-transitory computer-readable storage medium of claim 23, wherein the method further comprises: Receive a second input via the one or more input devices; Based on the determination that the second input corresponds to a request to move the 3D preview, the 3D preview is moved within the 3D environment based on the second input; Based on the determination that the second input corresponds to a request to change the size of the tray, the size of the tray is changed from a first size to a second size different from the first size based on the second input; Based on the determination that the second input corresponds to a request to rotate the tray, the tray and the first object disposed on the surface of the tray are rotated within the three-dimensional environment according to the second input; as well as Based on the determination that the second input corresponds to a request to lift the tray, the first object is moved from a first height to a second height, offset from the initial position of the tray in the three-dimensional environment, according to the second input.

26. The non-transitory computer-readable storage medium according to claim 25, wherein: The determination that the second input corresponds to the request to rotate the tray is based on the determination of the second input including: Interactive input provided by a predetermined portion of the user when the user's gaze is directed at the tray of the first electronic device for at least a threshold time, wherein, in response to detecting the interactive input, a representation of a pivot point is displayed at a corresponding location on the tray; and While maintaining the interactive input, the user's predetermined portion of the first electronic device moves in a corresponding direction relative to the representation of the pivot point on the tray; and The second input includes rotating the tray and the first object disposed on the surface of the tray about an axis passing through the pivot point, within the three-dimensional environment.

Images