SYSTEMS AND METHODS OF THE PRODUCTION WORKFLOW FOR SCENE CREATION

Abstract

Some examples of the disclosure relate to systems and methods for displaying a virtual three-dimensional environment on an electronic device. Some examples of the disclosure relate to displaying a virtual three-dimensional environment according to an environment template. Some examples of the disclosure relate to a virtual stage, a virtual preview of the virtual three-dimensional environment, a virtual viewbox, a virtual model, and an immersive display of the virtual three-dimensional environment.

Description

CROSS-REFERENCE TO RELATED REGISTRATIONS

[0001] This application claims the benefit of U.S. Preliminary Patent Application No. 63 / 700,402, filed on September 27, 2024, U.S. Patent Application No. 19 / 311,465, filed on August 27, 2025, and U.S. Patent Application No. 19 / 300,648, filed on September 16, 2025, the contents of which are incorporated herein by reference in their entirety for all purposes. AREA OF REVELATION

[0002] This refers generally to systems and methods for displaying virtual three-dimensional environments and, in particular, for displaying virtual scenes according to templates. BACKGROUND OF THE REVELATION

[0003] Some computer graphics environments provide two-dimensional and / or three-dimensional environments in which at least some objects displayed to a user are virtual and generated by a computer. In some examples, virtual three-dimensional environments may be based on one or more images of the computer's physical environment. In other examples, virtual three-dimensional environments do not include any images of the computer's physical environment. SUMMARY OF THE REVELATION

[0004] This generally refers to systems and methods for displaying virtual three-dimensional environments, and in particular, for displaying the virtual three-dimensional environment according to an environment template. In some examples, the template includes one or more virtual stages, virtual models, virtual preview and / or view portals, virtual view boxes, and / or immersive views of the virtual three-dimensional environment. In some examples, data and information used to display the virtual three-dimensional environment on an electronic device, such as a headset, are received and / or streamed from a computer system. In some examples, an electronic device exports virtual content from the virtual three-dimensional environment. In some examples, an electronic device modifies an active environment template.In some examples, an electronic device enables the viewing and interaction with a virtual three-dimensional environment for a media production workflow.

[0005] The complete descriptions of these examples are included in the drawings and the detailed description, and it is understood that this summary does not in any way limit the scope of the revelation. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] For a better understanding of the various examples described herein, please refer to the detailed description below, along with the accompanying drawings. The same reference numerals throughout the drawings often refer to corresponding parts. Fig. Figure 1 illustrates an electronic device that illustrates a three-dimensional environment according to some examples from the disclosure. Fig. Sections 2A to 2B illustrate block diagrams of exemplary architectures for electronic devices according to some examples of the disclosure. Fig. Figure 3 illustrates an example of an electronic device that represents a virtual three-dimensional environment template including a virtual stage according to some examples of the disclosure. Fig. Figure 4 illustrates the display of an environment template including a virtual model according to some examples from the revelation. Fig. Figure 5 illustrates the display of a virtual environment with a level of immersion that is above a threshold level of immersion, according to some examples from the Revelation. Fig. Figure 6 illustrates the display of a review of animations according to some examples from the Revelation. Fig. Figure 7 illustrates an example of electronic devices that represent a virtual three-dimensional environment template, which includes a virtual model and an environment preview according to some examples of the disclosure. Fig. Figure 8 illustrates an example of an electronic device that represents a virtual three-dimensional environment template including a viewbox according to some examples of the disclosure. Fig. Figure 9 is a flowchart of a procedure for representing a virtual three-dimensional environment according to a template based on some examples from the revelation. Fig. Figure 10 is a flowchart of a method for streaming information from a computer system to an electronic device to cause the electronic device to display a virtual three-dimensional environment according to a template according to some examples of the disclosure. DETAILED DESCRIPTION

[0007] This generally refers to systems and methods for displaying virtual three-dimensional environments, and in particular, for displaying the virtual three-dimensional environment according to an environment template. In some examples, the template includes one or more virtual stages, virtual models, virtual preview and / or view portals, virtual view boxes, and / or immersive views of the virtual three-dimensional environment. In some examples, data and information used to display the virtual three-dimensional environment on an electronic device, such as a headset, are received and / or streamed from a computer system. In some examples, an electronic device exports virtual content from the virtual three-dimensional environment. In some examples, an electronic device modifies an active environment template.In some examples, an electronic device enables the viewing and interaction with a virtual three-dimensional environment for a media production workflow.

[0008] In some examples, a three-dimensional object is displayed in a computer-generated three-dimensional environment with a specific orientation that controls one or more behaviors of the three-dimensional object (e.g., when the three-dimensional object is moved within the three-dimensional environment). In some examples, the orientation in which the three-dimensional object is displayed in the three-dimensional environment is selected by a user of the electronic device or automatically selected by the electronic device. For example, when the presentation of the three-dimensional object in the three-dimensional environment is initiated, the user can select a specific orientation for the three-dimensional object, or the electronic device can automatically select the orientation for the three-dimensional object (e.g., based on a type of three-dimensional object).

[0009] In some examples, a three-dimensional object can be displayed in the three-dimensional environment in a fixed orientation, a body-fixed orientation, a tilt-fixed orientation, or a head-fixed orientation, as described below. As used herein, an object displayed in a three-dimensional environment in a body-fixed orientation has a distance and orientation offset relative to a part of the user's body (e.g., the user's torso). Alternatively, in some examples, a body-fixed object has a fixed distance from the user without the orientation of its content being relative to any part of the user's body (e.g., it can be displayed in the same direction relative to the user regardless of head and / or body movements).Additionally or alternatively, in some examples, the fixed object can be configured to always remain aligned with gravity or the horizon (e.g., perpendicular to gravity), so that changes in the head and / or body's direction of roll would not cause the fixed object to move within its three-dimensional environment. Rather, in any configuration, a translational movement would cause the fixed object to reposition itself within its three-dimensional environment to maintain the distance offset.

[0010] As used herein, an object displayed in a head-fixed orientation within a three-dimensional environment has a distance and orientation offset relative to the user's head. In some examples, a head-fixed object moves within the three-dimensional environment as the user's head moves (as the user's viewing angle changes).

[0011] As used herein, an object displayed in a three-dimensional environment in a fixed orientation has no distance or orientation offset relative to the user.

[0012] As used herein, an object displayed in a three-dimensional environment in a tilt-fixed orientation (referred to herein as a tilt-fixed object) has a distance offset relative to the user, for example, to a part of the user's body (e.g., the user's torso) or the user's head. In some examples, a tilt-fixed object is displayed in a fixed orientation relative to the three-dimensional environment. In some examples, a tilt-fixed object moves according to a polar (e.g., spherical) coordinate system whose center point is a pole through the user (e.g., the user's head). For example, the tilt-fixed object moves in the three-dimensional environment based on the movement of the user's head within a spherical space surrounding the user's head (e.g., head as center point). Consequently, if the user tilts their head relative to gravity (e.g.,Up or down in the direction of tilt), the tilt-fixed object would follow the head tilt and move radially along a sphere, thus repositioning itself within the three-dimensional environment to be offset relative to the user by the same distance as before the head tilt, while optionally maintaining the same orientation relative to the three-dimensional environment. In some examples, the tilt-fixed object does not reposition itself within the three-dimensional environment when the user moves their head relative to gravity in the direction of roll (e.g., clockwise or counterclockwise).

[0013] Fig. Figure 1 illustrates an electronic device 101 that visualizes a three-dimensional environment (e.g., an augmented reality (XR) environment or a computer-generated reality (CGR) environment, optionally including representations of physical and / or virtual objects), according to some examples of the disclosure. In some examples, as in Fig. As shown in Figure 1, the electronic device 101 is a head-mounted display or other head-mounted device configured to be worn on the head of a user of the electronic device 101. Examples of the electronic device 101 are given below with reference to the architectural block diagram of Fig. 2A described. As in Fig. As shown in Figure 1, the electronic device 101 and table 106 are located in a physical environment. The physical environment can include physical features such as a physical surface (e.g., floor, walls) or a physical object (e.g., table, lamp, etc.). In some embodiments, the electronic device 101 can be configured to detect and / or capture images of the physical environment, including the table 106 (illustrated in the field of view of the electronic device 101).

[0014] In some examples, such as in Fig. As shown in Figure 1, the electronic device 101 includes one or more internal image sensors 114a oriented towards the user's face (e.g., eye-tracking cameras, which are described below in relation to Fig. (described in sections 2A to 2B). In some examples, internal image sensors 114a are used for eye tracking (e.g., to detect the user's gaze). Internal image sensors 114a are optionally arranged in the left and right areas of the display 120 to enable eye tracking of the user's left and right eyes. In some examples, the electronic device 101 also includes external image sensors 114b and 114c, which are directed outwards from the user to detect and / or capture the physical environment of the electronic device 101 and / or movements of the user's hands or other body parts.

[0015] In some examples, the display 120 has a user-visible field of view. In some examples, the user-visible field of view is the same as the field of view of the external image sensors 114b and 114c. For example, if the display 120 is optionally part of a head-mounted device, its field of view is optionally the same as or similar to the user's eye field of view. In some examples, the user-visible field of view differs from the field of view of the external image sensors 114b and 114c (e.g., narrower than the field of view of the external image sensors 114b and 114c). In other examples, the field of view of the display 120 may be smaller than the user's eye field of view. A user's viewing angle determines what content is visible in the field of view; a viewing angle generally specifies a location and direction relative to the three-dimensional environment.When a user's viewing angle changes, the field of view of the three-dimensional environment also changes accordingly. In some examples, the electronic device 101 can be an optically transparent device, where the display 120 is a transparent or translucent display through which parts of the physical environment can be viewed directly. In some examples, the display 120 can be enclosed in a transparent lens and overlap all or part of the transparent lens. In other examples, the electronic device can be a video passthrough device, where the display 120 is an opaque display configured to display images of the physical environment using images captured by the external image sensors 114b and 114c. While in . Fig. Where a single display is shown, it is understood that display 120 optionally includes more than one display. For example, display 120 optionally includes a stereo display pair (e.g., left and right display fields for the user's left and right eyes, respectively) with displayed outputs that are merged (e.g., by the user's brain) to represent the information in Fig. 1. To generate the view of the content shown. In some examples, as below with reference to Fig. As explained in more detail in sections 2A to 2B, display 120 includes a transparent or translucent surface (e.g. a lens) that is not equipped with display functions (and therefore, for example, is unable to generate and display the virtual object 104) and alternatively represents or corresponds to a direct view of the physical environment in the user's field of vision (e.g. the user's eye field of vision).

[0016] In some examples, the electronic device 101 is configured to display a virtual object 104 in the three-dimensional environment (e.g., in response to a trigger). The virtual object 104 is displayed by a Fig. Figure 1 illustrates a cube that is not present in the physical environment but is displayed in the three-dimensional environment, positioned on the top of the table 106 (e.g., a real table or a representation thereof). Optionally, the virtual object 104 can be displayed on the surface of the table 106 in the three-dimensional environment via the display 120 of the electronic device 101 in response to the detection of the flat surface of the table 106 in the physical environment 100.

[0017] It is understood that virtual object 104 is a representative virtual object, and one or more different virtual objects (e.g., of different dimensions, such as two-dimensional or other three-dimensional virtual objects) can be enclosed and rendered in a three-dimensional environment. For example, the virtual object can represent an application or a user interface displayed in the three-dimensional environment. In some examples, the virtual object can represent content corresponding to the application and / or displayed via the user interface in the three-dimensional environment. In some examples, virtual object 104 is optionally configured to be interactive and responsive to user input (e.g.,Air gestures, such as pinch air gestures, tap air gestures and / or touch air gestures), so that a user can virtually touch, tap, move, rotate or otherwise interact with the virtual object 104.

[0018] As discussed herein, one or more air pincer grip gestures performed by a user (e.g., with hand 103 in Fig. 1) is detected by one or more input devices of the electronic device 101 and interpreted as one or more user inputs directed at content displayed by the electronic device 101. Additionally or alternatively, in some examples, the one or more user inputs interpreted by the electronic device 101 as directed at content displayed by the electronic device 101 (e.g., the virtual object 104) are detected via one or more hardware input devices (e.g., controllers, touchpads, proximity sensors, keys, sliders, knobs, etc.) instead of via the one or more input devices configured to detect air gestures, such as the one or more air pincer gestures performed by the user. Such a representation is intended to be exemplary rather than limiting.The user optionally provides user input using different air gestures and / or other input methods.

[0019] In some examples, the electronic device 101 may be configured to communicate with a second electronic device, such as a companion device. For example, the electronic device 101, as in Fig. Figure 1 illustrates, optionally with the electronic device 160 in communication. In some examples, the electronic device 160 corresponds to a mobile electronic device, such as a smartphone, tablet computer, smartwatch, laptop computer, or other electronic device. In other examples, the electronic device 160 corresponds to a non-mobile electronic device that is generally stationary and cannot be easily moved within the physical environment (e.g., desktop computer, server, etc.). Additional examples of the electronic device 160 are given below with reference to the block diagram architecture of Figure 1. Fig. 2B described. In some examples, electronic device 101 and electronic device 160 are assigned to the same user. For example, in Fig. 1. The electronic device 101 may be positioned on (e.g., attached to) a user's head, and the electronic device 160 may be positioned near the electronic device 101, such as in the user's hand 103 (e.g., the hand 103 holds the electronic device 160), in the user's pocket or bag, or on a surface near the user. The electronic device 101 and the electronic device 160 are optionally associated with the same user account (e.g., the user is logged into the user account on both the electronic device 101 and the electronic device 160). Further details regarding the communication between the electronic device 101 and the electronic device 160 are described below with reference to Fig. 2A to 2B provided.

[0020] In some examples, displaying an object in a three-dimensional environment can be initiated or enabled by an interaction with one or more user interface objects within that environment. For instance, initiating the display of the object in the three-dimensional environment might involve an interaction with one or more virtual options / affordances displayed within that environment. In some examples, a user's gaze can be tracked as input to identify one or more virtual options / affordances available for selection when the display of an object in the three-dimensional environment is initiated. The gaze might, for example, be used to identify one or more virtual options / offers to be selected using a further selection input.In some examples, a virtual option or offer can be selected using hand tracking input, which is captured via an input device communicating with the electronic device. In some examples, objects displayed in the three-dimensional environment can be moved and / or reoriented in the three-dimensional environment in accordance with the motion input captured by the device.

[0021] The following descriptions describe an electronic device that communicates with one or more displays and one or more input devices. It is understood that the electronic device optionally communicates with one or more other physical user interface devices, such as a touch-sensitive surface, a physical keyboard, a mouse, a joystick, a hand tracking device, an eye tracking device, a stylus, etc. Furthermore, as described above, it is understood that the described electronic device, the display, and the touch-sensitive surface are optionally distributed among two or more devices.Therefore, as used in this disclosure, information displayed on or by the electronic device is optionally used to describe information output by the electronic device for display on a separate display device (touch-sensitive or not). Similarly, as used in this disclosure, input received by the electronic device (e.g., touch input received on a touch-sensitive surface of the electronic device or touch input received on the surface of an input pen) is optionally used to describe input received on a separate input device from which the electronic device receives input information.

[0022] The device typically supports a variety of applications, such as one or more of the following: a drawing application, a presentation application, a word processing application, a website creation application, a disc authoring software development application, a spreadsheet application, a gaming application, a telephone application, a video conferencing application, an email application, an instant messaging application, a training support application, a photo management application, a digital camera application, a digital video camera application, an internet browsing application, a digital music playback application, a television channel browsing application, and / or a digital video playback application.

[0023] Fig. Figures 2A to 2B illustrate block diagrams of exemplary architectures for electronic devices according to some examples of the disclosure. In some examples, the electronic device 201 and / or the electronic device 260 includes one or more electronic devices. For example, the electronic device 201 may be a portable device, an auxiliary device communicating with another device, a head-mounted display, a head-worn loudspeaker, etc. In some examples, the electronic device 201 corresponds to the one described above with reference to Fig. 1 electronic device 101 described above. In some examples, the electronic device 260 corresponds to the one described above with reference to Fig. 1 described electronic device 160.

[0024] As in Fig. As illustrated in Figure 2, the electronic device 201 optionally includes one or more sensors, such as one or more hand tracking sensors 202, one or more location sensors 204A, one or more image sensors 206A (optionally the internal image sensors 114a and / or external image sensors 114b and 114c in Fig. 1 accordingly), one or more touch-sensitive surfaces 209A, one or more motion and / or orientation sensors 210A, one or more eye-tracking sensors 212, one or more microphones 213A or other audio sensors, one or more body-tracking sensors (e.g., upper body and / or head-tracking sensors), etc. The electronic device 201 optionally includes one or more output devices, such as one or more display generation components 214A, optionally the display 120 in Fig. 1 accordingly, one or more loudspeakers 216A, one or more haptic output devices (not shown), etc. The electronic device 201 optionally includes one or more processors 218A, one or more memories 220A, and / or communication switching logic 222A. One or more communication buses 208A are optionally used for communication between the aforementioned components of the electronic device 201.

[0025] Furthermore, the electronic device 260 optionally includes the same or similar components as the electronic device 201. As in Fig. As shown in Figure 2B, the electronic device 260 optionally includes, for example, one or more location sensors 204B, one or more image sensors 206B, one or more touch-sensitive surfaces 209B, one or more orientation sensors 210B, one or more microphones 213B, one or more display generation components 214B, one or more loudspeakers 216B, one or more processors 218B, one or more memory units 220B, and / or communication switching logic 222B. One or more communication buses 208B are optionally used for communication between the aforementioned components of the electronic device 260.

[0026] Electronic devices 201 and 260 are optionally configured to communicate between the two devices via a wired or wireless connection (e.g., via communication switching logic 222A, 222B). For example, electronic device 260 can, as shown in Fig. 2A, as specified, to function as an accompanying device to the electronic device 201. In some examples, the electronic device 260, for instance, processes sensor inputs from the electronic devices 201 and 260 and / or generates content for display using the display generation components 214A of the electronic device 201.

[0027] The 222A and 222B communication switching logic optionally includes switching logic for communicating with electronic devices, networks such as the Internet, intranets, wired and / or wireless networks, cellular networks, and wireless local area networks (LANs). The 222A and 222B communication switching logic optionally includes switching logic for communicating using near-field communication (NFC) and / or short-range communication, such as Bluetooth®. In some examples, the 222A and 222B communication switching logic includes or supports Wi-Fi (e.g., an 802.11 protocol), Ethernet, ultra-wideband (UWB), radio frequency systems (e.g., 900 MHz, 2.4 GHz, and 5.6 GHz communication systems), or any other communication protocol or combination thereof.

[0028] The one or more processors 218A, 218B include one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, one or more processors 218A, 218B include one or more microprocessors, one or more central processing units, one or more application-specific integrated circuits, one or more field-programmable gate arrays, one or more programmable logic devices, or a combination of these devices. In some examples, memory 220A and / or memory 220B is a non-transient, computer-readable storage medium (e.g.,Flash memory, random-access memory, or other volatile or non-volatile memory or storage that stores computer-readable instructions configured to be executed by one or more processors 218A, 218B to perform the techniques, processes, and / or procedures described below. In some examples, memory 220A, 220B may include more than one non-transient computer-readable storage medium. A non-transient computer-readable storage medium may be any medium (except, for example, a signal) that can tangiblely contain or store computer-executable instructions for use by, or in connection with, the instruction execution system, instruction execution device, or instruction execution apparatus. In some examples, the storage medium is a transient computer-readable storage medium.In some examples, the storage medium is a non-transient, computer-readable storage medium. Non-transient, computer-readable storage mediums can include, but are not limited to, magnetic, optical, and / or semiconductor storage. Examples of such storage include magnetic disks, optical disks based on Compact Disc (CD), Digital Versatile Disc (DVD), or Blu-ray technologies, and persistent semiconductor storage such as flash memory, solid-state drives, and the like.

[0029] In some examples, one or more display-generating components 214A, 214B include a single display (e.g., a liquid crystal display (LCD), organic light-emitting diode (OLED), or other types of displays). In some examples, the one or more display-generating components 214A, 214B include multiple displays. In some examples, the one or more display-generating components 214A, 214B may include a touch-enabled display (e.g., a touchscreen), a projector, a holographic projector, a retina projector, a transparent or translucent display, etc. In some examples, the electronic device does not include any display-generating components 214A or 214B.For example, some electronic devices include transparent or translucent lenses or other surfaces not configured to display or represent virtual content, instead of one or more display generation components 214A or 214B. It is understood, however, that in these cases, the electronic device 201 and / or the electronic device 260 may optionally be combined with one or more of the other components described in the following. Fig. 2A and Fig. The electronic devices 201 and / or 260 are equipped with components illustrated in Figure 2B and described herein, such as one or more hand tracking sensors 202, one or more eye tracking sensors 212, one or more image sensors 206A, and / or one or more motion and / or orientation sensors 210A. Alternatively, in some examples, the one or more display generation components 214A or 214B are provided separately from the electronic devices 201 and / or 260. For example, the one or more display generation components 214A, 214B communicate with the electronic device 201 (and / or the electronic device 260) but are not integrated into the electronic device 201 and / or the electronic device 260 (e.g., within an enclosure of the electronic devices 201, 260).In some examples, the electronic devices 201 and 260 include one or more touch-sensitive surfaces 209A and 209B, respectively, for receiving user input, such as taps, swipes, or other gestures (e.g., hand- or finger-based gestures). In some examples, the one or more display-generating components 214A, 214B, and the one or more touch-sensitive surfaces 209A, 209B form one or more touch-sensitive displays (e.g., a touchscreen that is integrated into each of the electronic devices 201 and 260 or is located outside of each of the electronic devices 201 and 260 and communicates with each of the electronic devices 201 and 260).

[0030] The electronic devices 201 and 260 optionally include one or more image sensors 206A and 206B, respectively. The one or more image sensors 206A and 206B optionally include one or more visible light image sensors, such as CCD (charge-coupled device) sensors and / or CMOS (complementary metal oxide semiconductor) sensors, which can be operated to obtain images of physical objects from the real environment. The one or more image sensors 206A and 206B also optionally include one or more infrared (IR) sensors, such as a passive or an active IR sensor, for detecting infrared light from the real environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real environment.The one or more image sensors 206A, 206B optionally include one or more cameras configured to detect movement of physical objects in the real-world environment. The one or more image sensors 206A, 206B optionally also include one or more depth sensors configured to detect the distance of physical objects from the electronic device 201, 260. In some examples, information from one or more depth sensors can enable the device to identify objects in the real-world environment and distinguish them from other objects in the real-world environment. In some examples, one or more depth sensors can enable the device to determine the texture and / or topography of objects in the real-world environment.In some examples, the one or more image sensors 206A or 206B are enclosed in an electronic device that is distinct from the electronic devices 201 and / or 260. For example, the one or more image sensors 206A, 206B communicate with the electronic device 201, 260, but are not integrated into the electronic device 201, 260 (e.g., within a housing of the electronic device 201, 260). In particular, in some examples, the one or more cameras of the one or more image sensors 206A, 206B are integrated into and / or coupled to one or more devices separate from the electronic devices 201 and / or 260 (but communicate with the electronic devices 201 and / or 260, for example), such as one or more input and / or output devices (e.g.,one or more loudspeakers and / or one or more microphones (such as earphones or headphones) enclosing one or more image sensors 206A, 206B. In some examples, the electronic device 201 or the electronic device 260 corresponds to a head-worn loudspeaker (e.g., headphones or earphones). In these cases, the electronic device 201 or the electronic device 260 is connected to a subset of the others in . Fig. 2A and Fig. 2B illustrated and described herein are equipped with the components. In some of these examples, the electronic device 201 or the electronic device 260 is equipped with one or more image sensors 206A, 206B, one or more motion and / or orientation sensors 210A, 210B and / or loudspeakers 216A, 216B.

[0031] In some examples, the electronic device 201, 260 uses CCD sensors, event cameras, and depth sensors in combination to detect the physical environment around the electronic devices 201, 260. In some examples, the one or more image sensors 206A, 206B include a first image sensor and a second image sensor. The first and second image sensors work together and are optionally configured to capture different information about physical objects in the real-world environment. In some examples, the first image sensor is a visible light image sensor, and the second image sensor is a depth sensor. In some examples, the electronic device 201, 260 uses the one or more image sensors 206A, 206B to detect the position and orientation of the electronic device 201, 260 and / or the display generation components 214A, 214B in the real-world environment.For example, the electronic device 201, 260 uses the one or more image sensors 206A, 206B to track the position and orientation of the one or more display generation components 214A, 214B relative to one or more fixed objects in the real environment.

[0032] In some examples, the electronic devices 201 and 260 include one or more microphones 213A and 213B, respectively, or other audio sensors. The electronic device 201, 260 optionally uses the one or more microphones 213A, 213B to detect sound from the user and / or from the user's real-world environment. In some examples, the one or more microphones 213A, 213B include an array of microphones (e.g., a plurality of microphones) that optionally work together to identify ambient noise or to localize the sound source within the real-world environment.

[0033] The electronic devices 201 and 260 include one or more location sensors 204A and 204B, respectively, for detecting the location of the electronic device 201 and / or the one or more display generation components 214A, or the location of the electronic device 260 and / or the one or more display generation components 214B. The one or more location sensors 204A, 204B may include a GPS (Global Positioning System) receiver that receives data from one or more satellites and enables the electronic devices 201, 260 to determine their absolute position in the physical world.

[0034] The electronic devices 201 and 260 include one or more orientation sensors 210A and 210B, respectively, for detecting the orientation and / or movement of the electronic device 201 and / or the one or more display generation components 214A, or the orientation and / or movement of the electronic device 260 and / or the one or more display generation components 214B. For example, the electronic device 201, 260 uses one or more orientation sensors 210A, 210B to track changes in the position and / or orientation of the electronic device 201, 260 and / or the one or more display generation components 214A, 214B, as with respect to physical objects in the real environment. The one or more orientation sensors 210A, 210B optionally include one or more gyroscopes and / or one or more accelerometers.

[0035] In some examples, the electronic device 201 includes one or more hand tracking sensors 202 and / or one or more eye tracking sensors 212. It is understood that the electronic device 201, although referred to as hand tracking or eye tracking sensors, additionally or alternatively optionally includes one or more other body tracking sensors, such as one or more leg, one or more torso, and / or one or more head tracking sensors. The one or more hand tracking sensors 202 are configured to track the position and / or location of one or more sections of the user's hands and / or movements of one or more sections of the user's hands in relation to the three-dimensional environment relative to the one or more display generation components 214A and / or relative to another defined coordinate system.The one or more eye-tracking sensors 212 are configured to track the position and movement of a user's gaze (e.g., a user's attention, including eyes, face, or generally the head) with respect to the real or three-dimensional environment and / or relative to the one or more display-generating components 214A. In some examples, the one or more hand-tracking sensors 202 and / or the one or more eye-tracking sensors 212 are implemented together with the one or more display-generating components 214A. In other examples, the one or more hand-tracking sensors 202 and / or the one or more eye-tracking sensors 212 are implemented separately from the one or more display-generating components 214A.In some examples, the electronic device 201 alternatively does not include the one or more hand tracking sensors 202 and / or the one or more eye tracking sensors 212. In some of these examples, the one or more display generation components 214A can be used by the electronic device 260 to provide a three-dimensional environment, and the electronic device 260 can receive inputs and other data from the one or more other sensors (e.g.,The electronic device 201 uses the inputs and data acquired by the one or more location sensors 204A, the one or more image sensors 206A, the one or more touch-sensitive surfaces 209A, the one or more motion and / or orientation sensors 210A, and / or the one or more microphones 213A or other audio sensors) as inputs and data to be processed by the one or more processors 218B of the electronic device 260. Additionally or alternatively, the electronic device 260 optionally includes no other inputs. Fig. The components shown in Figure 2B include, for example, the one or more location sensors 204B, the one or more image sensors 206B, the one or more touch-sensitive surfaces 209B, etc. In some of these examples, the one or more display generation components 214A can be used by the electronic device 260 to provide a three-dimensional environment, and the electronic device 260 can use as input inputs and other data acquired via the one or more motion and / or orientation sensors 210A (and / or the one or more microphones 213A) of the electronic device 201.

[0036] In some examples, the one or more hand tracking sensors 202 (and / or other body tracking sensors, such as leg, torso, and / or head tracking sensors) can use the one or more image sensors 206 (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that detect three-dimensional information from the real world, including one or more body parts (e.g., hands, legs, or torso of a human user). In some examples, the hands can be resolved with sufficient resolution to distinguish fingers and their respective positions. In some examples, the one or more image sensors 206A are positioned relative to the user to define a field of view for the one or more image sensors 206A and an interaction space in which finger / hand position, orientation, and / or movement detected by the image sensors are used as inputs (e.g.,(to distinguish between the user's hand at rest and the hands of other people in the real environment). Tracking fingers / hands for input (e.g., gestures, touch, tapping, etc.) can be advantageous because the user does not need to touch, hold, or wear any kind of beacon, sensor, or other device.

[0037] In some examples, the one or more eye-tracking sensors 212 include at least one eye-tracking camera (e.g., IR cameras) and / or light sources (e.g., IR light sources, such as LEDs) that emit light toward a user's eyes. The eye-tracking cameras may be directed toward a user's eyes to receive reflected IR light from the light sources directly or indirectly from the eyes. In some examples, both eyes are tracked separately by respective eye-tracking cameras and light sources, and a gaze / view can be determined from tracking both eyes. In some examples, one eye (e.g., a dominant eye) is tracked by one or more respective eye-tracking cameras / light sources.

[0038] The electronic devices 201 and 260 are not related to the components and configuration of Fig. The electronic device 201 is limited to 2A to 2B, but may include fewer, different, or additional components in several configurations. In some examples, the electronic device 201 and / or the electronic device 260 may be implemented among several electronic devices (e.g., as a system). In some of these examples, each of the electronic devices (or devices) may include one or more of the components discussed above, such as various sensors, one or more display generation components, one or more loudspeakers, one or more processors, one or more memories, and / or communication switching logic. A person or persons using the electronic device 201 or the electronic device 260 are referred to herein as users of the device.

[0039] Some examples of the disclosure are directed to one or more electronic devices and / or computer systems configured to communicate information, to view virtual scenes, and / or to interact with virtual scenes. A virtual scene may include virtual content and / or other information rendered on a special device for viewing and / or interaction, such as a virtual campsite, a virtual office, a virtual meadow, a virtual city, and / or the like. In some examples, the virtual scene may include user-generated and / or computer-generated assets, such as a virtual floor, sky, grouping of objects, and / or metadata relating to these assets.

[0040] In some examples, the virtual scene can be displayed on a device. While the virtual scene is displayed, a user of the device can view, edit, and / or share comments about the content of the virtual scene. In some examples, the display device presents the virtual scene in an extended reality (XR) environment, virtual reality (VR) environment, and / or mixed reality (XR) environment that includes a portion of the virtual scene. In some examples, the virtual scene is displayed using an environment template, as described further herein. In some examples, the environment template is included in a user interface for creating content. For example, the user interface might be for an application that allows editing of Universal Scene Description (USD) files, and virtual assets are included in the USD files.The user interface and / or the virtual elements included in the environment template can enable a user of the device to inspect the virtual scene, comment on the virtual scene, and / or quickly collaborate with other users of other devices while inspecting the virtual scene.

[0041] In some examples, the display device can show virtual content within the elements it contains and / or be itself enclosed in an environment template to display a virtual environment. The virtual content can be displayed with a first level of detail, including resolution, appearance, simulated lighting, and / or a combination thereof, of the virtual assets displayed within the stage. The examples herein detail several operations related to how the virtual scene is rendered using an environment template and how users of the device can interact with a virtual scene while a content creation user interface is displayed according to the environment template.

[0042] The advent of digital display technology has enabled both professionals and hobbyists to create and share virtual content for consumers worldwide. For example, directors, cinematographers, visual effects artists, architects, designers, and software developers can use digital tools to create, share, and interact with virtual content such as virtual backgrounds, virtual scenes, virtual objects, and / or virtual user interfaces. Some devices are configured to display the virtual content within virtual reality (VR) environments, extended reality (XR) environments, and / or mixed reality (MR) environments. By displaying the virtual content within these environments, the devices can provide more intuitive, efficient, and novel ways to view and interact with the virtual content.However, it is understandable that current platforms for displaying virtual content require only limited or completely inflexible methods for displaying virtual content. Therefore, systems and methods for displaying virtual content in a variety of spatial and interactive formats may be desirable, especially in the context of creating different types of content during a media production workflow.

[0043] Some examples in this disclosure relate to the different ways in which virtual content can be displayed within a content creation user interface. The content creation user interface may include options for selecting a content creation project type, such as a real-time experience, immersive video, spatial video, three-dimensional movie, and / or a conventional movie project. In some examples, the devices may display the virtual content and / or scene according to an environment template. In some examples, the environment template may specify how the virtual content is displayed and / or what interactions are available to the users of the devices when the display of the virtual content and / or scene is initiated.In some examples, one or more devices can communicate via a multi-user communication session to interact with the virtual content simultaneously and / or jointly. In some examples, a host device, such as a desktop computer and / or a server, can stream data and metadata to one or more devices by offloading at least some of the processing required to render and / or display the virtual content. In this way, the examples of the disclosure considered herein provide novel approaches for rapidly creating, examining, and verifying content, such as creating three-dimensional computer graphics data.

[0044] In some examples, a device, such as a headset, can display a virtual scene according to an environment template format. In some examples, displaying the virtual scene using an environment format includes displaying a virtual stage and / or background. In some examples, displaying the virtual scene using an environment format includes displaying a model of the virtual scene. In some examples, displaying the virtual scene using an environment format includes displaying a preview of a virtual object, such as a viewport. In some examples, displaying the virtual scene using an environment format includes displaying the virtual scene as an environment with an immersion level above a threshold immersion level.In some examples, displaying the virtual scene using an environment format includes displaying a viewbox that corresponds to an area within the virtual scene.

[0045] By displaying the same virtual scene according to different environment templates, the display device can ensure that virtual content is presented in a way that allows for the review, editing, and / or export of assets, such as displaying an appropriate scale and / or view of the virtual scene. For example, displaying a virtual model of a virtual three-dimensional environment can visually emphasize the spatial arrangement of a large number of virtual objects, which may be comparatively less emphasized when displaying a virtual stage on which a virtual street and / or one or more of the virtual objects are shown.Additionally or alternatively, in some examples, the display of a view portal may represent a two-dimensional shape suitable for previewing a virtual three-dimensional environment when used as a background for generating two-dimensional and / or video content.

[0046] It should be noted that the specific order of inputs, provisions, presentation of information and other processes relating to Fig. The descriptions in sections 3 to 8 are merely exemplary, and examples in which the sequence of these processes may differ from that expressly described are also considered without going beyond the scope of this disclosure.

[0047] Fig. Figure 3 illustrates an example of an electronic device that represents a virtual three-dimensional environment template, including a virtual stage, according to some examples of the disclosure. In some examples, the electronic device 101 has the same architecture as the one described above with reference to Fig. 1 electronic device 101 described above and / or the above with reference to Fig. 2 described electronic device 201. In some examples, the device is described as in Fig. The virtual scene shown is displayed in response to receiving a specification of a virtual stage environment format. In some examples, the virtual stage defines an area in which created virtual content can be displayed and / or interacted with. Some examples related to Fig. The disclosures described in section 3 apply to additional or alternative examples, such as those relating to Fig. 4 to 8 described (e.g., description of attention, virtual environments in general, avatars, annotations, and / or the like). Some examples referring to Fig. The disclosures described in Section 3, such as those further described herein with reference to Stage 306, may be directed to the display of a virtual scene according to a virtual stage template when a specification of a virtual stage environment format and / or environment template is received, such as a request received by the computer system 312 at the electronic device 101.

[0048] In some examples, the electronic device 101 can be a first electronic device used by a user 308 to display user interfaces for viewing and interacting with virtual content and / or accessing and participating in a communication session. For example, the electronic device 101 can be a mobile device (such as a tablet, smartphone, media player, or portable device), a computer, or another electronic device. In some examples, the display generation component is a display integrated into the electronic device (optionally a touchscreen display), an external display such as a monitor, projector, television, and / or a hardware component (optionally integrated or external) for projecting a user interface or displaying a user interface to one or more users.In some examples, the one or more input devices include an electronic device or component capable of receiving user input (e.g., capturing user input, recognizing user input) and transmitting information associated with the user input to the electronic device. Examples of input devices include a touchscreen, a mouse (e.g., external), a trackpad (optionally integrated or external), a touchpad (optionally integrated or external), a remote control (e.g., external), another mobile device (e.g., separate from the electronic device), a handheld device (e.g., external), a controller (e.g., external), a camera, a depth sensor, an eye-tracking device, and / or a motion sensor (e.g., a hand-tracking device, a hand-motion sensor).In some examples, the electronic device communicates with a hand-tracking device (e.g., one or more cameras, depth sensors, proximity sensors, or touch sensors (e.g., a touchscreen or trackpad)). In some examples, the hand-tracking device is a wearable device, such as a smart glove. In some examples, the hand-tracking device is a handheld input device, such as a remote control or a stylus.

[0049] In some examples, Computer System 312 transmits and / or receives (e.g., streams) information such as data. In some examples, Computer System 312 includes part or all of the switching logic of Electronic Device 101 and / or Electronic Device 201 (e.g., with reference to Fig. (described in Sections 1 and 2A through 2B). In some examples, the electronic device 101 and the computer system 312 are different types of devices. For example, computer system 312 may be a desktop or laptop computer, and electronic device 101 may be a portable device, such as a headset. In some examples, computer system 312 streams the data using one or more data formats, such as JavaScript Object Notation (JSON), Extensible Markup Language (XML), and / or Graphics Library Transmission Format (GLTF). In some examples, the computer system and / or electronic device 101 use the streamed data to render and / or otherwise display scene diagrams, object models, animation data, and other graphics-related information.

[0050] In some examples, the electronic device 101 communicates with the computer system 312 using one or more protocols, such as UDP (User Datagram Protocol) and / or TCP / IP (Transmission Control Protocol / Internet Protocol), to send and / or receive data packets to and from the computer system 312. In some examples, the computer system 312 can host and / or manage XR applications that are used by the electronic device 101 to render displayed virtual content. For example, a computer system can implement a client-server architecture in which a central computing unit (e.g., computer system 312) manages the state of a virtual environment and sends updates to connected clients (e.g., electronic device 101 and additional or alternative devices).

[0051] In some examples, the three-dimensional environment is generated, displayed, or otherwise caused to be displayable by the device (e.g., a computer-generated augmented reality (CGR) environment, such as a virtual reality (VR) environment, a mixed reality (MR) environment, and / or an augmented reality (AR) environment). For example, the three-dimensional environment 302 may include the physical environment and / or the virtual environment of the user 308 and the electronic device 101.

[0052] In Fig. 3A includes the three-dimensional environment 302, a virtual three-dimensional environment (sometimes referred to herein, for example, as a virtual scene) that includes a variety of virtual objects and / or textures, the virtual scene being displayed via display 120 of the electronic device 101. In some examples, the electronic device 101 displays the virtual scene in a manner that completely replaces a view of the physical environment, as if the user were physically located in a physical equivalent of the virtual scene. In some examples, the display of the virtual scene that replaces the user's view of the physical environment may correspond to the display of the virtual scene with a level of immersion above a threshold immersion level, the immersion level(s) being further described herein.In some examples, the electronic device can, in response to the detection of a change in the user's viewing angle (e.g., changes in the user's position and / or orientation) in the physical environment, change the perspective view of the virtual scene as if the user were changing their position within the virtual scene. Fig. Figure 3 illustrates the three-dimensional environment 302 from the perspective of the electronic device 101 and additionally from a camera perspective in a glyph 310 below the perspective of the electronic device 101.

[0053] In some examples, the virtual scene is an immersive three-dimensional environment. For example, a user of the electronic device 101 can physically move within their physical environment (including areas of the physical environment that extend beyond the outer boundary of an enclosure of the electronic device 101). Fig. 3 are illustrated), and the device 101 optionally updates a simulated perspective of a virtual sky, a virtual floor, and objects in response to the detection of changes in the user's viewpoint (e.g., the user's position and / or orientation relative to their physical environment), similar to a physical perspective of a physical sky, a physical floor, and / or one or more physical objects as the user moves relative to their physical environment. In some examples, the virtual scene integrated into the three-dimensional environment 302 optionally includes a simulated texture overlaid on a physical representation of the floor of the user's physical environment, and / or a virtual floor with a simulated spatial profile (e.g., topography) that differs from that of the user's physical environment.Furthermore, the virtual scene can include a simulated atmosphere, such as a virtual sky (e.g., simulation of the lower atmosphere at dawn, during the day, at dusk, at night, and the like). It is understood that the virtual scene can be any suitable computer-generated environment without infringing upon the scope of protection afforded by the disclosure.

[0054] In Fig. 3A illustrates the user's physical environment beyond the outer boundaries of the housing of the electronic device 101. It is understood that the description of three-dimensional, virtual, and / or physical environments relating to Fig. 3 described, may apply at least in part to additional or alternative example processes and techniques described herein and relating to three-dimensional environments, virtual environments and / or the physical environments, as with reference to Fig. 4 to 8 described.

[0055] In some examples, the virtual scene is displayed as if it occupies one or more regions of the user's physical environment. The physical environment—in Fig. Figure 3, outside the housing of an electronic device 101, illustrates that the virtual scene can include a physical space occupied by the user 308. In some examples, the virtual scene can be displayed by display 120, which at least partially replaces a view of a representation of the user's physical environment and thus "consumes" a view of the physical environment. For example, the electronic device 101 can include one or more outward-facing cameras that receive images of the user's physical environment (e.g., image sensors 114a to 114b), and the images can be displayed via display 120 as if the user could view the physical environment directly without the assistance of the electronic device 101.At least part or all of the view of such a physical environment can be displayed at corresponding positions on the display 120 with an opacity level below a threshold (e.g., 0, 1, 5, 10, 15, 20, 25, 30, 35, 40, 45, or 50% opacity), and the virtual scene can be displayed at corresponding positions with an opacity level above a threshold (e.g., 0, 1, 5, 15, 25, 40, 50, 60, 65, 75, 90, or 100% opacity). Additionally or alternatively, in some examples, parts of the physical environment may be visible through a transparent portion of the display without the display actively showing these parts of the physical environment.

[0056] In some examples, the user's physical environment 308 may include one or more physical objects in the user's surroundings, physical people in the user's surroundings, physical walls, a physical floor, and the like. In some examples, the electronic device 101 may present representations of the user's physical environment. For example, the three-dimensional environment 302 in Fig. 3. An integrated virtual scene with at least a partial degree of light transmission is displayed, superimposed on a representation and / or view of the user's physical environment (e.g., images of the user's physical environment captured by sensors 114a to c) displayed on the display 120. In some examples, the representation of the three-dimensional environment 302 includes displaying virtual content and / or a view of the user's physical environment (e.g., via a passive optical passthrough that includes a lens and / or a partially or fully transparent material sheet such as glass). In some examples, a representation of the user's physical environment includes one or more images of the user's physical environment. In some examples, the electronic device 101 displays a real-time or near-real-time image stream (e.g.,Video) of one or more parts of the physical environment that correspond to a “representation” of the user’s physical environment.

[0057] As previously described, in some examples of the disclosure, the virtual scene may include one or more virtual objects. The virtual objects may include digital assets that model physical objects, virtual placeholder objects (e.g., polygons, prisms, and / or simulated two- or three-dimensional shapes), virtual objects including user interfaces for applications (e.g., stored in the memory of Electronic Device 101), and / or other virtual objects that can be displayed in a VR, XR, and / or MR environment.For example, the three-dimensional environment 302 includes a virtual street 328, a foreground 326 within the stage 306, a virtual background corresponding to the background 304, including virtual trees, which are optionally virtual objects displayed within the virtual scene at a simulated position similar to the physical position and orientation of a physical object relative to a viewpoint of the user 308. It is understood that a larger number, a smaller number, and / or alternative objects can be displayed without exceeding the scope of the disclosure.

[0058] In some examples, the electronic device 101 displays a virtual boundary and / or limit simultaneously with the virtual scene. For example, in Fig. 3 A boundary of the virtual stage 306 is displayed, enclosing a curved line that illustrates a circular border. In some examples, the border is displayed with visual properties, such as a color, brightness, saturation, opacity, hue, simulated lighting, and / or a glow effect that mimics the visual appearance of a light source illuminating the virtual scene, and / or a width to distinguish the border from the three-dimensional environment 302. For example, the virtual border (and therefore stage 306) is optionally triangular, pentagonal, circular, elliptical, and / or has any other suitable polygonal shape and / or set of curves. In some examples, the virtual border is volumetric and occupies one or more portions of the virtual floor and / or one or more portions of the virtual scene above the virtual floor.For example, the boundary is optionally a sphere, a cube, a rectangular prism, and / or another suitable volume shape, optionally enclosing one or more visually distinguishable edges. Displaying a virtual boundary reduces the likelihood that the user will mistakenly move relative to the three-dimensional environment in a way that would cause the electronic device 101 to cease displaying virtual content that lies within and / or corresponds to the virtual boundary, thereby reducing the processing overhead required to perform operations based on such mistaken movement.

[0059] In some examples, the electronic device 101 displays virtual content within stage 306 using a first rendering technique. For example, the electronic device 101 can use the first technique to display first content in the foreground 326 with one or more first levels of detail. It is understood that the specific rendering technique is not limited, but may include one or more of the following: multi-rendering of targeted virtual content, delayed and forward rendering, screen-space effects such as ambient occlusion, subsurface scattering and / or distortion and refraction, stereoscopic rendering, foveal rendering, asynchronous time warping, reprojection, late latching, delayed shading, ray tracing, ray casting, radiosity analysis, path tracing, neural rendering, and / or a combination thereof.It is further understood that virtual content can generally be rendered as relatively high-resolution sections of images, videos, and / or animations, displayed by the electronic device 101 for interaction by a user of the electronic device 101. By displaying the virtual content using the first rendering technique, a user of the electronic device 101 can examine a high-resolution section of the virtual scene, thereby reducing the time and effort required for the precise examination of virtual assets when creating virtual content using the virtual scene (e.g., images, videos, animations, immersive virtual experiences, and / or using the virtual scene as a background for traditional media such as television and / or film).

[0060] In some examples, the electronic device 101 displays virtual content outside the stage 306 using a second rendering technique that differs from the first. For example, the electronic device 101 can display the background 304 using the second rendering technique, which may include displaying a lower resolution, reduced detail in shading and / or textures, and / or a combination thereof. In some examples, the implementation of displaying virtual content using the second rendering technique includes displaying the virtual content outside the stage 306 with a second level of detail that differs from the first. It is understood that the second rendering technique may include one or more properties similar to or identical with those of the first rendering technique.In some examples, the second rendering technique may differ from the first by omitting one or more techniques, setting different thresholds for algorithms used to determine the display mode of virtual content, including one or more techniques not included in the first rendering technique, and / or using a combination of these. Furthermore, the mixed-rendering approach reduces the computational load on the electronic device 101 and / or a computer system communicating with the electronic device 101 by potentially using more computationally intensive techniques for virtual content closer to the user's viewpoint and / or less computationally intensive techniques for virtual content farther away from the user's viewpoint.

[0061] In general, it is understood that by displaying virtual content using the second technique, the electronic device 101 can reduce the computational effort required to display the virtual content compared to displaying the same virtual content using the first rendering technique. For example, the electronic device 101 can display a virtual cloud at a lower resolution than if the virtual cloud were displayed in stage 306. Additionally or alternatively, the cloud can move with a simulated parallax effect when a movement of the electronic device 101's viewing angle is detected. In some examples, the virtual content outside of stage 306 corresponds to a midground and / or background of the virtual scene.In some examples, the electronic device 101 can render other sections of the virtual scene simultaneously with stage 306 and / or background 304, such as by using a third rendering technique that differs from the first and / or second techniques. By simplifying the computational complexity that may be associated with displaying the virtual content by using a variety of rendering techniques, the electronic device 101 and / or a computer system that streams the data used to display the virtual content on the electronic device 101 reduces the amount of data that can be transmitted and / or reduces the processing required on the electronic device 101 and / or the computer system.

[0062] In some examples, the electronic device 101 can detect input directed toward a virtual content enclosed in the three-dimensional environment 302 (e.g., a glance, a voice command, an air gesture (e.g., an air pincer grip involving one or more contacts of a multitude of the user's fingers, pointing one or more fingers in the air, and curling one or more fingers together), contact with a trackpad, and / or selection with a stylus). In response to receiving the input directed toward the virtual object, the electronic device 101 can initiate scaling and / or movement of the virtual objects in a first dimension and with a first size corresponding to a direction and / or size of the user input.Additionally or alternatively, the electronic device 101 can initiate a scaling of a virtual object in a second different direction in response to an input directed at the virtual object.

[0063] In some examples, the electronic device 101 displays the virtual scene with a first visual appearance, while a position and / or orientation of the user 308 corresponds to an area of ​​the three-dimensional environment 302 that corresponds to the virtual scene. For example, the electronic device 101 displays the virtual scene with a first degree of immersion (e.g., total immersion, where the virtual scene replaces any representation of the user's physical environment). In some examples, the degree of immersion is equal to or includes the degree to which virtual content occupies a viewport of the electronic device 101. In some examples, the degree of immersion additionally or alternatively includes visual properties of the virtual scene, such as haze, brightness, saturation, resolution, and / or a combination thereof.For example, an immersion threshold can define the percentage and / or positions within the three-dimensional environment 302 occupied by the virtual three-dimensional environment, such as an immersion level of 5%, 15%, 25%, 35%, 50%, 75%, or 90%. For example, the electronic device 101 can display virtual content that replaces the display of a representation of the physical environment, and as the immersion level increases, the percentage of the viewport occupied by the virtual three-dimensional environment can increase. Accordingly, when displaying the virtual three-dimensional environment with an immersion level of 100%, the user 308 may optionally not see physical objects that may be located in their physical environment.Thus, increasing the level of immersion optionally results in more of the virtual environment being displayed, thereby replacing and / or obscuring a larger part of the physical environment, and reducing the level of immersion optionally results in less of the virtual environment being displayed, thereby revealing sections of the physical environment that were previously not displayed and / or obscured.

[0064] In some examples, prior to initiating the display, the electronic device 101 indicates the boundary of the virtual stage 306, which overlays a representation (e.g., image(s) and / or video) of the user's physical environment, and prompts the user 308 to remove physical objects from the area of ​​the three-dimensional environment 302 bounded by the boundary. Fig. 3 The electronic device 101 displays the virtual sky, the virtual ground and the previously described virtual objects with a relatively high degree of opacity (e.g. 100% opacity).

[0065] In some examples, the electronic device 101 displays one or more representations of persons other than the user 308 via the display 120. For example, the electronic device 101 can display virtual avatars and / or additional or alternative media, such as virtual windows, that include images of users corresponding to the avatars. Thus, the electronic device 101 can display representations of other persons who are virtually or physically enclosed in the user's three-dimensional environment 302. For example, the representations can optionally be expressive avatars, such as anthropomorphic avatars, that have one or more body parts that can move relative to each other. The body parts optionally include a head, hands, arms, shoulders, neck, legs, fingers, toes, facial features, and the like.In some examples, the representations may correspond to persons who are in the same physical environment as the user, such as persons who are in the user's physical space. In some examples, the representations are displayed via passive optical passthrough (e.g., a lens, a transparent material, and / or directly visible to the user's eyes) and correspond to a view of their respective physical users. The representation may include a variety of body parts, as an example of a fully expressive avatar or a representation of a physical individual who shares the user's physical environment. In some examples, the representation includes a partially expressive avatar or a representation of a user of an electronic device who does not physically share the user's physical environment.It is understood that such representations are merely exemplary and that additional or alternative representations may be included by users of corresponding electronic devices in the three-dimensional environment 302.

[0066] In some examples, the representations may correspond to persons who are not in the user's physical environment but are represented using spatial information. In some examples, the electronic device 101 uses the spatial information to map sections of the user's physical environment 308 to sections of the virtual scene and / or to map sections of the physical environments of the persons corresponding to the representations to parts of the virtual scene.For example, a communication session can take place between electronic device 101, a first electronic device used by a first user represented by a first representation, and a second electronic device used by a second user represented by a second representation, to enable mapping between the physical environments of the respective users of the respective electronic devices. In some examples, the communication session includes the communication of information, including real-time or near-real-time communication of sounds detected by the electronic devices (e.g., speech, user-generated sounds, and / or ambient sounds).In some examples, the communication session includes the communication of information that corresponds to real-time or near-real-time movements and / or requirements for the movement of representations (e.g., avatars) corresponding to the users participating in the communication session.

[0067] For example, the first electronic device can detect the movement of the first user according to the first representation in the user's physical environment (e.g., different from the physical environment of user 308) and can communicate information specifying this movement to the electronic devices participating in the communication session, including electronic device 101. Before detecting the movement, the first electronic device can display the virtual scene relative to a viewpoint of the first electronic device (e.g., a position and / or orientation relative to the virtual scene, similar to a physical position and / or orientation of the user relative to a physical equivalent of the virtual scene). In response to detecting the movement (e.g.,Upon receiving information indicating movement from the other electronic device, the first electronic device can update the user's viewpoint to reflect the physical movement (e.g., in a direction and / or by a certain amount), as if the user were physically moving through a physical equivalent of the virtual scene. It should be noted that requirements for such movement can be directed to an input device (e.g., a virtual joystick, trackpad, physical joystick, virtual button, physical button, and / or other suitable control) in addition to, or as an alternative to, the detection of the user's physical movement.The electronic device 101 can receive such information and, in response, move the representation relative to the virtual scene with a size and / or in a direction of movement that mimics the physical movement of the user of the first electronic device relative to the user's physical environment. It is understood that other electronic devices—such as an electronic device corresponding to the second representation and / or the electronic device 101—can also recognize similar inputs described with respect to the first electronic device and cause their corresponding representation to move within the virtual scene.

[0068] It is also understood that the movement and / or placement of representations of users participating in the communication session can be defined relative to a common coordinate system and not strictly relative to the virtual dimensions of the virtual scene. For example, the electronic device 101 can represent a view of the user's 308 physical environment without including a virtual scene, and can display representations of the user at positions within the view of the physical environment and / or movements of the user's representations within the view of the physical environment. It is to be assumed that the representations relating to Fig. Examples 3 to 8 described may occur during a communication session (described herein) and that information communicating positions, orientations, audio and / or other aspects of physical users, and / or information provided by physical users, may be exchanged via the communication session to devices participating in the communication session.

[0069] It is understood that the processes described with reference to virtual content, which are displayed relative to the virtual scene, may be displayed context-dependently relative to a representation of the user's physical environment, such as visual indications of the user's attention 308.

[0070] In some examples, the electronic device 101 displays one or more visual indicators that specify the user's attention within the three-dimensional environment 302. For example, the electronic device 101 detects a virtual position of a target of the user's attention (e.g., gaze) and displays a visual indicator at that virtual position, thus providing a visual indication of the portion of the three-dimensional environment 302 toward which the user's attention is directed. In some examples, the target of the user's attention is indicated by one or more body parts of the user other than the eyes. Although in Fig. Not shown in Figure 3, the electronic device 101 can, for example, detect a spatial relationship between a contact point between one or more fingers in the hand of the user 308 (e.g., by forming an air pinch grip or an air pointing gesture) and the electronic device 101. The spatial relationship can be based on a beam emitted from a part of the electronic device 101, such as its center, through parts of the user's body (e.g., through the air flick gesture, through a fingertip arranged in an air pointing gesture) and extending toward a position within the virtual scene.

[0071] In some examples, attention is detected and / or information regarding an attention target is received by the electronic device 101, and the electronic device 101 displays a visual indication of attention in response to the detection and / or receipt. According to a determination that a user's attention is directed toward a part of the virtual scene that is not visible (e.g., as if a physical user were looking at a portion of a physical object in the user's physical environment that corresponds to a representation which the user 308 cannot see from their perspective), the electronic device 101 refrains from displaying an indication of attention. Additionally or alternatively, the electronic device 101 may provide a visual indication of attention with a modified appearance (e.g.,a different spatial profile, such as a simulated glow surrounding a section of the virtual tree enclosed in foreground 326, an arrow with simulated depth tilting behind the virtual tree, and / or visual properties (e.g., haze, blur, saturation, and / or a simulated light effect), to indicate that a user attention target corresponds to the representation, which is not currently visible to user 308. It is understood that the visual attention cues may be displayed and, for the sake of simplicity, are sometimes omitted from the figures.

[0072] In some examples, the virtual scene has a simulated depth, and the visual cue is displayed at a position according to the user's attention and / or the spatial relationship between the electronic device 101 and the air gesture. As an example, the electronic device 101 can be in Fig. 3. Based on the user's view and / or the beam projected by device 101 through the air pincer grip gesture, which intersects the position on the virtual floor, display a visual indication at a position on a surface of the virtual floor enclosed in the virtual scene.

[0073] In some of the examples described herein, the electronic device 101 can display information about the attention of other users. For example, in Fig. 3. The attention (e.g., gaze) of a user, corresponding to the representation of another user of another electronic device, is directed toward the virtual floor. This user's electronic device can detect that the floor is the target of the user's attention and can communicate information indicating this target to the electronic device 101. In response to receiving this information, the electronic device 101 can display a visual indication of attention. It is understood that in some examples, corresponding electronic devices, in response to detecting information that a corresponding user's attention has changed relative to the three-dimensional environment 302 and / or the virtual scene, can communicate information about the movement of attention indicating an updated attentional target.In response to receiving such updated information, the electronic device 101 can move the visual attention indicator to an updated position and / or orientation relative to content included in the virtual scene, in accordance with the updated information. In some examples, the visual attention indicator(s) is / are displayed superimposed on representations of the user's 308 physical environment (e.g., without including a portion of the virtual scene).

[0074] In some examples, the electronic device 101 selectively displays the visual indicator of attention. For example, when an interaction mode is enabled relative to the virtual scene (such as an editing mode), the electronic device 101 may display the visual indicator of attention. In some examples, the electronic device refrains from displaying the visual indicator of attention when the interaction mode is disabled. Similarly, while the interaction mode is enabled, the electronic device 101 may display other visual indicators of attention from other users, and while the interaction mode is disabled, the electronic device 101 may refrain from displaying the visual indicators of attention from other users. In some examples, the electronic device 101 displays the visual indicator(s) of attention according to user preferences.For example, a user setting determined by the electronic device 101 may permit or prohibit the sharing of visual attention cues by user 308 with other users participating in a communication session with the electronic device 101. In some examples, the visual attention cue may be given in response to the detection of an explicit prompt to display the visual cue (e.g., a predefined air gesture performed by the user's body, a pose of one or more parts of the user's body, a verbal prompt to display the visual cue, and / or selection of a virtual and / or physical control (e.g.,button, slider and / or menu options)) within the three-dimensional environment 302 and / or to share the visual indication with other devices participating in the communication session with the electronic device 101.

[0075] Fig. Figure 4 illustrates the display of an environment template, including a virtual model, according to some examples from the disclosure. In some examples, the electronic device 101a displays a model 404. In other examples, the electronic device 101a displays a visual indication 406, which corresponds to a view of a portal. The portal, such as Portal 410, can provide a preview of at least one section of the virtual environment, similar to how a user could look through a physical window into a physical equivalent of the virtual environment. The scale of the virtual content in Model 404 can differ from the scale of the virtual content when the electronic device 101a displays the virtual environment immersively (e.g., the scale of Building 412 in Model 404 can be larger or smaller than the scale of Building 414 in Portal 410).In some examples, the electronic device 101a represents a view of the physical environment (e.g., similar to or the same as with reference to ). Fig. 3) while simultaneously displaying Model 404 and / or Portal 410. In some examples, the electronic device 101a represents the virtual environment according to one or more of the at least with reference to Fig. The 4 described examples illustrate when an environment format corresponds to a virtual model and / or a display portal format.

[0076] By displaying a virtual model and / or a view portal, a user of an electronic device can be provided with ways to quickly preview and / or verify changes to virtual content contained within a virtual three-dimensional environment. By displaying a virtual model, the user can quickly view sections of the virtual three-dimensional environment as a whole, for example, how a collection of buildings in an old western set might be arranged in a virtual scene. By displaying a view portal, the user can quickly view a collection of detailed virtual objects, textures, and / or the like contained within the virtual three-dimensional environment and / or verify that the virtual content is rendered according to the user's preferences in a scaled implementation.Furthermore, displaying a scaled model of the virtual three-dimensional environment can reduce the computing and power consumption that would otherwise be required to display the virtual three-dimensional environment at a scale comparable to the user.

[0077] In the context of media production workflows, the view portal can allow users of an electronic device to quickly preview the effects of editing virtual content in the virtual three-dimensional environment. When an electronic device receives data streamed from a computer system (as described in the section on...) Fig. (described in more detail in Section 5), a system that includes the electronic device and the computer system can additionally or alternatively enable the editing and / or verification of the virtual three-dimensional environment in real time, without requiring excessive time for rendering, subsequent sharing, and subsequent verification of a generated or edited virtual three-dimensional environment. Furthermore, by initiating the display of a virtual environment according to a virtual model and / or an environment template for a view portal, the electronic device reduces user input for visually configuring and / or highlighting sections of the virtual environment that may be useful for a particular phase of the media production workflow and / or for a corresponding type of media content. By simultaneously displaying the model 404 and the portal 410, a single device (e.g.,electronic device 101) display different views of the virtual content in order to quickly identify the appearance of the same virtual content as displayed in different environment formats.

[0078] In some examples, the environment template display includes the display of both Model 404 and Portal 410, such as the simultaneous display of Model 404 and Portal 410. In some examples, the electronic device 101a displays Model 404 at a first size and / or displays Portal 410 at a second size relative to the three-dimensional environment 402. Model 404 may represent a miniaturized version of the virtual environment, similar to a diorama of the three-dimensional environment. In some examples, Model 404 is displayed with a visual appearance according to a setting specified in the template. For example, a setting may be configured to display Model 404 without textures.Displaying the Model 404 without textures saves the processing required to check aspects of the Model 404, such as the dimension of virtual objects, the spatial arrangement of virtual objects, the quantity of virtual objects and / or the appearance of a simulated skyline.

[0079] In some examples, Portal 410 is a two-dimensional representation of the virtual environment. In some examples, the content displayed in Portal 410 corresponds to a projection of the visual display 406 relative to the model 404. For example, in Fig. As shown in Figure 4, Portal 410 includes Building 414, which corresponds to Building 412 in Model 404, since the projection and scaling of the visual indication 406 in the direction of the content included in Model 404 includes the front of Building 412.

[0080] In some examples, Portal 410 and Model 404 are rendered using different rendering techniques. In some examples, Portal 410 and Model 404 are rendered with different visual properties (e.g., due to different rendering techniques). For example, Model 404 might be displayed with a wireframe that does not include lighting effects, shading layers, textures, and / or the like. Portal 410, on the other hand, might be displayed with some or all of the aforementioned visual effects, making the content displayed in Portal 410 more similar to a finalized rendering and / or appear more realistic.

[0081] In some examples, the environment template defines the position of model 404 and portal 410 relative to each other and / or a viewing angle of the electronic device 101a. For example, model 404 may be displayed in a first position, and portal 410 may be displayed in a second, different position. In some examples, the first and / or second position is determined before the virtual environment is displayed using the Fig. The template shown in section 4 is predetermined. In some examples, the electronic device 101a recognizes one or more inputs requesting movement of the virtual model 404 and / or portal 410, and in response moves the virtual model 404 and / or portal 410 according to the one or more inputs.

[0082] In some examples, the electronic device 101a communicates with one or more other electronic devices, as with reference to Fig. 5 described. In some examples, each electronic device can display a preview of the virtual three-dimensional environment at the same location relative to the physical environment as the location where electronic device 101 displays the preview of the virtual three-dimensional environment. For example, the preview of the virtual three-dimensional environment, which may include both virtual model 404 and portal 410, can be frozen. In some examples, when the preview of the virtual three-dimensional environment is frozen, the position of the preview of the virtual three-dimensional environment does not change in response to the detection of movement by electronic device 101a (or any of the other electronic devices with access to the virtual three-dimensional environment).Thus, the three-dimensional environment cannot move with respect to any spatial reference point other than the viewing angle of the electronic device 101.

[0083] In some examples, the model 404 is bound to the user's body. For example, the model 404 can be visible to the user regardless of whether the electronic device 101a displays the model 404 without showing the view portal with the preview of the virtual three-dimensional environment. In other examples, the model 404 is world-bound. For example, the model 404 is frozen regardless of whether the electronic device displays the model without displaying the portal 410 and / or while simultaneously displaying the portal 410.

[0084] In some examples, Model 404 includes a specification of a virtual camera, such as an image, graphic, symbol, and / or lighting effect, in the three-dimensional environment that illustrates a viewpoint relative to the three-dimensional environment from which a two-dimensional image of the virtual three-dimensional environment can be generated. For example, Model 404 includes a specification of the virtual camera pointed at Building 412. The electronic device 101a and / or a computer system communicating with the electronic device 101a can generate a two-dimensional image of Building 412 that resembles the appearance of a physical camera pointed at a physical building. Additionally or alternatively, virtual background content present in the virtual three-dimensional environment behind Building 412 can be included in the image.In some examples, the virtual camera is not connected to the three-dimensional environment and / or differs from the view of user 408 relative to the three-dimensional environment. For example, the view of the virtual three-dimensional environment displayed by the virtual camera may differ from the perspective of user 408 displayed by the electronic device 101a. In some examples, in response to the detection of an input to update the position and / or orientation of the virtual camera, the electronic device 101a updates the virtual camera in model 404 and changes the displayed image so that it corresponds to the viewpoint displayed by the virtual camera relative to the virtual three-dimensional environment.In some examples, the electronic device 101 is able to store one or more virtual images of the virtual three-dimensional environment and / or the viewing angles of the virtual cameras. In some examples, the electronic device 101a exports part or all of the virtual environment, which may include the virtual model 404, portal 410, the position and / or orientation of the visual indicator 406 relative to the virtual model 404, the position and / or orientation of one or more virtual cameras, and / or a combination thereof. Additionally or alternatively, the electronic device 101a can generate media such as images reflecting the position of the virtual cameras, two- and / or three-dimensional videos corresponding to the virtual cameras, and / or immersive videos corresponding to the position of the virtual cameras within the model 404.

[0085] Fig. Figure 5 illustrates the display of a virtual environment with a level of immersion exceeding a threshold level of immersion, according to some examples from the revelation (e.g., as with reference to Fig. 3 described). In Fig. 5 The electronic device 101a displays immersive virtual content enclosed in a virtual environment 502 (e.g., a virtual three-dimensional environment) within an immersion area 512 while communicating with the electronic device 101b and / or the computer system 516. The electronic device 101a can display the virtual environment 502 according to an immersive environment template which, when implemented on the electronic device 101a, can cause the electronic device 101a to display the virtual environment 502 with a degree of immersion exceeding a threshold degree of immersion, as further described herein.

[0086] Computer system 516 can stream scenic data to electronic device 101a and / or 101b and can simultaneously display the virtual environment 502 on a display, while electronic device 101a displays an immersive view of the virtual environment 502 (e.g., a three-dimensional environment). Additionally or alternatively, electronic device 101b can simultaneously display a virtual object 514, which may include a perspective view corresponding to the viewing angle of electronic device 101a. If any of the electronic devices 101a, 101b, and / or computer system 516 provides input that modifies the virtual content in the virtual environment 502, this modification can be displayed by other electronic devices and / or the computer system.Thus, the electronic device 101a and 101b and computer system 516 can communicate to jointly develop virtual content that is enclosed in the virtual environment 502.

[0087] In some examples, the electronic device 101a and / or 101b has one or more features, such as switching logic and architecture, and / or the ability to perform operations related to the one described in Fig. 1 electronic device 101 shown and / or the in Fig. Electronic device 201 and / or electronic device 101a shown in 2A to 2B Fig. 3 were described. The electronic device 101a in Fig. Device 5 is used by user 518a to participate in the multi-user communication session, and electronic device 101 is used by user 518b to participate in the multi-user communication session. In some examples, the multi-user communication session has one or more properties that define the relationship between the Fig. 3 are similar or the same as described.

[0088] In some examples, the immersion area 512 corresponds to the sections of the physical environment 500 that are replaced by the display of the virtual environment 502. For example, the electronic device 101a can display a virtual environment 502 that replaces the visibility of walls, ceilings, floors, and / or physical objects. In some examples, the electronic device 101a detects an input to join the multi-user communication session and, in response, initiates the display of the virtual environment 502, as shown in Fig. Figure 5 shows that, since the environment format corresponds to an immersive format type, the electronic device 101a foregoes the display of a virtual model and / or a two-dimensional portal and instead displays a three-dimensional version of the virtual environment 502, which occupies at least a portion of the display of the electronic device 101a. In some examples, the immersion area 512 is a percentage of a viewport of the electronic device 101a (e.g., 50%, 60%, 70%, 80%, 90%, or 100%).

[0089] In some examples, the degree to which virtual content replaces the visibility of representations of viewpoints of the physical environment 500 corresponds to a degree of immersion. A degree of immersion can include the percentage of the display 120 that encompasses the virtual environment 502 (and does not include a view of the physical environment 500 via a transparent passthrough and / or a camera reproduction of the physical environment 500). For example, the degree of immersion can include a width and / or height of the display 120 that encompasses contiguous virtual content. The degree of immersion can additionally or alternatively include the position within the three-dimensional environment at which virtual content overlays the physical environment 500, such as a boundary that separates the beginning of the virtual environment 502 from the physical environment 500.In some examples, in response to the detection of a movement of a viewpoint of the electronic device 101a that changes a position and / or orientation relative to the virtual environment 502 and / or physical environment 500, the electronic device 101a updates the portion of the virtual environment 502 that is displayed according to the movement. Thus, the electronic device 101a can maintain the display of the virtual environment 502 (e.g., occupy a central area or the entire display of the electronic device 101a). Conversely, the electronic device 101a can terminate the display of a portal rendering of the virtual environment 502 in response to a movement of the viewpoint away from the portal rendering.

[0090] In some examples, the electronic device 101a and / or the electronic device 101b communicate to cause the electronic device 101b to display a view of the perspective of the electronic device 101a relative to the virtual environment 502. For example, the electronic device 101a can send information to the electronic device 101b, which, in response to receiving the information, can display the virtual object 514 and / or populate the virtual object 514 with the immersion area 512, the displayed sections of the virtual environment 502, and / or the visible sections of the physical environment 500. In some examples, the information can additionally or alternatively be transmitted to computer system 516, and computer system 516 can display the perspective of the electronic device 101a.

[0091] In some examples, Computer System 516 displays a model of the virtual environment. In some examples, Computer System 516 renders virtual environments with higher visual fidelity than other renderings of virtual environments on devices with less processing power (such as a headset or a mobile phone). For example, Computer System 516 may display a full rendering with shaders and textures, while Electronic Device 101a displays a simplified or schematic version of the same set of virtual content. Additionally or alternatively, Electronic Device 101a may display a preview of Virtual Environment 502 with some of the shaders and / or textures displayed on Computer System 516, but with a lower polygon count, texture resolution, shading accuracy, global illumination, and / or a combination thereof.

[0092] In some examples, computer system 516 can host collaborative review sessions in which the electronic devices participating in the communication session exchange information and leave annotations about their observations and comments on the virtual environment 502. In some examples, each device participating in the communication and / or the computer system can detect inputs that insert an annotation into the virtual environment 502 and / or into a recording of a session during which users are reviewing the virtual environment 502 (e.g., text, a virtual pin, a spatial recording of a user moving through the virtual environment 502, speech input, eye-tracking, user attention, and / or pointing with a finger or cursor, and / or a combination thereof).In response to input recognition, the recognizing device can share the annotation with other devices in the multi-user communication session. In response to receiving an annotation from another device, the receiving device can overlay the annotation onto the 502 virtual environment. Both the sending and receiving devices can share a common location within the 502 virtual environment, thus synchronizing annotation placement between devices.

[0093] In some examples, one or more of the electronic devices 101a, 101b, and / or computer system 516 define properties of an environment template. For example, computer system 516 may display a content creation user interface and a menu with different project types available. The project types may correspond to a likely intended output of virtual content generated during project use, such as spatial video (for example, rectilinear three-dimensional video rendered over a virtual frame that includes parallax simulation), immersive video (for example, video rendered immersively, occupying 180 or 360 degrees around the user's viewpoint), a three-dimensional environment project, and / or conventional two-dimensional video.When setting up the project based on a selected project type, Computer System 516 can automatically assume a specific environment format, such as the environment formats described herein. Additionally or alternatively, Computer System 516 can prompt a user of Computer System 516 for additional properties of the format, such as the size of a portal, the position of a portal, the size and / or position of a virtual model, the level of immersion, whether a virtual model and / or a portal should be included, the size of a viewbox, the dimensions of a virtual stage, a spatial profile of a virtual stage, and / or a background for projecting background content, and / or a combination thereof.

[0094] In some examples, Computer System 516 can send data and / or metadata to other devices in the communication session according to user-specified settings and / or default settings. In some examples, receiving devices can accept (or reject) an invitation to receive the data and / or metadata and display a view of an environment according to the environment template and / or settings specified by Computer System 516. In some examples, other devices, such as Electronic Device 101a and / or 101b, can define some or all of the environment format properties mentioned above and display Virtual Environment 502 (or other virtual environments) according to the specified properties and / or settings.By displaying three-dimensional representations in an immersive view simultaneously with the two-dimensional or simulated two-dimensional view of an environment, a single device may be able to display simultaneous representations of the same virtual scene in different ways, thereby reducing the processing overhead of displaying the different views of the same views on different devices and decreasing the number of devices used to preview the display of the different views on different display types (e.g., immersive and conventional two-dimensional displays such as a television).Furthermore, the scene can be displayed in a way that simulates the experience of being placed in the scene, while simultaneously providing visibility of the user's physical environment, thereby enhancing the user's awareness of their physical surroundings and simultaneously demonstrating how another electronic device could display an immersive view of the scene, like a preview of what the scene might look like when displayed by an XR-enabled electronic device.

[0095] Fig. Figure 6 illustrates the display of a review of animations according to some examples of the disclosure. In some examples, the electronic device 101 displays a portal, using similar or the same techniques as those relating to Fig. 4 described. In Fig. 6 initiates computer system 616 to display an animation of a virtual stock moving relative to a virtual scene in response to the detection of an input that initiates playback of the animation from user 618d. In some examples, portal 610 can display a view of a virtual three-dimensional environment. As in Fig. As shown in Figure 6, the environment format can correspond to a template in which the Portal 610 is displayed without simultaneously displaying a virtual model of a virtual three-dimensional environment.

[0096] Users 618a through 618c, each participating in a multi-user communication session with computer system 616 via electronic devices 101a through 101c, can simultaneously display Portal 610 via the electronic devices 101a through 101c. For example, computer system 616 can stream virtual scene and / or animation data to the electronic devices 101a through 101c to display the animation. The electronic devices 101a through 101c can display Portal 610 with appropriate orientations based on the respective viewing angles of the electronic devices 101a through 101c relative to a location within the physical environment specified by computer system 616.

[0097] In some examples, animated virtual content may extend beyond the dimensions of Portal 610, for example, into areas where the physical environment is depicted and / or displayed. For example, the virtual object 620, which corresponds to a virtual representation of a dinosaur, is located in Fig. 6 is shown extruded from the surface of portal 610. As with reference to Fig. As described in section 4, a portal, such as Portal 610, can be a two-dimensional virtual object. As in Fig. As shown in Figure 6, the virtual object 620 is displayed superimposed over sections of the three-dimensional environment of the electronic devices 101a to 101c that correspond to the physical environment (e.g., beyond and outside the surface of portal 610). In some examples, computer system 616 can share information during the ongoing communication session that reintroduces the animation into the virtual three-dimensional environment. In response to receiving information from computer system 616 during the communication session, the electronic devices 101a to 101c can initiate the animation as described above (e.g., move beyond portal 610). Thus, computer system 616 can cause the display of animated virtual content that extends beyond portal 610, which is initially created while the collaborative communication session between computer system 616 and the electronic devices 101a to 101c is underway.

[0098] In some examples, the electronic devices 101a to 101c and / or computer system 616 can detect attention targets while displaying portal 610 and share these attention targets with other participants in the multi-user communication session. By displaying attention targets, devices can efficiently point to and / or understand a discussion target. In some examples, the display of attention targets is used to show a anticipated animation of an object moving within the virtual three-dimensional environment.

[0099] The electronic device 101a can, for example, detect that a user is looking at positions within the portal 610, detect that a user is pointing at positions using their fingers and / or a pointing device such as a pen, and / or detect the movement of a joystick controlling a cursor. The electronic device 101a can display a visual indication of the attention target, such as a lit cursor at the target position, and can follow a path through the virtual environment according to the detected movement of attention through portal 610. The electronic device 101a can share information indicating attention with other electronic devices 101b and 101c and / or computer system 616.In response to receiving the information, the devices can each display visual indications of the attention target and / or the movement through the virtual environment, thus reflecting the movement of attention.

[0100] In some examples, an electronic device can detect inputs requesting the placement of an annotation in the virtual three-dimensional environment. In response to the input detection, the electronic device can prompt a user to provide information corresponding to an annotation and / or a description of a virtual object. For example, electronic device 101a can display a user interface that prompts the user to provide speech, air gesture(s), text input (e.g., via a virtual or physical keyboard), movement, attention, and / or other suitable information modalities. Such a user interface can include one or more virtual buttons to initiate and / or terminate text input, voice recording, movement recording, and / or attention recording.In some examples, the user-provided information includes a description associated with the virtual object, a name for the virtual object, metadata associated with the virtual object, such as a category for the virtual object, and / or other suitable information that might be of interest to a future reviewer of the virtual object. Once the user-provided text input and / or recording is complete, the electronic device may stop displaying the user interface and / or associate the provided information with a corresponding virtual object. In some examples, the electronic device begins recording and / or initiates text input without displaying a dedicated user interface in response to the insertion of a virtual object and / or in response to the detection of input directed at an animated virtual object.By displaying a virtual scene using the Portal 610, multiple devices can synchronize the view of the same set of virtual content and / or without the need for a dedicated external display occupying the physical environment, thereby increasing the likelihood that users can jointly review, edit and / or export media during and / or after collaborating with a workstation or other computer system such as Computer System 616.

[0101] Fig. Figure 7 illustrates an example of electronic devices that represent a virtual three-dimensional environment template, which includes a virtual model and an environment preview according to some examples of the disclosure. As with reference to Fig. 4 and Fig. As described in Section 6, electronic devices, such as electronic devices 101a to 101c, can be in a communication session with each other and / or with a computer system that streams data to the electronic devices 101a to 101c. In some examples, the data can be used to represent a virtual three-dimensional environment according to an environment template. For example, the electronic devices 101a to 101c can be in Fig. 7. Receive a notification that the computer system is requesting the display of the virtual three-dimensional environment according to an environment format that includes a preview of the virtual three-dimensional environment such as Portal 710 and a virtual model 704. It is understood that some or all of the techniques and / or procedures relating to the in Fig. 4 and Fig. The six templates shown were described, which were referenced to Fig. The 7 described examples are applicable.

[0102] In some examples, the environment format and / or the computer system data additionally or alternatively specify the spatial arrangement of Portal 710 and / or Virtual Model 704, and / or which section of the virtual environment is represented within Portal 710 and / or Virtual Model 704. For example, Portal 710 may represent a two-dimensional view of a section of the virtual three-dimensional environment that is represented three-dimensionally in Virtual Model 704. Additionally or alternatively, Portal 710 and / or Virtual Model 704 may include different sections of the virtual three-dimensional environment. For example, Virtual Model 704 includes in Fig. 7. An elongated virtual object 720 is entered, which is not displayed and / or represented in the portal 710. As in Fig. As shown in Figure 7, the virtual model 704 does not include a visual display corresponding to the portal 710, because the portal 710 corresponds to a section of the virtual three-dimensional environment that is different from the section displayed in the virtual model 704.

[0103] As in Fig. As shown in Figure 7, Portal 710 and the virtual model 704 are displayed at positions within a representation of a physical environment that is the same for each of the electronic devices 101a to 101c. Accordingly, Portal 710 and the virtual model 704 can be frozen virtual content. In some examples, the users 718a to 718c can move the Portal and the virtual model 704 relative to a three-dimensional environment of the users 718a to 718c. For example, the electronic devices 101a to 101c can detect inputs directed at selectable options, such as buttons and / or icons, and in response, move Portal 710 and / or the virtual model 704 separately or simultaneously according to the input.In some examples, the other electronic devices participating in the communication session move Portal 710 and / or the virtual model 704 to the updated position specified by the input detected by the first electronic device when a first electronic device moves Portal 710 and / or the virtual model 704. By simultaneously displaying the virtual model 704 and Portal 710, the electronic devices 101a to 101c can synchronize the spatial data that may dictate where virtual assets are displayed in Portal 710 and visualize the spatial data as if they occupied three dimensions of the three-dimensional environment, thus presenting the virtual scene in a manner that may be more intuitive and / or is not possible with conventional two-dimensional displays.Furthermore, the editing and / or previewing of spatial media via the communication session between the electronic devices 101a to 101c, as with reference to . Fig. 7 described, enabling the simultaneous verification of the spatial relationship of holdings in the spatial media using the virtual model 704, while simultaneously displaying a preview of how the view of a simulated camera could capture the holdings, as shown in Portal 710.

[0104] In some examples, the computer system streaming environment data to the electronic devices 101a to 101c, and / or the electronic devices 101a to 101c, can switch between environment templates. For example, the electronic devices 101a to 101c can detect an input that changes the environment template used to display the virtual three-dimensional environment. The input could be, for example, a voice command, a selection of one or more buttons, icons, prompts, text, and / or representations of an environment template while an initial environment template is displayed (e.g., the one shown in Fig. (7 template shown). In response to receiving the input, the electronic devices 101a to 101c can display the virtual three-dimensional environment according to the input. For example, the electronic device 101a can initially display the virtual three-dimensional environment while showing only a virtual model (e.g., model 404 in Figure 7). Fig. 4), and can detect the selection of a button from a menu requesting the display of a portal and a virtual mode. In response to the detection of the input by selecting the button, the electronic device 101a can stop displaying the model 404 and / or transfer the model 404 to the device specified in Fig. 7. Move the displayed position (e.g., according to the virtual model 704) and / or can simultaneously display the portal 710.

[0105] Additionally or alternatively, the electronic device 101a can recognize inputs that request the display of a portal-only environment template and can retrieve the environment template from the one in Fig. 7 shown in the Fig. 6 shown. In some examples, when switching between templates, the electronic device 101a retains the display of virtual content common to the selected template types. For example, if the newly selected environment template type has a portal (as in Fig. (as shown in Figure 6), the electronic device 101a can maintain the display of the portal corresponding to the same section of the virtual three-dimensional environment when the display of an environment template is initiated that includes a virtual model and a portal (as shown in Figure 6). Fig. 7 shown).

[0106] In some examples, the electronic devices 101a to 101c can modify the environment template used to display the virtual three-dimensional environment in response to the detection of inputs directed at the virtual three-dimensional environment. For example, the electronic devices 101a to 101c can detect movements of users 718a to 718c (e.g., a single electronic device can detect the movement of a user wearing and / or using the device), such as movements within a threshold distance of the virtual model 704 and / or portal 710. The threshold distance can be 0.05, 0.1, 0.25, 0.5, 0.75, 1, 1.5, 3, 4, or 5 m and can be used to trigger the display of an immersive and / or semi-immersive display of the virtual three-dimensional environment.For example, the electronic device 101a can detect that user 718a is moving within the threshold distance and / or through portal 710. In response to the detection of movement, the electronic device 101a can stop displaying the virtual model 704 and / or portal 710 and / or display the virtual three-dimensional environment according to an immersive environment template. For example, the electronic device 101a can animate and / or initiate the display of the virtual environment with a specific scale, as shown in [reference]. Fig. Figure 5 shows that, in addition or alternatively, sections of the physical environment displayed on the electronic device 101a can be replaced by sections of the virtual environment. For example, the electronic device 101a can display the virtual three-dimensional environment as if the user were standing in front of a physical equivalent of the virtual three-dimensional environment.

[0107] In some examples, the immersion level is 5%, 10%, 25%, 50%, 75%, 90%, or 100%. For example, in response to the detection of user 718a's movement by portal 710, electronic device 101a can display the virtual three-dimensional environment, which occupies the entire viewport of electronic device 101a. In some examples, the portion of the three-dimensional environment occupied by the virtual three-dimensional environment is mapped onto the physical environment. For example, the area in which the virtual three-dimensional environment is displayed immersively can be defined and mapped onto portions of the physical three-dimensional environment.Accordingly, in response to the detection of a movement of the viewpoint of the electronic device 101a away from the area in which the virtual three-dimensional environment is displayed, the electronic device 101a can display sections of the physical three-dimensional environment, similar to how the virtual three-dimensional environment at least temporarily occupies a section of the physical three-dimensional environment, but not the entire physical three-dimensional environment. In the example where the three-dimensional environment is displayed with a full degree of immersion (e.g., 100%), the electronic device 101a can dispense with displaying the physical environment, since the entire physical environment is virtually occupied by the virtual three-dimensional environment.

[0108] It is understood that the electronic devices 101a to 101c, according to various inputs between those relating to Fig. You can switch between the environment templates described in sections 3 to 8. The description of various operations below, for example, optionally applies to one, some, or all of the environment templates related to... Fig. Examples 3 to 8 are described. For example, during the immersive display of a virtual three-dimensional environment, the electronic device 101a can detect an input requesting the integration of a virtual stage, as described in reference to Fig. 3 described. In response to the recognition of the input, the electronic device 101a can display the virtual stage 306 and / or change the level of immersion to display physical sections of the three-dimensional environment 334 and / or 336.

[0109] Additionally or alternatively, the computer system can detect one or more inputs that load a new workflow and / or project for sharing with the electronic devices 101a to 101c. The new project can be associated with a specific environment format and / or a type of data to be exported, such as a spatial video, an immersive video, an environmental, and / or a two-dimensional content creation project. The computer system can send a notification to the electronic devices 101a to 101c to load the new project and / or display a new virtual three-dimensional environment. In some examples, the computer system also sends an indication that a specific environment format corresponds to the new project and / or the new virtual three-dimensional environment.In response to receiving the information from the computer system, the electronic devices 101a to 101c can stop displaying a currently displayed virtual three-dimensional environment and initiate the display of a new virtual three-dimensional environment using an environment template that matches the requested environment format and / or environment template.

[0110] In some examples, displaying the virtual scene using an environment format includes displaying a viewbox that corresponds to an area within the virtual scene. By rendering the same virtual scene according to different environment templates, the rendering device can ensure that virtual content is displayed in a way that allows for the review, editing, and / or export of assets, such as displaying an appropriate scale and / or view of the virtual scene.

[0111] Fig. Figure 8 illustrates an example of an electronic device that presents a virtual three-dimensional environment template, including a viewbox, according to some examples of the disclosure. A viewbox 824, as mentioned herein, can be a virtual volume that can restrict which section or sections of a virtual three-dimensional environment are presented to the user in order to direct the user's attention to aspects of the virtual three-dimensional environment. In the context of a media production workflow, the viewbox can limit the amount of visual stimuli presented to the user, thereby increasing the likelihood that a user can focus on a subset of the virtual three-dimensional environment, such as a group of virtual objects and / or on less than the entire virtual three-dimensional environment.Therefore, a Viewbox environment template can be useful when a user wants to export virtual objects, virtual textures, virtual lighting effects and / or sections of a virtual three-dimensional environment, especially when going through rough ideas and / or refining a design for such virtual content.

[0112] In some examples, the viewbox 824 includes a variety of markers 826 that indicate a scale of the virtual three-dimensional environment. Markers 826 can help the user 808 inspecting virtual content, such as the virtual objects 822. Virtual objects 822 include virtual boxes and a virtual barrel, which can be inserted into the virtual three-dimensional environment at a location bounded by the viewbox 824.

[0113] In some examples, viewbox 824 occupies a virtual space that corresponds to a physical area (e.g., in a manner similar to the virtual stage described above). In some examples, viewbox 824 corresponds to a volume within a virtual three-dimensional environment, and virtual assets bound within the volume are at least partially displayed within viewbox 824. In some examples, displaying a virtual environment using viewbox 824 involves omitting the display of part or all of the virtual three-dimensional environment. For example, one or more walls, the ceiling, and / or the floor of viewbox 824 may not contain any virtual content from the virtual three-dimensional environment.In some examples, some or all of the walls, ceiling, and / or floor of viewbox 824 enclose a two-dimensional image of the virtual three-dimensional environment. In some examples, some or all of the walls, ceiling, and / or floor of viewbox 824 are rendered using similar or the same techniques as described above in relation to the virtual stage.

[0114] In some examples, the electronic device 101a can detect inputs that change the scale of the view box 824 and can scale virtual content, such as virtual objects 822 and / or markers 826, according to the input. In this way, the electronic device 101a can make virtual content from the virtual three-dimensional environment larger or smaller according to the user's preference. In some examples, the view box 824 can have a different scale, dimension, shape, and / or a combination thereof. For example, the length, width, and / or height of the view box 824 can be defined and / or changed by the computer system that streams data to the electronic device 101a to display the view box 824.Additionally or alternatively, an environment template may include a spherical, rounded, other rectilinear and / or hybrid shaped viewbox that incorporates different lines, curves and / or surfaces with different spatial profiles and / or dimensions.

[0115] In one or more of the examples described herein, the electronic device 101a can change or maintain a perspective relative to the virtual three-dimensional environment according to a currently implemented environment template. For example, in response to the detection of movement of a view angle of the electronic device 101a relative to a virtual environment displayed in an immersive environment template, the electronic device 101a can update the displayed virtual content according to the movement of the view angle, similar to how the user can physically see their physical environment change according to movement relative to a physical environment.In response to the detection of a change in viewing angle relative to at least a partially immersive virtual three-dimensional environment, the electronic device 101a can terminate the display of virtual content and / or initiate the display of virtual content enclosed in the virtual three-dimensional environment.

[0116] In some examples, the electronic device 101a can maintain a perspective display relative to the virtual three-dimensional environment in response to the detection of a change in the viewing angle of the electronic device 101a. For example, when displaying a view portal, the electronic device 101a can refrain from changing which sections of the three-dimensional environment are displayed in the view portal in response to the detection of changes in the viewing angle of the electronic device 101a. In this way, a user of the electronic device 101a can always be shown a perspective into the virtual three-dimensional environment that is determined according to the positioning and / or orientation of the portal relative to the virtual three-dimensional environment.

[0117] Fig. Figure 9 is a flowchart of a procedure for representing a virtual three-dimensional environment according to a template based on some examples from the revelation.

[0118] In some examples, instructions for carrying out the procedure 900 are stored using a (e.g., non-transitory) computer-readable storage medium, and carrying out the instructions causes an electronic device (e.g., electronic device 101 or electronic device 201) to carry out the procedure 900.

[0119] In 902, Method 900, in some examples, includes the electronic device detecting an input while a user of the electronic device has a three-dimensional environment visible. This input includes a request to display a user interface for content creation, which includes a view of a section of a virtual environment, such as a user interface that includes some or all of the virtual content displayed in the three-dimensional environment 302. In 904, Method 900, in some examples, includes displaying the view of the virtual environment in the three-dimensional environment in response to the detection of the input. In 906, it includes displaying one or more displays, such as the display of the virtual model 404 and / or portal 410, as described in Fig. Figure 4 shows the display of the immersion range 512, as shown in Fig. 5 shown, and / or the display of view box 824, as in Fig. 8 shown. In 908, in some examples, when a computer system communicating with the electronic device indicates that the view is displayed according to a first type of environment template, the virtual environment is displayed with a first spatial profile, such as a spatial profile of the virtual stage 306 and / or background 304, as in Fig. 3 shown. In some examples at 910, when the computer system indicates that the view is displayed according to a second type of environment template that differs from the first type, the virtual environment view is displayed with a second spatial profile that differs from the first spatial profile, where the first spatial profile and the second spatial profile differ from an orientation of the virtual environment view, such as the display of Portal 710 and / or Virtual Model 704, as shown in Fig. 7 shown.

[0120] Additionally or alternatively, in some examples, the first spatial profile includes a first form of the view relative to the three-dimensional environment, and the second spatial profile includes a second form of the view relative to the three-dimensional environment, different from the first. Additionally or alternatively, in some examples, the first form corresponds to a two-dimensional shape, and the second form to a three-dimensional shape. Additionally or alternatively, in some examples, when displayed with the first spatial profile, the view occupies the first portion of a viewport of one or more displays, and when displayed with the second spatial profile, the view occupies a second portion of the viewport, different from the first. Additionally or alternatively, in some examples, the first portion of the viewport includes less than one viewport's field of view.Additionally or alternatively, in some examples, the first section of the viewport occupies an entire field of view of the viewport. Additionally or alternatively, in some examples, the user has a viewing angle relative to the three-dimensional environment, which is a first viewing angle when the input is detected and when the view of the virtual environment is displayed, wherein Method 900 further comprises: while the viewing angle is the first viewing angle, detecting, via one or more input devices, a request to change the viewing angle to a second viewing angle that differs from the first viewing angle; and in response to the detection of the request, updating the view of the virtual environment according to the change in viewing angle.Additionally or alternatively, in some examples, when the user's viewing angle is the first viewing angle, a portion of the electronic device's viewport is occupied by the virtual environment, and the portion of the electronic device's viewport remains occupied by the virtual environment when the user's viewing angle is the second viewing angle. Additionally or alternatively, in some examples, a first portion of the electronic device's viewport is occupied by the virtual environment when the user's viewing angle is the first viewing angle, and a second portion of the viewport, distinct from the first, is occupied by the virtual environment when the user's viewing angle is the second viewing angle. Additionally or alternatively, in some examples, the first spatial profile corresponds to a frozen viewing volume.Additionally or alternatively, in some examples, the input includes the selection of a selectable option, with Method 900 further comprising: during the display of the three-dimensional environment, receiving a notification that three-dimensional graphic data representing the virtual environment is available to the electronic device from the computer system; and in response to receiving the notification, displaying one or more selectable options, each selectable to choose the view of the virtual environment. Additionally or alternatively, in some examples, the three-dimensional graphic data includes one or more of a set of images or data corresponding to a virtual object.Additionally or alternatively, in some examples, Method 900 further comprises: receiving a message from the computer system regarding changes to the three-dimensional graphics data during the rendering of the three-dimensional environment and the display of the virtual environment view; and, in response to the detection of the message, updating the view of the virtual environment in accordance with the changes to the three-dimensional graphics data. Additionally or alternatively, in some examples, Method 900 further comprises: in response to the detection of the input and when the view corresponds to the first type of environment template, displaying, over the display, a virtual object corresponding to a representation of at least the section of the virtual environment, wherein the virtual object is distinct from the view of the virtual environment and is displayed concurrently with the view of the virtual environment.Additionally or alternatively, in some examples, Method 900 further includes: while displaying the view of the virtual environment, receiving one or more inputs requesting the export of content corresponding to the virtual environment; in response to receiving the one or more inputs: according to a determination that the view corresponds to the first type of environment template, exporting one or more first types of virtual content; and according to a determination that the view corresponds to the second type of environment template, exporting one or more second types of virtual content. Additionally or alternatively, in some examples, the one or more first types of virtual content include one or more virtual objects, images, videos, scene data, animation data, and virtual cameras.Additionally or alternatively, in some examples, Method 900 further comprises: in response to the detection of the input, joining a multi-user communication session between the computer system providing three-dimensional graphic data used to display the view of the virtual environment and a corresponding computer system that is different from the computer system, wherein: one or more features of the electronic device match one or more features of the corresponding computer system, and the corresponding computer system displays a corresponding view of the section of the virtual environment based on a viewpoint of a user of the corresponding computer system relative to the three-dimensional environment, while the communication session is in real time.Additionally or alternatively, in some examples, Method 900 further comprises: while the three-dimensional environment is visible and while the view of the virtual environment with the first spatial profile is displayed, receiving appropriate input, including a request to change the view of the section of the virtual environment; and in response to receiving the respective input, changing the view of the virtual environment from the display with the first spatial profile to the display with the second spatial profile. Additionally or alternatively, in some examples, when displaying the virtual environment with the first spatial profile, which includes a first shape relative to the three-dimensional environment, displaying a first virtual content associated with the virtual environment that extends beyond the dimensions of the first shape.Additionally or alternatively, in some examples, the first type of environment template corresponds to one or more of a view portal template, a virtual stage template, a view box template, a virtual model template, and an immersive template.

[0121] Some examples of the disclosure relate to an electronic device that includes a memory and one or more processors coupled to the memory and configured to perform Method 900. Some examples of the disclosure relate to a non-transitory, computer-readable storage medium that stores instructions which, when executed by an electronic device that includes a memory and one or more processors coupled to the memory, cause the electronic device to perform Method 900.

[0122] Fig.Figure 10 is a flowchart of a method for streaming information from a computer system to an electronic device to cause the electronic device to display a virtual three-dimensional environment according to a template according to some examples of the disclosure.

[0123] In some examples, instructions for carrying out the procedure 1000 are stored using a (e.g., non-transitory) computer-readable storage medium, and carrying out the instructions causes an electronic device (e.g., electronic device 101 or electronic device 201) to carry out the procedure 1000.

[0124] In 1002, Method 1000, in some examples, involves a computer system communicating with an electronic device, one or more input devices, and one or more displays: recognizing a request from the electronic device to display a virtual environment. In some examples, in 1004, the computer system, in response to recognizing the input, transmits information to the electronic device corresponding to a view of the virtual environment in 1006. In some examples, in 1008, when a virtual environment editing application on the computer system specifies the display of the virtual environment with a first type of environment template, the computer system causes the electronic device to display the virtual environment with a first spatial profile corresponding to the first type of environment template.In some examples, the computer system causes the electronic device at 1010 to display the virtual environment with a second spatial profile corresponding to the second type of environment template when the virtual environment editing application on the computer system specifies the display of the virtual environment with a second type of environment template. Some examples of the disclosure are directed to a computer system that includes a memory and one or more processors coupled to the memory and configured to perform method 1000. Some examples of the disclosure are directed to a non-transitory, computer-readable storage medium that stores instructions which, when executed by a computer system that includes a memory and one or more processors coupled to the memory, cause the computer system to perform method 1000.

[0125] Some examples of the disclosure relate to an electronic device comprising one or more processors, memory, and means for carrying out one of the foregoing methods.

[0126] Some examples of the disclosure relate to an information processing device for use in an electronic device, the information processing device comprising means for carrying out one of the methods disclosed above.

[0127] Additionally or alternatively, in some examples, the first spatial profile includes a first form of view relative to a three-dimensional environment, and the second spatial profile includes a second form of view, different from the first form, relative to the three-dimensional environment. Additionally or alternatively, in some examples, the first spatial profile corresponds to a frozen viewing volume. Additionally or alternatively, in some examples, the method may further include, during the display of a three-dimensional environment, transmitting a statement that three-dimensional graphic data representing the virtual environment is available from the computer system to the electronic device, and causing the electronic device to display one or more selectable options, each of which can be selected to choose the view of the virtual environment.Additionally or alternatively, in some examples, while the electronic device displays the view of the virtual environment, according to a provision that the view corresponds to the first type of environment template, the method may further include receiving one or more first types of virtual content exported by the electronic device, and according to a provision that the view corresponds to the second type of environment template, receiving one or more second types of virtual content exported by the electronic device.Additionally or alternatively, in some examples, the method may further comprise joining a multi-user communication session between the computer system providing three-dimensional graphics data used to display the view of the virtual environment and a corresponding computer system that is different from the computer system, wherein: one or more properties of the electronic device match one or more properties of the corresponding computer system, and the corresponding computer system displays a corresponding view of the section of the virtual environment based on a viewpoint of a user of the corresponding computer system relative to the three-dimensional environment while the multi-user communication session is running.In some examples, the procedure may further involve, while a three-dimensional environment is visible and while the view of the virtual environment with the first spatial profile is displayed, receiving appropriate input that includes a request to change the view of the section of the virtual environment, and in response to receiving the appropriate input, changing the view of the virtual environment from the display with the first spatial profile to the display with the second spatial profile. Additionally or alternatively, in some examples, the first type of environment template corresponds to one or more of a view portal template, a virtual stage template, a view box template, a virtual model template, and an immersive template.

[0128] Some examples of the disclosure relate to a computer system comprising one or more processors, memory, and one or more programs, wherein the one or more programs are stored in memory and configured for execution by the one or more processors, wherein the one or more programs include instructions for recognizing a request to display a virtual environment from an electronic device communicating with the computer system, and, in response to the recognition of the request, streaming information to the electronic device according to a view of the virtual environment, wherein, when an editing application of the virtual environment on the computer system determines the display of the virtual environment with a first type of environment template, the electronic device is caused toThe virtual environment is displayed with a first spatial profile corresponding to the first type of environment template when the editing application of the virtual environment on the computer system determines the display of the virtual environment with a second type of environment template and the electronic device is caused to display the virtual environment with a second spatial profile corresponding to the second type of environment template. It is understood that the one or more programs containing instructions for one or more operations may include examples in which the one or more programs contain instructions which, when executed by one or more processors of the computer system, can cause the computer system to perform the corresponding one or more operations.

[0129] Additionally or alternatively, in some examples, the first spatial profile includes a first form of view relative to the three-dimensional environment, and the second spatial profile includes a second form of view relative to the three-dimensional environment, different from the first. Additionally or alternatively, in some examples, one or more instructions serve, during the rendering of the three-dimensional environment, to convey an indication that three-dimensional graphic data representing the virtual environment is available from the computer system to the electronic device, and to cause the electronic device to display one or more selectable options, each of which can be selected to choose the view of the virtual environment.Additionally or alternatively, in some examples, one or more instructions further serve, while the electronic device displays the view of the virtual environment, according to a provision that the view corresponds to the first type of environment template, to receive one or more first types of virtual content exported by the electronic device, and according to a provision that the view corresponds to the second type of environment template, to receive one or more second types of virtual content exported by the electronic device.Additionally or alternatively, the one or more instructions in some examples further serve to join a multi-user communication session between the computer system providing three-dimensional graphics data used to display the view of the virtual environment and a corresponding computer system that is different from the computer system, wherein: one or more properties of the electronic device match one or more properties of the corresponding computer system, and the corresponding computer system displays a corresponding view of the section of the virtual environment based on a viewpoint of a user of the corresponding computer system relative to the three-dimensional environment while the multi-user communication session is running.Additionally or alternatively, in some examples, one or more instructions further serve to receive, while the three-dimensional environment is visible and the view of the virtual environment with the first spatial profile is displayed, a corresponding input that includes a request to change the view of the section of the virtual environment, and in response to receiving the corresponding input, to change the display of the virtual environment from the display with the first spatial profile to the display with the second spatial profile. Additionally or alternatively, in some examples, the first type of environment template corresponds to one or more of a view portal template, a virtual stage template, a view box template, a virtual model template, and an immersive template.

[0130] Some examples of the disclosure relate to a non-transitory, computer-readable storage medium on which instructions are stored which, when executed by one or more processors of a computer system communicating with a display generation component and one or more input devices, cause the computer system to recognize a request to display a virtual environment from an electronic device communicating with the computer system and, in response to recognizing the request, to stream information to the electronic device corresponding to a view of the virtual environment, wherein, if an editing application of the virtual environment on the computer system determines the display of the virtual environment with a first type of environment template, the electronic device is caused to display the virtual environment with a first spatial profile.that corresponds to the first type of environment template, and if the editing application of the virtual environment on the computer system determines the display of the virtual environment with a second type of environment template, the electronic device is caused to display the virtual environment with a second spatial profile that corresponds to the second type of environment template.

[0131] Additionally or alternatively, in some examples, the first spatial profile includes a first form of view relative to a three-dimensional environment, and the second spatial profile includes a second form of view, different from the first, relative to the three-dimensional environment. Additionally or alternatively, in some examples, the instructions, when executed, further cause the computer system, during the rendering of a three-dimensional environment, to transmit an indication that three-dimensional graphic data representing the virtual environment is available from the computer system to the electronic device, and cause the electronic device to display one or more selectable options, each of which can be selected to choose the view of the virtual environment.Additionally or alternatively, in some examples, the instructions, when executed, further cause the computer system, while the electronic device displays the view of the virtual environment, to receive one or more first types of virtual content exported by the electronic device, according to a provision that the view corresponds to the first type of environment template, and to receive one or more second types of virtual content exported by the electronic device, according to a provision that the view corresponds to the second type of environment template.Additionally or alternatively, in some examples, the instructions, when executed, further cause the computer system to join a multi-user communication session between the computer system providing three-dimensional graphics data used to display the view of the virtual environment and a corresponding computer system that is different from the first computer system, wherein: one or more properties of the electronic device match one or more properties of the corresponding computer system, and the corresponding computer system displays a corresponding view of the section of the virtual environment based on a viewpoint of a user of the corresponding computer system relative to the three-dimensional environment, while the multi-user communication session is running.

[0132] This disclosure considers that in some examples, the data used may include personal data that can uniquely identify a specific individual or be used to contact or locate that individual. This personal data may include demographic data, content-related activity, location data, telephone numbers, email addresses, Twitter IDs, home addresses, data or records relating to a user's health or fitness (e.g., vital signs measurements, medication information, exercise information), date of birth, or other identifying or personal information. In particular, one aspect of this disclosure, as described herein, is the tracking of a user's biometric data.

[0133] This disclosure recognizes that the use of such personal data in the present technology can be to the benefit of users. For example, personal data can be used to display text suggestions that change depending on changes in a user's biometric data. For example, the suggested text is updated based on changes in the user's age, height, weight, and / or health history.

[0134] This disclosure assumes that entities responsible for the collection, analysis, disclosure, transfer, storage, or other use of such personal data will adhere to generally established privacy policies and / or practices. Specifically, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or regulatory requirements for maintaining and protecting the confidentiality of personal data. Such policies should be readily accessible to users and should be updated whenever the collection and / or use of data changes. Personal information of users should be collected for legitimate and reasonable uses by the entity and should not be shared or sold outside of those legitimate uses.Furthermore, such collection / disclosure should only occur after obtaining informed consent from users. In addition, such organizations should consider taking all necessary steps to protect and secure access to such personal data and ensure that others who have access to the personal data comply with their privacy policies and procedures. Furthermore, such organizations may submit to a third-party evaluation to confirm their adherence to generally accepted privacy policies and practices. Moreover, the policies and practices should be tailored to the specific types of personal data being collected and / or accessed and adapted to applicable laws and standards, including court-specific considerations.For example, in the US, the collection of or access to certain health data may be regulated by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); whereas in other countries, health data may be subject to different regulations and guidelines and should be handled accordingly. Therefore, different data protection practices should be followed for the various types of personal data in each country.

[0135] Despite the foregoing, this disclosure also considers examples where users selectively block the use of or access to personal data. That is to say, this disclosure assumes that hardware and / or software elements can be provided to prevent or block access to such personal data. For example, the technology in question can be configured to allow users to "give" or "withhold" their consent to participate in the collection of personal data during registration for services. In another example, users can choose not to allow the recording of personal data in a specific application (e.g., the first application and / or the second application).In addition to providing options to consent (“opt in”) or decline (“opt out”), this disclosure considers providing notifications regarding access to or use of personal data. For example, a user may be notified when initiating a collection that their personal data is being accessed, and then reminded again shortly before the personal data is accessed by one or more devices.

[0136] Furthermore, the intention of this disclosure is that personal data be managed and handled in a manner that minimizes the risks of unintentional or unauthorized access or use. This risk can be minimized by limiting data collection and deleting data once it is no longer needed. Additionally, and where appropriate, including in certain health-related applications, data anonymization may be used to protect a user's privacy. De-identification may be aided, where appropriate, by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the place level rather than the address level), and the manner in which data is stored (e.g.,Aggregation of data across users) is controlled, and / or by other methods.

[0137] The foregoing description was provided for explanatory purposes with reference to specific examples. However, the foregoing illustrative descriptions are neither intended to be exhaustive nor to limit revelation to the exact forms revealed. In light of the foregoing teachings, many modifications and variations are possible. The examples were chosen and described to best explain the principles of revelation and their practical application, thus enabling other experts to make the best possible use of revelation and the various described examples with different modifications, as appropriate for their intended particular use. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 63 / 700,402

[0001] US 19 / 311,465

[0001] US 19 / 300,648

[0001]

Claims

[1] Procedure, encompassing: on a computer system that communicates with an electronic device, one or more input devices and one or more displays: Recognizing a request to display a virtual environment from the electronic device; and in response to the recognition of the request: Streaming information to the electronic device corresponding to a view of the virtual environment, wherein: If an editing application of the virtual environment on the computer system specifies the display of the virtual environment with a first type of environment template, cause the electronic device to display a section of a view of the virtual environment with a first spatial profile that corresponds to the first type of environment template; and If the virtual environment editing application on the computer system specifies that the virtual environment should be displayed with a second type of environment template, cause the electronic device to display the section of the virtual environment view with a second spatial profile corresponding to the second type of environment template. [2] Method according to claim 1, wherein the first spatial profile includes a first form of view relative to a three-dimensional environment and the second spatial profile includes a second form of view relative to the three-dimensional environment that differs from the first form. [3] Method according to claim 1 or 2, wherein the first spatial profile corresponds to a world-locked viewing volume. [4] Method according to any one of claims 1 to 3, further comprising: during the presentation of a three-dimensional environment, transmitting a statement from the computer system indicating that three-dimensional graphic data representing the virtual environment is available to the electronic device; and To cause the electronic device to display one or more selectable options, each of which can be selected to choose the view of the virtual environment. [5] Method according to any one of claims 1 to 4, further comprising: Participating in a multi-user communication session between the computer system that provides three-dimensional graphics data used to display the view of the virtual environment and a corresponding computer system that is different from the computer system, where: one or more properties of the electronic device match one or more properties of the respective computer system and The respective computer system displays a view of the section of the virtual environment based on a user's perspective of the respective computer system relative to a three-dimensional environment while the multi-user communication session is running. [6] Method according to any one of claims 1 to 5, further comprising: While a three-dimensional environment is visible and while the view of the virtual environment is displayed with the first spatial profile, receiving a corresponding input that includes a request to change the view of the section of the virtual environment; and In response to receiving the respective input, the view of the virtual environment changes from the display with the first spatial profile to the display with the second spatial profile. [7] Method according to any one of claims 1 to 6, wherein the first type of environment template corresponds to one or more of a view portal template, a virtual stage template, a view box template, a virtual model template and an immersive template. [8] Method according to any one of claims 1 to 7, wherein the view of the virtual environment displayed on the electronic device changes according to a change in the viewing angle of the electronic device relative to a three-dimensional environment. [9] Method according to any one of claims 1 to 8, wherein, when the view corresponds to the first type of environment template, the electronic device displays a virtual object corresponding to a representation of at least the section of the view of the virtual environment, wherein the virtual object is distinct from the section of the view of the virtual environment and is displayed simultaneously with it. [10] Method according to any one of claims 1 to 9, further comprising: Receiving one or more types of virtual content from the electronic device when the electronic device receives one or more inputs requesting the export of one or more types of virtual content, wherein: if the view corresponds to the first type of environment template, the received one or more types of virtual content include one or more first types of virtual content; and If the view corresponds to the second type of environment template, the received one or more types of virtual content include one or more second types of virtual content that differ from the one or more first types of virtual content. [11] Method according to any one of claims 1 to 10, wherein the computer system participates in a multi-user communication session between the electronic device and a respective computer system, wherein: one or more properties of the electronic device match one or more properties of the respective computer system and The respective computer system displays a view of the section of the virtual environment based on the perspective of a user of the respective computer system relative to the three-dimensional environment while the multi-user communication session is running. [12] Method according to any one of claims 1 to 11, wherein the electronic device is configured to change the section of the view of the virtual environment from the first spatial profile to the second spatial profile in response to the detection of a corresponding input which includes a request to change the view of the section of the virtual environment. [13] Method according to any one of claims 1 to 12, wherein displaying the virtual environment on the electronic device with the first spatial profile comprises displaying with a first shape relative to the three-dimensional environment and displaying first virtual content that is linked to the virtual environment and extends beyond the dimensions of the first shape. [14] Electronic device, including a memory and one or more processors coupled to the memory and configured to perform the method according to any one of claims 1 to 13. [15] Non-transitory computer-readable storage medium which stores instructions which, when issued by an electronic device comprising a memory and one or more processors coupled to the memory, cause the electronic device to perform the method according to any one of claims 1 to 13.

Images