System and method for capturing and viewing spatial images

Abstract

The disclosure relates to systems and methods for capturing and viewing spatial images. In some examples, a first electronic device communicates with a plurality of displays while also interfacing with two external cameras, each capturing a different viewpoint. In some examples, a first external camera continuously captures first image data, and a second external camera simultaneously captures second image data, both contributing to generating spatial image data. In some examples, the first electronic device obtains spatial image data from the two external cameras, or generates spatial image data based on the first image data and the second image data. In some examples, when one or more first criteria are met, the first electronic device renders spatial image data on one or more displays.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 700,655, filed September 28, 2024; U.S. Provisional Application No. 63 / 879,567, filed September 10, 2025; U.S. Patent Application No. 19 / 329,256, filed September 15, 2025; and U.S. Patent Application No. 19 / 329,277, filed September 15, 2025, the contents of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] The present invention relates generally to systems and methods for providing extended reality experiences, and more particularly to systems and methods for presenting spatial images in extended reality based on images captured by one or more external cameras. Background Technology

[0004] Some computer graphics environments provide two-dimensional and / or three-dimensional environments in which at least some of the objects displayed for the user to view are virtual and computer-generated. For example, objects include images captured using a camera. Summary of the Invention

[0005] Providing a convenient way to display images captured by multiple external cameras enhances user interaction with electronic devices by offering real-time display of images captured on an auxiliary display (such as a head-mounted display) and reduces the need to view the captured images later.

[0006] In some examples, a first electronic device with multiple displays communicates with multiple external cameras, each with a different viewpoint. In some examples, the multiple cameras capture spatial image data of a three-dimensional environment and transmit the spatial image data as a spatial image at the first electronic device based on determining that one or more criteria are met.

[0007] In some examples, the multiple external cameras are integrated into a second electronic device (such as a mobile phone) that communicates with the first electronic device.

[0008] In some examples, the multiple external cameras are integrated into a separate camera that communicates with the first electronic device.

[0009] In some examples, the second electronic device generates a spatial image based on spatial image data captured from the plurality of external cameras and sends the spatial image to the first electronic device.

[0010] In some examples, the first electronic device receives spatial image data from the second electronic device and generates a spatial image based on the received spatial image data.

[0011] In some examples, the second electronic device includes a display, such as a touch panel display, configured to display a two-dimensional rendering of the captured spatial image data. In some examples, the two-dimensional rendering of the captured spatial image data corresponds to the rendering of a three-dimensional environment lacking depth information associated with the spatial image data.

[0012] In some examples, the first electronic device displays the received spatial image data as a spatial image or a two-dimensional rendering of the captured spatial image data discussed above.

[0013] In some examples, the first electronic device does not display the spatial image until it receives a command from the second electronic device to display the spatial image.

[0014] In some examples, multiple external cameras capture spatial video or spatial images of the three-dimensional environment.

[0015] In some examples, spatial image data captured by multiple external cameras includes multiple images of a three-dimensional environment from different viewpoints. In some examples, a first electronic device combines multiple images from varying viewpoints to generate a single spatial image that includes the varying viewpoints (e.g., depth information discussed above).

[0016] In some examples, the varying viewpoints of the multiple images discussed above differ too much. If this occurs, the first electronic device cannot generate a spatial image; instead, it displays a two-dimensional rendering of the captured spatial image data discussed above. In some examples, the varying viewpoints of the multiple images are a result of the different focal lengths of the associated external cameras among the multiple external cameras. In some examples, the first electronic device determines the focal length difference between the focal lengths of each of the multiple external cameras, and if the focal length difference is too large, the electronic device determines that the spatial image data captured by the external cameras cannot be used to generate a spatial image.

[0017] In some examples, the multiple external cameras capture spatial image data only when the second electronic device is oriented parallel to the three-dimensional environment (e.g., "landscape mode"). In some examples, if the second electronic device detects that it is not in "landscape mode," it sends a notification to the first electronic device, such as a visual pop-up, to inform the user of the first electronic device of the second electronic device's orientation. In some examples, if the second electronic device is rotated while remaining parallel to the three-dimensional environment, the multiple external cameras capture spatial image data reflecting the new orientation of the second electronic device. In some examples, the first electronic device receives the captured spatial image data reflecting the new orientation of the second electronic device and updates the displayed spatial image in response.

[0018] In some examples, the display of the second electronic device includes a control panel configured to change various aspects of the captured spatial image data, such as a playback menu. In some examples, the user of the second electronic device touches the display and "drags" the playback of the captured spatial image data (e.g., moving the spatial video from a first time point to a second time point). In some examples, the control panel is an editing interface configured to modify the spatial video data, such as changing the saturation of the spatial image.

[0019] In some examples, the first electronic device displays spatial image data as a portion of the display covering a three-dimensional environment at the display of the first electronic device, such as a rectangular frame in the upper portion of the display of the first electronic device. In some examples, a user of the first electronic device may expect to view the spatial image at a larger scale and make an input to the first electronic device (such as a scroll wheel) to "zoom in" on the spatial image, thereby increasing the size of the spatial image on the display (e.g., the spatial image completely covers the three-dimensional environment at the display of the first electronic device). In some examples, a user may expect to move the spatial image's position on the display and make an input to the touch panel display of the second electronic device, such as a swipe motion to the left on the display. In response, the first electronic device moves the spatial image to the left on the display at the first electronic device, thereby mirroring the gesture (e.g., input) made at the touch panel display of the second electronic device.

[0020] In some examples, the touch panel display of the second electronic device responds to spatial image data captured by the multiple external cameras by applying filters (such as tints, e.g., darkening the screen) to the touch panel display. In some examples, the filters serve as a visual indication to the user that the multiple external cameras are capturing spatial image data.

[0021] A full description of these examples is provided in the accompanying drawings and detailed embodiments, and it should be understood that the content of this invention does not limit the scope of this disclosure in any way.

[0022] As is widely recognized, the use of personally identifiable information should comply with privacy policies and practices that are generally accepted to meet or exceed industry or governmental requirements for protecting user privacy. Specifically, personally identifiable information data should be managed and processed to minimize the risk of unintentional or unauthorized access or use, and the nature of authorized use should be clearly explained to users. Attached Figure Description

[0023] To better understand the various examples described herein, reference should be made to the following detailed embodiments and the accompanying drawings. Throughout the drawings, similar reference numerals generally refer to corresponding parts.

[0024] Figure 1Examples of electronic devices that present extended real-world environments according to some examples of this disclosure are illustrated.

[0025] Figures 2A to 2B Various block diagrams illustrating example architectures of devices according to some examples of this disclosure are shown.

[0026] Figures 3A to 3K Various examples of capturing spatial images in a three-dimensional environment and simultaneously displaying the captured spatial images on an electronic device are illustrated according to some examples of this disclosure.

[0027] Figures 4A to 4H Various examples of spatial images of previously captured three-dimensional environments are illustrated according to some examples of this disclosure.

[0028] Figure 5 This is a flowchart illustrating example methods for displaying spatial images in a three-dimensional environment according to some examples of this disclosure.

[0029] Figures 6A to 6J Examples of capturing spatial images in a three-dimensional environment at an electronic device communicating with a stand-alone camera, according to some examples of this disclosure, are illustrated.

[0030] Figure 7 This is a flowchart illustrating an example method for displaying an updated spatial image of a three-dimensional environment captured by a standalone camera, according to some examples of this disclosure. Detailed Implementation

[0031] In the following description of the examples, reference is made to the accompanying drawings that form part of this document, which illustrate specific examples of optional practice by way of example. It should be understood that other examples and structural changes may be optionally used without departing from the scope of the disclosed examples.

[0032] In some examples, a first electronic device with multiple displays communicates with multiple external cameras, each with a different viewpoint. In some examples, the multiple cameras capture spatial image data of a three-dimensional environment and transmit the spatial image data as a spatial image at the first electronic device based on determining that one or more criteria are met.

[0033] In some examples, the multiple external cameras are integrated into a second electronic device (such as a mobile phone) that communicates with the first electronic device.

[0034] In some examples, the second electronic device generates a spatial image based on spatial image data captured from the plurality of external cameras and sends the spatial image to the first electronic device.

[0035] In some examples, the first electronic device receives spatial image data from the second electronic device and generates a spatial image based on the received spatial image data.

[0036] In some examples, the second electronic device includes a display, such as a touch panel display, configured to display a two-dimensional rendering of the captured spatial image data. In some examples, the two-dimensional rendering of the captured spatial image data corresponds to the rendering of a three-dimensional environment lacking depth information associated with the spatial image data.

[0037] In some examples, the first electronic device displays the received spatial image data as a spatial image or a two-dimensional rendering of the captured spatial image data discussed above.

[0038] In some examples, the first electronic device does not display the spatial image until it receives a command from the second electronic device to display the spatial image.

[0039] In some examples, multiple external cameras capture spatial video or spatial images of the three-dimensional environment.

[0040] In some examples, spatial image data captured by multiple external cameras includes multiple images of a three-dimensional environment from different viewpoints. In some examples, a first electronic device combines multiple images from varying viewpoints to generate a single spatial image that includes the varying viewpoints (e.g., depth information discussed above).

[0041] In some examples, the varying viewpoints of the multiple images discussed above differ too much. If this occurs, the first electronic device cannot generate a spatial image; instead, it displays a two-dimensional rendering of the captured spatial image data discussed above. In some examples, the varying viewpoints of the multiple images are a result of the different focal lengths of the associated external cameras among the multiple external cameras. In some examples, the first electronic device determines the focal length difference between the focal lengths of each of the multiple external cameras, and if the focal length difference is too large, the electronic device determines that the spatial image data captured by the external cameras cannot be used to generate a spatial image.

[0042] In some examples, the multiple external cameras capture spatial image data only when the second electronic device is oriented parallel to the three-dimensional environment (e.g., "landscape mode"). In some examples, if the second electronic device detects that it is not in "landscape mode," it sends a notification to the first electronic device, such as a visual pop-up, to inform the user of the first electronic device of the second electronic device's orientation. In some examples, if the second electronic device is rotated while remaining parallel to the three-dimensional environment, the multiple external cameras capture spatial image data reflecting the new orientation of the second electronic device. In some examples, the first electronic device receives the captured spatial image data reflecting the new orientation of the second electronic device and updates the displayed spatial image in response.

[0043] In some examples, the display of the second electronic device includes a control panel configured to change various aspects of the captured spatial image data, such as a playback menu. In some examples, the user of the second electronic device touches the display and "drags" the playback of the captured spatial image data (e.g., moving the spatial video from a first time point to a second time point). In some examples, the control panel is an editing interface configured to modify the spatial video data, such as changing the saturation of the spatial image.

[0044] In some examples, the first electronic device displays spatial image data as a portion of the display covering a three-dimensional environment at the display of the first electronic device, such as a rectangular frame in the upper portion of the display of the first electronic device. In some examples, a user of the first electronic device may expect to view the spatial image at a larger scale and make an input (such as a scroll wheel) to the first electronic device to "zoom in" on the spatial image, thereby increasing the size of the spatial image on the display (e.g., the spatial image completely covers the three-dimensional environment at the display of the first electronic device). In some examples, a user may expect to move the spatial image's position on the display and make an input at the touch panel display of the second electronic device, such as a swipe motion to the left on the display. In response, the first electronic device moves the spatial image to the left on the display at the first electronic device, thereby mirroring the gesture (e.g., input) made at the touch panel display of the second electronic device.

[0045] In some examples, the touch panel display of the second electronic device responds to spatial image data captured by the multiple external cameras by applying filters (such as tints, e.g., darkening the screen) to the touch panel display. In some examples, the filters serve as a visual indication to the user that the multiple external cameras are capturing spatial image data.

[0046] Providing a convenient way to display images captured by multiple external cameras enhances user interaction with electronic devices by offering real-time display of images captured at an auxiliary display (such as a head-mounted display) and reduces the need to view the captured images at a later time. In one or more examples, displaying content captured by multiple cameras on a display capable of displaying spatial images (e.g., images with depth) allows for previewing spatial content, even when the device associated with the multiple cameras (e.g., a mobile phone) only includes a display capable of displaying 2D images. Additionally, by previewing images captured by cameras on a device separate from the device used to capture the images, flexibility is provided in the type of camera used to capture spatial images. For example, in one or more examples, the camera can be portable (e.g., mobile) and can capture scenes that might not typically be visible to one or more cameras that are part of a head-mounted device.

[0047] Although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, a first touch may be referred to as a second touch, and similarly, a second touch may be referred to as a first touch, without departing from the scope of the various embodiments described. Both the first touch and the second touch are touches, but they are not the same touch.

[0048] The terminology used in the description of the various examples herein is for the purpose of describing particular examples only and is not intended to be limiting. As used in the description of the various described examples and in the appended claims, the singular forms “an,” “a,” and “the” are intended to include the plural forms as well, unless the context expressly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms “comprising” and / or “including” as used in this specification specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0049] Depending on the context, the term "if" may optionally be interpreted as meaning "when," "in," or "in response to determination" or "in response to detection." Similarly, depending on the context, the phrases "if it is determined..." or "if [the stated condition or event] is detected" may optionally be interpreted as meaning "in response to determination..." or "in response to detection of [the stated condition or event]."

[0050] Figure 1An electronic device 101 is illustrated according to some examples of this disclosure, which presents an extended reality (XR) environment (e.g., a computer-generated environment that optionally includes representations of physical and / or virtual objects). In some examples, such as Figure 1 As shown, electronic device 101 is a head-mounted display or other head-mountable device configured to be worn on the head of a user of electronic device 101. See below for reference. Figure 2A An example of an architecture block diagram to describe electronic device 101. For example... Figure 1 As shown, electronic device 101 and table 106 are located in a physical environment. The physical environment may include physical features such as physical surfaces (e.g., floor, wall) or physical objects (e.g., table, lamp, etc.). In some examples, electronic device 101 may be configured to detect and / or capture images of the physical environment including table 106 (exemplified in the field of view of electronic device 101).

[0051] In some examples, such as Figure 1 As shown, the electronic device 101 includes one or more internal image sensors 114a oriented toward the user's face (e.g., referred to below). Figures 2A to 2B (The described eye-tracking camera). In some examples, an internal image sensor 114a is used for eye tracking (e.g., detecting the user's gaze). The internal image sensor 114a is optionally arranged on the left and right portions of the display 120 to enable eye tracking of the user's left and right eyes. In some examples, the electronic device 101 also includes external image sensors 114b and 114c facing outwards from the user to detect and / or capture the physical environment of the electronic device 101 and / or movement of the user's hands or other body parts.

[0052] In some examples, display 120 has a field of view visible to the user (e.g., it may or may not correspond to the field of view of external image sensors 114b and 114c). Because display 120 is optionally part of a head-mounted device, the field of view of display 120 may be the same as or similar to the field of view of the user's eyes. In other examples, the field of view of display 120 may be smaller than the field of view of the user's eyes. In some examples, electronics 101 may be an optical pass-through device, through which display 120 is a transparent or translucent display through which parts of the physical environment can be directly viewed. In some examples, display 120 may be included within a transparent lens and may overlap with all or only a portion of the transparent lens. In other examples, electronics may be a video pass-through device, through which display 120 is an opaque display configured to display images of the physical environment captured by external image sensors 114b and 114c. Although a single display 120 is shown, it should be understood that display 120 may include a stereoscopic display pair.

[0053] In some examples, in response to a trigger, electronic device 101 can be configured to display in an XR environment... Figure 1 The illustrated cube represents a virtual object 104 that does not exist in the physical environment but is displayed in an XR environment positioned on top of a real-world table 106 (or a representation thereof). Optionally, in response to detecting a flat surface of the table 106 in the physical environment 100, the virtual object 104 may be displayed on the surface of the table 106 in the XR environment displayed via a display 120 of the electronic device 101.

[0054] It should be understood that virtual object 104 is a representative virtual object and may include and render one or more different virtual objects (e.g., virtual objects with various dimensions, such as two-dimensional or other three-dimensional virtual objects) in a three-dimensional XR environment. For example, a virtual object may represent an application or user interface displayed in an XR environment. In some examples, a virtual object may represent content corresponding to an application and / or displayed via a user interface in an XR environment. In some examples, virtual object 104 may optionally be configured to be interactive and responsive to user input (e.g., air gestures, such as air pinch gestures, air tap gestures, and / or air touch gestures), allowing the user to virtually touch, tap, move, rotate, or otherwise interact with virtual object 104.

[0055] In some examples, electronic device 101 may be configured to communicate with a second electronic device that is communicatively coupled (e.g., via wired or wireless means) to electronic device 101. For example, such as Figure 1As illustrated, electronic device 101 can communicate with second electronic device 160. In some examples, second electronic device 160 corresponds to mobile electronic device, such as a smartphone, tablet computer, smartwatch, or other electronic device. See below for reference. Figure 2B The following is an architectural block diagram to describe additional examples of the second electronic device 160. In some examples, electronic device 101 and second electronic device 160 are associated with the same user. For example, in Figure 1 In this configuration, electronic device 101 may be positioned (e.g., mounted) on a user's head, and second electronic device 160 may be positioned near electronic device 101, such as in the user's hand 103 (e.g., hand 103 is holding second electronic device 160), and electronic device 101 and second electronic device 160 are associated with the same user account (e.g., the user is logged into a user account on electronic device 101 and second electronic device 160). See below for further details. Figures 2A to 2B Additional details regarding the communication between electronic device 101 and second electronic device 160 are provided.

[0056] In some examples, displaying an object in a 3D environment may include interaction with one or more user interface objects in the 3D environment. For example, initiating the display of an object in a 3D environment may include interaction with one or more virtual option / power representations displayed in the 3D environment. In some examples, when initiating the display of an object in a 3D environment, the electronic device may track the user's gaze as input for identifying one or more virtual option / power representations as a target for selection. For example, a gaze may be used to identify one or more virtual option / power representations as a target for selection using another selection input. In some examples, a virtual option / power representation may be selected using hand-tracking input detected via an input device communicating with the electronic device. In some examples, an object displayed in a 3D environment may move and / or reorient itself in the 3D environment based on movement input detected via an input device.

[0057] In the following discussion, an electronic device communicating with a display generation component and one or more input devices is described. It should be understood that the electronic device may optionally communicate with one or more other physical user interface devices, such as a touch-sensitive surface, physical keyboard, mouse, joystick, hand-tracking device, eye-tracking device, stylus, etc. Furthermore, as described above, it should be understood that the described electronic device, display, and touch-sensitive surface may optionally be distributed among two or more devices. Therefore, as used in this disclosure, information on or displayed by an electronic device may optionally be used to describe information output by the electronic device for display on a separate display device (touch-sensitive or non-touch-sensitive). Similarly, as used in this disclosure, input received on an electronic device (e.g., touch input received on a touch-sensitive surface of the electronic device, or touch input received on the surface of a stylus) may optionally be used to describe input received on a separate input device from which the electronic device receives input information.

[0058] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, TV channel browsing applications, and / or digital video player applications.

[0059] Figures 2A to 2B Block diagrams illustrating example architectures for electronic devices 201 and 260 according to some examples of this disclosure are shown. In some examples, electronic device 201 and / or electronic device 260 include one or more electronic devices. For example, electronic device 201 may be a portable device, an auxiliary device for communicating with another device, a head-mounted display, etc. In some examples, electronic device 201 corresponds to the above reference. Figure 1 The described electronic device 101. In some examples, electronic device 260 corresponds to the above reference. Figure 1 The second electronic device 160 described.

[0060] like Figure 2A As illustrated, electronic device 201 may optionally include various sensors, such as one or more hand tracking sensors 202, one or more position sensors 204A, and one or more image sensors 206A (optionally corresponding to...). Figure 1The internal image sensor 114a and / or external image sensors 114b and 114c, one or more touch-sensitive surfaces 209A, one or more motion and / or orientation sensors 210A, one or more eye-tracking sensors 212, one or more microphones 213A or other audio sensors, one or more body tracking sensors (e.g., torso tracking sensors and / or head tracking sensors), and one or more display generation components 214A (optionally corresponding to...) Figure 1 The electronic device 201 includes a display 120), one or more speakers 216A, one or more processors 218A, one or more memories 220A, and / or communication circuitry 222A. One or more communication buses 208A are optionally used for communication between the aforementioned components of the electronic device 201. Additionally, as... Figure 2B As shown, electronic device 260 optionally includes one or more position sensors 204B, one or more image sensors 206B, one or more touch-sensitive surfaces 209B, one or more orientation sensors 210B, one or more microphones 213B, one or more display generating components 214B, one or more speakers 216B, one or more processors 218B, one or more memories 220B, and / or communication circuitry 222B. One or more communication buses 208B are optionally used for communication between the aforementioned components of electronic device 260. Electronic devices 201 and 260 are optionally configured to communicate via a wired or wireless connection between the two electronic devices (e.g., via communication circuitry 222A, 222B). For example, as Figure 2A As indicated, electronic device 260 can be used as an accessory device to electronic device 201.

[0061] Communication circuits 222A and 222B optionally include circuitry for communicating with electronic devices and networks such as the Internet, intranets, wired and / or wireless networks, cellular networks, and wireless local area networks (LANs). Communication circuits 222A and 222B optionally include circuitry for using near-field communication (NFC) and / or short-range communication such as... The circuit used for communication.

[0062] Processors 218A and 218B include one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, memory 220A or 220B is a non-transitory computer-readable storage medium (e.g., flash memory, random access memory, or other volatile or non-volatile memory or storage device) storing computer-readable instructions configured to be executed by processor 218A or 218B to perform the techniques, processes, and / or methods described below. In some examples, memory 220A and / or 220B may include more than one non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium can be any medium (e.g., excluding signals) that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, and device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include hard disks, optical discs based on compact disc (CD), digital versatile optical disc (DVD), or Blu-ray technology, and persistent solid-state storage (such as flash memory, solid-state drives, etc.).

[0063] In some examples, display generating components 214A, 214B include a single display (e.g., a liquid crystal display (LCD), an organic light-emitting diode (OLED), or other type of display). In some examples, display generating components 214A, 214B include multiple displays. In some examples, display generating components 214A, 214B may include a display with touch capability (e.g., a touchscreen), a projector, a holographic projector, a retinal projector, a transparent or translucent display, etc. In some examples, electronic devices 201 and 260 respectively include touch-sensitive surfaces 209A and 209B for receiving user input such as tap input and swipe input or other gestures. In some examples, display generating components 214A, 214B and touch-sensitive surfaces 209A, 209B form a touch-sensitive display (e.g., a touchscreen integrated with each of electronic devices 201 and 260 or a touchscreen external to each of electronic devices 201 and 260 that communicates with each of electronic devices 201 and 260).

[0064] In some examples, electronic devices 201 and 260 may optionally include image sensors 206A and 206B, respectively. Image sensors 206A and 206B may optionally include one or more visible light image sensors (such as charge-coupled device (CCD) sensors) and / or complementary metal-oxide-semiconductor (CMOS) sensors operable to obtain images of physical objects from the real-world environment. Image sensors 206A and 206B may also optionally include one or more infrared (IR) sensors, such as passive or active IR sensors, for detecting infrared light from the real-world environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real-world environment. Image sensors 206A and 206B may also optionally include one or more cameras configured to capture movement of a physical object in the real-world environment. Image sensors 206A and 206B may also optionally include one or more depth sensors configured to detect the distance between the physical object and electronic devices 201 and 260. In some examples, information from one or more depth sensors allows a device to identify objects in a real-world environment and distinguish them from other objects in the real-world environment. In some examples, one or more depth sensors allow a device to determine the texture and / or shape of objects in a real-world environment.

[0065] In some examples, electronic devices 201 and 260 combine a CCD sensor, an event camera, and a depth sensor to detect the physical environment surrounding them. In some examples, image sensors 206A and 206B include a first image sensor and a second image sensor. The first and second image sensors work cooperatively and are optionally configured to capture different information about physical objects in the real-world environment. In some examples, the first image sensor is a visible light image sensor, and the second image sensor is a depth sensor. In some examples, electronic devices 201 and 260 use image sensors 206A and 206B to detect the location and orientation of electronic devices 201 and 260 and / or display generation components 214A and 214B in the real-world environment. For example, electronic devices 201 and 260 use image sensors 206A and 206B to track the location and orientation of display generation components 214A and 214B relative to one or more stationary objects in the real-world environment.

[0066] In some examples, electronic devices 201 and 260 include microphones 213A and 213B, or other audio sensors, respectively. Electronic devices 201 and 260 may optionally use microphones 213A and 213B to detect sound from a user and / or the user's real-world environment. In some examples, microphones 213A and 213B include arrays (multiple microphones) of microphones that may optionally operate in cooperation, such as to identify ambient noise or locate sound sources in the space of a real-world environment.

[0067] In some examples, electronic devices 201 and 260 include position sensors 204A and 204B, respectively, which are used to detect the positions of electronic device 201A and / or display generation component 214A and the positions of electronic device 260 and / or display generation component 214B. For example, position sensors 204A and 204B may include Global Positioning System (GPS) receivers that receive data from one or more satellites and allow electronic devices 201 and 260 to determine the absolute location of the device in the physical world.

[0068] In some examples, electronic devices 201 and 260 include orientation sensors 210A and 210B, respectively, for detecting the orientation and / or movement of electronic device 201 and / or display generation component 214A, and the orientation and / or movement of electronic device 260 and / or display generation component 214B. For example, electronic devices 201 and 260 use orientation sensors 210A and 210B to track changes in the positioning and / or orientation of electronic devices 201 and 260 and / or display generation components 214A and 214B, such as changes relative to physical objects in a real-world environment. Orientation sensors 210A and 210B may optionally include one or more gyroscopes and / or one or more accelerometers.

[0069] In some examples, electronic device 201 includes a hand tracking sensor 202 and / or an eye tracking sensor 212 (and / or other body tracking sensors, such as a leg tracking sensor, a torso tracking sensor, and / or a head tracking sensor). The hand tracking sensor 202 is configured to track the localization / position of one or more portions of a user's hand, and / or the movement of one or more portions of the user's hand relative to the extended reality environment, relative to the display generation component 214A, and / or relative to another defined coordinate system. The eye tracking sensor 212 is configured to track the localization and movement of the user's gaze (more generally, the eyes, face, or head) relative to the real world or extended reality environment and / or relative to the display generation component 214A. In some examples, the hand tracking sensor 202 and / or the eye tracking sensor 212 are implemented together with the display generation component 214A. In some examples, the hand tracking sensor 202 and / or the eye tracking sensor 212 are implemented separately from the display generation component 214A. In some examples, electronic device 201 optionally does not include the hand tracking sensor 202 and / or the eye tracking sensor 212. In some such examples, the generation component 214A is shown to be used by the electronic device 260 to provide an extended reality environment and utilizes input and other data collected via other sensors of the electronic device 201 (e.g., one or more position sensors 204A, one or more image sensors 206A, one or more touch-sensitive surfaces 209A, one or more motion and / or orientation sensors 210A, and / or one or more microphones 213A or other audio sensors) as input and data to be processed by the processor 218B of the electronic device 260. Additionally or alternatively, the electronic device 201 may optionally not include... Figure 2B Other components shown include position sensor 204B, image sensor 206B, touch-sensitive surface 209B, etc. In some such examples, generation component 214A is shown to be used by electronic device 260 to provide an extended reality environment, and electronic device 260 uses input and other data collected via one or more motion and / or orientation sensors 210A (and / or one or more microphones 213A) of electronic device 201 as input.

[0070] In some examples, the hand tracking sensor 202 (and / or other body tracking sensors, such as leg tracking sensors, torso tracking sensors, and / or head tracking sensors) may use image sensors 206 (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that capture 3D information from the real world, including one or more body parts (e.g., a human user's hand, leg, torso). In some examples, sufficient resolution is available to distinguish the hand to differentiate the fingers and their corresponding positions. In some examples, one or more image sensors 206A are positioned relative to the user to define the field of view and interaction space of the image sensors 206A, in which the finger / hand positions, orientations, and / or movements captured by the image sensors are used as input (e.g., to differentiate from the user's resting hand or other hands of other people in the real-world environment). Tracking the fingers / hands used for input (e.g., gestures, touches, taps, etc.) may be advantageous because it does not require the user to touch, hold, or wear any type of beacon, sensor, or other marker.

[0071] In some examples, the eye-tracking sensor 212 includes at least one eye-tracking camera (e.g., an infrared (IR) camera) and / or an illumination source (e.g., an IR light source, such as an LED) that emits light toward the user's eyes. The eye-tracking camera may be pointed at the user's eyes to receive reflected IR light from the light source directly or indirectly from the eyes. In some examples, both eyes are tracked separately by the respective eye-tracking camera and illumination source, and focus / gaze can be determined by tracking both eyes. In some examples, one eye (e.g., the dominant eye) is tracked by one or more respective eye-tracking cameras / illumination sources.

[0072] Electronic devices 201 and 260 are not limited to Figures 2A to 2B The components and configurations may include, but are not limited to, a few components, other components, or additional components in various configurations. In some examples, electronic device 201 and / or electronic device 260 may each be implemented among multiple electronic devices (e.g., as a system). In some such examples, each(s) electronic device may each include one or more of the same components discussed above, such as various sensors, one or more display generation components, one or more speakers, one or more processors, one or more memories, and / or communication circuitry. One or more persons using electronic device 201 and / or electronic device 260 may optionally be referred to herein as one or more users of the devices.

[0073] Now turn to the interaction of capturing and displaying spatial images in a three-dimensional environment presented at an electronic device (e.g., corresponding to electronic device 201). In some examples, the spatial image of one or more physical objects is captured by a mobile electronic device (e.g., a second electronic device 160) and transmitted for presentation at electronic device 101. Due to the nature of displays typically included at mobile devices, while spatial images can be captured (e.g., typically using one or more cameras), mobile devices typically cannot present spatial images using their corresponding two-dimensional displays (e.g., without a depth component). The depth component of the spatial image is most often implemented by using two or more displays (such as the display provided at electronic device 101). In the following examples, various configurations for presenting spatial images within a portion of the user's field of view at a head-mounted display (e.g., electronic device 101) are presented.

[0074] Figure 3A An example is illustrated whereby electronic device 101 presents a three-dimensional environment 700 from a first perspective when communicating with a second electronic device 160. This three-dimensional environment includes elements related to the physical environment (e.g., as referenced above). Figure 1 Multiple objects corresponding to physical objects within the physical environment under discussion. In some examples, such as Figure 3A As shown, electronic device 101 and second electronic device 160 are in wireless communication 161 and / or wired communication 162. In some examples, wireless communication 161 may optionally correspond to Bluetooth connectivity, cellular broadband, Wi-Fi, or radio. In one or more examples, both electronic device 101 and second electronic device 160 include one or more outward-facing cameras viewing a common three-dimensional scene. In one or more examples, electronic device 101 displays the scene captured by its outward-facing camera on display 120. Similarly, second electronic device 160 displays the scene captured from one or more outward-facing cameras (described in further detail below) on display 164 of second electronic device 160. In some examples, as shown in top view 200, the outward-facing camera positioned at second electronic device 261 (e.g., second electronic device 160) includes an outwardly projected field of view 271, the direction of which is the same as the field of view 270 corresponding to the field of view of electronic device 211 (e.g., electronic device 101). In some examples, fields of view 271 and 270 overlap depending on the orientation of the second electronic device 261, but are not necessarily the same fields of view, as discussed in further detail below. In some examples, such as Figure 3AAs shown, the second electronic device 160 includes a button 163 optionally configured to receive user input. In some examples, the second electronic device 160 includes a display 164 configured to display a representation of the three-dimensional environment 700 from a second perspective different from the first perspective (the difference in perspective is attributed to the fact that the cameras of the electronic devices 101 and 160 are positioned differently relative to the common scene from which they capture image data). In some examples, the representation of the three-dimensional environment 700 includes a representation of a person 164a and a representation of a tree 164b. In some examples, the aforementioned representations correspond to physical objects within the three-dimensional environment 700, as discussed in further detail below. In some examples, the second electronic device 160 obtains the representation of the three-dimensional environment 700 from one or more external cameras (not shown), as referenced below. Figure 3B This will be discussed in further detail below. In some examples, the representation of the 3D environment 700 corresponds to a real-time video feed of the 3D environment 700, and the representation may optionally be updated based on updates in the 3D environment, as discussed in further detail below. In some examples, such as Figure 3A As shown, the second electronic device displays a representation of the three-dimensional environment 700 in the same manner as the electronic device 101 displays the three-dimensional environment 700.

[0075] In some examples, such as Figure 3A As shown, the electronic device displays the plurality of objects (as described above), including a person 710 centrally positioned within the field of view of the electronic device 101 in the three-dimensional environment 700, and a tree 720 positioned on the left side of the field of view of the electronic device 101. In some examples, the plurality of physical objects correspond to a representation of person 164a and a representation of tree 164b. In some examples, the electronic device 101 displays only a portion of the person 710 included in the three-dimensional environment. For example, as... Figure 3A As shown, the display 120 optionally includes the upper portion of the person 710. In some examples, the electronic device 101 displays only the upper portion of the person 710 to indicate a closer positioning relationship between the user of the electronic device 101 and the tree 720, as further detailed in the top view 200.

[0076] In some examples, as shown in top view 200, the user of electronic device 101 (e.g., a representation of user 230) faces the three-dimensional environment 700. In some examples, as shown in top view 200, electronic device 211 includes a field of view 271 that encompasses a representation of tree 220 (e.g., tree 720, and / or a representation of tree 164b) and a representation of person 210 (e.g., person 710, and / or a representation of person 164a). In some examples, field of view 271 corresponds to the three-dimensional environment 700 at display 120. In some examples, user 230 is positioned within the three-dimensional environment 700, directly facing the representation of user 230 and centered within the field of view of electronic device 101. In some examples, as shown in top view 200, the representation of tree 220 is located further away from the representation of user 230 relative to the position of user 230. In some examples, user 230 positions the second electronic device 261 such that display 120 does not show the second electronic device 160 within the three-dimensional environment 700 (e.g., the second electronic device is low enough to be outside the field of view of electronic device 211).

[0077] In some examples, a first external camera 171 and a second external camera 170 capture a common scene (e.g., a 3D environment 700) from a first perspective and a second perspective, respectively. When capturing the scene from multiple angles, a second electronic device 160 is able to generate an image that captures depth information associated with the spatial relationships between the various objects in the scene.

[0078] Figure 3B An example is illustrated where a second electronic device captures spatial image data (e.g., data that can be used to create a spatial image including three-dimensional depth information) via a first external camera 171 and a second external camera 170. This spatial image data includes first image data 171a captured by the first external camera 171 and second image data 170a captured by the second external camera 170 when the electronic device 101 views the three-dimensional environment 700. In some examples, such as... Figure 3BAs shown, the first external camera 171 and the second external camera 170 capture images of the three-dimensional environment 700 from different perspectives because they are positioned at slightly different locations relative to the three-dimensional environment 700. In some examples, as compared between first image data 171a and second image data 170a, the first external camera 171 captures the three-dimensional environment 700 from a first perspective, which includes the upper portion of the person 710 and the entirety of the tree 720 (e.g., first image data 171a), while the second external camera 170 captures the three-dimensional environment 700 from a second perspective, which includes the entirety of the person 710 and the lower portion of the tree 720 (e.g., second image data 170a). In some examples, a second electronic device records the first image data 171a and the second image data 170a to produce spatial image data displayed by the viewfinder, as discussed in further detail below. In some examples, as... Figure 3B As shown, a first external camera 171 and a second external camera 170 are mounted on the back 165 of the second electronic device. In some examples, the back 165 of the second electronic device, including the first external camera 171 and the second external camera 172, faces directly towards the person 710, as shown by the corresponding representation of the second electronic device 261 in the top view 200. In some examples, the second electronic device 160 combines first image data 171a and second image data 170a into spatial image data as discussed above, and transmits the spatial image data to the electronic device 101 via wireless communication 161 and / or wired communication 162, or transmits the first image data 171a and / or the second image data to the electronic device 101, and the electronic device 101 combines the received spatial images into spatial image data as discussed in further detail below. In some examples, the second electronic device 160 displays the spatial image data as a two-dimensional representation of the three-dimensional environment 700 at a display 164, while simultaneously transmitting the spatial image data to an electronic device for display as a spatial image at a viewfinder 300.

[0079] In some examples, as discussed above, display 164 is a two-dimensional display (lacking multiple displays) and cannot display the first image data 171a and / or the second image data 170a as a spatial image. Instead, the second electronic device 160 sends the first image data 171a and / or the second image data 170a for display by a display at electronic device 101 (e.g., display 120 including viewfinder 300), as illustrated in further detail below.

[0080] Figure 3CAn example is illustrated where a second electronic device 160, in response to user input pointing to button 163, sends spatial image data (e.g., a combination of first image data 171a and second image data 170a) to the viewfinder 300 at display 120, while electronic device 101 displays a three-dimensional environment 700. In some examples, such as Figure 3C As shown, user input is provided by hand 103, optionally corresponding to a user of electronic device 101 and second electronic device 160. As discussed above, second electronic device 160 includes only a two-dimensional display (e.g., display 164) and cannot include depth information associated with spatial image data (e.g., first image data 171a, second image data 170a). To display the depth information associated with the spatial image data, multiple displays are required, such as the display provided at electronic device 101. Therefore, in response to the first external camera 171 and / or the second external camera 170 capturing spatial image data, second electronic device 160 sends the spatial image data to electronic device 101, as discussed in further detail below. In some examples, in response to user input provided by hand 103, second electronic device 160 transmits commands to the first external camera 171 and / or the second external camera 170 (as shown above) to begin capturing first image data 171a and / or second image data 170a (as shown above). In some examples, such as... Figure 3C As shown, the second electronic device 160, in response to user input provided at button 163, displays a recording instruction 166 via display 164, indicating that the first external camera 171 and the second external camera 170 are capturing first image data 171a and second image data 170a. In some examples, electronic device 101 receives spatial image data from the second electronic device 160 via wireless communication 161 and / or wired communication 162, and displays the spatial image data in the viewfinder 300 at display 120. In some examples, such as... Figure 3C As shown, the viewfinder 300 covers at least a portion of the three-dimensional environment 700. It should be noted that the location covered by the viewfinder 300 is not necessarily limited to what is presented in the following figures and should be considered as examples of possible locations of the viewfinder 300 at the display 120. In some examples, the electronic device 101 displays spatial image data as a spatial image at the viewfinder 300 (e.g., including those referenced above). Figure 3B The depth information discussed in the image is as indicated by cube 301. The display 120 at the electronic device 101 may optionally include multiple displays, such that a first display associated with the first image data 171a and a second display associated with the second image data 170a are configured to be relative to the user's eyes to produce a depth illusion (e.g., depth information). In some examples, such as... Figure 3CAs shown, the viewfinder 300 displays the 3D environment 700 from a perspective (e.g., display 120) corresponding to the user's perspective relative to the electronic device 101. In some examples, such as Figure 3C As shown, the viewfinder includes representations of person 310 and tree 320 corresponding to person 710 and tree 720 in a three-dimensional environment. In some examples, the spatial image data at viewfinder 300 and the three-dimensional environment 700 correspond to a real-time video feed of the real-world environment.

[0081] Figure 3D Examples Figure 3C In another example, user input provided by hand 103 points to button 115 on the first electronic device. In some examples, in response to user input provided by hand 103 pointing to button 115, electronic device 101 sends a command to second electronic device 160 to begin capturing spatial image data (e.g., a combination of first image data 171a and second image data 170a) via first external camera 171 and second external camera 170 (see [link to documentation]). Figure 3B In some examples, in response to receiving a command to begin capturing spatial image data, the second electronic device 160 engages with the above-described... Figure 3C The spatial image data to be displayed at the viewfinder 300 on the electronic device 101 is transmitted in a similar manner to that discussed in the paper.

[0082] In some examples, the user of the second electronic device 160 may optionally modify the orientation of the second electronic device 160, causing the external camera to capture different portions of the three-dimensional environment 700. Therefore, the spatial image data sent to the electronic device 101 includes updated depth information and / or objects in the three-dimensional environment. In response to receiving the updated spatial image data, the electronic device 101 may optionally update the viewfinder 300 to display the updated spatial image data including the updated depth information and / or objects, as referenced below. Figure 3E Further detailed discussion is needed.

[0083] Figure 3E An example illustrates how a user updates the orientation (e.g., rotation) of the second electronic device 160 while the electronic device 101 maintains its orientation from the second electronic device 160. Figures 3A to 3D The perspective displays an example of a 700-degree 3D environment. In some examples, such as... Figure 3E As shown, the orientation of the second electronic device 160 is updated so that it is aligned with... Figure 3D Compared to the example, the field of view of the second electronic device (specifically, the camera of the second electronic device) is shifted to the left. In some examples, the user of electronic device 101 issues user input (not shown) to the second electronic device 160 to orient it from... Figure 3D The orientation shown in the top view 200 is updated to Figure 3E The orientation shown in the top view 200 (e.g., by rotating the second electronic device by hand (e.g., on the yaw axis)). In some examples, such as Figure 3E As shown, the second electronic device 160 continues to capture spatial image data with an updated orientation (as indicated by recording indicator 166). In some examples, due to the change in the orientation of the second electronic device 160, the first external camera 171 and the second external camera 170 capture the three-dimensional environment from a perspective different from that of the display 120. In some examples, as shown by top view 200, the orientation of the second electronic device 160 is rotated (e.g., to the right, such as a rotation on the yaw axis) such that the field of view of the second electronic device 160 is outside the right-hand portion of the field of view of the first electronic device (e.g., capturing the right-hand portion of the three-dimensional environment 700). In some examples, due to the change in the orientation of the second electronic device 160, the display 164 includes the right-hand portion of the representation of tree 164b. In some examples, the orientation changes discussed above cause the first external camera 171 and the second external camera 170 to capture only the right-hand portion of tree 720, as shown in the representation of tree 164b. In some examples, the orientation change of the second electronic device is a rotation of the second electronic device from a position directly facing the user of the second electronic device 160 outwards to an orientation 45 degrees to the right relative to the user. In some examples, when the second electronic device 160 is rotated, the user of the electronic device 101 maintains a forward orientation, as shown in top view 200, resulting in the display 120 showing the three-dimensional environment 700 (e.g., field of view 270) unchanged compared to the previous figures. In some examples, as shown in top view 200, the field of view 271 (the area of ​​the three-dimensional environment 700 captured by the external camera) extends beyond the field of view 270, resulting in the first image data 171a and / or the second image data 170a capturing aspects of the right side of the three-dimensional environment 700 that are not visible from the display 120 (e.g., corresponding to field of view 270). In some examples, during and / or after the orientation change, the second electronic device 160 sends an updated view of the three-dimensional environment 700 (e.g., spatial image data) to the electronic device 101 for display by the viewfinder 300. In some examples, in response to receiving an updated viewpoint from the 3D environment 700, the electronic device 101 replaces the updated viewpoint with that of the 3D environment 700. Figure 3D The spatial image data shown is located at 300 in the viewfinder.

[0084] Figure 3F Examples Figure 3EIn alternative examples, users of electronic devices 101 and 160 optionally update the orientation (e.g., rotation) of the second electronic device 160 to view the left side of the 3D environment 700. In some examples, the user of electronic device 101 inputs to the second electronic device 160 to update its orientation in a manner similar to that discussed above. In some examples, in response to a change in orientation, electronic device 101 receives an updated viewpoint (e.g., spatial image data) of the 3D environment 700 and updates the viewfinder 300 in a manner similar to that discussed above. In some examples, similar to... Figure 3E As discussed above, the updated orientation of the second electronic device 160 results in a field of view 271 encompassing the left portion of the three-dimensional environment 700 that cannot be viewed by the field of view 270 (e.g., display 120). In some examples, the user of electronic device 101 maintains a forward orientation in a similar manner to that discussed above. In some examples, as shown by top view 200, the orientation of the second electronic device 160 is rotated (e.g., to the left, such as on the yaw axis) such that the field of view of the second electronic device 160 is outside the left portion of the electronic device's field of view (e.g., capturing the left portion of the three-dimensional environment 700). In some examples, due to the change in the orientation of the second electronic device 160, display 164 includes the left portion of the representation of person 164a. In some examples, the orientation change discussed above results in the first external camera 171 and the second external camera 170 capturing only the left portion of person 710, as shown in the representation of person 164a. In some examples, the orientation change is a rotation of the second electronic device from a position directly facing the user of the second electronic device 160 outwards to an orientation at a 45-degree angle to the left relative to the user. In some examples, the user of electronic device 101 can optionally update the orientation of the second electronic device 160 (e.g., rotate it so that the second electronic device 160 is perpendicular to the three-dimensional environment 700 presented by the first electronic device), such that the first external camera 171 and the second external camera cannot capture the first image data 171a and the second image data 170a, as discussed in further detail below.

[0085] Figure 3G An example orientation of a second electronic device 160 is illustrated, which triggers an indication 400b (e.g., "Error") on the viewfinder 300 displayed on electronic device 101 and / or an indication 400a (e.g., "Error") on the display 164 of the second electronic device 160 in response to a change in orientation of the second electronic device while electronic device 101 maintains the display of the three-dimensional environment 700. In some examples, the orientation of the second electronic device 160 is changed via user input in a manner similar to that discussed above. In some examples, such as Figure 3G As shown, the orientation of the second electronic device 160 is from an orientation parallel to the three-dimensional environment 700 (e.g., a lateral mode, see...). Figure 3D The orientation is updated to be perpendicular to the 3D environment (e.g., portrait mode), and in response, the second electronic device 160 displays indication 400a. In some examples, the second electronic device 160 stops displaying (via display 164) a representation of the 3D environment 700 and generates indication 400a (e.g., "error"). In some examples, in response to detecting the updated orientation, the second electronic device 160 stops capturing spatial image data via the first external camera 171 and the second external camera 170, and sends indication 400b to electronic device 101. In some examples, such as Figure 3G As shown, electronic device 101 displays indication 400a at viewfinder 300 in response to detecting an updated orientation of second electronic device 160. In some examples, such as Figure 3G As shown, the second electronic device 160 displays indication 400a in an orientation corresponding to the portrait mode and concurrently transmits commands to the electronic device 101 to display indication 400b. In some examples, indication 400a and / or indication 400b may optionally be accompanied by haptic and / or auditory feedback (not shown). In some examples, as shown in top view 200, the representation of the second electronic device 261 maintains an orientation parallel to the user 230 when in portrait mode. In some examples, the second electronic device 160 continues to command the first external camera 171 and / or the second external camera 170 to capture the three-dimensional environment 700 in an updated orientation, as discussed in further detail below.

[0086] Figure 3H An example orientation is illustrated whereby the second electronic device 160 triggers a non-spatial image 302 at viewfinder 300 and / or a non-spatial image of the three-dimensional environment 700 at display 164 in response to a change in the orientation of the second electronic device 160 (e.g., Figure 3G (As illustrated in the example), while the electronic device 101 maintains the display of the three-dimensional environment 700. With Figure 3G Compared to the example, in Figure 3H In one example, the viewfinder 300 of electronic device 101 maintains the display of image data provided by the second electronic device 160, but instead of displaying the image as a spatial image (e.g., a stereoscopic image), it displays the image as a two-dimensional (e.g., a single field of view) image. In some examples, in conjunction with the above reference... Figure 3G The orientation of the second electronic device 160 is updated in a similar manner as discussed. In some examples, such as Figure 3HAs shown, the second electronic device 160 displays a recording instruction 166, which indicates that the external camera is continuing to capture the 3D environment 700, even though it is capturing a non-spatial image of the 3D environment 700. In some examples, the second electronic device 160 sends the non-spatial image of the 3D environment 700 to the electronic device 101, and in response, the electronic device 101 displays the non-spatial image of the 3D environment 700 in the viewfinder 300. In some examples, such as Figure 3H As shown, the display 120 includes an updated shape (e.g., length and width dimensions) of the viewfinder 300 to reflect the updated orientation of the image (non-spatial) captured and transmitted by the second electronics 160. In some examples, as shown by top view 200, the representation of the second electronics 261 maintains an orientation parallel to the user 230 when in portrait mode. In some examples, in response to the restoration of the orientation of the second electronics 160 to an orientation favorable for capturing spatial image data, the second electronics 160 transmits a command to the electronics 101 to display the spatial image data at the viewfinder 300, as discussed in further detail below.

[0087] In some examples, when the second electronic device 160 is in an orientation conducive to capturing spatial image data, the second electronic device 160 may optionally display a control user interface 360, which is configured to control various aspects of the spatial image data, including the manner in which the spatial image data is displayed at the viewfinder 300.

[0088] Figure 3I An example is illustrated where a second electronic device 160 displays a control user interface 360 ​​in response to input from a user's hand 103 to a button 163, while the electronic device 101 (at the display 120) displays a three-dimensional environment 700 and spatial image data at the viewfinder 300. In some examples, such as Figure 3I As shown, the second electronic device 160 is positioned to facilitate the capture of spatial image data and transmits the spatial image data to the electronic device 101 for display at the viewfinder 300, as indicated by the cube 301. In some examples, the second electronic device 160 captures spatial image data as indicated by the recording instruction 166 and displays a three-dimensional environment 700 partially covered by the control user interface 360 ​​at the display 164. In some examples, such as Figure 3I As shown, the control user interface 360 ​​includes multiple controls 361 to 365. In some examples, the multiple controls 361 to 365 are optionally configured to change one or more aspects of the capture of the spatial image, as discussed in further detail below, such as displaying... Figure 4B The spatial image shown. In some examples, such as... Figure 3IAs shown, the control user interface 360 ​​is opaque and covers the lower portion of the 3D environment 700 at the display 164. In some examples, the second electronic device 160 sends spatial image data (e.g., a spatial image of the 3D environment 700) while only a portion of the 3D environment 700 is displayed at the display 164. In some examples, the control user interface 360 ​​is displayed by the second electronic device 160 at the display 164 in response to the second electronic device 160 (via an external camera) capturing spatial image data. Figure 3I As shown, in response to the detection of user input from the hand 103 pointing to the button 163, this capture of the spatial image can be optionally triggered.

[0089] In some examples, the second electronic device 160 alternately displays hue 515 on the display 164 in response to user input via hand input to button 163 provided by hand 103. In one or more examples, and as described in further detail below, hue 515 is configured to provide the user with a visual indication that electronic device 101 is displaying an image captured by the second electronic device 160. In some examples, the detection of input via hand 103 signals the second electronic device 160 to command the first external camera 171 and / or the second external camera 170 to begin capturing image data.

[0090] Figure 3J An alternative example is illustrated whereby, when the second electronic device 160 captures spatial image data and sends that spatial image data to the electronic device 101 for display at the viewfinder 300, a three-dimensional environment 700 with hue 515 is displayed on the display 164 in response to input provided by the hand 103 at button 163, while the electronic device displays the three-dimensional environment 700 on the display 120. In some examples, in response to input provided by the user (e.g., corresponding to hand 103) to begin capturing spatial image data (e.g., via the first external camera 171 and the second external camera 170), the second electronic device 160 automatically applies hue 515 to the image of the three-dimensional environment 700 displayed on the display 164, thereby instructing the user of the second electronic device 160 that the second electronic device is capturing spatial image data and optionally sending the spatial image data to the electronic device 101. In some examples, such as Figure 3J As shown, display 164 displays hue 515 as a partially opaque shading around the perimeter of display 164, thereby overlaying the image of the 3D environment 700. In some examples, hue 515 is automatically configured such that representations of people 164a and trees 164b are partially visible through hue 515. In some examples, such as Figure 3JAs shown, the second electronic device 160 is configured to display a hue 515 of an image that partially covers the three-dimensional environment 700, while simultaneously sending spatial image data to the electronic device 101 for display in the viewfinder 300 without the hue 515. In some examples, the second electronic device 160 omits the hue 515 in response to initiating the capture of the spatial image and optionally displays a still image of the three-dimensional environment 700, as discussed in further detail below.

[0091] Figure 3K An alternative example is illustrated whereby, when the second electronic device 160 captures spatial image data and sends that spatial image data to the electronic device 101 for display at the viewfinder 300, the user is indicated to be informing the user that spatial image data is being captured (e.g., in response to a hand 103 as discussed above), while the electronic device displays the three-dimensional environment 700 at the display 120. In some examples, the second electronic device 160 detects input (not shown, optionally a hand 103 as discussed above) to begin capturing spatial image data via the first external camera 171 and / or the second external camera 170, and in response, displays a still (e.g., frozen, static) image of the three-dimensional environment 700 at the moment the input is detected. In some examples, the display 164 continues to include a recording instruction 166, but does not include an updated image of the three-dimensional environment 700 as shown on the display 120. In some examples, as... Figure 3K As shown, electronic device 101 receives updated spatial image data from second electronic device 160, wherein person 710 is positioned in the three-dimensional environment 700 in relation to... Figure 3K Compared to different positions (e.g., to the left of the user's viewpoint on electronic device 101), this indicates that a person has moved in the real world. In some examples, this movement (e.g., person 710) is not reflected on display 164; instead, second electronic device 160 displays an image of the three-dimensional environment 700 at the moment the input is detected (e.g., before the movement of person 710). In some examples, while second electronic device 160 maintains a static image of the three-dimensional environment 700, it simultaneously sends (updated) spatial image data to electronic device 101, as indicated by cube 301.

[0092] In some examples, after capturing spatial image data, the user of electronic device 101 may want to view the captured spatial image data in a playback application. In some examples, after capturing spatial image data, the user of the electronic device inputs the control user interface 360 ​​discussed above, thereby triggering the second electronic device 160 to display the captured spatial image data on display 164 and / or viewfinder 300, as discussed in further detail below. In some examples, once the second electronic device 160 displays the captured spatial image data (e.g., optionally in a playback application), the user of the second electronic device performs various operations related to the captured spatial image data, as discussed in further detail below.

[0093] Figure 4A An example is illustrated where a user of electronic device 101 and second electronic device 160 is in a second three-dimensional environment 800, with electronic device 101 displaying the second three-dimensional environment 800 and displaying spatial image data received from second electronic device 160 (e.g., corresponding to three-dimensional environment 700) at viewfinder 300, while second electronic device 160 displays an image of three-dimensional environment 700. In some examples, as referenced above... Figure 3I As discussed, the user of electronic device 101 can optionally input to any of controls 361 to 365, and in response, second electronic device 160 displays previously captured spatial image data (e.g., 3D environment 700) at display 164 and sends the spatial image data to electronic device 101 for display at viewfinder 300 as a spatial image as indicated by cube 301. In some examples, such as Figure 4A As shown, relative to the user's viewpoint of the electronic device 101, the display 120 shows a portion of the second three-dimensional environment 800, including window 801, while simultaneously displaying a viewfinder 300 that covers a portion of the second three-dimensional environment 800. In some examples, such as Figure 4A As shown, the display 120 is configured to display the viewfinder 300 in a manner similar to the figures above (e.g., covering the upper portion of the 3D environment 700 / second 3D environment 800). In some examples, as shown in top view 200, the user 230 (e.g., the user corresponding to the electronic device 101) is positioned facing the window 801 in a manner similar to the user of electronic device 101 facing the person 710 and the tree 720 as discussed above. In some examples, the second electronic device 160 detects input to the pointing button 163 provided by the hand 103 and, in response, sends a command to electronic device 101 to increase the size of the viewfinder 300, as indicated by the arrow at the corner of the viewfinder 300. In some examples, electronic device 101 continues to display the second 3D environment 800 while increasing the size of the viewfinder 300. In some examples, as... Figure 4AAs shown, the electronic device 101 maintains the display of spatial image data as indicated by the cube 301 while increasing the size of the viewfinder 300. In some examples, the electronic device 101 increases the size of the viewfinder 300 to completely occupy the size of the display 120, as discussed in further detail below.

[0094] Figure 4B An example is illustrated where a user of electronic device 101 and a second electronic device 160 are in a second three-dimensional environment 800, while electronic device 101 displays the second three-dimensional environment 800. When electronic device 101 optionally displays the second three-dimensional environment 800, electronic device 101 optionally displays spatial image data received from the second electronic device 160 (e.g., corresponding to the three-dimensional environment 700) at viewfinder 300. In some examples, spatial image data is displayed while the second electronic device 160 displays an image of the three-dimensional environment 700. In some examples, electronic device 101 increases the size of viewfinder 300 in response to input directed at electronic device 101 (as discussed above). In some examples, such as... Figure 4B As shown, electronic device 101 detects input to pointing button 115 provided by hand 103, and in response, initiates an increase in the size of viewfinder 300, as indicated by the arrow at the corner of viewfinder 300. In some examples, when electronic device 101 detects input (e.g., hand 103), second electronic device 160 maintains the display of an image of the three-dimensional environment 700.

[0095] Figure 4C An example is illustrated where electronic device 101 displays a viewfinder 300 via display 120, the viewfinder covering a second three-dimensional environment 800 relative to the user's field of view (e.g., a spatial image is shown in a fully immersive manner, such that the spatial image occupies the user's entire viewport), while a second electronic device 160 transmits the spatial image to electronic device 101. In some examples, with Figure 4A In contrast, the representations of person 310 and tree 320 are displayed at the same positioning intervals. In some examples, electronic device 101 receives commands from second electronic device 160 to display the spatial image in a fully immersive manner (e.g., viewfinder 300), as referenced above. Figure 4A The subject of discussion. In some examples, such as Figure 4C As shown, the electronic device 101 continues to display the three-dimensional environment 700 at the viewfinder 300 as a spatial image as indicated by the cube 301. In some examples, the viewfinder 300 displays the three-dimensional environment 700 from the same perspective as the second electronic device 160 displays the three-dimensional environment 700 at the display 164. In some examples, the electronic device 101 modifies the display of the viewfinder 300 in several different ways, as discussed in further detail below.

[0096] Figure 4D An example is illustrated of a second electronic device 160 that displays an image of a three-dimensional environment 700 (e.g., captured spatial image data) and detects input provided by hand 103 at button 163. Electronic device 101 continues to display the image discussed above (e.g., a second three-dimensional environment 800, viewfinder 300), while the second electronic device 160 displays an image of the three-dimensional environment 700 and detects input. In some examples, electronic device 101 returns the viewfinder 300 in response to user input at electronic device 101 and / or the second electronic device 160 (not shown). Figure 4A The display size and configuration status. In some examples, the user of the electronic device additionally provides input pointing to button 163 (e.g., hand 103), and in response, a second electronic device 160 sends a command to electronic device 101 to update the position of viewfinder 300 in display 120. In some examples, such as Figure 4D As shown, electronic device 101 receives a command to update the positioning of viewfinder 300 (e.g., input provided by hand 103) and begins to move viewfinder 300 as indicated by the arrow attached to viewfinder 300. In some examples, the input includes the user's intention to move viewfinder 300 in the direction (not shown) within display 120. For example, as... Figure 4D As shown, in response to receiving input from the second electronic device 160, the electronic device 101 begins to move the viewfinder downward relative to the viewpoint of the user of the electronic device 101.

[0097] Figure 4E Examples are illustrated where electronic device 101 continues to display the images discussed above (e.g., the second 3D environment 800, viewfinder 300) and the updated positioning of viewfinder 300, while second electronic device 160 displays an image of the 3D environment 700. In some examples, electronic device 101 responds to the above references Figure 4D The input discussed will position the viewfinder from the upper center of the display 120 (in... Figure 4D (As shown in the image) The viewfinder 300 is updated to the lower right corner of the display 120. In some examples, the updated positioning of the viewfinder 300 makes the window 801 fully visible in the three-dimensional environment 700 relative to the user's viewpoint of the electronic device 101. It should be understood that the positioning of the viewfinder 300 is not limited to... Figure 4E The illustrated positioning, and can be referenced above. Figure 4DThe input discussed may optionally be located at any position on display 120. In some examples, the second electronic device 160 continues to send spatial image data to electronic device 101 and display the three-dimensional environment 700. In some examples, as indicated by cube 301, the electronic device continues to display the spatial image data (e.g., the three-dimensional environment 700) as a spatial image as described in the accompanying figures above. In some examples, electronic device 101 updates the display of the spatial image data (e.g., the three-dimensional environment 700) based on additional input sent by the second electronic device 160, as discussed in further detail below.

[0098] In some examples, the user of electronic device 101 can also manipulate various aspects of the display of the captured spatial image data. For example, as discussed in further detail below, the user can begin playback of the spatial image data and optionally wish to utilize, for example... Figure 4F and Figure 4G The illustrated image control user interface begins replaying spatial image data at various points in time.

[0099] Figure 4F Examples are shown where electronic device 101 continues to display the images discussed above (e.g., second 3D environment 800, viewfinder 300), while second electronic device 160 displays an image of 3D environment 700 and an image control user interface 168. In some examples, in Figure 4F Previously, the second electronic device 160 detected the pointing controls 361 to 365 (provided by the hand 103) Figure 3I (Example) Input from one of the elements, and in response, display an image control user interface 168 at display 164 that partially covers the three-dimensional environment 700. This image control user interface 168 (e.g., a scroll bar) can optionally be configured to modify the time point at which the captured spatial image data is displayed, as previously mentioned, such as updating the captured spatial image data from a first time point to a second time point. In some examples, such as... Figure 4F As shown, the second electronic device 160 displays the image control user interface 168 as a bar (including a dark bar indicating a time point in the spatial image) in the lower portion of the display 164, which is configured to receive input provided by the hand 103. In some examples, the input provided by the hand 103 corresponds to a swipe gesture (exemplified by the right arrow) pointing to the dark bar at the image control user interface 168, which triggers the second electronic device 160 to send a command to the electronic device 101 to update the spatial image data (viewed in the viewfinder 300) from a first time point to a second time point, as discussed in further detail below.

[0100] Figure 4G Examples are shown as follows Figure 4FAn alternative example illustrating the updating of spatial image data from a first time point to a second time point is provided, wherein electronic device 101 detects input provided by hand 103 at button 115 rather than at image control user interface 168. In some examples, electronic device 101 and second electronic device 160 respond to the input provided by hand 103 in accordance with the above... Figure 4F Similar approaches are discussed. In some examples, electronic device 101 sends input provided by hand 103 based on the magnitude of input detected by button 115 (e.g., a scrolling gesture at button 115), and in response, a second electronic device initiates an update of spatial image data from a first time point to a second time point, as described below. Figure 4H exemplified.

[0101] Figure 4H Examples are shown where electronic device 101 continues to display what has been discussed above (e.g., second 3D environment 800, viewfinder 300), while second electronic device 160 displays an updated image of the 3D environment 700 (e.g., spatial image data) and an image control user interface 168. In some examples, based on input provided by hand 103, second electronic device 160 positions the display of the dark bar at image control user interface 168 from the left (see...). Figure 4F Update the positioning to the right, thus indicating that the spatial image data being displayed is at a second time point. In some examples, such as... Figure 4H As shown, the second electronic device 160 is based on the above... Figure 4F The direction of the input provided by the illustrated hand 103 updates the display 164 to show a dark bar at a left-hand positioning at the image control user interface 168. In some examples, the second electronic device 160 detects that the input provided by the hand 103 is at a positioning at the image control user interface 168 corresponding to a second time point, and in response, sends spatial image data (e.g., 3D environment 700) at the second time point to the electronic device 101. In some examples, such as Figure 4H As shown, the electronic device receives spatial image data at a second time point, and updates the viewfinder 300 to display the spatial image data at the second time point when displaying the second three-dimensional environment 800. In some examples, such as Figure 4H As shown, the updated spatial image data at viewfinder 300 includes the updated spatial location of person 710 (e.g., a representation of person 310), displayed as a jumping person.

[0102] Figure 5 A flowchart illustrating a method 500 for displaying spatial image data obtained from a second electronic device 160, according to some examples of this disclosure, is shown. This method may optionally be referenced above. Figures 1 to 4HThis method 500 may be executed at any of the described electronic devices. In some examples, executing method 500 includes executing instructions stored using a non-transitory computer-readable storage medium at an electronic device having one or more processors. Some operations in method 500 may be optionally combined, and / or the order of some operations may be optionally changed.

[0103] In some examples, at box 502, according to method 500, a first electronic device (e.g., electronic device 101) communicates with a first external camera (e.g., first external camera 171) and a second external camera (e.g., second external camera 170), while the first and second external cameras are respectively capturing first image data (e.g., first image data 171a) and second image data (e.g., second image data 170a). In some examples, while the first and second external cameras are capturing the first and second image data, as discussed above, reference is made to at least... Figures 3A to 4H The first electronic device further communicates with one or more displays corresponding to display 120. In some examples, a first external camera is positioned to view the 3D environment 700 from a first viewpoint. In some examples, a second external camera is positioned to view the 3D environment 700 from a second viewpoint different from the first viewpoint. In some examples, the first and second external cameras are located at the second electronic device 160, as referenced above. Figures 3A to 4H The subject of discussion.

[0104] In some examples, at block 504, according to method 500, and according to some examples of this disclosure, a second electronic device 160 may optionally be used to obtain first image data (e.g., first image data 171a) and second image data (e.g., second image data 170a). In some examples, the second electronic device 160 obtains the first image data and the second image data, and optionally sends the first image data and the second image data to a first electronic device (e.g., electronic device 101) to obtain spatial image data based on the first image data and the second image data. In some examples, the second electronic device 160 obtains spatial image data based on the first image data and the second image data.

[0105] In some examples, at 506, method 500 determines that one or more first criteria are met, and in response, displays a spatial image at a first electronic device according to some examples in this disclosure. In some examples, the first electronic device (e.g., electronic device 101) determines that the one or more first criteria are met. In some examples, a second electronic device 160 determines that the one or more first criteria are met. In some examples, the one or more first criteria are met when the second electronic device 160 is in landscape mode, as referenced above. Figure 3GAs discussed above at box 504, the second electronic device 160 may optionally obtain spatial image data based on the first image data and the second image data. In this example, when the first electronic device detects the transmission of spatial image data from the second electronic device 160, it may optionally satisfy one or more of the first criteria.

[0106] It should be understood that method 500 is an example, and more, fewer, or different operations may be performed in the same or different order. Additionally, the operations in method 500 described above may optionally be performed by running an information processing device such as a general-purpose processor (e.g., as per [reference to...]). Figures 2A to 2B (as described) or one or more functional modules in a dedicated chip and / or through Figures 2A to 2B It is implemented using other components.

[0107] Now let’s turn our attention to electronic devices (e.g., corresponding to…) Figure 1 The interaction involves capturing and displaying spatial images in a three-dimensional environment at an electronic device 101. In some examples, spatial images of one or more physical objects in the physical environment are captured by a standalone camera (e.g., a consumer electronics camera or other mirrorless camera, or an external camera communicating with the electronic device and including multiple cameras (e.g., a stereo camera pair)) and transmitted for presentation at the electronic device 101. Due to the nature of the display typically included in the standalone camera (e.g., a mirrorless camera) and / or the absence of a display, standalone cameras are generally unable to present spatial images using their corresponding two-dimensional displays (e.g., without a depth component). The depth component of the spatial image is most often achieved by using two or more displays (such as the display provided at the electronic device 101). In the following examples, various configurations for presenting previews and capturing spatial images within a portion of the user's field of view on a head-mounted display (e.g., electronic device 101) are presented.

[0108] Figures 6A to 6J Examples of capturing spatial images in a three-dimensional environment at an electronic device communicating with a stand-alone camera, according to some examples of this disclosure, are illustrated.

[0109] Figure 6A An example is illustrated whereby electronic device 101 presents a three-dimensional environment 650 from a first viewpoint while communicating with a standalone camera 660 (e.g., a mirrorless camera or other consumer electronics camera), the three-dimensional environment including a physical environment 600 (e.g., referenced above). Figures 3A to 3K Multiple objects corresponding to physical objects within the physical environment under discussion. In some examples, such as Figure 6AAs shown, electronic device 101 and independent camera 660 are in wireless communication 661 and / or wired communication 662. In some examples, wireless communication 661 may optionally correspond to Bluetooth connectivity, cellular broadband, Wi-Fi, or radio connectivity. In some examples, both electronic device 101 and independent camera 660 include one or more outward-facing cameras that view a common physical scene (e.g., portions of physical environment 600). In some examples, electronic device 101 displays and / or presents on display 120 (e.g., in a 3D environment 650) the outward-facing cameras of electronic device 101 (e.g., corresponding to...). Figure 6A The external image sensors 114b and 114c) capture portions of the physical scene. Similarly, the standalone camera 660 optionally displays portions of the physical scene captured by one or more outward-facing cameras of the standalone camera 660 on a display 664, such as... Figure 6A As shown.

[0110] In some examples, such as top view 601, an outward-facing camera positioned at a standalone camera 660 includes an outward-projecting field of view 671, the direction of which corresponds to... Figure 6A The field of view 670 of the electronic device 101 is the same. In some examples, the fields of view 671 and 670 overlap depending on the orientation of the independent camera 660, but are not necessarily the same fields of view, as discussed in further detail below.

[0111] In some examples, such as Figure 6A As shown, the standalone camera 660 includes a button 663 (e.g., a hardware button or element) optionally configured to receive user input, such as a click or press. In some examples, the standalone camera 660 includes a display 664 configured to display a representation (e.g., a two-dimensional representation) of the three-dimensional environment 600 presented at the electronic device 101 from a second viewpoint different from a first viewpoint of the electronic device 101 (e.g., where the difference in viewpoint is due to the fact that the camera of the electronic device 101 and the standalone camera 660 are in different locations and / or orientations relative to the common physical scene from which the camera captures image data). Additionally, in some examples, the standalone camera 660 includes a viewfinder 666.

[0112] In some examples, such as Figure 6AAs shown, the representation of the physical environment 600 displayed on display 664 includes a representation of a person 610b and a representation of a tree 620b. In some examples, the aforementioned representations correspond to physical objects within the physical environment 600, which are included in the three-dimensional environment 600, as discussed in further detail below. In some examples, a standalone camera 660 obtains the representation of the physical environment 600 from one or more external lenses (not shown) of the standalone camera 660. In some examples, the representation of the physical environment 600 corresponds to a real-time video feed of the physical environment 600, which may optionally be updated based on updates in the physical environment 600 and / or updates in a second viewpoint of the standalone camera 660, as discussed in further detail below. In some examples, such as Figure 6A As shown, the standalone camera 660 displays a representation of the physical environment 600 in the same or similar manner as the electronic device 101 displays the 3D environment 650. In some examples, such as Figure 6A As shown, display 664 also displays information 665 (e.g., overlaid on and / or beneath the representation of person 610b and tree 620b). For example, as Figure 6A As illustrated, the display 664 includes image capture information and / or camera operation information, such as the battery level indication of the standalone camera 660, aperture settings, focus modes, shutter speeds, brightness settings, etc.

[0113] In some examples, such as Figure 6A As shown, electronic device 101 is displaying or presenting multiple objects (as discussed above) within the field of view of electronic device 101 in a three-dimensional environment 600, including a person 610 and a tree 620. For example, as Figure 6A As shown, the 3D environment 650 includes a representation (e.g., a computer-generated representation or a transparent representation) of a person 610 centrally positioned within the field of view of electronic device 101, and a representation of a tree 620 positioned from a first viewpoint of electronic device 101 in the left portion of the field of view of electronic device 101. In some examples, these multiple physical objects correspond to representations of the person 610b and the tree 620b displayed on a display 664 of a separate camera 660. However, as... Figure 6A As illustrated and described in more detail below, due to the different viewpoints of electronic device 101 and independent camera 660, the view of the person 610 and tree 620 in the three-dimensional environment 600 at electronic device 101 may optionally differ from the view of the representation of person 610b and tree 620b on display 664 at independent camera 660. For example, as Figure 6A As shown, display 120 may optionally include all or the entire portion of tree 620, while display 664 of stand-alone camera 660 includes portions of tree 620 that have been cropped or cut off, as indicated by the representation of tree 620b.

[0114] In some examples, such as Figure 6A As shown in the top view 601, the user 602 of the electronic device 101 is directly facing the person 610 in the physical environment 600. For example, as shown in the top view 601, the person 610 (e.g., a representation of person 610 in the three-dimensional environment 650 and / or person 610b on the display 664) is located at the center of the field of view 670 of the electronic device 101. In some examples, the field of view 670 corresponds to the three-dimensional environment 650 at the display 120. In some examples, as shown in the top view 601, the tree 620 is located further away from the user 602 and / or the electronic device 101 in the physical environment 600. In some examples, as shown in the top view 601, the tree 620 is located further away from the user 602 relative to the position of the user 602 and / or the electronic device 101 in the physical environment 600. Figure 6A As illustrated, user 602 positions (e.g., holds) standalone camera 660 such that standalone camera 660 is not visible in the field of view of electronic device 101 and is therefore not included in the three-dimensional environment 650 (e.g., standalone camera 660 is held low enough to be outside the field of view of electronic device 101).

[0115] In some examples, as discussed above, the display 664 of the standalone camera 660 is a two-dimensional display (lacking multiple displays), and therefore cannot display the image data captured by the one or more cameras of the standalone camera 660 as a spatial image. Therefore, in some examples, when the standalone camera 660 communicates with the electronic device 101, the standalone camera 660 sends image data to the electronic device 101 for display by the display 120 of the electronic device 101 (e.g., the display 120 including the virtual viewfinder 615), as discussed in further detail below.

[0116] In some examples, such as Figure 6A As shown, electronic device 101 is displaying a virtual viewfinder 615 in a three-dimensional environment 650 via display 120. For example, as... Figure 6AAs illustrated, electronic device 101 is displaying a virtual viewfinder (e.g., as a virtual window or similar user interface element displayed in a head-lock orientation) that overlays portions of the physical environment 600 that are visible and / or represented in the three-dimensional environment 650. Specifically, in some examples, as described above, a standalone camera 660 sends image data captured by one or more of its cameras to electronic device 101, which uses this image data to generate a virtual viewfinder 615 and displays it on display 120. In some examples, electronic device 101 displays the virtual viewfinder 615 in response to user input (such as launching a specific application associated with the standalone camera 660 on electronic device 101) detected by electronic device 101 (e.g., previously) while electronic device 101 is displaying the three-dimensional environment 650. In some examples, electronic device 101 displays a virtual viewfinder 615 on display 120 (e.g., automatically) in response to the detection of the establishment of communication (e.g., wireless communication 661 or wired communication 662) between electronic device 101 and independent camera 660.

[0117] In some examples, as discussed above, the standalone camera 660 includes only a two-dimensional display (e.g., display 664) and therefore cannot include depth information associated with image data captured by one or more cameras of the standalone camera 660. To display the depth information associated with the image data, multiple displays may be required, such as those provided at the electronic device 101 (e.g., including display 120). Therefore, in response to the capture of image data corresponding to the physical environment 600 by the one or more cameras of the standalone camera 660, the standalone camera 660 sends image data including spatial image data to the electronic device 101, which is used to generate and display the image data. Figure 6A The virtual viewfinder 615 in the image. In some examples, the virtual viewfinder 615 includes and / or corresponds to a spatial image of (e.g., two-dimensional) image displayed on the display 664 of the standalone camera 660. For example, in Figure 6A In this context, the virtual viewfinder 615 includes and / or corresponds to an image that includes depth information. In some examples, such as... Figure 6A As shown, the virtual viewfinder 615 includes a representation of the physical environment 600 captured from the perspective of a standalone camera 660 (e.g., the second viewpoint discussed above). For example, as Figure 6A As shown, the virtual viewfinder 615 includes representations of person 610a and tree 620a corresponding to person 610 and tree 620 in the physical environment 600. In some examples, the spatial image data at the virtual viewfinder 615 corresponds to a real-time video feed of the physical environment 600 (e.g., captured by one or more cameras of a standalone camera 660). For example, as... Figure 6AAs indicated, the image provided in the virtual viewfinder 615 is the same as or similar to the image displayed on the monitor 664 of the standalone camera 660. Therefore, in some examples, as an advantage, the virtual viewfinder 615 displayed at the electronic device 101 provides a virtual representation of the physical environment 600, which can be viewed from a second viewpoint of the standalone camera 660 via its physical viewfinder 666, without requiring the user to physically view it through the physical viewfinder 666. This improves the user interaction and operation of the standalone camera 660. Additionally, in some examples, such as... Figure 6A As illustrated, the virtual viewfinder 615 includes or displays information 617, which corresponds to information 665 displayed on or configured to be displayed on the display of the independent camera 660. It should be noted that... Figure 6A The position of the virtual viewfinder 615 illustrated is not necessarily limited to what is shown in the following figures, and should be considered as an example of the possible positions of the virtual viewfinder 615 on the display 120.

[0118] In some examples, when the virtual viewfinder 615 is displayed on the monitor 120 of the electronic device 101, the monitor 664 of the standalone camera 660 may be optionally turned off or set to a low-power mode or state. For example, as Figure 6B As shown, when the standalone camera 660 is communicating with the electronic device 101 and when the electronic device 101 is displaying the virtual viewfinder 615, the display 664 does not display the image illustrated in the virtual viewfinder 615. In some examples, the standalone camera 660 remains powered on even though the display 664 is powered off or set to a low-power mode or state (e.g., to enable the standalone camera 660 to continue receiving user input, such as user input for capturing one or more images, as discussed in more detail below). In some examples, the standalone camera 660 turns off the display 664 or powers it off in response to receiving data or other instructions or commands from the electronic device 101 for turning off the display 664 or powering it off. As an advantage, when a virtual viewfinder 615 is provided on the display 120 of the electronic device 101, the power-off of the display 664 by the standalone camera 660 helps to save power and battery life of the standalone camera and / or helps to avoid the repeated display of information (e.g., particularly the image provided in the virtual viewfinder 615) that could otherwise distract the user 602 when using the electronic device 101 and / or the standalone camera 660.

[0119] In some examples, user 602 can provide user input to a standalone camera 660 for capturing one or more images, such as spatial images, while using a virtual viewfinder 615 presented at electronic device 101 as visual guidance within a three-dimensional environment 650. For example, as Figure 6BAs illustrated, while the electronic device 101 is displaying the virtual viewfinder 615, the independent camera 660 detects input corresponding to a request to capture a first spatial image, such as a selection (e.g., pressing or pushing) of the capture button 663 provided by the hand 603 of the user 602.

[0120] Additionally or alternatively, in some examples, user 602 may provide user input to electronic device 101 for capturing one or more images, such as spatial images, while using a virtual viewfinder 615 presented at electronic device 101 as visual guidance within the three-dimensional environment 650. For example, as Figure 6C As shown, when electronic device 101 is displaying virtual viewfinder 615, electronic device 101 detects input corresponding to a request to capture a first spatial image. In some examples, this input includes selection of hardware components or buttons on electronic device 101 (e.g., pressing or pushing). For example, in Figure 6C In this process, electronic device 101 detects a selection made by user 602's hand 603a to a hardware button 115 of electronic device 101, the selection corresponding to a request for independent camera 660 to capture a spatial image corresponding to the view provided in virtual viewfinder 615. Alternatively, this input may include selection of selectable options (e.g., virtual buttons or elements) displayed in the three-dimensional environment 650 along with virtual viewfinder 615. For example, as... Figure 6C As shown, electronic device 101 detects the selection of virtual capture option 618 displayed in the three-dimensional environment 650 together with virtual viewfinder 615 (e.g., superimposed on the virtual viewfinder), such as an air pinch gesture performed by user 602's hand 603b, optionally while user 602's gaze 626 is directed at virtual capture option 618 in the three-dimensional environment 650.

[0121] In some examples, in response to the above description by Figure 6B or Figure 6C User 602 provides user input, and standalone camera 660 initiates the capture of image data corresponding to a spatial image (e.g., the first spatial image discussed above). In some examples, while capturing image data corresponding to the spatial image, and / or during this process, the standalone camera displays a visual indication (e.g., on display 664) indicating that standalone camera 660 is capturing image data, as previously stated in [the previous text]. Figure 3C The recording instruction 166 is shown by displaying it at the second electronic device 160. In some examples, when and / or after image data is captured by the independent camera 660, the electronic device 101, such as via... Figure 6D Wireless communication 661 and / or wired communication 662 receive image data (e.g., transmitted by independent camera 660) from independent camera 660.

[0122] In some examples, such as Figure 6D As shown, in response to receiving image data from the independent camera 660, the electronic device 101 (e.g., automatically) displays a spatial image 622 corresponding to the image data captured by the independent camera 660 and / or uses the image data to display the spatial image. In some examples, such as Figure 6D As shown, electronic device 101 displays a spatial image 622 on display 120 instead of viewfinder 615. For example, in Figure 6D In this process, the electronic device 101 stops displaying the virtual viewfinder 615 in the three-dimensional environment 650 while displaying the spatial image 622. In some examples, such as Figure 6D As shown, electronic device 101 displays spatial image 622 at a predetermined location on display 120 (such as a location corresponding to the center of the field of view of electronic device 101). Additionally or alternatively, in some examples, electronic device 101 displays spatial image 622 in response to an instruction received from independent camera 660 to preview (e.g., display) spatial image 622 on electronic device 101. For example, as... Figure 6D As shown, the standalone camera 660 has detected a selection (e.g., a push or press) of the play button 667 provided by the user 602's hand 603, causing the standalone camera 660 to display image 622a (e.g., corresponding to spatial image 622) on the display 664 and send data or other instructions to the electronic device 101 for displaying spatial image 622 in the three-dimensional environment 650. In some examples, such as Figure 6D As illustrated, when input provided by user 602 is detected, spatial image 622 corresponds to Figures 6B to 6C The view of the illustrated virtual viewfinder 615. For example, in Figure 6D In this context, the spatial image 622 includes representations of people 610a and trees 620a, which were previously included in a virtual viewfinder 615 in the three-dimensional environment 650 (e.g., according to a second viewpoint of a standalone camera 660 as discussed above).

[0123] In some examples, such as Figure 6D As shown, when a spatial image 622 is displayed in a three-dimensional environment 650, the electronic device 101 applies a visual effect 609 to at least a portion of the three-dimensional environment 650 surrounding the spatial image 622. For example, as Figure 6DAs instructed, electronic device 101 displays and / or applies darkening, coloring, blurring, or other visual processing to the transparency of the physical environment 600 (e.g., including a person 610 and / or a tree 620) included in and surrounding the spatial image 622 within the three-dimensional environment 650, such that the visual salience of the transparency of the physical environment 600 is reduced relative to the spatial image 622. In some examples, as a benefit, electronic device 101 applies visual effect 609 to at least a portion of the three-dimensional environment 650 to draw the attention of user 602 to the spatial image 622 being displayed in the three-dimensional environment 650, and / or to promote clear and focused visibility of the content of the spatial image 622 to user 602.

[0124] In some examples, the focus associated with capturing one or more spatial images at the independent camera 660 (e.g., the focus within the corresponding image being captured) can be controlled and / or adjusted based on input detected by the electronic device 101. Specifically, in some examples, the focus of the independent camera 660 can be adjusted in response to gaze-based input directed at the virtual viewfinder 615 in the three-dimensional environment 650. For example, in Figure 6E In this process, electronic device 101 detects that user 602's gaze 626 is directed to a corresponding location within the virtual viewfinder 615 in the three-dimensional environment 650, such as the center of the virtual viewfinder 615 in the three-dimensional environment 650. In some examples, electronic device 101 detects that user 602's gaze 626 is directed to the corresponding location within the virtual viewfinder 615 for at least a threshold amount of time, such as 1 second, 1.5 seconds, 2 seconds, 3 seconds, 4 seconds, 5 seconds, 10 seconds, etc.

[0125] In some examples, in response to detecting a gaze 626 from user 602 pointing to a corresponding location within the virtual viewfinder 615, electronic device 101 sends data or other instructions to standalone camera 660, causing standalone camera 660 to adjust its lens based on the gaze-based input to update its focus. For example, in Figure 6E In this process, electronic device 101 (e.g., via wireless communication 661 or wired communication 662) sends instructions to standalone camera 660 to adjust the camera lens, thereby adjusting the focus of standalone camera 660 to the center point of the field of view of standalone camera 660, as illustrated by focus 668 on display 664.

[0126] In some examples, movement of the independent camera 660 relative to the first viewpoint of the electronic device 101 allows the electronic device 101 to selectively update the display of the virtual viewfinder 615 in the three-dimensional environment 650. For example, in Figure 6FIn this context, electronic device 101 detects an indication of movement of a standalone camera 660 (e.g., in physical environment 600) relative to a first viewpoint of electronic device 101, such movement being in a direction toward the first viewpoint of electronic device 101 and / or toward the field of view of electronic device 101, as indicated by arrow 669. In some examples, the indication of detecting movement of standalone camera 660 includes detecting data from standalone camera 660 (e.g., transmitted via wireless communication 661 or wired communication 662) that informs electronic device 101 of the movement of standalone camera 660. In some examples, the indication of detecting movement of standalone camera 660 includes detecting, via one or more input devices (e.g., image sensors, cameras, or other motion sensors) communicating with electronic device 101, that standalone camera 660 is physically moving relative to the first viewpoint of electronic device 101. As previously discussed similarly above and as... Figure 6F As illustrated, electronic device 101 detects an indication of movement of independent camera 660, while independent camera 660 is outside the field of view of electronic device 101.

[0127] In some examples, such as Figure 6G As illustrated, in response to and / or upon detection of an indication that the independent camera 660 has moved relative to a first viewpoint of the electronic device 101, the electronic device 101 updates the display of the virtual viewfinder 615 in the three-dimensional environment 650. Specifically, as Figure 6G As illustrated, the movement of the stand-alone camera 660 within the physical environment 600 (e.g., by the hand 603 of the user 602 holding the stand-alone camera 660) causes a change in the second viewpoint of the stand-alone camera 660. For example, in Figure 6G In this context, the independent camera 660 has an updated second viewpoint (e.g., a third viewpoint) in the physical environment 600, such that the view of the physical environment 600 changes according to the updated positioning of the independent camera 660. Therefore, in some examples, image data sent to the electronic device 101 is updated according to the updated second viewpoint of the independent camera 660 in the physical environment 600, causing the spatial image displayed in the virtual viewfinder 615 to be updated accordingly in the three-dimensional environment 650 at the electronic device 101. For example, as... Figure 6G As shown, based on updated image data received from the independent camera 660, the positions of the representations of person 610a and tree 620a are updated (e.g., shifted) within the virtual viewfinder 615 in the three-dimensional environment 650. This updated image data corresponds to the updated second viewpoint of the independent camera 660 and the image data currently being captured by the independent camera 660.

[0128] In addition to updating the spatial image within the virtual viewfinder 615 in the three-dimensional environment 650 when an indication of movement of the independent camera 660 is detected, or as an alternative, in some examples, the electronic device 101 completely stops displaying the virtual viewfinder 615 in the three-dimensional environment 650 when one or more criteria are met based on the indication that the independent camera 660 has moved. In some examples, the one or more criteria for causing the electronic device 101 to stop displaying the virtual viewfinder in the three-dimensional environment 650 include a criterion that is met when movement of the independent camera 660 makes the display 664 visible from the first viewpoint of the electronic device 101 in the field of view of the electronic device 101. Figure 6G In this case, the display 664 is not currently within the field of view of the electronic device 101, making the display 664 invisible from the first viewpoint of the electronic device 101 in the three-dimensional environment 650; therefore, one or more criteria are not met, and the electronic device 101 maintains the display of the virtual viewfinder in the three-dimensional environment 650.

[0129] In some examples, the one or more criteria for stopping the display of a virtual viewfinder 615 in the three-dimensional environment 650 include criteria satisfied when movement of the independent camera 660 causes the physical viewfinder 666 of the independent camera to become visible from the first viewpoint of the electronic device 101 within the field of view of the electronic device 101. In some examples, the one or more criteria for stopping the display of a virtual viewfinder 615 in the three-dimensional environment 650 include criteria satisfied when movement of the independent camera 660 causes the hand 603 of the user 602 holding the independent camera 660 to become visible from the first viewpoint of the electronic device 101 within the field of view of the electronic device 101. For example, as... Figure 6G As shown, the hand 603 holding the independent camera 660 is not visible in the field of view of the electronic device 101 in the three-dimensional environment 650; therefore, the one or more criteria are not met. Additionally or alternatively, in some examples, the one or more criteria include those satisfied when movement of the independent camera 660 causes the physical viewfinder 666 of the independent camera to be within a threshold distance of the first viewpoint of the electronic device 101 and / or within a threshold distance of the electronic device 101 (e.g., within 0.15 m, 0.25 m, 0.5 m, 0.75 m, 1 m, 1.5 m, 2 m, etc.). In some examples, the one or more criteria for causing the electronic device 101 to stop displaying a virtual viewfinder in the three-dimensional environment 650 include those satisfied when movement of the independent camera 660 causes the independent camera 660 to be within a threshold distance of the first viewpoint of the electronic device 101 (e.g., 0.15 m, 0.25 m, 0.5 m, 0.75 m, 1 m, 1.5 m, 2 m, etc.) and / or within a threshold distance of the electronic device 101. For example, such as Figure 6GAs illustrated in the top view 601, the electronic device 101 determines, for example, based on visual detection by the independent camera 660 in one or more images captured by external image sensors 114b and 114c, the strength of the wireless signal transmitted between the electronic device 101 and the independent camera 660, and / or other indications detected by the electronic device 101 and / or the independent camera 660, whether the independent camera 660 and / or the independent camera 660's physical viewfinder 666 is outside a threshold distance 612 of the electronic device 101 (e.g., and / or the first viewpoint of the electronic device 101). Therefore, as... Figure 6G As illustrated in the example, electronic device 101 maintains the display of virtual viewfinder 615 in three-dimensional environment 650 based on the determination that one or more criteria are not met.

[0130] exist Figure 6G In the context of displaying the virtual viewfinder 615 in the 3D environment 650, the electronic device 101 detects an indication of further movement of the independent camera 660 relative to a first viewpoint of the electronic device 101 within the physical environment 600. For example, as discussed similarly above, in... Figure 6G In this process, electronic device 101 detects that user 602 of electronic device 101 is moving further toward the first viewpoint of electronic device 101 and / or toward the field of view of electronic device 101 (e.g., using hand 603) independent camera 660, as indicated by arrow 669.

[0131] In some examples, Figure 6H In response to and / or upon detecting an indication that the independent camera 660 has moved relative to the first viewpoint of the electronic device 101, the electronic device 101 updates the display of the virtual viewfinder 615 in the three-dimensional environment 650. Specifically, as Figure 6H As illustrated, electronic device 101 determines that the movement of stand-alone camera 660 in physical environment 600 (e.g., by the hand 603 of user 602 holding stand-alone camera 660) causes one or more criteria for stopping electronic device 101 from displaying a virtual viewfinder in three-dimensional environment 650 to be met. Therefore, as Figure 6H As shown, the electronic device 101 stops displaying the virtual viewfinder 615 in the three-dimensional environment 650.

[0132] In some examples, electronic device 101 determines that one or more criteria are met because the movement of independent camera 660 causes the display 664 of independent camera 660 to move and / or become detectable within the field of view of electronic device 101. For example, as Figure 6HAs shown, the three-dimensional environment 650 includes a display 664 of a standalone camera 660 visible from a first viewpoint of the electronic device 101, which satisfies one or more criteria. In some examples, the electronic device 101 determines that one or more criteria are satisfied because movement of the standalone camera 660 causes the physical viewfinder 666 of the standalone camera 660 to move and / or become detectable within the field of view of the electronic device 101. In some examples, the electronic device 101 determines that one or more criteria are satisfied because movement of the standalone camera 660 causes the physical viewfinder 666 and / or the standalone camera 660 to be within a threshold distance of the first viewpoint of the electronic device 101 and / or within a threshold distance of the electronic device 101. For example, as Figure 6H As shown in top view 601, the standalone camera 660 is positioned within a threshold distance 612 of the electronic device 101, which allows one or more criteria to be met.

[0133] Additionally, in some examples, when the electronic device 101 stops displaying the virtual viewfinder 615 in the three-dimensional environment 650, the standalone camera 660 may optionally power on its display 664, which is visible from the first viewpoint of the electronic device 101 in the three-dimensional environment 650. Specifically, in some examples, as previously discussed above, if the display 664 of the standalone camera 660 is de-powered and / or operating at a low power level, while the virtual viewfinder 615 is displayed in the three-dimensional environment 650, the electronic device 101 sends an instruction or other command (e.g., via wireless communication 661 or wired communication 662) to the standalone camera 660, which causes the standalone camera 660 to power on the display 664. For example, in Figure 6HWhen electronic device 101 determines that one or more of the above criteria are met and stops displaying the virtual viewfinder 615 in the three-dimensional environment 650, electronic device 101 causes independent camera 660 to power on display 664, such that the image displayed on display 664 (e.g., including a representation of person 610b and a representation of tree 620b) corresponds to a view of the physical environment 600 that can be viewed from the current (e.g., updated) viewpoint of independent camera 660 and / or via the physical viewfinder 666 of independent camera 660 and / or display 664. In some examples, if display 664 is powered on when electronic device 101 stops displaying the virtual viewfinder 615 in the three-dimensional environment 650, independent camera 660 waives the ability to perform any operation in response to an instruction or other command received from electronic device 101 to power on display 664 of independent camera 660. Therefore, as a benefit, since the content of the virtual viewfinder 615 is otherwise provided to the user 602 via the display 664 of the independent camera 660 (e.g., visible to the user), stopping the display of the virtual viewfinder 615 in the three-dimensional environment 650 (e.g., in response to determining that one or more criteria are met) avoids the repeated display of information to the user 602 that could obstruct or distract the visibility of the display 664 of the independent camera 660 and / or the physical viewfinder 666, thereby improving user-device interaction.

[0134] In some examples, as an alternative to stopping the display of the virtual viewfinder 615 in the three-dimensional environment 650 based on determining that one or more of the criteria discussed above are met, the electronic device 101 updates the display of the virtual viewfinder 615 in the three-dimensional environment 650 in a manner that maintains the visibility of the display 664 of the independent camera 660 and / or the physical viewfinder 666 from the first viewpoint of the electronic device 101. For example, as Figure 6I As shown, electronic device 101 minimizes the display of virtual viewfinder 615 in three-dimensional environment 650. In some examples, such as Figure 6I As shown, minimizing the display of the virtual viewfinder 615 includes updating the size of the virtual viewfinder 615 displayed in the three-dimensional environment 650, such as reducing the size and / or scale of the virtual viewfinder 615 on the display 120. In some examples, such as Figure 6I As shown, minimizing the display of the virtual viewfinder 615 includes updating the position of the virtual viewfinder 615 displayed in the 3D environment 650, such as moving and / or repositioning the virtual viewfinder 615 to the edge or corner of the display 120. In some examples, such as Figure 6IAs shown, minimizing the display of the virtual viewfinder 615 includes updating the content and / or information displayed in the 3D environment 650 along with the virtual viewfinder 615, such as stopping the display of information 617 on the display 120 that is displayed with (e.g., overlaid on) the virtual viewfinder 615. In this way, the display of the virtual viewfinder 615 is maintained in the 3D environment 650 while maintaining the visibility of the display 664 of the standalone camera 660 and / or the physical viewfinder 666 in the 3D environment 650 from the first viewpoint of the electronic device 101.

[0135] It should be understood that, in the above text Figures 6H to 6I In the illustrated example, after movement of the independent camera 660 satisfying one or more of the criteria discussed above is detected, and the display of the virtual viewfinder 615 in the three-dimensional environment 650 has been updated (e.g., the virtual viewfinder 615 stops displaying or the virtual viewfinder 615 is minimized on the display 120), subsequent movement of the independent camera 660 causes one or more criteria to no longer be satisfied, causing the electronic device 101 to restore the previous display of the virtual viewfinder 615 in the three-dimensional environment 650. For example, in Figure 6H In the event that the virtual viewfinder 615 is not displayed in the three-dimensional environment 650, if the electronic device 101 detects an indication of further movement of the independent camera 660 relative to the first viewpoint of the electronic device 101, such further movement causing one or more criteria to no longer be met, then the electronic device 101 redisplays the virtual viewfinder 615 in the three-dimensional environment 650. Figure 6G As shown similarly in [the text]. For example, in [the text]... Figure 6I In the event that the virtual viewfinder 615 is minimized on the display 120, if the electronic device 101 detects an indication of further movement of the independent camera 660 relative to the first viewpoint of the electronic device 101, such further movement causing one or more criteria to no longer be met, the electronic device 101 restores the display of the virtual viewfinder 615 in the three-dimensional environment 650 (e.g., inverts its minimized state), as... Figure 6G As shown similarly in the middle.

[0136] In some examples, multiple independent cameras (e.g., a multi-camera system, workstation, or similar setup) are able to communicate with electronic device 101, enabling multiple virtual viewfinders to be provided on display 120 of electronic device 101 corresponding to the fields of view of the multiple independent cameras. For example, as Figure 6JAs shown, electronic device 101 (e.g., concurrently) communicates with a first independent camera 660a and a second independent camera 660b. In some examples, the first independent camera 660a and the second independent camera 660b have one or more characteristics of the aforementioned independent camera 660. In some examples, the first independent camera 660a is different from the second independent camera 660b. For example, the first independent camera 660a and the second independent camera 660b are different types of consumer electronic cameras, such as mirrorless cameras of different brands, models, generations, with different components and / or accessories. In some examples, the first independent camera 660a and the second independent camera 660b are the same type of camera.

[0137] In some examples, such as Figure 6J As shown, the first independent camera 660a communicates wirelessly 661a or via wired communication 662a with the electronic device 101, and the second independent camera 660b communicates wirelessly 661b or via wired communication 662b with the electronic device 101. Additionally, as... Figure 6J As shown, the first independent camera 660a includes a first display 664a and a first capture button 663a, and the second independent camera 660b includes a second display 664b and a second capture button 664b. Figure 6J In this context, the first display 664a of the first independent camera 660a is displaying a digital representation of the current view of the first independent camera 660a, and the second display 664b of the second independent camera 660b is displaying a digital representation of the current view of the second independent camera 660b of the physical environment 600. For example, as... Figure 6J As shown, the first display 664a and the second display 664b include representations of a person 610b and a tree 620b, but from unique viewpoints of the first independent camera 660a and the second independent camera 660b, respectively. Specifically, as Figure 6J As illustrated in the top view 601, both the person 610 and the tree 620 are located within the first field of view 671a of the first independent camera 660a and the second field of view 671b of the second independent camera 660b. It should be understood that, although... Figure 6J The top view 601 illustrates a first independent camera 660a and a second independent camera 660b being held (e.g., in the hand of user 602) by user 602 of electronic device 101, but the first independent camera 660a and / or the second independent camera 660b are alternatively positioned in a tripod or other bracket-based arrangement in physical environment 600.

[0138] In some examples, such as Figure 6J As shown, the electronic device 101 is configured to provide a virtual viewfinder for each of the first independent camera 660a and the second independent camera 660b. For example, as Figure 6JAs shown, electronic device 101 is (e.g., concurrently) displaying in a three-dimensional environment 650 a first virtual viewfinder 615a associated with a view of the physical environment 600 from a first independent camera 660a, and a second virtual viewfinder 615b associated with a view of the physical environment 600 from a second independent camera 660b. In some examples, as similarly described above, the first virtual viewfinder 615a includes a first spatial image corresponding to the view of the physical environment 600 being captured by the first independent camera 660a, and the second virtual viewfinder 615b includes a second spatial image corresponding to the view of the physical environment 600 being captured by the second independent camera 660b. For example, as... Figure 6J As shown, the first virtual viewfinder 615a includes a representation of a person 610a and a representation of a tree 620a corresponding to a digital image displayed on the first display 664a of the first independent camera 660a, and the second virtual viewfinder 615b includes a representation of a person 610a and a representation of a tree 620a corresponding to a digital image displayed on the second display 664b of the second independent camera 660b. In some examples, such as Figure 6J As shown, a first virtual viewfinder 615a and a second virtual viewfinder 615b are displayed at predetermined locations on the display 120. For example, the first virtual viewfinder 615a and the second virtual viewfinder 615b are displayed as overlays on a capture portion of the physical environment 600 (e.g., captured by external image sensors 114b and 114c), which includes the three-dimensional environment 650 and / or is visible in the three-dimensional environment from a first viewpoint of the electronic device 101. In some examples, the first virtual viewfinder 615a and / or the second virtual viewfinder 615 have one or more of the characteristics of the virtual viewfinder 615 described above.

[0139] In some examples, when the first virtual viewfinder 615a and the second virtual viewfinder 615b are displayed in the three-dimensional environment 650, the user can interact with the standalone cameras 660a and 660b in a manner similar to that described above with reference to the standalone camera 660. For example, in Figure 6J In this context, user 602 is able to: capture spatial images that are viewable (e.g., and / or previewable) in the three-dimensional environment 650 (e.g., via selecting capture buttons 663a and 663b), such as Figures 6B to 6D Similarly, as shown in the diagram; the focus of the first independent camera 660a and / or the second independent camera 660b is adjusted (e.g., via gaze-based input, which is detected as pointing to positions within the first virtual viewfinder 660a and / or the second virtual viewfinder 660b in the 3D environment 650, respectively), as... Figure 6ESimilarly, as shown in the diagram; adjusting and / or altering the display of the spatial image presented within the first virtual viewfinder 615a and / or the second virtual viewfinder 615b (e.g., via movement of the first independent camera 660a and / or the second independent camera 660b, which alters the corresponding viewpoint of the first independent camera 660a and / or the second independent camera 660b in the physical environment 600), such as Figures 6F to 6G Similarly, as shown in the diagram; and / or updating the display of the first virtual viewfinder 615a and / or the second virtual viewfinder 615b in the three-dimensional environment 650 (e.g., stopping the display and / or minimizing the display) (e.g., via movement of the first independent camera 660a and / or the second independent camera 660b satisfying one or more of the criteria previously described herein), as Figures 6H to 6I As similarly illustrated in the diagram. Therefore, it should be understood that, as described above, one or more of the operations performed by the electronic device 101 and / or the independent camera 660 are similarly and / or correspondingly applied to... Figure 6J In the examples, the first independent camera 660a and / or the second independent camera 660b (e.g., and / or an additional or alternative consumer electronic camera (e.g., a mirrorless camera) communicating with electronic device 101). Additionally, it should be understood that in some examples, one or more interactions of the interactions previously described above with reference to electronic device 101 and the second electronic device 160 (e.g., Figures 3A to 4H (As illustrated) This is similarly and / or correspondingly applied to the standalone camera 660.

[0140] Therefore, as described above, providing a virtual viewfinder in a three-dimensional environment, comprising a spatial image corresponding to the view of the physical environment of a standalone camera at the electronic device, allows the user of the electronic device to more easily and effectively capture and save spatial images from the standalone camera without relying on the limited display capabilities of the standalone camera, thereby improving user-device interaction. Additionally, as another benefit, providing a preview of the captured spatial image in the three-dimensional environment at the electronic device (e.g., automatically) after the standalone camera captures an image in response to user input provides the user with immediate visual feedback that the spatial image has been captured and / or reduces the amount of input required to preview the spatial image captured by the standalone camera.

[0141] Figure 7 This is a flowchart illustrating an example method for displaying an updated spatial image in a three-dimensional environment captured by a standalone camera, according to some examples of this disclosure. In some examples, process 702 begins with an electronic device communicating with one or more displays, one or more input devices, a first external camera having a first viewpoint, and a second external camera having a second viewpoint different from the first viewpoint. In some examples, the electronic device may optionally be similar to or corresponding to... Figure 2AThe electronic device 201 is a head-mounted display or similar to or corresponding to Figure 2B 260 mobile electronic devices. For example... Figure 7 As shown, in some examples, at 704, when a first external camera is capturing first image data and a second external camera is capturing second image data, the electronic device obtains the first image data from the first external camera and the second image data from the second external camera. For example, as referenced... Figure 6A As described, electronic device 101 is capturing image data of physical environment 600 from a first viewpoint using external image sensors 114b and 114c, and independent camera 660 is capturing image data of physical environment 600 from a second viewpoint.

[0142] In some examples, at point 706, the electronic device displays a spatial image based on first and second image data in a three-dimensional environment via one or more displays. For example, such as Figure 6A As shown, electronic device 101 is displaying a virtual viewfinder 615 in a three-dimensional environment 650, which overlays portions of the physical environment 600 visible and / or represented from a first viewpoint of electronic device 101 within the three-dimensional environment 650. In some examples, at 708, when displaying a spatial image in the three-dimensional environment, the electronic device detects a second external camera in the field of view of the first external camera in the three-dimensional environment via the one or more input devices or via the first external camera. For example, as referenced... Figures 6F to 6G The electronic device 101 detects at least partially the movement of the independent camera 660 within the field of view of the electronic device 101 in the three-dimensional environment 650.

[0143] In some examples, at 710, in response to the detection of a second external camera in the field of view of the first external camera, at 712, based on the determination that one or more criteria are met, the electronic device stops displaying a spatial image in the three-dimensional environment via the one or more displays. For example, as Figure 6H As shown, the electronic device 101 stops displaying the virtual viewfinder 615 in the three-dimensional environment 650. In some examples, such as reference... Figures 6G to 6H As described, the one or more criteria include those satisfied when the display 664 of the standalone camera 660 is visible in the field of view of the electronic device 101 in the three-dimensional environment 650. In some examples, the one or more criteria include those satisfied when the standalone camera 660 moves to within a threshold distance 612 (e.g., illustrated in top view 601) of a first viewpoint of the electronic device 101, such as... Figure 6H exemplified. Referenced above Figures 6G to 6H Additional examples of this one or more standards are described.

[0144] It should be understood that process 702 is an example, and more, fewer, or different operations may be performed in the same or different order. Additionally, the operations in method 702 described above may optionally be performed by running an information processing device such as a general-purpose processor (e.g., as per [reference to...]). Figures 2A to 2B (as described) or one or more functional modules in a dedicated chip and / or through Figures 2A to 2B It is implemented using other components.

[0145] In some examples, a first electronic device (optionally wearable) communicates with one or more displays and with a first external camera having a first viewpoint and a second external camera having a second viewpoint different from the first viewpoint, such as Figure 3B The illustrated electronic device 101, first external camera 171, and second external camera 170. In some examples, the first electronic device is a head-mounted display or device (HMD) and / or a body-worn display or device, a mobile device (e.g., a tablet, smartphone, media player, or wearable device) that optionally communicates with one or more of headphones and / or earbuds, including wireless communication circuitry. These headphones and / or earbuds optionally include one or more cameras and / or inertial measurement units (IMUs), (e.g., external) mice, (optionally integrated or external) touchpads, (optionally integrated or external) remote control devices, another mobile device (e.g., independent of the first electronic device), (e.g., external) handheld devices, and / or (e.g., external) controllers. In some examples, the first electronic device includes one or more sensors configured to detect the positioning and / or orientation of the first electronic device. For example, the first electronic device may optionally include multiple orientation sensors (e.g., accelerometers, gyroscopes, magnetometers, inertial measurement units (IMUs), tilt sensors / inclinometers, optical sensors, electromechanical gyroscopes, fiber optic gyroscopes (FOGs), ring laser gyroscopes (RLGs), and / or microelectromechanical systems (MEMS) gyroscopes) configured to optionally detect changes in the orientation of a user of the wearable device, such as the user tilting their head upwards. In some examples, the first electronic device includes one or more display generation components, which are displays integrated with the first electronic device (optionally touchscreen displays), external displays (such as monitors, projectors, televisions), or (optionally integrated or external) hardware components for projecting a user interface or making the user interface visible to a user of the system. In some examples, the one or more display generation components include display generation components configured to view a three-dimensional environment. For example, the display generation components may optionally be configured to be semi-transparent, thereby allowing a user of the system (e.g., the first electronic device) to view a real-world environment (e.g., a three-dimensional environment).

[0146] In some examples, the first electronic device corresponds to a head-mounted display, such as wearable glasses. For example, the first electronic device may optionally be included within the housing of wearable reading glasses, wherein the one or more displays may optionally be disposed in the upper portion of the lenses of the reading glasses, thereby allowing a user to simultaneously view a three-dimensional environment and content displayed by the one or more displays. In some examples, a first external camera and / or a second external camera are included on a mobile device (e.g., a mobile phone or other mobile computing device, such as a tablet and / or laptop computer) configured to communicate with other first electronic devices (e.g., the first electronic device). In some examples, the first and second external cameras are part of a common electronic device (e.g., the second electronic device discussed below), and each camera is positioned at a different location on the electronic device such that the viewpoint of each camera is different from each other when viewing the three-dimensional environment. In some examples, the first external camera and / or the second external camera each correspond to a camera configured to detect / record image data of the three-dimensional environment, which is discussed in further detail below. In some examples, the first external camera is disposed within the three-dimensional environment such that the first external camera captures the three-dimensional environment from a first viewpoint.

[0147] In some examples, a second external camera is positioned within the three-dimensional environment, such that the second external camera captures the three-dimensional environment from a second viewpoint different from the first viewpoint. In some examples, the first and second external cameras are included in auxiliary electronic equipment communicating with a first electronic equipment. For example, the first external camera may be optionally positioned on a first side (e.g., the display side) of the second electronic equipment, such as a mobile device, and the second external camera may be optionally positioned on a second side (e.g., the back) of the mobile device. In this configuration, the first and second external cameras are positioned at locations where each camera records different viewpoints (e.g., a first viewpoint, a second viewpoint) of the three-dimensional environment. In some examples, the first viewpoint / second viewpoint each includes a predetermined field of view corresponding to the camera associated with the first external camera. For example, the first external camera may optionally include a 180-degree field of view configured to optionally capture the three-dimensional environment from the front side of the auxiliary electronic equipment. On the rear side of the auxiliary electronic equipment, the second external camera may optionally include a 180-degree field of view optionally configured to capture the three-dimensional environment. In some examples, the first electronic device is communicatively coupled to a first external camera and / or a second external camera via a wired connection (e.g., a High Definition Multimedia Interface (HDMI) cable, an auxiliary cable). In some examples, the first electronic device is communicatively coupled to the first external camera and / or the second external camera via a wireless connection (e.g., Wi-Fi, Bluetooth). For example, the first electronic device may optionally send a connection request via a local and / or global Wi-Fi network. When the request is sent, the first external camera and / or the second external camera may optionally detect the request and automatically accept the connection request. In some examples, the first external camera is capturing first image data (e.g., one or more images from a first viewpoint, one or more videos from a first viewpoint), and the second external camera is capturing second image data (e.g., one or more images from a second viewpoint, one or more videos from a second viewpoint) (at box 502).

[0148] In some examples, the first external camera and the second external camera simultaneously capture first image data and second image data. In some examples, the first external camera and the second external camera capture first image data and second image data during different time periods. For example, the first external camera may optionally capture first image data at a first time, such as immediately after communication is optionally established with the first electronic device, and the second external camera may optionally capture second image data at a second time, such as after a time threshold is reached after communication is optionally established with the first electronic device. In some examples, in response to detecting established communication between the first electronic device, the first external camera, and the second external camera, the camera begins to capture first image data and second image data, respectively. In some examples, the first external camera and / or the second external camera respond to user input at the first external device and / or the corresponding external camera (such as by hand 103 at...). Figure 3CThe first electronic device captures first image data and / or second image data by means of input provided at button 163 shown. For example, the first electronic device may optionally detect user touch input at the one or more displays and may optionally send a command to a first external camera and / or a second external camera to optionally begin capturing first image data and / or second image data. In another example, the user touch input may optionally be directed to the first external camera and / or the second external camera. In response to the corresponding external camera receiving the user touch input, the corresponding external camera may optionally send a command to the other external camera to begin capturing the corresponding image data. In some examples, the first external camera and / or the second external camera capture first image data and second image data corresponding to images / videos of the three-dimensional environment in which the first electronic device resides. In some examples, the first external camera and the second external camera capture first image data / second image data within a predetermined time period (e.g., set by communication from the first electronic device). In some examples, the first external camera and / or the second external camera capture first image data / second image data until the first electronic device sends a communication to stop capturing. In some examples, the first electronic device acquires first image data from a first external camera, second image data from a second external camera, or spatial image data generated based on the first and second image data (at box 504). In some examples, the first and / or second image data correspond to image and / or video data configured to be viewable by a user of the first electronic device, as discussed in further detail below. For example, the first and / or second external cameras may optionally record video (e.g., first and second image data) of the environment (e.g., a three-dimensional environment) corresponding to the location of the first electronic device. In some examples, the first external camera includes a high-resolution sensor and a wide-angle lens. The external camera may optionally be mounted on a gimbal and may optionally rotate to optionally capture the entire scene of the three-dimensional environment while continuously capturing spatial data (e.g., first and / or second image data). By optionally using image stitching and panorama techniques, the external camera combines multiple images into a spatial image map. In some examples, the second external camera is configured to supplement the first external camera by capturing depth perception and fine details of the scene. The external camera optionally utilizes a telephoto lens to capture high-detail images of a scene in a three-dimensional environment from different perspectives. The external camera can optionally utilize depth sensing techniques (e.g., light detection and ranging (LIDAR), stereo imaging) to optionally calculate distances between objects in the scene. This information is optionally added by the first external camera to the captured scene to optionally create a depth layer of a spatial map (e.g., spatial image data).

[0149] In some examples, the first image data and / or the second image data each include depth data. In some examples, the first image data and / or the second image data are received at a respective external camera and simultaneously transmitted to a second electronic device. In some examples, the first image data and / or the second image data are received and processed at the respective external camera before being transmitted to the first electronic device. In some examples, the first electronic device obtains the first image data and / or the second image data from a first external camera and / or a second external camera via wireless communication, as described similarly above, such as... Figure 3C The wireless communication 161 and / or wired communication 162 are shown. In some examples, the first electronic device combines the acquired first image data and second image data to generate spatial image data to be displayed on the one or more displays (discussed in further detail below) (504). For example, the spatial image may optionally be a pair of two slightly different images (e.g., first image data, second image data) that, when viewed together (e.g., at the first electronic device), optionally create a depth illusion in the spatial image data. In some examples, the spatial image data includes depth data of the first image data and / or the second image data.

[0150] In some examples, the first electronic device utilizes multiple spatial video processing algorithms to combine first image data and second image data to obtain spatial image data (at box 504). In some examples, the spatial image data corresponds to a 3D model of a 3D environment (e.g., a real-world environment including the first electronic device). In some examples, the first electronic device combines first image data from a first viewpoint and second image data from a second viewpoint based on a combination of a first viewpoint and a second viewpoint to obtain spatial image data. For example, the first electronic device optionally acquires first image data from a first external camera, which optionally includes a field of view relative to a user of the first electronic device that encapsulates a left portion of the 3D environment (e.g., the first viewpoint). During this process, the first electronic device optionally acquires second image data from a second external camera, which optionally includes a field of view relative to a user of the first electronic device that encapsulates a right portion of the 3D environment (e.g., the second viewpoint). Using the combination of the left portion (e.g., the first image data) and the right portion (e.g., the second image data) of the 3D environment, the first electronic device generates spatial image data optionally corresponding to a field of view including the left and right portions of the 3D environment. In some examples, a first electronic device displays (506) a spatial image based on first image data and second image data, or spatial image data generated based on the first image data and second image data, in a three-dimensional environment via the one or more displays, based on the determination that one or more first criteria are met. In some examples, the one or more first criteria are met when the first electronic device acquires the first image data and / or the second image data. In some examples, the one or more first criteria are met according to one or more characteristics discussed in further detail below. In some examples, the first electronic device determines that the one or more first criteria are met via communication from a first external camera and / or a second external camera. For example, the first electronic device may optionally receive an error transmission from the first external camera and / or the second external camera, the error transmission optionally indicating a failure to capture the first image data and / or the second image data. In some examples, the electronic device does not display the spatial image if the one or more first criteria are not met. In some examples, the spatial image is displayed on a first display among the one or more displays. In some examples, the spatial image is displayed on multiple displays among the one or more displays. In some examples, the one or more displays include a display configured to display a three-dimensional environment, a display configured to display the first image data, and a display configured to display the second image data. In some examples, one or more displays are configured to simultaneously display first image data, second image data, and a 3D environment.

[0151] In some examples, a first external camera and a second external camera are included in a second electronic device that communicates with a first electronic device (at box 502). In some examples, the second electronic device includes one or more features of the auxiliary electronic devices discussed above. In some examples, after the first external camera and / or the second external camera acquires first image data and / or second image data, the second electronic device stores the first image data and / or second image data before transmitting the corresponding image data to the first electronic device (at box 504). In some examples, the second electronic device corresponds to a mobile device such as those discussed above. In some examples, the first external camera and the second external camera are each located at different positions on the second electronic device. In some examples, the first external camera and the second external camera view a common scene in a three-dimensional environment from their respective different positions on the second electronic device. In some examples, the first external camera and the second external camera view the common scene from different perspectives (e.g., a first viewpoint, a second viewpoint). In some examples, the first external camera, the second external camera, and the second electronic device communicate concurrently with the first electronic device.

[0152] In some examples, the second electronic device generates spatial image data based on the first image data and the second image data, and transmits the spatial image data to the first electronic device. In some examples, the second electronic device generates spatial image data based on the first image data / second image data in a manner similar to that discussed above. In some examples, the second electronic device continuously acquires the first image data and the second image data over a time period, and continuously updates the spatial image data in response. For example, a first external camera and a second external camera may optionally acquire first image data and second image data of a public scene during a first time period, the public scene optionally including objects at a first location in a three-dimensional environment. During this time period, the second electronic device may optionally generate spatial image data based on the first image data and the second image data. At a second time period, the first external camera and the second external camera may optionally acquire first image data and second image data of the public scene, the public scene including objects at a second location in a three-dimensional environment, and in response, the second electronic device updates the spatial image from including objects at the first location to including objects at the second location based on the first image data and the second image data from the second time period. In some examples, the second electronic device transmits the spatial image data to the first electronic device via one or more wired and / or wireless methods as discussed above. In some examples, the second electronic device processes first image data and second image data to generate spatial image data in response to user input instructing a corresponding device to generate spatial image data. For example, when a first external camera and a second external camera optionally capture first image data and second image data, the user optionally selects the first electronic device to display the spatial image data. In response to this input, the second electronic device automatically begins combining the acquired first image data and second image data to generate spatial image data, and subsequently sends the spatial image data to the first electronic device. In some examples, the first electronic device automatically displays the spatial image data in response to receiving a transmission from the second electronic device including at least spatial image data. For example, the second electronic device optionally combines first image data and second image data to generate spatial image data, and optionally sends the spatial image data to the first electronic device, including a command to display the spatial image data at one or more displays of the first electronic device.

[0153] In some examples, the first electronic device obtains first image data and second image data from the second electronic device, and the first electronic device generates spatial image data based on the first image data and second image data. In some examples, the second electronic device (discussed above) sends the first image data and second image data to the first electronic device after obtaining the corresponding image data. In some examples, the first electronic device generates spatial image data in a manner similar to that discussed above regarding the second electronic device generating spatial image data. In some examples, the first electronic device obtains the first image data and / or second image data while the first external camera and / or the second external camera is capturing the first image data and / or second image data. In some examples, the first electronic device obtains the first image data and / or second image data after the first external camera and / or the second external camera have captured their respective image data. In some examples, the first electronic device generates spatial image data after the first external camera and / or the second external camera has stopped capturing the first image data and / or second image data, and sends the corresponding image data to the first electronic device.

[0154] In some examples, the second electronic device includes a display different from the one or more displays, which can be configured to display two-dimensional image data on the display of the second electronic device when a first external camera is capturing first image data and a second external camera is capturing second image data, such as... Figure 3DThe display 164 is shown. In some examples, the display at the second electronic device includes at least one or more of the features of the one or more displays discussed above. In some examples, the display corresponds to the display at the mobile device. In some examples, in response to the first external camera and the second external camera capturing first image data / second image data, the second electronic device compiles the first image data / second image data to create two-dimensional image data (e.g., generating a two-dimensional image based on the first image data and the second image data). In some examples, the two-dimensional image data includes at least one or more features of the first image data and the second image data. For example, the two-dimensional image data may optionally include a first viewpoint associated with the first external camera and a second viewpoint associated with the second external camera. In some examples, the two-dimensional image data corresponds to a real-time video feed of a three-dimensional environment. In some examples, the two-dimensional image data corresponds to a still image of a three-dimensional environment. In some examples, the two-dimensional image data is continuously updated to correspond to a combination of the currently captured first image data and the second image data. In some examples, the second electronic display shows the 3D environment before the first external camera and / or the second external camera captures the 3D environment, and in response to the first external camera and / or the second external camera starting to capture first image data and / or second image data, the second electronic device sends the display of the 3D environment to the one or more displays of the first electronic device. In some examples, in response to detecting user input (e.g., button press) at the second electronic device and / or the first electronic device, the second electronic device sends the display of the 3D environment to the one or more displays of the first electronic device. In some examples, in response to the first electronic device connecting to the second electronic device, the second electronic device automatically sends the display of the 3D environment to the one or more displays of the first electronic device. In some examples, after sending the display of the 3D environment from the second electronic device to the first electronic device, the second electronic device updates its display to include a blurred or static image of the 3D environment, and fills the display with one or more controls at a control user interface configured to control the first external camera and / or the second external camera. In some examples, the one or more controls correspond to one or more physical controls (e.g., buttons, switches) at the second electronic device.

[0155] In some examples, when spatial image data is displayed in a three-dimensional environment via a display, the two-dimensional image data has an appearance different from the first or second image data. In some examples, the two-dimensional image data is displayed as a blurred image of the first and / or second image data to indicate that image data generated by the camera of the second device is being displayed on the display of the first electronic device. In some examples, in response to acquiring the first and / or second image data, the second electronic device updates the display from a representation of the three-dimensional environment to the aforementioned blurred image of the first and / or second image data. In some examples, the appearance of the two-dimensional image data indicates that the second electronic device is receiving the first and / or second image data. For example, the display at the second electronic device may optionally display a user interface including multiple applications, and in response to optionally acquiring the first and / or second image data, may optionally display a representation of the three-dimensional environment (e.g., the two-dimensional image data) with a darker appearance compared to the first and / or second image data. In some examples, the appearance of the two-dimensional image data includes a user interface with one or more icons indicating that the second electronic device is acquiring the first and / or second image data. For example, the appearance of two-dimensional image data may optionally include a glowing recording icon indicating that a second electronic device is acquiring first image data and / or second image data, such as Figure 3K The record shown is indicated by 166. In some examples, the second electronic device displays the appearance of the two-dimensional image data as a static image of the first image data and / or the second image data. In some examples, the appearance of the two-dimensional image data is displayed as blank on the display of the second electronic device. In some examples, the appearance of the two-dimensional image data includes a colored outer portion encapsulating the two-dimensional image data.

[0156] In some examples, when spatial image data is displayed in a three-dimensional environment via a display, the two-dimensional image data includes a single-view representation of the first image data or the second image data, or a stereoscopic representation of the first image data and the second image data. In some examples, the second electronic device simultaneously displays two-dimensional image data having both a single-view representation of the first image data / second image data and a stereoscopic representation of the first image data / second image data. In some examples, the single-view representation of the first image data / second image data corresponds to a planar image visible only from one viewpoint without providing depth perception or 3D effects. In some examples, the stereoscopic representation of the first image / second image corresponds to a pair of two slightly different images (e.g., first image data and second image data), one of which is configured to be viewed by each eye of the user of the first electronic device, producing depth illusion and 3D perception when viewed together (e.g., spatial image data). In some examples, the stereoscopic representation includes one or more characteristics of the spatial image data discussed above. In some examples, the two-dimensional image data includes a first portion corresponding to a single-view representation of the first image data or second image data displayed at the second electronic device, and a second portion corresponding to a stereoscopic representation of the first image data or second image data displayed at the first electronic device. In some examples, the second electronic device determines a corresponding representation (e.g., single-view or stereoscopic) of the first or second image data based on user input. For example, the user may optionally input a preferred representation at the second electronic device before displaying the two-dimensional image data.

[0157] In some examples, the one or more first criteria include criteria satisfied when the first electronic device receives an instruction from an external electronic device to display spatial image data generated based on first image data and second image data in a three-dimensional environment. In some examples, the first electronic device receives the instruction from the external electronic device via wireless and / or wired communication. In some examples, the external electronic device corresponds to a second electronic device. In some examples, the external electronic device corresponds to a first external camera and / or a second electronic camera. In some examples, the first electronic device displays the instruction as a visual instruction at the one or more displays that can be configured to receive user input. For example, the one or more displays may optionally display the instruction as a visual instruction that optionally includes a power indication that initiates the display of spatial image data at the one or more displays upon detection of user input. In some examples, the first electronic device displays the instruction at the one or more displays while displaying spatial image data at the one or more displays. In some examples, the first electronic device receives the instruction while the first external camera and / or the second external camera is capturing first image data and / or second image data. In some examples, the criteria are satisfied after the user of the first electronic device interacts with the instruction. For example, the first electronic device may optionally receive user input at the indication, and in response, the first electronic device displays spatial image data on the one or more displays.

[0158] In some examples, spatial image data includes visual depth information. In some examples, visual depth information includes one or more characteristics of the depth layer of the spatial map discussed above with reference to first image data obtained from a first external camera and second image data obtained from a second external camera. In some examples, the first external camera and / or the second external camera captures visual depth data associated with the first image data and / or the second image data before combining the first and second image data to generate spatial image data. In some examples, visual depth refers to three-dimensional perception achieved by simulating the natural depth perception of human vision by presenting two slightly different images (e.g., first image data and second image data) to each eye of a user of a first electronic device. The variations between the images seen by each of the user's eyes enable the brain to interpret spatial relationships and distances of objects, thus creating a depth illusion (e.g., visual depth information).

[0159] In some examples, the first image data and the second image data correspond to video data, and the spatial image data corresponds to spatial video. This spatial image data corresponding to the spatial video can be displayed by viewfinder 300. In some examples, when spatial video is displayed at viewfinder 300, this can optionally be referred to as displaying a spatial image. In some examples, the video data corresponds to a compilation of one or more captured first image data and / or second image data. In some examples, the spatial video includes one or more characteristics of the first image data and / or second image data as discussed above. For example, the video data optionally includes depth information that can optionally be perceived by a user of the first electronic device. In some examples, the video data includes one or more characteristics of a video of a three-dimensional environment as discussed above. In some examples, the spatial video includes depth and spatial information (similar to the spatial image data discussed above), thereby allowing for the representation of a three-dimensional environment and objects. Compared to conventional two-dimensional video, spatial video optionally includes data defining the location, orientation, and movement of objects within a three-dimensional environment.

[0160] In some examples, the first image data includes a plurality of first pixels, and the second image data includes a plurality of second pixels, wherein the display spatial image data includes: in some examples, the plurality of first pixels corresponds to a two-dimensional array of pixels configured to display the first image data. In some examples, the plurality of second pixels corresponds to a two-dimensional array of pixels configured to display the second image data. In some examples, the first electronic device displays the plurality of first pixels and / or the plurality of second pixels, as discussed in further detail below. In some examples, the first electronic device applies a pixel matching process to the first image data and the second image data before displaying the spatial image. In some examples, the first electronic device determines one or more matching pixels between the plurality of first pixels and the plurality of second pixels. In some examples, the pixel matching process corresponds to a machine learning algorithm configured to detect associated pixels between the acquired first image data and the second image data. For example, the acquired first image data and the acquired second image data may optionally include representations of a chair in a three-dimensional environment from a first viewpoint and a second viewpoint, respectively. After capturing the representation of the chair, the plurality of first pixels and the plurality of second pixels may optionally include a plurality of pixels associated with the representation of the chair. Using a pixel matching process, the first electronic device may optionally identify a plurality of pixels associated with a representation of a chair from a first plurality of pixels from a first viewpoint and a second plurality of pixels from a second viewpoint. In some examples, the spatial image data includes one or more matching pixels between the plurality of first pixels and the plurality of second pixels. In some examples, the first electronic device applies the pixel matching process to the first image data and / or the second image data when a first external camera and / or a second external camera captures first image data and / or second image data.

[0161] In some examples, the one or more first criteria include a criterion satisfied when the stereo disparity between the first image data and the second image data is below a first threshold. In some examples, the stereo disparity corresponds to the difference in the localization of objects in the first image data and the second image data due to different camera (e.g., a first external camera, a second external camera) viewpoints. This difference may optionally produce a parallax effect that can be used to perceive depth and three-dimensional structure in the spatial image data. In some examples, stereo disparity corresponds to the difference between a first viewpoint and a second viewpoint as discussed above. For example, a first electronic device may optionally detect an object in the first image data that can be viewed from a first angle (e.g., a first viewpoint) and may optionally detect an object in the second image data that can be viewed from a second angle (e.g., a second viewpoint). When it is determined that the first angle and the second angle are below the first threshold, the first electronic device may optionally combine the first image data and the second image data to generate spatial image data. In some examples, based on the determination that the stereo disparity does not meet the criteria, the first electronic device displays a non-spatial image via the one or more displays based on the first image data and the second image data or the spatial image data. In some examples, the viewpoint of the first image data and the viewpoint of the second image data (e.g., stereo parallax) are greater than a first threshold, and in response, the first electronic device combines the first image data and the second image data in a manner that excludes depth data (as described above). In some examples, the non-spatial image includes one or more characteristics of the spatial image discussed above. In some examples, the non-spatial image includes one or more characteristics of the two-dimensional image data discussed above. In some examples, when the first external camera and / or the second external camera acquires the first image data and / or the second image data, the first electronic device determines that the spatial parallax does not meet the criteria. In some examples, the non-spatial image is displayed on the second electronic device.

[0162] In some examples, the one or more first criteria include a criterion that is met when the focal length difference between the first image data and the second image data is less than a first threshold. In some examples, the focal length difference corresponds to the difference in focal length between the cameras used to capture the first and second image data. The focal length of the corresponding camera lens (e.g., a first external camera, a second external camera) determines the angle of view and magnification of the image, thereby essentially controlling how "magnified" or "reduced" the 3D environment appears. When comparing (e.g., a pixel-matching process) the first and second image data, the focal length difference may mean that one photograph was taken using a wide-angle lens, capturing more of the 3D environment, while another photograph may have been taken using a longer focal length, focusing on a narrower area and providing more detail for specific elements. This parallax can affect the visual characteristics of the photograph, such as depth of field, sense of space, and the relative size of objects within the frame. In some examples, the first electronic device detects the first focal length associated with the first external camera and the second focal length associated with the second external camera, and calculates the focal length difference based on the first and second focal lengths. In some examples, the second electronic device calculates the focal length difference based on the first and second focal lengths. In some examples, the first threshold is a predetermined threshold generated by the first electronic device and / or the second electronic device. In some examples, the user of the first electronic device determines the first threshold before the first external camera and / or the second external camera acquires the first image data and / or the second image data. In some examples, based on the determination that the focal length difference does not meet the criterion, the first electronic device displays a non-spatial image via the one or more displays based on the first image data and the second image data or spatial image data. In some examples, the first electronic device and / or the second electronic device determines that the focal length difference is higher than the first threshold, thus failing to meet the criterion, and in response, abandons the display of spatial image data (as discussed above) and displays non-spatial image data. In some examples, the first electronic device and / or the second electronic device determines that the criterion is not met when the first external camera and / or the second external camera acquires the first image data and / or the second image data. In some examples, the non-spatial image data includes one or more characteristics of two-dimensional image data as discussed above. In some examples, in response to the focal length difference not meeting the criterion, the first electronic device displays a non-spatial image and / or video based on the first image data, the second image data, or the spatial image data. In some examples, the first electronic device displays first image data, second image data, or spatial image data until the focal length difference no longer meets a criterion. For example, a first external camera and a second external camera may optionally capture first image data and second image data at a first time point when the focal length difference between them is below a first threshold. During this time period, the first electronic device may optionally display the first image data and / or the second image data.At a second time, the first external camera and the second external camera capture first image data and second image data, at which point the focal length difference between them exceeds a first threshold. During this time, the first electronic device stops displaying the first image data and / or the second image data and generates a non-spatial image display.

[0163] In some examples, a first external camera and a second external camera are included in a second electronic device that communicates with a first electronic device. In some examples, the one or more first criteria include criteria satisfied based on determining that the second electronic device is in a first orientation. In some examples, based on determining that the second electronic device is in a second orientation different from the first orientation, the first electronic device displays a visual indication of the orientation of the second electronic device via the one or more displays. In some examples, the second electronic device includes one or more characteristics of the second electronic device discussed above. In some examples, the second electronic device includes one or more sensors configured to detect the positioning and / or orientation of the second electronic device. For example, the second electronic device may optionally include multiple orientation sensors (e.g., accelerometers, gyroscopes, magnetometers, inertial measurement units (IMUs), tilt sensors / inclinometers, optical sensors, electromechanical gyroscopes, fiber optic gyroscopes (FOGs), ring laser gyroscopes (RLGs), and / or MEMS gyroscopes) configured to optionally detect changes in the orientation of the second electronic device, such as a user tilting the second electronic device upwards. In some examples, the multiple orientation sensors discussed above detect changes in the orientation of the device. For example, the user of the device may optionally change the viewpoint of the second electronic device from facing directly forward in the three-dimensional environment (e.g., the first orientation) to facing left (e.g., the second orientation). In some examples, the first orientation corresponds to the orientation of the second electronic device in a Cartesian coordinate system (e.g., the XYZ coordinate system). For example, the first orientation may optionally correspond to an orientation along the x-axis and y-axis (e.g., facing forward). In some examples, the first orientation corresponds to a range of acceptable orientations that meet criteria. For example, the first orientation may optionally include an orientation range from 0 degrees to 89 degrees relative to the horizontal line of the three-dimensional environment. If the second electronic device optionally detects an orientation outside the orientation range (e.g., 90 degrees), the first electronic device may optionally display a visual indication. In some examples, the second electronic device determines that the detected orientation is outside the orientation range discussed above and, in response, sends a command to the first electronic device to display a visual indication. In some examples, the second electronic device records the orientation and sends the orientation data to the first electronic device. If the first electronic device determines that the orientation data corresponds to an orientation outside the orientation range (as discussed above), the first electronic device displays a visual indication at the one or more displays. In some examples, the one or more displays continue to display first image data and / or second image data when they detect a change in orientation of the second electronic device to a second orientation. In some examples, the one or more display generating components stop displaying first image data and / or second image data in response to detecting a change in orientation to a second orientation, and do not display a visual indication of orientation.In some examples, in response to a detected change in orientation, the one or more displays cease displaying first image data and / or second image data while maintaining the display of the three-dimensional environment. In some examples, the first electronic device displays a visual indication covering at least a portion of the first image data and / or second image data. In some examples, the visual indication corresponds to a visual warning, thereby indicating an incorrect orientation used to capture the first image data and / or second image data. In some examples, the visual indication includes textual and / or visual cues to the user of the second electronic device to change the device's orientation to an orientation that meets standards.

[0164] In some examples, a first electronic device displays spatial image data from a first viewpoint relative to a user of the first electronic device via the one or more displays, where the first viewpoint corresponds to a first orientation. In some examples, the first viewpoint corresponds to the orientation of a second electronic device (e.g., the first orientation discussed above). In some examples, a first external camera and a second external camera capture first image data and second image data from an orientation corresponding to the orientation of the second electronic device. In some examples, the user and the second electronic device share the same orientation (e.g., the first orientation). For example, the user may optionally position the second electronic device directly facing outwards in the three-dimensional environment, parallel to the user's face. In some examples, after displaying spatial image data from a first viewpoint relative to the user, based on determining that the second electronic device has changed from the first orientation to a second orientation different from the first orientation, the first electronic device modifies the display of the spatial image data to be displayed from the first viewpoint relative to the user to a second viewpoint different from the first viewpoint via the one or more displays. In some examples, the second electronic device detects user input that causes the orientation of the second electronic device to change from the first orientation to the second orientation. In some examples, the first external camera and / or the second external camera update to the second orientation along with the second electronic device. In some examples, the second electronic device modifies the display during and / or after the second electronic device changes from the first orientation to the second orientation. In some examples, as the second electronic device changes from a first orientation to a second orientation, the first and second external cameras continue to capture first and / or second image data. For example, a user may optionally begin capturing the 3D environment to their left (e.g., in a first orientation), and while the camera is capturing the scene, the phone is continuously rotated to the right of the 3D environment (e.g., in a second orientation), such that the external camera optionally captures a panoramic image of the 3D environment. In some examples, the second electronic device will display a change from a first to a second viewpoint via an updated animation (e.g., dissipation, swipe).

[0165] In some examples, spatial image data is displayed based on the determination that a first external camera has stopped capturing first image data and a second external camera has stopped capturing second image data. In some examples, a first electronic device and / or a second electronic device determines that the first external camera and / or the second external camera has stopped capturing first image data and / or second image data. In some examples, the first external camera / second external camera sends an indication to the first electronic device and / or the second electronic device that the respective electronic device has stopped capturing the corresponding image data. In some examples, the spatial image data is displayed within a graphical user interface associated with the spatial image data. In some examples, the graphical user interface includes multiple controls configured to change one or more aspects of the spatial image data, as discussed in further detail below. In some examples, a first gesture input (e.g., swipe, press, pinch) is received from the second electronic device when the spatial image data is displayed at a first time point. In some examples, the first gesture input corresponds to user input detected at the second electronic device. In some examples, the first gesture input includes one or more characteristics of touch input as discussed above. In some examples, the second electronic device receives the first gesture input at a display. In some examples, the first gesture input points to the graphical user interface discussed above. For example, the graphical user interface may optionally display multiple controls configured to change one or more aspects of spatial image data, and optionally detect a first gesture input pointing to a first control configured to change a point in time in the spatial image data, as discussed in further detail below. In some examples, the first gesture input (optionally pointing to one of the multiple controls at the graphical user interface) causes the second electronic device to stop displaying the spatial image data. In some examples, the first gesture input reduces the display size of the spatial image data from covering the entire display to a portion of the display at the second electronic device. In some examples, the first gesture input includes a series of gestures. In some examples, in response to receiving the first gesture input from the second electronic device, the display of the spatial image data is updated to correspond to a second point in time within the spatial image data that differs from the first point in time. In some examples, the first gesture input points to a control of the graphical user interface associated with controlling the display time of the spatial image data (e.g., the first point in time, the second point in time). For example, the user may optionally point the first gesture input to a control, and in response, the second electronic device may optionally begin playback of the spatial image data (e.g., spatial video) at the first point in time. The second electronic device may optionally detect the first gesture input again, and in response, the second electronic device may begin playback of the spatial image data at the second point in time.

[0166] In some examples, upon determining that a first external camera has stopped capturing first image data and a second external camera has stopped capturing second image data, a first electronic device displays spatial image data and sends a command to a second electronic device to display an editing user interface on a display of the second electronic device. In some examples, the second electronic device detects that the first and second external cameras have stopped capturing first and second image data. In some examples, the first electronic device sends a command to the second electronic device based on the fact that at least one of the external cameras (e.g., the first or second external camera) has stopped capturing its corresponding image data. In some examples, the editing user interface includes multiple controls configured to manipulate various aspects of the spatial image data. For example, the editing user interface may optionally include controls for editing the depth parallax of the spatial image data (e.g., specifying a specific depth of an object in the spatial image data). In some examples, the editing user interface includes one or more selection controls configured to allow a user to select images captured by the first and / or second external cameras for manipulation. In some examples, the editing user interface includes controls for specifying the spatial image data as a single field of view or stereo, as discussed in further detail above. In some examples, the editing user interface includes metadata information associated with the spatial image data (e.g., location labels, local time, file size).

[0167] In some examples, when generated spatial data (e.g., a combination of first and second image data) is displayed at a first immersion level via one or more displays, and based on detected user input, the first electronic device modifies the display of the space from the first immersion level to a second immersion level different from the first immersion level. In some examples, the first immersion level corresponds to displaying the generated spatial image data such that it partially covers at least a portion of the three-dimensional environment relative to the user's viewpoint of the first electronic device. In some examples, the first immersion level refers to the amount by which the generated spatial data covers the three-dimensional environment. In some examples, the user input corresponds to a user gesture interacting with physical hardware (e.g., a button, a switch) at the first electronic device. In some examples, the magnitude of the user input corresponds to the degree of immersion at the second immersion level. For example, the first electronic device may optionally detect user input as a single press of a button at the first electronic device within a first time range (e.g., 1 second, 2 seconds, 3 seconds). This user input may optionally cause the generated spatial image data to cover 10% of the three-dimensional environment (e.g., the first immersion level) to cover 50% of the three-dimensional environment (e.g., the second immersion level). If the first electronic device optionally detects user input as a single press within a second time range (e.g., 4 seconds, 5 seconds, 6 seconds), the user input may optionally cause the generated spatial image data to cover 10% of the three-dimensional environment (e.g., a first immersion level) to change to the generated spatial image data covering 80% of the three-dimensional environment (e.g., a second immersion level).

[0168] In some examples, when a spatial image is displayed at a first location in a 3D environment and in response to a user gesture detected via one or more input devices, a first electronic device displays the spatial image at a second location in the 3D environment, different from the first location. In some examples, the spatial image is displayed in the 3D environment while a first external camera and / or a second external camera is capturing first image data and / or second image data. In some examples, the spatial image is displayed to cover at least a portion of the 3D environment. In some examples, the first location is a predetermined location. In some examples, the first location is determined by user input, as discussed in further detail below. In some examples, the one or more input devices correspond to one or more other physical user interface devices, such as touch-sensitive surfaces, physical keyboards, mice, joysticks, hand-tracking devices, eye-tracking devices, styluses, etc. In some examples, the one or more input devices correspond to non-physical methods of input capture, such as motion detection, LiDAR, one or more cameras, etc. In some examples, user gestures (e.g., including single-input-element gestures, multi-input-element gestures, etc.) include one or more tap gestures, swipe gestures, swipe gestures, etc. In some examples, the spatial image moves from the first location to the second location in a mirror manner to the user gesture. For example, the user gesture may optionally correspond to a swipe motion from the left to the right of the user's viewpoint. In response, the first electronic device may optionally display a spatial image that moves continuously across the user's viewpoint (from left to right) in a swipe animation similar to the user gesture. In some examples, in response to detecting a user gesture, the first electronic device stops displaying the spatial image at a first position and redisplays the spatial image at a second position based on the user gesture. For example, the user gesture may optionally correspond to a pinch gesture at the first position and a drag gesture to the second position, and in response, the second electronic device stops displaying spatial image data at the first position and redisplays the spatial image data at a second position corresponding to the direction of the pinch and drag gesture. In some examples, the first electronic device displays the spatial image as moving from the first position to the second position in a continuous motion animation in the direction of the user gesture (e.g., a user gesture including a pinch and drag motion from left to right).

[0169] In some examples, a first external camera and a second external camera are included in a second electronic device (e.g., an auxiliary electronic device, the second electronic device discussed above) that communicates with a first electronic device, wherein the second electronic device includes a display (e.g., a display discussed above with reference to the second electronic device). In some examples, when displaying (e.g., a display discussed above with reference to the second electronic device) a spatial image, the first electronic device sends a command to the second electronic device to apply a hue to an image (e.g., first image data, second image data, spatial image data) displayed on the display of the second electronic device. In some examples, the first electronic device sends the command via a wired and / or wireless connection. In some examples, the first electronic device sends the command in response to user input (e.g., input to begin capturing first image data and / or second image data). In some examples, the hue corresponds to a slight tint or hue applied to a portion or sub-portion of an image, thereby optionally altering its overall color balance. In some examples, the command includes an opacity level associated with the hue. For example, the command may optionally include an instruction to reduce the opacity of the image by 20%. In some examples, the command applies the hue to only a portion of the image. For example, in response to receiving a command, the second electronic device applies a hue as a ring around the outer portion of the image. In some examples, the image corresponds to a still image and / or video of a three-dimensional environment. In some examples, the command applies a hue to the entire image. In some examples, the command to apply a hue corresponds to darkening at least a portion of the image. In some examples, the image corresponds to the graphical user interface discussed above, which updates the display of spatial image data from a first time point to a second time point. In some examples, the image corresponds to a single-field-of-view image of a three-dimensional environment. In some examples, the image corresponds to the two-dimensional image data discussed above, which displays two-dimensional image data at the second electronic device when a first external camera is capturing first image data and a second external camera is capturing second image data.

[0170] Some examples of this disclosure relate to a method comprising: at a first electronic device communicating with one or more displays and with a first external camera having a first viewpoint and a second external camera having a second viewpoint different from the first viewpoint: while the first external camera is capturing first image data and the second external camera is capturing second image data, obtaining at least a portion of the first image data from the first external camera, obtaining at least a portion of the second image data from the second external camera, or obtaining spatial image data generated based on the first image data and the second image data; and displaying, via the one or more displays, a spatial image based on at least a portion of the first image data and at least a portion of the second image data or the spatial image data in a three-dimensional environment, according to determining that one or more first criteria are met.

[0171] Additionally or alternatively, in some examples, a first external camera and a second external camera are included in a second electronic device that communicates with the first electronic device. Additionally or alternatively, in some examples, the second electronic device generates spatial image data based on the first image data and the second image data, and transmits the spatial image data to the first electronic device. Additionally or alternatively, in some examples, the first electronic device obtains at least a portion of the first image data and at least a portion of the second image data from the second electronic device, and the first electronic device generates spatial image data based on the at least a portion of the first image data and the at least a portion of the second image data. Additionally or alternatively, in some examples, the second electronic device includes a display different from the one or more displays, which can be configured to display two-dimensional image data when the first external camera is capturing first image data and the second external camera is capturing second image data. Additionally or alternatively, in some examples, when the spatial image data is displayed in a three-dimensional environment via the one or more displays of the first electronic device, the two-dimensional image data has an appearance different from the first image data or the second image data. Additionally or alternatively, in some examples, when a spatial image is displayed in a three-dimensional environment via the one or more displays of the first electronic device, the two-dimensional image data includes a single-field-of-view representation of the first image data or the second image data, or a stereoscopic representation of the first image data and the second image data. Additionally or alternatively, in some examples, the one or more first criteria include criteria satisfied when the first electronic device receives from an external electronic device an instruction to display a spatial image generated based on the first image data and the second image data in a three-dimensional environment. Additionally or alternatively, in some examples, the spatial image data includes visual depth information.

[0172] Additionally or alternatively, in some examples, the first image data and the second image data correspond to video data, and wherein the spatial image data corresponds to spatial video. Additionally or alternatively, in some examples, the first image data includes a plurality of first pixels and the second image data includes a plurality of second pixels, and wherein displaying the spatial image includes applying a pixel matching process to the first image data and the second image data prior to displaying the spatial image. Additionally or alternatively, in some examples, the one or more first criteria include a criterion satisfied when the stereo disparity between the first image data and the second image data is less than a first threshold, and wherein the method further includes displaying a non-spatial image via the one or more displays based on at least a portion of the first image data and at least a portion of the second image data or the spatial image data, based on determining that the stereo disparity is not less than the first threshold. Additionally or alternatively, in some examples, the one or more first criteria include a criterion satisfied when the focal length difference between the first image data and the second image data is less than a first threshold, and wherein the method further includes displaying a non-spatial image via the one or more displays based on the at least a portion of the first image data and the at least a portion of the second image data or the spatial image data, based on determining that the focal length difference is not less than the first threshold. Additionally or alternatively, in some examples, a first external camera and a second external camera are included in a second electronic device communicating with a first electronic device, and wherein the one or more first criteria include criteria satisfied based on determining that the second electronic device is in a first orientation, and wherein the method further includes, based on determining that the second electronic device is in a second orientation different from the first orientation, displaying a visual indication of the second orientation of the second electronic device via the one or more displays. Additionally or alternatively, in some examples, the method further includes: displaying a spatial image from a first viewpoint relative to a user of the first electronic device via the one or more displays, wherein the first viewpoint corresponds to the first orientation; and after displaying spatial image data from the first viewpoint relative to the user, modifying the display of the spatial image to be displayed from the first viewpoint relative to the user to a second viewpoint different from the first viewpoint based on determining that the second electronic device has changed from the first orientation to a second orientation different from the first orientation.

[0173] Additionally or alternatively, in some examples, the method further includes: displaying a representation of a spatial image via the one or more displays based on determining that a first external camera has stopped capturing first image data and a second external camera has stopped capturing second image data; receiving a first gesture input from a second electronic device when the representation of the spatial image is displayed at a first time point; and updating the display of the representation of the spatial image to correspond to a second time point within the spatial image that is different from the first time point in response to receiving the first gesture input from the second electronic device. Additionally or alternatively, in some examples, the method further includes displaying a representation of the spatial image via the one or more displays based on determining that a first external camera has stopped capturing first image data and a second external camera has stopped capturing second image data, and sending a command to the second electronic device to display an editing user interface on a display at the second electronic device. Additionally or alternatively, in some examples, the spatial image is displayed at a first immersion level, and the method further includes, when the spatial image is displayed at the first immersion level: modifying the display of the spatial image from the first immersion level to a second immersion level different from the first immersion level based on detected user input. Additionally or alternatively, in some examples, the method further includes: when displaying a spatial image at a first location in a three-dimensional environment, in response to detecting a user gesture via one or more input devices, displaying the spatial image at a second location in the three-dimensional environment different from the first location. Additionally or alternatively, in some examples, a first external camera and a second external camera are included in a second electronic device communicating with a first electronic device, wherein the second electronic device includes a display, and wherein the method further includes, when displaying the spatial image, sending a command to the second electronic device to apply a hue to the image displayed on the display of the second electronic device.

[0174] Some examples of this disclosure relate to a method comprising: at an electronic device communicating with one or more displays, one or more input devices, a first external camera having a first viewpoint, and an image capturing device having a second external camera and a third external camera (the second external camera having a second viewpoint different from the first viewpoint, and the third external camera having a third viewpoint different from the first and second viewpoints); obtaining at least a portion of the first image data from the first external camera, at least a portion of the second image data from the second external camera, and at least a portion of the third image data from the third external camera while the first external camera is capturing first image data, the second external camera is capturing second image data, and the third external camera is capturing third image data; displaying a spatial image based on at least a portion of the first image data, at least a portion of the second image data, and at least a portion of the third image data in a three-dimensional environment via the one or more displays; detecting the image capturing device in the field of view of the first external camera in the three-dimensional environment via the one or more input devices or via the first external camera; and stopping the display of the spatial image in the three-dimensional environment via the one or more displays in response to the detection of the image capturing device in the field of view of the first external camera, based on determining that one or more criteria are met.

[0175] Additionally or alternatively, in some examples, the one or more criteria include a criterion satisfied when a physical viewfinder of the image capturing device is detected in the field of view of a first external camera in a three-dimensional environment. Additionally or alternatively, in some examples, the one or more criteria include a criterion satisfied when a physical display of the image capturing device is detected in the field of view of a first external camera in a three-dimensional environment. Additionally or alternatively, in some examples, the one or more criteria include a criterion satisfied when the image capturing device is within a threshold distance of a first viewpoint of the first external camera in the field of view of the first external camera in a three-dimensional environment. Additionally or alternatively, in some examples, the image capturing device includes a physical display configured to display a representation of a spatial image, and the method further includes: while the first external camera is capturing first image data, the second external camera is capturing second image data, and the third external camera is capturing third image data, and after displaying a spatial image based on at least a portion of the first image data, at least a portion of the second image data, and at least a portion of the third image data in a three-dimensional environment, sending one or more instructions to the image capturing device, the one or more instructions causing the image capturing device to stop operation of the physical display, such that the physical display does not display a representation of the spatial image. Additionally or alternatively, in some examples, the method further includes: when displaying a spatial image in a three-dimensional environment, detecting an indication of movement of an image capturing device via the one or more input devices or via a first external camera, the movement causing a second viewpoint of a second external camera to be an updated second viewpoint and causing a third viewpoint of a third external camera to be an updated third viewpoint; and in response to detecting the indication of movement of the image capturing device, obtaining at least a portion of updated second image data captured relative to the updated second viewpoint from the second external camera, and obtaining at least a portion of updated third image data captured relative to the updated third viewpoint from the third external camera, and updating the display of the spatial image in the three-dimensional environment via the one or more displays based on at least a portion of the first image data, at least a portion of the updated second image data, and at least a portion of the updated third image data.

[0176] Additionally or alternatively, in some examples, the method further includes: receiving an indication of a request to save the spatial image when it is displayed in a three-dimensional environment; and, upon receiving the indication, receiving data from an image capture device corresponding to a representation of the spatial image. Additionally or alternatively, in some examples, the request to save the spatial image includes user input selecting a capture button on the image capture device. Additionally or alternatively, in some examples, receiving the request to save the spatial image includes detecting a selection of a button via the one or more input devices, the button being selectable to cause the image capture device to generate a representation of the spatial image. Additionally or alternatively, in some examples, the button corresponds to a selectable option displayed along with the spatial image in the three-dimensional environment. Additionally or alternatively, in some examples, the method further includes displaying a representation of the spatial image in the three-dimensional environment via the one or more displays in response to receiving data corresponding to the representation of the spatial image. Additionally or alternatively, in some examples, displaying a representation of the spatial image in the three-dimensional environment includes reducing the visual salience of portions of the representation of the spatial image in the three-dimensional environment from a first viewpoint of a first external camera. Additionally or alternatively, in some examples, the method further includes: detecting, via the one or more input devices, a user's gaze toward a first location in the spatial image in the three-dimensional environment when the spatial image is displayed in a three-dimensional environment; and, in response to detecting the user's gaze toward the first location in the spatial image, sending one or more instructions to an image capturing device, the one or more instructions causing the image capturing device to adjust the focus of the lenses of a second external camera and / or a third external camera based on the first location in the spatial image.

[0177] Additionally or alternatively, in some examples, the spatial image is a first spatial image, and the electronic device also communicates with a second image capturing device, the second image capturing device including a fourth external camera having a fourth viewpoint different from the second viewpoint and the third viewpoint, and a fifth external camera having a fifth viewpoint different from the second viewpoint, the third viewpoint, and the fourth viewpoint. The method further includes, when the first external camera is capturing first image data, the second external camera is capturing second image data, the third external camera is capturing third image data, the fourth external camera is capturing fourth image data, and the fifth external camera is capturing fifth image data: obtaining at least a portion of the fourth image data from the fourth external camera, and obtaining at least a portion of the fifth image data from the fifth external camera; and displaying, concurrently with the first spatial image in a three-dimensional environment, a second spatial image based on at least a portion of the first image data, at least a portion of the fourth image data, and at least a portion of the fifth image data via the one or more displays.

[0178] Some examples of this disclosure relate to an electronic device comprising: one or more processors; a memory; and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described above.

[0179] Some examples of this disclosure relate to a non-transitory computer-readable storage medium that stores one or more programs, the one or more programs including instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform any of the methods described above.

[0180] Some examples of this disclosure relate to an electronic device that includes one or more processors, a memory, and components for performing any of the methods described above.

[0181] Some examples of this disclosure relate to an information processing apparatus used in an electronic device, the information processing apparatus including components for performing any of the methods described above.

[0182] For purposes of explanation, the foregoing description has been given by way of reference to specific examples. However, the illustrative discussion above is not intended to be exhaustive or to limit this disclosure to the precise form disclosed. Many modifications and variations are possible in accordance with the foregoing teachings. The examples were chosen and described in order to best elucidate the principles of this disclosure and its practical application, thereby enabling others skilled in the art to make optimal use of this disclosure with various modifications suitable for the particular intended purpose, as well as the various described examples.

Claims

1. A method, the method comprising: At a first electronic device that communicates with one or more displays and with a first external camera having a first viewpoint and a second external camera having a second viewpoint different from the first viewpoint: When the first external camera is capturing first image data and the second external camera is capturing second image data: The first image data is obtained from the first external camera, and the second image data is obtained from the second external camera, or spatial image data generated based on the first image data and the second image data is obtained; as well as Based on determining that one or more first criteria are met, a spatial image based on the first image data and the second image data, or the spatial image data generated based on the first image data and the second image data, is displayed in a three-dimensional environment via the one or more displays.

2. The method of claim 1, wherein the first external camera and the second external camera are included in a second electronic device that communicates with the first electronic device.

3. The method of claim 2, wherein the second electronic device generates the spatial image data based on the first image data and the second image data, and transmits the spatial image data to the first electronic device.

4. The method of claim 2, wherein the first electronic device obtains the first image data and the second image data from the second electronic device, and the first electronic device generates the spatial image data based on the first image data and the second image data.

5. The method of claim 2, wherein the second electronic device includes a display different from the one or more displays, the display being configurable to display two-dimensional image data on the display of the second electronic device when the first external camera is capturing first image data and the second external camera is capturing second image data.

6. The method of claim 5, wherein when the spatial image data is displayed in the three-dimensional environment via the display of the first electronic device, the two-dimensional image data has an appearance different from the first image data or the second image data.

7. The method of claim 5, wherein when the spatial image data is displayed in the three-dimensional environment via the display of the first electronic device, the two-dimensional image data includes a single-view representation of the first image data or the second image data, or a stereoscopic representation of the first image data and the second image data.

8. The method of claim 1, wherein the one or more first criteria include criteria satisfied when the first electronic device receives from an external electronic device an instruction to display spatial image data generated based on the first image data and the second image data in the three-dimensional environment.

9. The method according to claim 1, wherein the spatial image data includes visual depth information.

10. The method according to claim 1, wherein the first image data and the second image data correspond to video data, and wherein the spatial image data corresponds to spatial video.

11. The method of claim 1, wherein the first image data comprises a plurality of first pixels and the second image data comprises a plurality of second pixels, and wherein displaying the spatial image data comprises: Before displaying the spatial image, a pixel matching process is applied to the first image data and the second image data.

12. The method of claim 1, wherein the one or more first criteria include criteria satisfied when the stereo disparity between the first image data and the second image data is less than a first threshold, and wherein the method further comprises: If the stereo parallax does not meet the standard, a non-spatial image is displayed via the one or more displays based on the first image data and the second image data or the spatial image data.

13. The method of claim 1, wherein the one or more first criteria include criteria satisfied when the focal length difference between the first image data and the second image data is less than a first threshold, and wherein the method further comprises: Based on the determination that the focal length difference does not meet the standard, a non-spatial image is displayed via the one or more displays based on the first image data and the second image data or the spatial image data.

14. The method of claim 1, wherein the first external camera and the second external camera are included in a second electronic device communicating with the first electronic device, and wherein the one or more first criteria include criteria satisfied according to determining that the second electronic device is in a first orientation, and wherein the method further comprises: Based on the determination that the second electronic device is in a second orientation different from the first orientation, a visual indication of the orientation of the second electronic device is displayed via the one or more displays.

15. The method according to claim 14, further comprising: The spatial image data is displayed via the one or more displays from a first perspective of a user relative to the first electronic device, wherein the first perspective corresponds to the first orientation; as well as After displaying the spatial image data from the first perspective relative to the user, based on determining that the second electronic device has changed from the first orientation to a second orientation different from the first orientation, the display of the spatial image data to be displayed is modified from the first perspective to a second perspective relative to the user via the one or more displays.

16. The method of claim 1, wherein the first external camera and the second external camera are included in a second electronic device communicating with the first electronic device, the method further comprising: Based on the determination that the first external camera has stopped capturing the first image data and the second external camera has stopped capturing the second image data, the spatial image data is displayed; When the spatial image data is displayed at the first time point, a first gesture input is received from the second electronic device; as well as In response to receiving the first gesture input from the second electronic device, the display of the spatial image data is updated to correspond to a second time point within the spatial image data that is different from the first time point.

17. The method of claim 1, wherein the first external camera and the second external camera are included in a second electronic device communicating with the first electronic device, the method further comprising: Based on the determination that the first external camera has stopped capturing the first image data and the second external camera has stopped capturing the second image data, the spatial image data is displayed and a command is sent to the second electronic device to display the editing user interface at the display of the second electronic device.

18. The method of claim 1, wherein the first external camera and the second external camera are included in a second electronic device communicating with the first electronic device, wherein the second electronic device includes a display, and wherein the method further comprises: When the spatial image is displayed: Send a command to the second electronic device to apply a hue to an image displayed on the display of the second electronic device.

19. A first electronic device, the first electronic device comprising: One or more processors; Memory; and One or more programs, the programs being stored in the memory and configured to be executed by the one or more processors, the programs including instructions for performing the method according to any one of claims 1 to 18.

20. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by one or more processors of a first electronic device, cause the first electronic device to perform the method according to any one of claims 1 to 18.

Images