Skeletal data tracking of participants in multi-user communication sessions

Abstract

To provide a system and a method that facilitate interaction based on skeletal data of participants in a multi-user communication session.SOLUTION: The present invention is directed to a system and a method for displaying virtual objects within a multi-user communication session and visual indications of participants within the multi-user communication session based on skeletal data associated with one or more participants in the multi-user communication session. While a first electronic device 101a is in a communication session with a second electronic device 101b co-located with the first electronic device in a physical environment 400, the first electronic device receives first data provided by the second electronic device and including skeletal data associated with a user of the second electronic device.SELECTED DRAWING: Figure 4A-2

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Patent Application No. 63 / 640,792, filed April 30, 2024, U.S. Provisional Patent Application No. 63 / 758,960, filed February 14, 2025, and U.S. Provisional Patent Application No. 19 / 083,245, filed March 18, 2025, the contents of which are incorporated herein by reference in their entirety for all purposes.

[0002] (Technical field) This relates generally to systems and methods for tracking skeletal data of participants in a multi-user communication session co-located within the same physical environment and facilitating interaction based on the skeletal data. [Background technology]

[0003] Some computer graphical environments provide computer-generated two-dimensional and / or three-dimensional environments in which at least some objects displayed for user viewing are virtual. In some examples, the three-dimensional environment is presented by multiple devices communicating in a multi-user communication session. In some examples, an avatar (e.g., a representation) of each non-co-located user participating in the multi-user communication session (e.g., via a computing device) is displayed within the three-dimensional environment of the multi-user communication session. In some examples, content may be shared within the three-dimensional environment for viewing and interaction by multiple users participating in the multi-user communication session. Summary of the Invention

[0004] Some examples of the present disclosure are directed to systems and methods for displaying virtual objects within a multi-user communication session based on skeletal data associated with one or more participants in the multi-user communication session. In some examples, the method is executed on a first electronic device in communication with one or more displays and one or more input devices, the first electronic device being co-located with a second electronic device in a first physical environment. In some examples, while the first electronic device is in a communication session with the second electronic device, the first electronic device receives first data provided by the second electronic device, the first data including skeletal data associated with a user of the second electronic device. In some examples, after receiving the first data, the first electronic device detects an indication of a request to share content in a three-dimensional environment. In some examples, in response to detecting the indication, the first electronic device presents, via one or more displays, a first object corresponding to the shared content within the three-dimensional environment. In some examples, while presenting the first object corresponding to the shared content, the first electronic device detects a change in one or more spatial properties between the user of the second electronic device and the first object within the three-dimensional environment. In some examples, in response to detecting a change in one or more spatial properties between a user of the second electronic device and the first object, the first electronic device updates, via one or more displays, a display of the visual appearance of the first object in accordance with the change in the one or more spatial properties and based on the first data.

[0005] Some examples of the present disclosure are directed to systems and methods for displaying visual indications of participants in a multi-user communication session based on skeletal data associated with the participants in the multi-user communication session. In some examples, the method is performed on a first electronic device in communication with one or more displays and one or more input devices, the first electronic device being co-located with a second electronic device in a first physical environment. In some examples, while the first electronic device is in a communication session with the second electronic device, the first electronic device receives first data provided by the second electronic device, the first data including skeletal data associated with a user of the second electronic device. In some examples, after receiving the first data, the first electronic device detects movement of the user of the second electronic device in the first physical environment. In some examples, in response to detecting movement of a user of the second electronic device and determining that the movement of the user of the second electronic device causes at least a portion of the user of the second electronic device to be occluded by at least a portion of the first physical environment relative to a viewpoint of the first electronic device, the first electronic device displays, via one or more displays, a visual indication corresponding to at least a portion of the user of the second electronic device at a location within the three-dimensional environment that corresponds to at least a portion of the first physical environment based on the first data.

[0006] A full description of these examples is set forth in the Drawings and Detailed Description, and it should be understood that this Summary is not intended to limit the scope of the present disclosure in any way. [Brief explanation of the drawings]

[0007] For a better understanding of the various examples described herein, reference should be made to the following detailed description in conjunction with the following drawings, in which like reference numerals refer to corresponding parts throughout:

[0008] [Figure 1] 1 illustrates an electronic device presenting an augmented reality environment, according to some examples of the present disclosure.

[0009] [Figure 2] 1 shows a block diagram of an example architecture for a system according to some examples of the present disclosure.

[0010] [Figure 3] 1 illustrates an example of a spatial group in a multi-user communication session including a first electronic device and a second electronic device, according to some examples of the present disclosure.

[0011] [Figure 4A-1] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 4A-2] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 4B-1] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 4B-2] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 4C-1] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 4C-2] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 4D-1] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 4D-2]1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 4E] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 4F] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 4G-1] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 4G-2] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 4H] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 4I] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure.

[0012] [Figure 5A-1] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 5A-2] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 5B] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 5C]1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 5D] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 5E] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 5F] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. [Figure 5G] 1 illustrates exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure.

[0013] [Figure 6] 1 is a flow diagram illustrating an example process for updating the visual appearance of a virtual object using skeletal data associated with one or more participants in a multi-user communication session, according to some examples of the present disclosure.

[0014] [Figure 7] 1 is a flow diagram illustrating an example process for presenting a visual indication of participants in a multi-user communication session based on skeletal data associated with the participants, according to some examples of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0015] Some examples of the present disclosure are directed to systems and methods for displaying virtual objects within a multi-user communication session based on skeletal data associated with one or more participants in the multi-user communication session. In some examples, the method is executed on a first electronic device in communication with one or more displays and one or more input devices, the first electronic device being co-located with a second electronic device in a first physical environment. In some examples, while the first electronic device is in a communication session with the second electronic device, the first electronic device receives first data provided by the second electronic device, the first data including skeletal data associated with a user of the second electronic device. In some examples, after receiving the first data, the first electronic device detects an indication of a request to share content in a three-dimensional environment. In some examples, in response to detecting the indication, the first electronic device presents, via one or more displays, a first object corresponding to the shared content within the three-dimensional environment. In some examples, while presenting the first object corresponding to the shared content, the first electronic device detects a change in one or more spatial properties between the user of the second electronic device and the first object within the three-dimensional environment. In some examples, in response to detecting a change in one or more spatial properties between a user of the second electronic device and the first object, the first electronic device updates, via one or more displays, a display of the visual appearance of the first object in accordance with the change in the one or more spatial properties and based on the first data.

[0016] Some examples of the present disclosure are directed to systems and methods for displaying visual indications of participants in a multi-user communication session based on skeletal data associated with the participants in the multi-user communication session. In some examples, the method is performed on a first electronic device in communication with one or more displays and one or more input devices, the first electronic device being co-located with a second electronic device in a first physical environment. In some examples, while the first electronic device is in a communication session with the second electronic device, the first electronic device receives first data provided by the second electronic device, the first data including skeletal data associated with a user of the second electronic device. In some examples, after receiving the first data, the first electronic device detects movement of the user of the second electronic device in the first physical environment. In some examples, in response to detecting movement of a user of the second electronic device and determining that the movement of the user of the second electronic device causes at least a portion of the user of the second electronic device to be occluded by at least a portion of the first physical environment relative to a viewpoint of the first electronic device, the first electronic device displays, via one or more displays, a visual indication corresponding to at least a portion of the user of the second electronic device at a location within the three-dimensional environment that corresponds to at least a portion of the first physical environment based on the first data.

[0017] As used herein, a spatial group corresponds to a group or number of participants (e.g., users) in a multi-user communication session. In some examples, a spatial group within a multi-user communication session has a spatial arrangement that determines the location of users and content located within the spatial group. In some examples, users within the same spatial group within a multi-user communication session experience spatial truth according to the spatial arrangement of the spatial group. In some examples, when a user of a first electronic device is in a first spatial group and a user of a second electronic device is in a second spatial group in a multi-user communication session, the users experience spatial truth localized to their respective spatial groups. In some examples, a user of a first electronic device and a user of a second electronic device are grouped into separate spatial groups within a multi-user communication session, but when the first electronic device and the second electronic device return to the same operating state, the user of the first electronic device and the user of the second electronic device are regrouped into the same spatial group within the multi-user communication session.

[0018] As used herein, a hybrid spatial group corresponds to a group or number of participants (e.g., users) in a multi-user communication session in which at least a subset of the participants are not co-located in a physical environment. For example, as described through one or more examples in this disclosure, a hybrid spatial group includes at least two participants co-located in a first physical environment and at least one participant not co-located with the at least two participants in the first physical environment (e.g., at least one participant is located in a second physical environment different from the first physical environment). In some examples, a hybrid spatial group within a multi-user communication session has a spatial arrangement that determines the location of users and content located within the spatial group. In some examples, users in the same hybrid spatial group within a multi-user communication session experience spatial authenticity according to the spatial arrangement of the spatial group, as similarly described above.

[0019] In some examples, initiating a multi-user communication session may include interaction with one or more user interface elements. In some examples, a user's gaze may be tracked by the electronic device as input for targeting selectable options / affordances within individual user interface elements displayed within the three-dimensional environment. For example, the gaze may be used to identify one or more options / affordances for selection using another selection input. In some examples, individual options / affordances may be selected using hand-tracking input detected via an input device in communication with the electronic device. In some examples, objects displayed within the three-dimensional environment may be moved and / or reoriented within the three-dimensional environment according to movement input detected via the input device.

[0020] FIG. 1 illustrates an electronic device 101 presenting an extended reality (XR) environment (e.g., a computer-generated environment that optionally includes representations of physical and / or virtual objects) according to some examples of the present disclosure. In some examples, as shown in FIG. 1, the electronic device 101 is a head-mounted display or other head-mountable device configured to be worn on the head of a user of the electronic device 101. Examples of the electronic device 101 are described below with reference to the architectural block diagram of FIG. 2. As shown in FIG. 1, the electronic device 101 and a table 106 are located in a physical environment. The physical environment may include physical features such as physical surfaces (e.g., floors, walls) or physical objects (e.g., tables, lamps, etc.). In some examples, the electronic device 101 may be configured to detect and / or capture images of the physical environment, including the table 106 (shown within the field of view of the electronic device 101).

[0021] 1, electronic device 101 includes one or more internal image sensors 114a (e.g., eye-tracking cameras described below with reference to FIG. 2) oriented toward the user's face. In some examples, internal image sensor 114a is used for eye tracking (e.g., detecting the user's gaze). Internal image sensor 114a is optionally positioned on left and right portions of display 120 to enable eye tracking of the user's left and right eyes. In some examples, electronic device 101 also includes external image sensors 114b and 114c facing outward from the user to detect and / or capture the physical environment of electronic device 101 and / or movements of the user's hands or other body parts.

[0022] In some examples, display 120 has a field of view visible to the user (e.g., which may or may not correspond to the field of view of external image sensors 114b and 114c). Because display 120 is optionally part of a head-mounted device, the field of view of display 120 is optionally the same as or similar to the field of view of the user's eyes. In other examples, the field of view of display 120 may be smaller than the field of view of the user's eyes. In some examples, electronic device 101 may be an optical see-through device in which display 120 is a transparent or translucent display through which a portion of the physical environment can be viewed directly. In some examples, display 120 may be contained within a transparent lens or may overlap all or only a portion of a transparent lens. In other examples, electronic device 101 may be a video pass-through device in which display 120 is an opaque display configured to display images of the physical environment captured by external image sensors 114b and 114c. While a single display 120 is shown, it should be understood that display 120 may include a stereo pair of displays.

[0023] 1 , which is not present in the physical environment but is displayed in the XR environment positioned on top of a real-world table 106 (or a representation thereof). Optionally, the virtual object 104 may be displayed on a surface of the table 106 in the XR environment displayed via the display 120 of the electronic device 101 in response to detecting the plane of the table 106 in the physical environment 100.

[0024] It should be understood that virtual object 104 is exemplary of a virtual object, and that one or more different virtual objects (e.g., of various dimensionalities, such as two-dimensional or other three-dimensional virtual objects) can be included and rendered within the three-dimensional XR environment. For example, the virtual object may represent an application or a user interface displayed within the XR environment. In some examples, the virtual object may represent content corresponding to an application and / or displayed via a user interface in the XR environment. In some examples, virtual object 104 is optionally interactive and configured to respond to user input (e.g., air gestures such as an air pinch gesture, an air tap gesture, and / or an air touch gesture) such that a user may virtually touch, tap, move, rotate, or otherwise interact with virtual object 104.

[0025] In some examples, displaying an object within the three-dimensional environment may include interaction with one or more user interface objects within the three-dimensional environment. For example, initiating the display of an object within the three-dimensional environment may include interaction with one or more virtual options / affordances displayed within the three-dimensional environment. In some examples, a user's gaze may be tracked by the electronic device as input to identify one or more virtual options / affordances for selection when initiating the display of the object within the three-dimensional environment. For example, the gaze may be used to identify one or more virtual options / affordances for selection using another selection input. In some examples, the virtual options / affordances may be selected using hand-tracking input detected via an input device in communication with the electronic device. In some examples, an object displayed within the three-dimensional environment may be moved and / or reoriented within the three-dimensional environment according to movement input detected via the input device.

[0026] In the following description, an electronic device is described that communicates with display generating components and one or more input devices. It should be understood that the electronic device optionally communicates with one or more other physical user interface devices, such as a touch-sensitive surface, a physical keyboard, a mouse, a joystick, a hand-tracking device, an eye-tracking device, a stylus, etc. It should also be understood that, as noted above, the described electronic device, display, and touch-sensitive surface are optionally distributed across two or more devices. Thus, as used in this disclosure, information displayed on or by an electronic device may optionally be used to describe information output by the electronic device for display on another display device (touch-sensitive or non-touch-sensitive). Similarly, as used in this disclosure, input received at an electronic device (e.g., touch input received on a touch-sensitive surface of the electronic device or touch input received on the surface of a stylus) is optionally used to describe input received at a separate input device from which the electronic device receives input information.

[0027] The device typically supports a variety of applications, such as one or more of a drawing application, a presentation application, a word processing application, a website creation application, a disc authoring application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a training support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music playback application, a television channel browsing application, and / or a digital video playback application.

[0028] 2 shows a block diagram of an example architecture of a system 201 according to some examples of the present disclosure. In some examples, the system 201 includes multiple devices. For example, the system 201 includes a first electronic device 260 and a second electronic device 270, where the first electronic device 260 and the second electronic device 270 communicate with each other. In some examples, the first electronic device 260 and the second electronic device 270 are portable devices, such as a mobile phone, a smartphone, a tablet computer, a laptop computer, an auxiliary device that communicates with another device, a head-mounted display, etc. In some examples, the first electronic device 260 and the second electronic device 270 correspond to the electronic device 101 described above with reference to FIG. 1.

[0029] As shown in FIG. 2 , first electronic device 260 optionally includes various sensors (e.g., one or more hand tracking sensors 202A, one or more location sensors 204A, one or more image sensors 206A, one or more touch-sensitive surface 209A, one or more movement and / or orientation sensors 210A, one or more eye tracking sensors 212A, one or more microphones 213A or other audio sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors), one or more display generating components 214A, one or more speakers 216A, one or more processors 218A, one or more memories 220A, and / or communications circuitry 222A. In some examples, second electronic device 270 may include various sensors (e.g., one or more hand tracking sensors 202A, one or more location sensors 204A, one or more image sensors 206A, one or more touch-sensitive surface 209A, one or more movement and / or orientation sensors 210A, one or more eye tracking sensors 212A, one or more microphones 213A or other audio sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors), one or more display generating components 214A, one or more speakers 216A, one or more processors 218A, one or more memories 220A, and / or communications circuitry 222A). In some examples, second electronic device 270 may include various sensors (e.g., one or more hand tracking sensors 202A, one or more location sensors 204A, one or more image sensors 206A, one or more touch-sensitive surface 209A, one or more movement and / or orientation sensors 210A, one or more eye tracking sensors 212A, one or more microphones 213A or other audio sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors), For example, one or more hand tracking sensors 202B, one or more location sensors 204B, one or more image sensors 206B, one or more touch-sensitive surfaces 209B, one or more movement and / or orientation sensors 210B, one or more eye tracking sensors 212B, one or more microphones 213B or other audio sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors), one or more display generating components 214B, one or more speakers 216, one or more processors 218B, one or more 1. In some examples, one or more display generation components 214A, 214B correspond to display 120 of FIG. 1. One or more communication buses 208A and 208B are optionally used for communication between the above-mentioned components of electronic devices 260 and 270, respectively. First electronic device 260 and second electronic device 270 optionally communicate via a wired or wireless connection between the two devices (e.g., via communication circuitry 222A, 222B).

[0030] Communications circuitry 222A, 222B optionally includes circuitry for communicating with electronic devices, networks such as the Internet, an intranet, wired and / or wireless networks, cellular networks, and wireless local area networks (LANs). Communications circuitry 222A, 222B optionally includes circuitry for communicating using short-range communications such as near field communication (NFC) and / or Bluetooth.

[0031] The processor(s) 218A, 218B include one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, the memory 220A, 220B is a non-transitory computer-readable storage medium (e.g., flash memory, random access memory, or other volatile or non-volatile memory or storage) that stores computer-readable instructions configured to be executed by the processor(s) 218A, 218B to perform the techniques, processes, and / or methods described below. In some examples, the memory 220A, 220B can include two or more non-transitory computer-readable storage media. A non-transitory computer-readable storage medium can be any medium (other than a signal) that can tangibly store or carry computer-executable instructions used by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic, optical, and / or semiconductor storage devices. Examples of such storage include magnetic disks, compact discs (CDs), digital versatile discs (DVDs), or optical discs based on Blu-ray technology, as well as persistent solid-state memory such as flash and solid-state drives.

[0032] In some examples, display generating component(s) 214A, 214B include a single display (e.g., a liquid crystal display (LCD), organic light emitting diode (OLED), or other type of display). In some examples, display generating component(s) 214A, 214B include multiple displays. In some examples, display generating component(s) 214A, 214B can include a display with touch capabilities (e.g., a touchscreen), a projector, a holographic projector, a retina projector, a transparent or translucent display, etc. In some examples, electronic devices 260 and 270 include touch-sensitive surface(s) 209A and 209B, respectively, for receiving user inputs such as tap and swipe inputs or other gestures. In some examples, display generating component(s) 214A, 214B and touch-sensitive surface(s) 209A, 209B form touch-sensitive display(s) (e.g., touchscreens integrated with or external to electronic devices 260 and 270, respectively, in communication with electronic devices 260 and 270, respectively).

[0033] Electronic devices 260 and 270 optionally include image sensor(s) 206A and 206B, respectively. Image sensor(s) 206A / 206B optionally include one or more visible light image sensors, such as charge-coupled device (CCD) sensors and / or complementary metal-oxide semiconductor (CMOS) sensors, operable to acquire images of physical objects from the real-world environment. Image sensor(s) 206A / 206B also optionally include one or more infrared (IR) sensors, such as passive or active IR sensors, for detecting infrared light from the real-world environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real-world environment. Image sensor(s) 206A / 206B also optionally include one or more cameras configured to capture movement of physical objects in the real-world environment. Image sensor(s) 206A / 206B also optionally include one or more depth sensors configured to detect the distance of a physical object from electronic device 260 / 270. In some examples, information from the one or more depth sensors can enable the device to identify an object in the real-world environment and distinguish it from other objects in the real-world environment. In some examples, the one or more depth sensors can enable the device to determine the texture and / or topography of an object in the real-world environment.

[0034] In some examples, the electronic devices 260 and 270 use a combination of a CCD sensor, an event camera, and a depth sensor to detect the physical environment around the electronic devices 260 and 270. In some examples, the image sensor(s) 206A / 206B include a first image sensor and a second image sensor. The first image sensor and the second image sensor cooperate and are optionally configured to capture different information of physical objects in the real-world environment. In some examples, the first image sensor is a visible light image sensor and the second image sensor is a depth sensor. In some examples, the electronic devices 260 / 270 use the image sensor(s) 206A / 206B to detect the position and orientation of the electronic devices 260 / 270 and / or the display generating component(s) 214A / 214B within the real-world environment. For example, the electronic device 260 / 270 uses the image sensor(s) 206A / 206B to track the position and orientation of the display generating component(s) 214A / 214B relative to one or more fixed objects in the real-world environment.

[0035] In some examples, electronic device 260 / 270 includes microphone(s) 213A / 213B or other audio sensors. Device 260 / 270 uses microphone(s) 213A / 213B to detect sounds from the user and / or the user's real-world environment. In some examples, microphone(s) 213A / 213B include an array of microphones (multiple microphones) optionally working in concert, such as to identify ambient noise or to localize a sound source within a space in the real-world environment.

[0036] In some examples, the device 260 / 270 includes location sensor(s) 204A / 204B and / or display generation component(s) 214A / 214B for detecting the location of the device 260 / 270. For example, the location sensor(s) 204A / 204B may include a Global Positioning System (GPS) receiver that receives data from one or more satellites, allowing the electronic device 260 / 270 to determine the device's absolute position in the physical world.

[0037] In some examples, the electronic device 260 / 270 includes orientation sensor(s) 210A / 210B to detect the orientation and / or movement of the electronic device 260 / 270 and / or the display generating component(s) 214A / 214B. For example, the electronic device 260 / 270 uses the orientation sensor(s) 210A / 210B to track changes in the position and / or orientation of the electronic device 260 / 270 and / or the display generating component(s) 214A / 214B relative to physical objects in a real-world environment, etc. The orientation sensor(s) 210A / 210B optionally include one or more gyroscopes and / or one or more accelerometers.

[0038] Electronic device 260 / 270, in some examples, includes hand tracking sensor(s) 202A / 202B and / or eye tracking sensor(s) 212A / 212B (and / or other body tracking sensor(s), such as leg, torso, and / or head tracking sensor(s)). Hand tracking sensor(s) 202A / 202B are configured to track the position / location of one or more parts of a user's hand and / or the movement of one or more parts of a user's hand relative to the extended reality environment, relative to display generating component(s) 214A / 214B, and / or relative to another defined coordinate system. The eye tracking sensor(s) 212A / 212B are configured to track the position and movement of a user's gaze (more generally, eyes, face, or head) relative to the real world or extended reality environment and / or relative to the display generation component(s) 214A / 214B. In some examples, the hand tracking sensor(s) 202A / 202B and / or the eye tracking sensor(s) 212A / 212B are implemented together with the display generation component(s) 214A / 214B. In some examples, the hand tracking sensor(s) 202A / 202B and / or the eye tracking sensor(s) 212A / 212B are implemented separately from the display generation component(s) 214A / 214B.

[0039] In some examples, hand tracking sensor(s) 202A / 202B (and / or other body tracking sensor(s), such as leg, torso, and / or head tracking sensor(s)) can use image sensor(s) 206A / 206B (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) to capture three-dimensional information from the real world, including one or more body parts (e.g., a human user's hand, leg, or torso). In some examples, the hand can be resolved with sufficient resolution to distinguish fingers and their respective positions. In some examples, the one or more image sensor(s) 206A / 206B are positioned relative to the user to define the field of view of the image sensor(s) 206A / 206B and an interaction space in which the position, orientation, and / or movement of the fingers / hands captured by the image sensor(s) are used as input (e.g., to distinguish from the user's stationary hand or other hands of other people in the real-world environment). Tracking fingers / hands for input (e.g., gestures, touches, taps, etc.) can be advantageous in that it does not require the user to touch, hold, or wear any kind of beacon, sensor, or other marker.

[0040] In some examples, the eye tracking sensor(s) 212A / 212B include at least one eye tracking camera (e.g., an infrared (IR) camera) and / or an illumination source (e.g., an IR light source such as an LED) that emits light toward the user's eyes. The eye tracking camera may be aimed at the user's eyes to receive reflected IR light from the light source directly or indirectly from the eyes. In some examples, both eyes are tracked separately by respective eye tracking cameras and illumination sources, and focus / gaze can be determined from tracking of both eyes. In some examples, one eye (e.g., a dominant eye) is tracked by one or more respective eye tracking cameras / illumination sources.

[0041] Electronic devices 260 / 270 and system 201 are not limited to the components and configuration of FIG. 2 and may include fewer, other, or additional components in multiple configurations. In some examples, system 201 may be implemented in a single device. One or more people using system 201 are optionally referred to herein as one or more users of the device(s). Attention now turns to exemplary simultaneous displays of three-dimensional environments on a first electronic device (e.g., corresponding to electronic device 260) and a second electronic device (e.g., corresponding to electronic device 270). As described below, the first electronic device can communicate with a second electronic device in a multi-user communication session. In some examples, an avatar (e.g., a representation) of a user of the first electronic device may be displayed within the three-dimensional environment at the second electronic device, and an avatar of a user of the second electronic device may be displayed within the three-dimensional environment at the first electronic device. In some examples, a user of the first electronic device and a user of the second electronic device may be associated with a spatial group in the multi-user communication session. In some examples, interaction with content within the three-dimensional environment while the first electronic device and the second electronic device are within a multi-user communication session can associate the user of the first electronic device and the user of the second electronic device with different spatial groups within the multi-user communication session.

[0042] 3 illustrates an example of a spatial group 340 in a multi-user communication session including a first electronic device 360 ​​and a second electronic device 370, according to some examples of the present disclosure. In some examples, the first electronic device 360 ​​may present a three-dimensional environment 350A, and the second electronic device 370 may present a three-dimensional environment 350B. The first electronic device 360 ​​and the second electronic device 370 may be similar to the electronic devices 101 or 260 / 270 and / or may be head-mountable and / or projection-based systems / devices (including hologram-based systems / devices) configured to generate and present three-dimensional environments, such as, for example, a head-up display (HUD), a head-mounted display (HMD), a window with integrated display capabilities, a display formed as a lens designed to be placed over a person's eye (e.g., similar to a contact lens), etc. In the example of FIG. 3, a first user is optionally wearing a first electronic device 360 ​​and a second user is optionally wearing a second electronic device 370, so that the three-dimensional environment 350A / 350B can be defined by X, Y, and Z axes as seen from the perspective of the electronic devices (e.g., a viewpoint associated with electronic device 360 / 370, which may be a head-mounted display).

[0043] 3 , first electronic device 360 ​​may be in a first physical environment that includes table 306 and window 309. Thus, three-dimensional environment 350A presented using first electronic device 360 ​​optionally includes captured portions of the physical environment surrounding first electronic device 360, such as a representation of table 306′ and a representation of window 309′. Similarly, second electronic device 370 may be in a second physical environment that is different from (e.g., separate from) the first physical environment, including floor lamp 307 and coffee table 308. Thus, three-dimensional environment 350B presented using second electronic device 370 optionally includes captured portions of the physical environment surrounding second electronic device 370, such as a representation of floor lamp 307′ and a representation of coffee table 308′. Additionally, the three-dimensional environments 350A and 350B may include representations of the floor, ceiling, and walls of the rooms in which the first electronic device 360 ​​and the second electronic device 370 are located, respectively.

[0044] As mentioned above, in some examples, the first electronic device 360 ​​is optionally in a multi-user communication session with the second electronic device 370. For example, the first electronic device 360 ​​and the second electronic device 370 are configured (e.g., via the communication circuits 222A / 222B) to present a shared three-dimensional environment 350A / 350B that includes one or more shared virtual objects (e.g., content such as images, video, audio, a representation of an application's user interface, etc.). As used herein, the term "shared three-dimensional environment" refers to a three-dimensional environment independently presented, displayed, and / or visible on two or more electronic devices, in which content, applications, data, etc. may be shared and / or presented to users of the two or more electronic devices. In some examples, while the first electronic device 360 ​​is in a multi-user communication session with the second electronic device 370, an avatar corresponding to a user of one electronic device is optionally displayed within the three-dimensional environment displayed via the other electronic device. 3, at first electronic device 360, avatar 315 corresponding to the user of second electronic device 370 is displayed within three-dimensional environment 350A. Similarly, at second electronic device 370, avatar 317 corresponding to the user of first electronic device 360 ​​is displayed within three-dimensional environment 350B.

[0045] In some examples, the presentation of avatar 315 / 317 as part of the shared three-dimensional environment is optionally accompanied by audio effects corresponding to the voice of the user of electronic device 370 / 360. For example, avatar 315 displayed within three-dimensional environment 350A using first electronic device 360 ​​is optionally accompanied by audio effects corresponding to the voice of the user of second electronic device 370. In some such examples, when the user of second electronic device 370 speaks, the user's voice may be detected by second electronic device 370 (e.g., via microphone(s) 213B) and transmitted to first electronic device 360 ​​(e.g., via communications circuitry 222B / 222A), such that the detected voice of the user of second electronic device 370 may be presented as audio (e.g., using speaker(s) 216A) to the user of first electronic device 360 ​​within three-dimensional environment 350A. In some examples, audio effects corresponding to the voice of the user of the second electronic device 370 may be spatialized to appear to the user of the first electronic device 360 ​​as emanating from the location of the avatar 315 within the shared three-dimensional environment 350A (e.g., despite being output from speakers of the first electronic device 360). Similarly, avatar 317 displayed within the three-dimensional environment 350B using the second electronic device 370 is, optionally, accompanied by audio effects corresponding to the voice of the user of the first electronic device 360. In some such examples, when a user of the first electronic device 360 ​​speaks, the user's voice may be detected by the first electronic device 360 ​​(e.g., via microphone(s) 213A) and transmitted to the second electronic device 370 (e.g., via communication circuitry 222A / 222B), so that the detected voice of the user of the first electronic device 360 ​​may be presented as audio (e.g., using speaker(s) 216B) to the user of the second electronic device 370 within the three-dimensional environment 350B.In some examples, audio effects corresponding to the voice of the user of the first electronic device 360 ​​may be spatialized so that they appear to the user of the second electronic device 370 to emanate from the location of the avatar 317 within the shared three-dimensional environment 350B (e.g., despite being output from the speakers of the first electronic device 360).

[0046] In some examples, during a multi-user communication session, avatars 315 / 317 are displayed in three-dimensional environments 350A / 350B with respective orientations that correspond to and / or are based on the orientation of electronic devices 360 / 370 (and / or the users of electronic devices 360 / 370) within the physical environment surrounding electronic devices 360 / 370. For example, as shown in FIG. 3 , in three-dimensional environment 350A, avatar 315 optionally faces toward the viewpoint of a user of first electronic device 360, and in three-dimensional environment 350B, avatar 317 optionally faces toward the viewpoint of a user of second electronic device 370. As a particular user moves their electronic device (and / or themselves) within the physical environment, the user's viewpoint changes accordingly, and therefore the orientation of the user's avatar within the three-dimensional environment may also change. For example, referring to FIG. 3 , if the user of the first electronic device 360 ​​looks leftward within the three-dimensional environment 350A such that the first electronic device 360 ​​is rotated (e.g., by a corresponding amount) to the left (e.g., counterclockwise), the user of the second electronic device 370 will see the avatar 317 corresponding to the user of the first electronic device 360 ​​rotate to the right (e.g., clockwise) relative to the perspective of the user of the second electronic device 370 as the first electronic device 360 ​​moves.

[0047] Additionally, in some examples, during a multi-user communication session, the viewpoint of three-dimensional environment 350A / 350B and / or the location of the viewpoint of three-dimensional environment 350A / 350B optionally change in accordance with movement of electronic device 360 / 370 (e.g., by the user of electronic device 360 / 370). For example, if during a communication session, first electronic device 360 ​​is moved closer toward table 306′ and / or representation of avatar 315 (e.g., because the user of first electronic device 360 ​​has moved forward in the physical environment surrounding first electronic device 360), the viewpoint of three-dimensional environment 350A changes accordingly, such that representation of table 306′, representation of window 309′, and avatar 315 appear larger in the field of view. In some examples, each user may interact with three-dimensional environment 350A / 350B independently, such that changes in perspective of three-dimensional environment 350A and / or interactions with virtual objects in three-dimensional environment 350A by first electronic device 360, optionally, do not affect what is shown in three-dimensional environment 350B at second electronic device 370, and vice versa.

[0048] In some examples, the avatar 315 / 317 is a representation (e.g., a full-body rendering) of the user of the electronic device 370 / 360. In some examples, the avatar 315 / 317 is a representation of a portion of the user of the electronic device 370 / 360 (e.g., a rendering of the head, face, head and torso, etc.). In some examples, the avatar 315 / 317 is a user-personalized, user-selected, and / or user-created representation displayed within the three-dimensional environment 350A / 350B representing the user of the electronic device 370 / 360. While the avatars 315 / 317 shown in FIG. 3 correspond to full-body representations of the user of the electronic device 370 / 360, respectively, it should be understood that alternative avatars, such as those described above, may be provided.

[0049] As described above, while first electronic device 360 ​​and second electronic device 370 are in a multi-user communication session, three-dimensional environment 350A / 350B may be a shared three-dimensional environment presented using electronic devices 360 / 370. In some examples, content viewed by one user at one electronic device may be shared with another user at another electronic device in the multi-user communication session. In some such examples, content may be experienced (e.g., viewed and / or interacted with) by both users (e.g., via their respective electronic devices) within the shared three-dimensional environment. For example, as shown in FIG. 3 , three-dimensional environment 350A / 350B includes a shared virtual object 310 (e.g., optionally a three-dimensional virtual sculpture) that is viewable by and interactive with both users. As shown in FIG. 3 , shared virtual object 310 may be displayed with a grabber affordance (e.g., handlebars) 335 that can be selected to initiate movement of shared virtual object 310 within three-dimensional environment 350A / 350B.

[0050] In some examples, three-dimensional environment 350A / 350B includes non-shared content that is private to one user in a multi-user communication session. For example, in FIG. 3 , first electronic device 360 ​​displays a private application window 330 within three-dimensional environment 350A, which is, optionally, an object that is not shared between first electronic device 360 ​​and second electronic device 370 in the multi-user communication session. In some examples, private application window 330 may be associated with an individual application (e.g., a media player application, a web browsing application, a messaging application, etc.) running on first electronic device 360. Because private application window 330 is not shared with second electronic device 370, second electronic device 370 optionally displays a representation of private application window 330″ within three-dimensional environment 350B. As shown in FIG. 3 , in some examples, the representation of the private application window 330″ may be a faded, occluded, discolored, and / or translucent representation of the private application window 330 that prevents a user of the second electronic device 370 from viewing the content of the private application window 330.

[0051] As described above, in some examples, the user of the first electronic device 360 ​​and the user of the second electronic device 370 are in a spatial group 340 within a multi-user communication session. In some examples, the spatial group 340 may be a baseline (e.g., first or default) spatial group within the multi-user communication session. For example, when the user of the first electronic device 360 ​​and the user of the second electronic device 370 initially join the multi-user communication session, the user of the first electronic device 360 ​​and the user of the second electronic device 370 are automatically (and initially, as described in more detail below) associated (e.g., grouped) with the spatial group 340 within the multi-user communication session. In some examples, while the users are in the spatial group 340 as shown in FIG. 3 , the user of the first electronic device 360 ​​and the user of the second electronic device 370 have a first spatial arrangement (e.g., a first spatial template) within the shared three-dimensional environment. For example, the user of the first electronic device 360 ​​and the user of the second electronic device 370, including the objects displayed in the shared three-dimensional environment, have spatial authenticity within the spatial group 340. In some examples, spatial authenticity requires consistent spatial placement between the users (or their representations) and the virtual objects. For example, the distance between the viewpoint of the user of the first electronic device 360 ​​and the avatar 315 corresponding to the user of the second electronic device 370 may be the same as the distance between the viewpoint of the user of the second electronic device 370 and the avatar 317 corresponding to the user of the first electronic device 360. As described herein, if the location of the viewpoint of the user of the first electronic device 360 ​​moves, the avatar 317 corresponding to the user of the first electronic device 360 ​​moves within the three-dimensional environment 350B according to the movement of the location of the user's viewpoint relative to the viewpoint of the user of the second electronic device 370.Additionally, when a user of first electronic device 360 ​​performs an interaction on shared virtual object 310 (e.g., moves virtual object 310 within three-dimensional environment 350A), second electronic device 370 changes the display of shared virtual object 310 within three-dimensional environment 350B in accordance with the interaction (e.g., moves virtual object 310 within three-dimensional environment 350B).

[0052] It should be understood that in some examples, more than two electronic devices may be communicatively linked in a multi-user communication session. For example, in a situation where three electronic devices are communicatively linked in a multi-user communication session, a first electronic device will display two avatars, rather than just one avatar, corresponding to the users of the other two electronic devices. Thus, it should be understood that the various processes and example interactions described herein with reference to first electronic device 360 ​​and second electronic device 370 in a multi-user communication session optionally apply to a situation where more than two electronic devices are communicatively linked in a multi-user communication session.

[0053] In some examples, it may be advantageous to provide mechanisms for facilitating multi-user communication sessions involving co-located users (e.g., co-located electronic devices associated with users). For example, it may be desirable to enable co-located users in a first physical environment to establish a multi-user communication session to share and present virtual content within a three-dimensional environment that is optionally viewable and / or interactive by the co-located users in the multi-user communication session. As used herein, with respect to a first electronic device, a co-located user corresponds to a local user, and a non-co-located user corresponds to a remote user. As also described above, the three-dimensional environment optionally includes avatars corresponding to remote users of electronic devices that are not co-located in the multi-user communication session. In some examples, avatars corresponding to remote users are generated based on (e.g., using) skeletal data associated with the remote users and presented in the three-dimensional environment. For example, as described in more detail herein, the skeletal data is used to at least partially define one or more visual characteristics of the avatar in the three-dimensional environment (e.g., size (e.g., height) and / or relative thickness of parts of the avatar, such as hands and / or limbs). Additionally, the skeletal data is optionally used to track the movements of remote users, which causes their corresponding avatars to shift and / or move within the three-dimensional environment relative to the viewpoint of the first electronic device, as described above with reference to Figure 3. In some examples, as described below, skeletal data associated with a local user may also be tracked and shared among co-located electronic devices in a multi-user communication session to help facilitate the presentation of and interaction with virtual objects (e.g., avatars and shared virtual content) in the three-dimensional environment.

[0054] 4A-4I illustrate exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. In some examples, while a first electronic device 101a is in a multi-user communication session with a second electronic device 101b, a three-dimensional environment 450A is presented using the first electronic device 101a (e.g., via display 120a), as shown in FIG. 4A-1, and a three-dimensional environment 450B is presented using the second electronic device 101b (e.g., via display 120b), as shown in FIG. 4A-2. In some examples, electronic devices 101a / 101b optionally correspond to or are similar to electronic devices 360 / 370 described above and / or electronic devices 260 / 270 of FIG. 2. In some examples, as shown in FIGS. 4A-1 and 4A-2, a first electronic device 101a is used (e.g., worn on the head) by a first user 402, and a second electronic device 101b is used (e.g., worn on the head) by a second user 404.

[0055] 4A-1 and 4A-2, as shown in an overhead view 410, the first electronic device 101a and the second electronic device 101b are co-located in the physical environment 400. For example, the first electronic device 101a and the second electronic device 101b are both located in the same room, which includes a wall / partition 407, a window 409, and a door 408. In some examples, the determination that the first electronic device 101a and the second electronic device 101b are co-located in the physical environment 400 is based on the distance between the first electronic device 101a and the second electronic device 101b. 4A-1 and 4A-2, the first electronic device 101a and the second electronic device 101b are co-located in the physical environment 400 because the first electronic device 101a is within a threshold distance (e.g., 0.1, 0.5, 1, 2, 3, 5, 10, 15, 20 meters, etc.) of the second electronic device 101b. In some examples, the determination that the first electronic device 101a and the second electronic device 101b are co-located in the physical environment 400 is based on communication between the first electronic device 101a and the second electronic device 101b. For example, in FIGS. 4A-1 and 4A-2, the first electronic device 101a and the second electronic device 101b are configured to communicate (e.g., wirelessly via Bluetooth, Wi-Fi, or a server (e.g., a wireless communication terminal), etc.). In some examples, the first electronic device 101a and the second electronic device 101b are connected to the same wireless network within the physical environment 400. In some examples, the determination that the first electronic device 101a and the second electronic device 101b are co-located within the physical environment 400 is based on the strength of wireless signals transmitted between the electronic devices 101a and 101b. For example, in FIGS. 4A-1 and 4A-2, the first electronic device 101a and the second electronic device 101b are co-located in the physical environment 400 because the strength of the Bluetooth signal (or other wireless signal) transmitted between the electronic devices 101a and 101b is greater than a threshold strength.In some examples, the determination that the first electronic device 101a and the second electronic device 101b are co-located in the physical environment 400 is based on visual detection of the electronic devices 101a and 101b in the physical environment 400. For example, as shown in FIG. 4A-1, the second electronic device 101b is positioned within the field of view of the first electronic device 101a (e.g., because the second user 404 is standing within the field of view of the first electronic device 101a), which enables the first electronic device 101a to visually detect (e.g., identify or scan, such as via object detection or other image processing techniques) the second electronic device 101b (e.g., within one or more images captured by the first electronic device 101a, such as via external image sensors 114b-i and 114c-i). Similarly, as shown in FIG. 4A-2, the first electronic device 101a is optionally positioned within the field of view of the second electronic device 101b (e.g., because the first user 402 is standing within the field of view of the second electronic device 101b), thereby enabling the second electronic device 101b to visually detect the first electronic device 101a (e.g., in one or more images captured by the second electronic device 101b, such as via external image sensors 11b-ii and 114c-ii).

[0056] In some examples, the three-dimensional environment 450A / 450B includes a captured portion of the physical environment 400 in which the electronic device 460 / 470 is located. For example, because the first electronic device 101a and the second electronic device 101b are co-located within the physical environment 400, the three-dimensional environments 450A and 450B include a wall 407 (e.g., a representation of a wall), a window 409 (e.g., a representation of a window), and a door 408 (e.g., a representation of a door), as shown in FIGS. 4A-1 and 4A-2, but from the unique perspectives of the first electronic device 101a and the second electronic device 101b, as shown in the figures. In some examples, the representations may include portions of the physical environment 400 that are viewable through transparent or translucent displays of the electronic devices 101a and 101b. In some examples, the three-dimensional environment 450A / 450B has one or more characteristics of the three-dimensional environment 350A / 350B described above with reference to FIG. 3.

[0057] As described above with reference to FIG. 3, while electronic devices are communicatively linked in a multi-user communication session, users may be represented by avatars corresponding to the users of the electronic devices. In FIGS. 4A-1 and 4A-2, the first electronic device 101a and the second electronic device 101b are co-located in the physical environment 400, so the users of the electronic devices 101a and 101b are represented in the multi-user communication session via their physical personas (e.g., bodies) that are visible in a pass-through of the physical environment 400 (e.g., rather than via virtual avatars). For example, while the first electronic device 101a and the second electronic device 101b are in the multi-user communication session as shown in FIG. 4A-1, the second user 404 is visible in the field of view of the first electronic device 101a, and the first user 402 is visible in the field of view of the second electronic device 101b, as shown in FIG. 4A-2. Similar to what was described above with reference to FIG. 3, when a third user (e.g., a remote user) who is not co-located within physical environment 400 joins the multi-user communication session, the third user is represented via an avatar (e.g., a visual representation) within three-dimensional environments 450A and 450B.

[0058] As also described above with reference to FIG. 3, while a first user 402 of a first electronic device 101a and a second user 404 of a second electronic device 101b are co-located in physical environment 400, and while the first electronic device 101a is in a multi-user communication session with the second electronic device 101b, the first user 402 and the second user 404 may be in a first spatial group within the multi-user communication session. In some examples, the first spatial group has one or more characteristics of spatial group 340 described above with reference to FIG. 3. As also described above, while the first user 402 and the second user 404 are in the first spatial group within the multi-user communication session, the users have a first spatial arrangement within the shared three-dimensional environment (e.g., represented by the locations of and / or the distance between users 402 and 404 within the overhead view 410 of FIG. 4A ) that is determined by the physical locations of electronic devices 101a and 101b within physical environment 440. Specifically, the first electronic device 101a and the second electronic device 101b experience spatial reality within the first spatial group as dictated by the physical locations and / or orientations of the first user 402 and the second user 404, respectively.

[0059] 4B , while the first electronic device 101a is co-located with the second electronic device 101b within the physical environment 400 (e.g., and optionally while the first electronic device 101a is in a multi-user communication session with the second electronic device 110b), the first electronic device 101a and the second electronic device 101b share skeletal data associated with the first user 402 and the second user 404, respectively (e.g., despite the first electronic device 101a and the second electronic device 101b being co-located within the physical environment 400). For example, as described below, by tracking skeletal data associated with the local user (e.g., despite the fact that an avatar corresponding to the local user is not necessarily displayed, as described above), the first electronic device 101a and the second electronic device 101b can more efficiently facilitate and synchronize operations within the multi-user communication session, such as interactions related to displaying and / or interacting with virtual content. Specifically, in some examples, the first electronic device 101a transmits skeletal data associated with the first user 402 to the second electronic device 101b (e.g., wirelessly directly or indirectly via a server), and the second electronic device 101b transmits skeletal data associated with the second user 404 to the first electronic device 101a.

[0060] In some examples, the skeletal data associated with the first user 402 and the second user 404 includes body (e.g., skeletal) pose data corresponding to the first user 402 and the second user 404. For example, the skeletal data associated with the first user 402 includes information corresponding to the position and / or orientation of the first user 402 relative to the origin of the first spatial group (e.g., the geometric center of the shared coordinate system of the first spatial group, such as origin 530 described below with reference to FIG. 5B ), and the skeletal data associated with the second user 404 includes information corresponding to the position and / or orientation of the second user 404 relative to the origin of the first spatial group. For example, as shown in the overhead view 410 of FIG. 4A-2, the first electronic device 101a is located at a first location relative to the origin of the first spatial group, and the second electronic device 101b is located at a second location, different from the first location, relative to origin 431. Further, the first electronic device 101a is located at a first distance from the origin, and the second electronic device 101b is located at a second distance (e.g., different from or equal to the first distance) from the origin. Additionally, in some examples, the origin enables virtual content (e.g., shared applications, user interfaces, three-dimensional objects / models, etc.) presented in the shared three-dimensional environment to be positioned at the same location within the first spatial group for all local users (e.g., by positioning the virtual content relative to the origin).

[0061] In some examples, the above-mentioned origin (e.g., and therefore the shared coordinate system) is defined based on the physical environment 400 (e.g., the physical room in which the first electronic device 101a and the second electronic device 101b are located). In some examples, the first electronic device 101a and the second electronic device 101b are each configured to analyze the physical environment 400 and determine the origin (e.g., and therefore the shared coordinate system) based on simultaneous localization and mapping (SLAM) data exchanged between the first electronic device 101a and the second electronic device 101b (e.g., SLAM data stored individually on the electronic devices 101a and 101b or SLAM data stored on one of the electronic devices 101a and 101b). For example, the first electronic device 101a and the second electronic device 101b may utilize the SLAM data to facilitate a shared understanding of one or more physical characteristics of the physical environment 400, such as the dimensions of the physical environment, the physical objects within the physical environment, the visual appearance of the physical environment (e.g., color and lighting characteristics), and an origin may be defined in the first spatial group accordingly. In some examples, the first electronic device 101a and the second electronic device 101b are each configured to analyze the physical environment 400 to determine an origin based on one or more characteristics of other electronic devices as individually perceived by the electronic devices. For example, based on one or more images captured via external image sensors 114b-i and 114c-i, the first electronic device 101a analyzes the position of the second electronic device 101b in the physical environment relative to the viewpoint of the first electronic device 101a, and based on one or more images captured via external image sensors 114b-ii and 114c-ii, the second electronic device 101b analyzes the position of the first electronic device 101a in the physical environment 400 relative to the viewpoint of the second electronic device 101b to establish spatial reality within the first spatial group and thus define the origin.

[0062] In some examples, the skeletal data associated with the first user 402 and the second user 404 includes information corresponding to one or more physical characteristics of the first user 402 and the second user 404. For example, the skeletal data associated with the first user 402 includes information corresponding to the height and / or size (e.g., weight) of the first user 402, and the skeletal data associated with the second user 404 includes information corresponding to the height and / or size of the second user 404. In some examples, the skeletal data associated with the first user 402 and the second user 404 includes information corresponding to the movement of one or more parts of the first user 402 and the second user 404. For example, the skeletal data associated with the first user 402 includes information corresponding to one or more joints of the first user's 402 upper body in space (e.g., joints in the first user's 402 fingers, hands, and / or arms), and the skeletal data associated with the second user 404 includes information corresponding to one or more joints of the second user's 404 upper body in space (e.g., joints in the second user's 404 fingers, hands, and / or arms). In some examples, the skeletal data associated with the first user 402 and the second user 404 includes information corresponding to one or more physical objects associated with the first user 402 and the second user 404. For example, the one or more physical objects include eyeglasses worn by the first user 402 and / or the second user 404, a chair or wheelchair in which the first user 402 and / or the second user 404 are seated, a cane, walker, or other assistive object used by the first user 402 and / or the second user 404, etc.

[0063] In some examples, the first electronic device 101a and the second electronic device 101b determine skeletal data associated with the first user 402 and the second user 404 via one or more input devices and / or sensors (e.g., cameras) of the first electronic device 101a and the second electronic device 101b, respectively. For example, in FIG. 4A-1, the first electronic device 101a determines skeletal data associated with the first user 402 based on images captured using one or more inward-facing cameras (e.g., one or more internal image sensors 114a-i) and / or one or more outward-facing cameras (e.g., one or more external image sensors 114b-i and 114c-i). 4A-2, the second electronic device 101b optionally determines skeletal data associated with the second user 404 based on images captured using one or more inward-facing cameras (e.g., one or more internal image sensors 114a-ii) and / or one or more outward-facing cameras (e.g., one or more external image sensors 114b-ii and 114c-ii). It should be understood that in some examples, the skeletal data associated with the first user 402 and the second user 404 may additionally or alternatively be determined using any one or combination of the sensors described with reference to FIG. 2. In some examples, the skeletal data associated with the first user 402 and the second user 404 is determined based on information provided by the first user 402 and the second user 404. For example, information corresponding to the height and / or size (e.g., weight) of the first user 402 is provided to the first electronic device 101a by the first user 402 (e.g., via user input, such as when setting up a user profile associated with the first user 402) and stored in memory of the first electronic device 101a. Similarly, information corresponding to the height and / or size of the second user 404 is, optionally, provided to the second electronic device 101b by the second user 404 (e.g., via user input, such as when setting up a user profile associated with the second user 404) and stored in memory of the second electronic device 101b.

[0064] In some examples, when the first electronic device 101a initially enters a multi-user communication session with the second electronic device (e.g., upon creation and / or synchronization of the shared coordinate space of the first spatial group described above), skeletal data associated with the first user 402 and the second user 404 is exchanged between the first electronic device 101a and the second electronic device 101b. In some examples, while the first electronic device 101a is in a multi-user communication session with the second electronic device 101b, skeletal data associated with the first user 402 and the second user 404 is exchanged between the first electronic device 101a and the second electronic device 101b periodically (e.g., multiple times per second, or at regular and / or predetermined time intervals such as every second, every 5 seconds, every 30 seconds, every 60 seconds, every 120 seconds, every 3 minutes, every 5 minutes, every 10 minutes, every 30 minutes, etc.). In some examples, while the first electronic device 101a is in a multi-user communication session with the second electronic device 101b, skeletal data associated with the first user 402 and the second user 404 is exchanged between the first electronic device 101a and the second electronic device 101b in response to detecting user input (e.g., hand-based input and / or gaze-based input), as described in more detail herein.

[0065] In some examples, skeletal data exchanged between electronic devices in a multi-user communication session including collocated participants (e.g., local users) corresponds to and / or is the same as skeletal data exchanged between electronic devices in a multi-user communication session including non-collocated participants (e.g., remote users). For example, in FIGS. 4A-1 and 4A-2, if the multi-user communication session includes a third electronic device associated with a third user, and the third electronic device is not collocated with the first electronic device 101a and the second electronic device 101b in physical environment 400 (e.g., the third user is located in a second, separate physical environment), the skeletal data exchanged between the first electronic device 101a and the second electronic device 101b (e.g., collocated electronic devices) is the same as (e.g., includes the same information as) the skeletal data exchanged between the first electronic device 101a or the second electronic device 101b and the third electronic device (e.g., non-collocated electronic device). Alternatively, in some examples, skeletal data exchanged between electronic devices in a multi-user communication session including co-located participants (e.g., local users) does not correspond to and / or is at least partially different from skeletal data exchanged between electronic devices in a multi-user communication session including non-co-located participants (e.g., remote users). For example, in FIGS. 4A-1 and 4A-2, if the multi-user communication session includes a third electronic device associated with a third user, and the third electronic device is not co-located with the first electronic device 101a and the second electronic device 101b in physical environment 400 (e.g., the third user is located in a second, separate physical environment), the skeletal data exchanged between the first electronic device 101a and the second electronic device 101b (e.g., the co-located electronic devices) will differ (e.g., include different information and / or include a subset of information) from the skeletal data exchanged between the first electronic device 101a or the second electronic device 101b and the third electronic device (e.g., the non-co-located electronic device).As another example, skeletal data exchanged between electronic devices in a multi-user communication session including co-located participants may be exchanged at a different rate (e.g., faster or slower) than skeletal data exchanged between electronic devices in a multi-user communication session including non-co-located participants. As an example, skeletal data associated with a local user (e.g., first user 402 and second user 404) may include information related to the local user's pose, the local user's height and / or size, and / or the local user's upper body joint movements, but may not include information related to the local user's facial texture and / or expressions (e.g., smile, frown, laugh, etc.) (e.g., because an avatar corresponding to the local user is not rendered / displayed for the local user as described above). However, in such a case, skeletal data associated with a third user (e.g., a remote user) may include information related to the third user's facial texture and / or expressions (e.g., in addition to other information described above).

[0066] In some examples, as described above, skeletal data associated with the first user 402 and the second user 404 exchanged between the first electronic device 101a and the second electronic device 101b assists the first electronic device 101a and the second electronic device 101b in facilitating interaction within the multi-user communication session (e.g., by helping to improve device responsiveness to user input). In Figures 4B-1 and 4B-2, while the first electronic device 101a and the second electronic device 101b are in a multi-user communication session, the first electronic device 101a detects input provided by the first user 402 that corresponds to a request to display one or more session controls for the multi-user communication session. 4B-1 and 4B-2, the first electronic device 101a detects an air gesture (e.g., an air pinch gesture, an air tap, or a touch gesture) provided by the hand 403 of the first user 402, optionally while the gaze 425 is directed toward the second user 404 within the three-dimensional environment 450A. It should be appreciated that in some examples, the first electronic device 101a detects an alternative input provided by the first user 402 corresponding to a selection of the second user 404 within the three-dimensional environment 450A, such as a gaze dwell or a verbal command directed toward the second user 404.

[0067] In some examples, as shown in FIG. 4C-1, in response to detecting an input corresponding to a selection of the second user 404 in the three-dimensional environment 450A, the first electronic device 101a displays a communication session user interface 415 in the three-dimensional environment 450A. In some examples, as shown in FIG. 4C-1, the communication session user interface 415 includes one or more controls for a multi-user communication session. For example, as shown in FIG. 4C-1, the communication session user interface 415 includes a video option 416a selectable to initiate video communication with the second user 404 (e.g., “Jill”) during the multi-user communication session (e.g., display a video conferencing user interface in the three-dimensional environment 450A including a (e.g., live) video stream of the second user 404 (e.g., captured via one or more cameras of the second electronic device 101b)). Additionally, in some examples, the communication session user interface 415 includes a mute option 416b selectable to disable / enable audio captured by the first electronic device 101a (e.g., deactivate / activate the microphone of the first electronic device 101a), a subscription option 416c selectable to initiate a process for sharing content (e.g., with the second user 404) in a multi-user communication session, and an end option 416d selectable to end the multi-user communication session between the first electronic device 101a and the second electronic device 101b. In some examples, as shown in FIG. 4C-1 , the communication session user interface 415 is displayed with a moving element 435 (e.g., a grabber bar) within the three-dimensional environment 450A. In some examples, the moving element 435 is selectable to trigger spatial refinement in the three-dimensional environment 450A / 450B. In some examples, the spatial refinement corresponds to movement and / or rearrangement of avatars and / or shared objects (e.g., triggered by movement of a shared object or moving element 435) that enables spatial authenticity to be maintained within the first spatial group of the first user 402 and the second user 404.In some examples, the communication session user interface 415 (e.g., and the moving element 435) is displayed at a location within the three-dimensional environment 450A that is selected based on the location of the second user 404 within the three-dimensional environment 450A from the perspective of the first electronic device 101a. For example, as shown in the overhead view 410 of FIG. 4C-1, the communication session user interface 415 is displayed in the three-dimensional environment 450A in front of the second user 404 from the perspective of the first electronic device 101a and / or at a predetermined distance from the second user 404 from the perspective of the first electronic device 101a.

[0068] Additionally, in some examples, the communication session user interface 415 includes an information option or a participants option 417, as shown in FIG. 4C-1. In some examples, the information option 417 is selectable to cause the first electronic device 101a to display a list and / or other visual indication of the participants currently active in the multi-user communication session (e.g., within the communication session user interface 415), as described in more detail below.

[0069] In some examples, the display of the communication session user interface 415 (e.g., and movement element 435) is performed using skeletal data associated with the first user 402 and the second user 404. For example, in FIGS. 4B-1 and 4B-2, the first electronic device 101a uses the skeletal data associated with the first user 402 to determine a gaze direction and / or location in the three-dimensional environment 450A (e.g., to determine the location of the line of sight 425 in FIG. 4B-1), for example, based on information corresponding to the movement and / or focus of the first user's 402's eyes. In some examples, utilizing skeletal data associated with the first user 402 to identify the second user 404 as the target of the selection input provided by the hand 403 described above with reference to FIGS. 4B-1 and 4B-2 (in addition to sensor input, such as input detected via one or more internal image sensors 114a-i) assists the first electronic device 101a in distinguishing the second user 404 from other users (e.g., people) present in the physical environment 400, such as the third user 406. For example, as shown in FIGS. 4A-1-4C-1, the physical environment 400 includes the third user 406 located within the field of view of the first electronic device 101a and the second electronic device 101b. Additionally, as shown in FIG. 4B-1, for example, the third user 406 is located adjacent to (e.g., to the right of) the second user 404 within the three-dimensional environment 450A from the perspective of the first electronic device 101a. 4B-1, the third user 406 is not a participant in the multi-user communication session including the first electronic device 101a and the second electronic device 101b when the first electronic device 101a detects the air gesture provided by the hand 403 of the first user 402. Thus, the skeletal data associated with the first user 402 enables the first electronic device 101a to identify the second user 404 as the target of the input provided by the hand 403 of the first user 402 (e.g., as opposed to the third user 406).Additionally, in some examples, skeletal data associated with the second user 404 (e.g., provided by the second electronic device 101b to the first electronic device 101a) further assists the first electronic device 101a in identifying the second user 404 as a target of input provided by the hand 403 of the first user 402. For example, as described above, the skeletal data associated with the second user 404 includes information corresponding to the location of the second user 404 within the three-dimensional environment 450A relative to the viewpoint of the first electronic device 101a (e.g., relative to the origin of the first spatial group), thereby enabling the first electronic device 101a to determine an intersection point from a line-of-sight vector corresponding to the line of sight 425 of the first user 402 that extends to a point cloud or volumetric data map corresponding to the second user 404 within the three-dimensional environment 450A. In some examples, the communication session user interface 415 (and, for example, the moving element 435) is also positioned within the three-dimensional environment 450A using skeletal data associated with the second user 404 (e.g., based on information corresponding to the position of the second user 404).

[0070] 4C-1, the first electronic device 101a detects a selection input directed toward the information option 417 in the communication session user interface 415. For example, as shown in FIG. 4C-1, the first electronic device 101a optionally detects an air gesture, such as an air pinch gesture or an air tap gesture, performed by the hand 403 of the first user 402 while the gaze 425 of the first user 402 is directed toward the information option 417 in the communication session user interface 415.

[0071] In some examples, as shown in FIG. 4C-2, in response to detecting selection of the information option 417, the first electronic device 101a displays a list of participants currently active in the multi-user communication session within the three-dimensional environment 450A. For example, as shown in FIG. 4C-2, because the first user 402 and the second user 404 are currently active in the multi-user communication session including the first electronic device 101a and the second electronic device 101b, the first electronic device 101a updates the display of the communication session user interface 415 to include a visual indication of the second user 404 (e.g., Jill) and a visual indication of the first user 402 (e.g., Jack). In some examples, as shown in FIG. 4C-2, the list of participants includes images (e.g., photos, contact posters, icons, or other visual representations) representing the users and / or names (or usernames) corresponding to the users currently active in the multi-user communication session. Alternatively, in some examples, the first electronic device 101a displays the list of participants as and / or within a separate user interface element within the three-dimensional environment 450A. For example, the first electronic device 101a displays the list of participants within a user interface element overlaid on or adjacent to (e.g., above, below, or to the side of) the communication session user interface 415 in the three-dimensional environment 450A from the perspective of the first electronic device 101a. As another example, the first electronic device 101a can update the display of a portion of the communication session user interface 415 to include the list of participants in the three-dimensional environment 450A (e.g., while maintaining the display of other portions of the communication session user interface 415, such as the information option 417).In some examples, the first electronic device 101a returns to displaying the communication session user interface 415 shown in FIG. 4C-1 in response to detecting a second selection of the information option 415 (e.g., when the information option 415 is displayed within the communication session user interface 415 when a list of participants is displayed) or in response to detecting a selection input directed toward the second user 404, as described above.

[0072] 4C-2, when the first electronic device 101a updates the display of the communication session user interface 415 to include a list of participants who are currently active in the multi-user communication session, the first electronic device 101a displays a user interface element 419 in the three-dimensional environment 450A that provides a visual indication of where the participants who are currently active in the multi-user communication session are spatially located within the three-dimensional environment 450A relative to the viewpoint of the first electronic device 101a. For example, as shown in FIG. 4C-2, the first electronic device 101a displays a user interface element 419 in the three-dimensional environment 450A that provides a visual indication that the second user 404 corresponds to “Jill” who is listed in the list of active participants in the communication session user interface 415. 4C-2, the first electronic device 101a displays the user interface element 419 at a location within the three-dimensional environment 450A that is based on the physical location of the second user 404 (e.g., and / or the second electronic device 101b) within the physical environment 400 from the perspective of the first electronic device 101a. For example, as shown in the overhead view 410 of FIG. 4C-2, the first electronic device 101a displays the user interface element 419 spatially above the second user 404 within the three-dimensional environment 450A from the perspective of the first electronic device 101a. In some examples, as also described above, the first electronic device 101a displays the user interface element 419 within the three-dimensional environment 450A based on and / or using skeletal data provided by the second electronic device 101b.

[0073] Additionally, in some examples, as shown in FIG. 4C-2, when the first electronic device 101a displays the user interface element 419 (e.g., simultaneously with a list of active participants in the communication session user interface 415), the first electronic device 101a displays an add option 421 within the three-dimensional environment 450A. In some examples, the add option 421 is selectable to initiate a process of adding a third electronic device 101c (e.g., associated with a third user 406) to a current multi-user communication session between the first electronic device 101a and the second electronic device 101c, as described in more detail below. In some examples, as shown in FIG. 4C-2, the first electronic device 101a displays the add option 421 at a location within the three-dimensional environment 450A that is based on the physical location of the third user 406 (e.g., and / or the third electronic device 101c) within the physical environment 400 from the perspective of the first electronic device 101a. For example, as shown in the overhead view 410 of FIG. 4C-2, the first electronic device 101a displays the add option 421 spatially above the third user 406 in the three-dimensional environment 450A from the perspective of the first electronic device 101a. In some examples, because the third user 406 (e.g., and / or the third electronic device 101c) is known and / or recognized by the first electronic device 101a, the first electronic device 101a displays the add option 421 in the three-dimensional environment 450A when displaying the list of active participants in the multi-user communication session in the communication session user interface 415 of FIG.For example, the third user 406 is associated with a contact in a contact list on the first electronic device 101a (e.g., associated with a separate application such as a phone application, a messaging application, an email application, a social media application, a contacts application, etc.), and the first electronic device 101a recognizes that the third user 406 is associated with the contact in the contact list based on a recognition of the third electronic device 101c (e.g., as described herein above) and / or a visual association between the face of the second user 404 and an image included in the list of participants in the communication session user interface 415. In some examples, the first electronic device 101a displays the add option 421 in the three-dimensional environment 450A because the first electronic device 101a and the third electronic device 101c have previously participated in and / or communicated within a multi-user communication session. In some examples, as also described herein, the first electronic device 101a displays the additional options 421 in the three-dimensional environment 450A because the third electronic device 101c is co-located with the first electronic device 101a in the physical environment 400. In some examples, as also described above, the first electronic device 101a displays the additional options 421 in the three-dimensional environment 450A based on and / or using skeletal data provided by the third electronic device 101c. Alternatively, in some examples, the first electronic device 101a displays the additional options 421 in the three-dimensional environment 450A based on image data captured via the external image sensors 114b-i and / or 114c-i. For example, the first electronic device 101a uses the image data to determine the location and / or orientation of the third user 406 (e.g., and / or the third electronic device 101c) within the three-dimensional environment 450A relative to the viewpoint of the first electronic device 101a and displays additional options 421 within the three-dimensional environment 450A based on the determined position and / or orientation of the third user 406.

[0074] 4C-2 as being displayed above the second user 404 and the third user 406, respectively, it should be understood that alternative display locations are possible. For example, the first electronic device 101a may alternatively display the user interface element 419 and / or the additional option 421 in front of (e.g., overlaid on a portion of the body of), in addition to, and / or below the second user 404 and / or the third user 406, respectively, from the perspective of the first electronic device 101a within the three-dimensional environment 450A.

[0075] 4D-1 and 4D-2, the first electronic device 101a and the second electronic device 101b detect an indication of a request to add a third electronic device 101c (e.g., associated with a third user 406) to a current multi-user communication session between the first electronic device 101a and the second electronic device 101b. For example, in FIGS. 4D-1 and 4D-2, the aforementioned third user 406 is wearing the third electronic device 101c (e.g., the third electronic device 101c is powered on and being used by the third user 406) and is providing input to the third electronic device 101c to join the current multi-user communication session. In some examples, this indication corresponds to selection of an affordance displayed by the third electronic device 101c to join the multi-user communication session including the first electronic device 101a and the second electronic device 101b. In some examples, the first electronic device 101a and the second electronic device 101b generate a notification corresponding to a request prompting the first user 402 and / or the second user 404 to accept / approve a request from the third electronic device 101c to join the multi-user communication session. Alternatively, in some examples, this indication corresponds to a selection of an affordance displayed by the first electronic device 101a and / or the second electronic device 101b that is selectable to add the third electronic device 101c to the multi-user communication session, such as selecting the add option 421 in FIG. 4C-2 described above.

[0076] 4D-1 and 4D-2, when the third electronic device 101c participates in the multi-user communication session, the third electronic device is co-located with the first electronic device 101a and the second electronic device 101b. For example, as shown in the overhead view 410 of FIG. 4D-2, the third electronic device 101c is located (e.g., with the third user 406) within the physical environment 400 in which both the first electronic device 101a and the second electronic device 101b are located. In some examples, as also described above, while the third electronic device 101c is within the physical environment 400, the third electronic device 101c is within a threshold distance (e.g., as described above) of the first electronic device 101a and / or the second electronic device 101b. Additionally, in some examples, as shown in FIGS. 4D-1 and 4D-2, the third electronic device 101c is within the field of view of the first electronic device 101a and / or the second electronic device 101b.

[0077] In some examples, as also described above, when the third electronic device 101c joins a multi-user communication session including the first electronic device 101a and the second electronic device 101b, the third electronic device 101c shares (e.g., transmits) skeletal data associated with the third user 406 with the first electronic device 101a and the second electronic device 101b. For example, as also described herein, the third electronic device 101c provides (e.g., wirelessly) to the first electronic device 101a and the second electronic device 101b information corresponding to the pose (e.g., orientation and / or position) of the third user 406 relative to an origin within the first spatial group, the height and / or size (e.g., weight) of the third user 406, one or more physical objects (e.g., eyeglasses, a wheelchair or wheel scooter, a cane, etc.) associated with the third user 406, and / or the position and / or movement of one or more joints of the upper body of the third user 406 (e.g., joints of the fingers, hands, and / or arms of the third user 406). In some examples, the third electronic device 101c determines skeletal data associated with the third user 406 and determines skeletal data associated with the first user 402 and the second user 404 in the same or similar manner as described above with reference to the first electronic device 101a and the second electronic device 101b. Additionally, in some examples, as also described above, the third electronic device 101c receives skeletal data associated with the first user 402 and skeletal data associated with the second user 404 (e.g., from the first electronic device 101a and the second electronic device 101b) when the third electronic device 101c joins the multi-user communication session.

[0078] 4D-1 and 4D-2, when a third electronic device 101c joins a multi-user communication session including the first electronic device 101a and the second electronic device 101b, the first electronic device 101a and the second electronic device 101b display a message element 412 (e.g., a notification) indicating that the third electronic device 101c has joined the multi-user communication session (e.g., that the third user 406 is now participating in the multi-user communication session). In some examples, the first electronic device 101a and the second electronic device 101b display the message element 412 in the three-dimensional environment 450A / 450B using skeletal data associated with the third user 406. 4D-1 and 4D-2, the first electronic device 101a and the second electronic device 101b use information corresponding to the location of the third user 406 included in skeletal data associated with the third user 406 to display the message element 412 at a location corresponding to the location of the third user 406 in the three-dimensional environment 450A / 450B (e.g., rather than and / or in addition to positioning the message element 412 based on the approximate location of the third user 406 visually detected via one or more cameras of the first electronic device 101a and the second electronic device 101b). Specifically, the first electronic device 101a and the second electronic device 101b optionally display the message element 412 above the third user 406 in the three-dimensional environment 450A / 450B from the perspectives of the first electronic device 101a and the second electronic device 101b, as shown in the overhead view 410.

[0079] 4D-2, the first electronic device 101a and the third electronic device 101c detect an indication that the second electronic device 101b has left and / or exited (e.g., is no longer participating in) the multi-user communication session. For example, as shown in the overhead view 410 of FIG. 4D-2, the second electronic device 101b detects movement of the second electronic device 101b in the physical environment 400 as indicated by arrow 471, optionally caused by movement of the second user 404 in the physical environment 400. In some examples, the movement of the second user 404 in the physical environment 400 causes the second electronic device 101b to be located farther than a threshold distance (e.g., 1, 2, 5, 10, 15, 20, 30, 40 meters, etc.) from the first electronic device 101a and the second electronic device 101b, thereby causing the second electronic device 101b to leave the multi-user communication session. In some examples, the second electronic device 101b detects disassociation from the second user 404 (e.g., the second user 404 unplugs the second electronic device 101b so that the second electronic device 101b is no longer being used and / or logs off / logs out of the user profile associated with the second user 404 on the second electronic device 101b), which causes the second electronic device 101b to leave the multi-user communication session. In some examples, the second electronic device 101b is powered off (e.g., by the second user 404), which causes the second electronic device 101b to leave the multi-user communication session. In some examples, the second electronic device 101b detects selection of the end option 416d of the communication session user interface 415 described above with reference to FIG. 4C-1, which causes the second electronic device 101b to end the multi-user communication session.

[0080] 4E, in response to detecting an indication that the second electronic device 101 has left the multi-user communication session, the first electronic device 101a (e.g., and the third electronic device 101c) displays a message element 414 (e.g., a notification) in the three-dimensional environment 450A indicating that the second electronic device 101b has left the multi-user communication session (e.g., that the second user 404 is no longer participating in the multi-user communication session). In some examples, the first electronic device 101a (e.g., and the third electronic device 101c) displays the message element 414 in the three-dimensional environment 450A using skeletal data associated with the second user 404. 4E, the first electronic device 101a displays the message element 414 at a location corresponding to the second user's 404's previous location in the three-dimensional environment 450A (e.g., the second user's 404's location before leaving the multi-user communication session) using information corresponding to the second user's 404's location included in skeletal data associated with the second user 404 (e.g., instead of and / or in addition to positioning the message element 414 based on the second user's 404's last approximate location visually detected via one or more cameras of the first electronic device 101a before the second electronic device 101b left the multi-user communication session). Specifically, as shown in the overhead view 410 of FIG. 4E, the first electronic device 101 (e.g., and the third electronic device 101c) optionally displays the message element 414 at the second user's 404's location shown in the overhead view 410 of FIG. 4D-2.

[0081] 4F, the third electronic device 101c detects movement of the third electronic device 101c in the physical environment 400. For example, as shown in the overhead view 410 of FIG. 4F, the third user 406 moves within the physical environment 400 as indicated by arrow 472, causing the third electronic device 101c to move relative to the viewpoint of the first electronic device 101a within the physical environment 400. In some examples, as shown in the overhead view 410 of FIG. 4F, the movement corresponds to movement of the third electronic device 101c (e.g., and the third user 406) behind a wall 407 within the physical environment 400 relative to the viewpoint of the first electronic device 101a.

[0082] In some examples, as shown in FIG. 4G-1, when the third user 406 moves behind a wall 407 in the physical environment 400, the third user 406 (e.g., and the third electronic device 101c) is no longer visible from the perspective of the first electronic device 101a. For example, as shown in FIG. 4G-1, the third user 406 and the third electronic device 101c are blocked / occluded by the wall 407 relative to the perspective of the first electronic device 101a. In addition, as shown in FIG. 4G-2, after the third user 406 moves behind the wall 407 in the physical environment 400 relative to the perspective of the first electronic device 101a, the view of the three-dimensional environment 450C is updated at the third electronic device 101c based on the updated perspective of the third electronic device 101c. For example, as shown in FIG. 4G-2, three-dimensional environment 450C includes painting 413 (eg, a representation of a painting) that is visible from the perspective (eg, in a pass-through) of third electronic device 101c.

[0083] In some examples, during a multi-user communication session, pursuant to a determination that movement of an individual user within physical environment 400 causes the individual user to be at least partially occluded by a portion of physical environment 400 (e.g., by a physical object within physical environment 400), first electronic device 101a provides a visual indication of the individual user indicating the presence of the individual user through / behind the portion of physical environment 400. In some examples, as described below, the visual indication of the individual user is presented in a three-dimensional environment using skeletal data associated with the individual user.

[0084] 4G-1, as described above, when the third user 406 moves behind the wall 407 in the physical environment 400, the third user 406 is occluded by the wall 407 from the perspective of the first electronic device 101a. Accordingly, as alluded to above, the first electronic device 101a optionally displays a visual indication of the third user 406 within the three-dimensional environment 450A indicating the location of the third user 406 relative to the perspective of the first electronic device 101a. In some examples, as shown in FIG. 4H, displaying the visual indication of the third user 406 includes presenting an avatar 426 corresponding to the third user 406 within the three-dimensional environment 450A, as shown in FIG. 4H. For example, as shown in FIG. 4H, the first electronic device 101a displays the avatar 426 in the three-dimensional environment 450A at a location and / or orientation that corresponds to the location and / or orientation of the third user 406 in the physical environment 400. Thus, in some examples, avatar 426 provides first user 402 with a visual indication of third user 406's position in three-dimensional environment 450A while first electronic device 101a and third electronic device 101c are in a multi-user communication session, even though third user 406 is physically obstructed from the first electronic device 101a's perspective by wall 407. In some examples, avatar 426 has one or more characteristics of avatar 315 / 317 described above with reference to FIG.

[0085] 4I, displaying the visual indication of the third user 406 alternatively includes displaying a user interface object 418 within the three-dimensional environment 450A. For example, as shown in FIG. 4I, the first electronic device 101a displays the user interface object 418 at a location in the three-dimensional environment 450A that corresponds to the location of the third user 406 in the physical environment 400. In some examples, the first electronic device 101a displays the user interface object 418 rather than the avatar 426 described above based on the user activity of the third user 406. For example, the third user 406 may be interacting with an application private to the third user 406 on the third electronic device 101c and / or may be participating in a private phone or video call on the third electronic device 101c, causing the first electronic device 101a to display the user interface object 418 rather than the avatar 426 to protect the privacy of the third user 406. As another example, the third user 406 may be speaking with another person in the physical environment 400 (e.g., a person who may not necessarily be participating in the multi-user communication session), which causes the first electronic device 101a to display the user interface object 418 rather than the avatar 426 to protect the privacy of the third user 406 and / or the other person.

[0086] In some examples, as described above, the first electronic device 101a displays a visual indication of the third user 406 (e.g., the avatar 426 of FIG. 4H or the user interface object 418 of FIG. 4I) based on skeletal data associated with the third user 406 (e.g., provided to the first electronic device 101a by the third electronic device 101c). For example, similar to what has been described herein, the skeletal data associated with the third user 406 includes information corresponding to the position and / or orientation of the third user 406 relative to the viewpoint of the first electronic device 101a (e.g., relative to the origin in the first spatial group). Thus, the skeletal data associated with the third user 406 allows the first electronic device 101a to determine the precise location and / or orientation of the third user 406 behind the wall 407, even though the third user 406 is not visible from the perspective of the first electronic device 101a, thereby enabling the first electronic device 101a to display a visual indication (e.g., an avatar 426 or a user interface object 418) at a location within the three-dimensional environment 450A that indicates the third user's 406's physical location within the physical environment 400 relative to the perspective of the first electronic device 101a. For example, information corresponding to the location and / or orientation of the third user 406 allows the first electronic device 101a to accurately render the avatar 426 corresponding to the third user 406 as being located at the same location and / or having the same orientation within the three-dimensional environment 450A. It should be understood that in some examples, the third electronic device 101c similarly displays a visual indication of the first user 402 (e.g., an avatar or user interface object corresponding to the first user 402) in the three-dimensional environment 450C (e.g., FIG. 4G-2) based on skeletal data associated with the first user 402 provided to the third electronic device 101c by the first electronic device 101a.

[0087] Thus, as outlined above, providing systems and methods for displaying and / or interacting with virtual objects (e.g., avatars and / or virtual content) in a shared three-dimensional environment during a multi-user communication session based on skeletal data associated with the co-located participants of the multi-user communication session advantageously enables the co-located participants of the multi-user communication session to experience synchronized interaction with the content and other users, thereby improving user-device interaction. Additionally, presenting a visual indication of an individual participant using skeletal data associated with an individual participant in a multi-user communication session pursuant to a determination that the individual participant is at least partially occluded by a portion of the co-located participant's physical environment, as another advantage, enables other participants to retain spatial awareness of the individual participant. We now turn our attention to additional examples of displaying and / or interacting with virtual objects in a multi-user communication session involving co-located users and electronic devices based on skeletal data associated with the co-located participants.

[0088] 5A-5G illustrate exemplary interactions with virtual content based on skeletal data associated with participants in a multi-user communication session, according to some examples of the present disclosure. In FIGS. 5A-1 and 5A-2, a first electronic device 101a (e.g., associated with a first user 502) and a second electronic device 101b (e.g., associated with a second user 504) are engaged in a multi-user communication session. In some examples, the first user 502 and the second user 504 correspond to the first user 402 and the second user 404, respectively, in FIGS. 4A-4I. As shown in the overhead view 510 of FIG. 5A-2, the first electronic device 101a and the second electronic device 101b are co-located in a physical environment 500, similar to that described above.

[0089] As shown in FIG. 5A-1, the first electronic device 101a presents a three-dimensional environment 550A (e.g., via the display 120a). In FIG. 5A-1, as also described above, the three-dimensional environment 550A includes a representation (e.g., a pass-through representation or a computer-generated representation) of the first electronic device 101a's physical environment 500. Thus, as shown in FIG. 5A-1, the three-dimensional environment 550A presented using the first electronic device 101a includes a representation of the window 509 and the back and side walls of the physical environment 500 (e.g., the window 509 and the back and side walls are visible within the field of view of the first electronic device 101a). Additionally, as shown in FIG. 5A-1, a second user 504 (e.g., the second electronic device 101b) is currently visible in the three-dimensional environment 550A from the first electronic device 101a's current viewpoint. In some examples, three-dimensional environment 550A has one or more characteristics of three-dimensional environment 450A described above.

[0090] Similarly, as shown in FIG. 5A-2, the second electronic device 101b presents a three-dimensional environment 550B (e.g., via the display 120b). In FIG. 5A-2, as also described above, the three-dimensional environment 550B includes a representation (e.g., a pass-through representation or a computer-generated representation) of the physical environment 500 of the second electronic device 101b. Thus, as shown in FIG. 5A-2, the three-dimensional environment 550B presented using the second electronic device 101b includes a representation of the door 508 and sidewalls of the physical environment 500 (e.g., the door 508 and sidewalls are visible within the field of view of the second electronic device 101b). Additionally, as shown in FIG. 5A-2, the first user 502 (e.g., and the first electronic device 101a) is currently visible in the three-dimensional environment 550B from the current viewpoint of the second electronic device 101b. In some examples, three-dimensional environment 550B has one or more characteristics of three-dimensional environment 450B described above.

[0091] 5A-1 and 5A-2, the second electronic device 101b detects input corresponding to a request to share content in a multi-user communication session (e.g., share content with the first electronic device 101a in a shared three-dimensional environment). As shown in FIG. 5A-2, the second electronic device 101b optionally displays a user interface object 520 in the three-dimensional environment 550B. In some examples, the user interface object 520 is associated with a media player application (e.g., a movie player application) running on the second electronic device 101b. In some examples, the user interface object 520 includes one or more selectable options for sharing content (e.g., movie A) in the multi-user communication session. For example, as shown in FIG. 5A-2, the user interface object 520 includes a selectable option 521 selectable to share movie A with “user 1” (e.g., the first user 502) in the multi-user communication session. In some examples, the input corresponding to the request to share content in the multi-user communication session corresponds to a selection of the selectable option 521. For example, as shown in FIG. 5A-2, the second electronic device 101b optionally detects the hand 503 of the second user 504 providing an air pinch gesture while the second user's 504 gaze 525 is directed at an option 521 within the user interface object 520.

[0092] In some examples, in response to detecting a selection of selectable option 521, second electronic device 101b initiates a process for displaying the shared virtual object in the shared three-dimensional environment. In some examples, when displaying the shared virtual object in the shared three-dimensional environment, first electronic device 101a and second electronic device 101b coordinate to select a placement location for the shared virtual object in the shared three-dimensional environment (e.g., based on the spatial arrangement of first electronic device 101a and second electronic device 101b in the first spatial group of first user 502 and second user 504). 5B , when the first electronic device 101a and the second electronic device 101b identify a placement location 532 of a shared virtual object in a first spatial group as shown in an overhead view 510, the first electronic device 101a and the second electronic device 101b analyze / identify their physical locations within the shared (e.g., synchronized) coordinate space / system of the first spatial group. For example, as shown in the overhead view 510 of FIG. 5B , the first electronic device 101a is located at a first location relative to an origin 530 of the first spatial group (e.g., a geometric center as described above with reference to FIGS. 4A-4I ), and the second electronic device 101b is located at a second location relative to the origin 530 that is different from the first location. Further, the first electronic device 101a is located at a first distance from the origin 530, and the second electronic device 101b is located at a second distance (e.g., different from or equal to the first distance) from the origin 530. In some examples, as described above with reference to FIGS. 4A-4I , the location of the first electronic device 101a relative to the origin 530 is determined using skeletal data associated with the first user 502 (e.g., provided by the first electronic device 101a), and the location of the second electronic device 101b relative to the origin 530 is determined using skeletal data associated with the second user 504 (e.g., provided by the second electronic device 101b).

[0093] 5B , when the first electronic device 101a and the second electronic device 101b identify a placement location 532 of the shared virtual object in the first spatial group, as shown in the overhead view 510, the first electronic device 101a and the second electronic device 101b analyze / identify one or more physical characteristics of the physical environment 500. For example, as described above, the physical environment 500 includes a physical wall, a door 508, and a window 509. In some examples, the first electronic device 101a and the second electronic device 101b select a placement location of an avatar corresponding to a user of the third electronic device based on one or more physical characteristics of the physical environment 500. For example, the location at which the shared virtual object is positioned in the shared three-dimensional environment is selected to not correspond to (e.g., not intersect and / or extend beyond) boundaries of walls (e.g., including the door 508 and the window 509) in the physical environment 500.

[0094] 5B , when the first electronic device 101a and the second electronic device 101b identify the placement location 532 of the shared virtual object in the first spatial group as shown in the overhead view 510, the first electronic device 101a and the second electronic device 101b analyze / identify the orientations of the first electronic device 101a and the second electronic device 101b in the first spatial group. For example, the orientation of the first electronic device 101a defines the forward direction of the first electronic device 101a (e.g., the forward head direction of the first user 502), and the orientation of the second electronic device 101b defines the forward direction of the second electronic device 101b (e.g., the forward head direction of the second user 504). 5B , as an example, the forward direction of the first electronic device 101a and the forward direction of the second electronic device 101b are indicated by arrows extending from the first electronic device 101a and the second electronic device 101b, respectively, in an overhead view 510. In some examples, the first electronic device 101a and the second electronic device 101b use the forward directions of the electronic devices 101a and 101b to determine an average forward direction of the electronic devices 101a and 101b in the first spatial group (e.g., an average forward head direction of the users 502 and 504). For example, as shown in the overhead view 510 of FIG. 5B , the first electronic device 101a and the second electronic device 101b determine an average forward direction 531 in the first spatial group based on averaging the forward directions of the first electronic device 101a and the second electronic device 101b. In some examples, as described above with reference to Figures 4A-4I, the orientation of the first electronic device 101a relative to the origin 530 is determined using skeletal data associated with the first user 502 (e.g., provided by the first electronic device 101a), and the orientation of the second electronic device 101b relative to the origin 530 is determined using skeletal data associated with the second user 504 (e.g., provided by the second electronic device 101b).

[0095] In some examples, the first electronic device 101a and the second electronic device 101b select / adjust a placement location for the shared virtual object based on any one or a combination of the factors described above. In FIG. 5C, after selecting a placement location 532 for the shared virtual object, the first electronic device 101a and the second electronic device 101b display the shared virtual object at the selected placement location. For example, as shown in FIG. 5C, the shared virtual object is displayed as a shared application window 540 in the shared three-dimensional environment (e.g., a media player user interface displaying Movie A). Additionally, in some examples, the shared application window 540 is displayed with and / or includes a grabber bar 535 within the three-dimensional environment 550A. In some examples, the grabber bar 535 is selectable to initiate movement of the shared application window 540 within the three-dimensional environment 550A. In some examples, when shared application window 540 is displayed within three-dimensional environment 550A, shared application window 540 is displayed at a first location (e.g., corresponding to placement location 532 described above) and at a first orientation relative to the viewpoint of first electronic device 101a. In some examples, shared application window 540 has one or more characteristics of shared virtual object 310 described above with reference to FIG. 3 .

[0096] 5C, the first electronic device 101a detects a movement input to the shared application window 540 in the three-dimensional environment 550A. For example, as shown in FIG. 5C, the first electronic device 101a detects that the hand 503 of the first user 502 provides an air pinch gesture while the line of sight 525 of the first user 502 is directed toward the grabber bar 535 in the three-dimensional environment 550A. In some examples, as shown in FIG. 5C, the movement input includes movement of the hand 503 of the first user 502 (e.g., movement forward and left in space relative to the viewpoint of the first electronic device 101a).

[0097] 5D , in response to detecting a movement input directed at the shared application window 540, the first electronic device 101a moves the shared application window 540 within the three-dimensional environment 550A according to the movement input. For example, as shown in the overhead view 510 of FIG. 5D , the shared application window 540 is moved backward and leftward in space relative to the viewpoint of the first electronic device 101a (e.g., positioned farther away (e.g., behind the second user 504) than the second user 504 relative to the viewpoint of the first electronic device 101a within the three-dimensional environment 550A). In some examples, as shown in FIG. 5D , when the shared application window 540 is moved within the three-dimensional environment 550A according to the above-described movement input, the shared application window 540 is displayed at a second location different from the first location of FIG. 5C and in a second orientation different from the first orientation of FIG. 5C relative to the viewpoint of the first electronic device 101a.

[0098] In some examples, the display of the shared application window 540 at a second location and / or a second orientation within the three-dimensional environment 550A relative to the viewpoint of the first electronic device 101a is based on skeletal data associated with the first user 502 and skeletal data associated with the second user 504. For example, when the shared application window 540 is moved within the shared three-dimensional environment according to movement input provided by the first user 502 in the manner described above, the orientation of the shared application window 540 is updated to point toward the average location of the first user 502 (e.g., and the first electronic device 101a) and the second user 504 (e.g., and the second electronic device 101b) within the shared three-dimensional environment. For example, when the shared application window 540 is moved according to the movement input, the shared application window 540 is rotated such that a vector extending from the front-facing surface of the shared application window 540 (e.g., extending from a point at the center of the front-facing surface) intersects with and / or extends to the average location of the first user 502 and the second user 504 within the shared three-dimensional environment of the overhead view 510. As described above, the locations of the first user 502 and the second user 504 (e.g., according to which the above-mentioned average locations are determined) are determined / recognized based on skeletal data associated with the first user 502 and the second user 504, respectively. Additionally, as shown in FIGS. 5D and 5E, skeletal data associated with the first user 502 and the second user 504 is utilized by the first electronic device 101a and / or the second electronic device 101b to determine what portions of the shared application window 540 are occluded by the individual users relative to the unique viewpoints of the first electronic device 101a and the second electronic device 101b.For example, in FIG. 5D , based on skeletal data associated with the second user 504, the first electronic device 101a determines that the user 504 is spatially located in front of and overlapping a portion (e.g., the right edge) of the shared application window 540 relative to the perspective of the first electronic device 101a, which allows the first electronic device 101a to render / display the portion of the shared application window 540 as being occluded by the second user 504 relative to the perspective of the first electronic device 101a.

[0099] 5E, the second electronic device 101b detects a movement of the second electronic device's 101b's viewpoint within the shared three-dimensional environment. For example, as shown in the overhead view 510 of FIG. 5E, the second user 504 moves within the physical environment 500 as indicated by arrow 571, causing the second electronic device's 101b's viewpoint to update according to the second user's 504's movement. In some examples, as shown in the overhead view 510, the movement of the second user 504 (e.g., and therefore the second electronic device 101b) is in the direction of the shared application window 540 relative to the viewpoint of the first electronic device 101a.

[0100] 5F, after the second user 504 moves through the physical environment 500 as described above, the second user 504 (e.g., and the second electronic device 101b) is positioned behind the shared application window 540 relative to the perspective of the first electronic device 101a, as shown in the overhead view 510. In some examples, the movement of the second user 504 behind the shared application window 540 within the three-dimensional environment 550A relative to the perspective of the first electronic device 101a may cause and / or create a depth / spatial conflict between the second user 504 and the shared application window 540 from the perspective of the first electronic device 101a. For example, because the second user 504 has moved behind the shared application window 540 relative to the perspective of the first electronic device 101a, at least a portion of the second user 504 is overlaid (e.g., occluded) by the shared application window 540 in the three-dimensional environment 550A from the perspective of the first electronic device 101a.

[0101] Accordingly, to help address (e.g., mitigate) and / or prevent instances of depth conflicts in the three-dimensional environment 550A from the perspective of the first electronic device 101a, the first electronic device 101a applies a breakthrough effect 541 to the shared application window 540 in the three-dimensional environment 550A. Specifically, as shown in FIG. 5F , the first electronic device 101a alters the visual appearance of the shared application window 540 such that the second user 504 remains visible and / or perceivable through the shared application window 540 relative to the perspective of the first electronic device 101a within the three-dimensional environment 550A. In some examples, as shown in FIG. 5F , applying the breakthrough effect 541 includes feathering (e.g., ceasing to be displayed) one or more portions of the shared application window 540 that visually overlap and / or correspond to the second user 504 from the perspective of the first electronic device 101a within the three-dimensional environment 550A such that the one or more portions of the shared application window 540 visually appear to no longer obscure / overlap the second user 504 from the perspective of the first electronic device 101a within the three-dimensional environment 550A. In some examples, as shown in FIG. 5F , applying breakthrough effect 541 includes adjusting the opacity (e.g., decreasing the opacity) of one or more portions of shared application window 540 that visually overlap and / or correspond to second user 504 from the perspective of first electronic device 101a in three-dimensional environment 550A so that one or more portions of shared application window 540 no longer appear to obscure / overlap the perspective of first electronic device 101a in three-dimensional environment 550A to second user 504.In some examples, as shown in FIG. 5F , applying breakthrough effect 541 includes adjusting the brightness (e.g., decreasing the brightness) of one or more portions of shared application window 540 that visually overlap and / or correspond to second user 504 from the perspective of first electronic device 101a within three-dimensional environment 550A so that one or more portions of shared application window 540 no longer visually obscure / overlap second user 504 from the perspective of first electronic device 101a within three-dimensional environment 550A.

[0102] In some examples, the first electronic device 101a uses skeletal data associated with the second user 504 to apply the breakthrough effect 541 to the shared application window 540. For example, as described herein above, the skeletal data associated with the second user 504 includes information corresponding to the location of the second user 504 relative to the viewpoint of the first electronic device 101a (e.g., relative to the origin 530 in the first spatial group described above). Thus, the first electronic device 101a, optionally, determines that the second user 504 is at least partially overlapped and / or obscured by the shared application window 540 relative to the viewpoint of the first electronic device 101a in accordance with a determination that the location of the second user 504 known from the skeletal data associated with the second user 504 provided by the second electronic device 101b at least partially corresponds to (e.g., intersects, overlaps, etc.) the location of the shared application window 540 relative to the viewpoint of the first electronic device 101a in the three-dimensional environment 550A.

[0103] In addition to using the skeletal data associated with the second user 504 to determine that a depth conflict has occurred as described above (e.g., based on the location of the second user 504 relative to the viewpoint of the first electronic device 101a), the first electronic device 101a also utilizes the skeletal data associated with the second user 504 to determine the amount of the shared application window 540 to apply the breakthrough effect 541 to. For example, as described herein above, the skeletal data associated with the second user 504 includes information corresponding to the size (e.g., height and / or weight) of the second user 504, as well as volumetric data associated with parts of the second user's 504's body, such as the size (e.g., volume or thickness) of the second user's 504's limbs, torso, and / or head. In some examples, the skeletal data associated with the second user 504 thus enables the first electronic device 101a to apply breakthrough effect 541 to one or more portions of the shared application window 540 based on size and / or volume data associated with the body portions of the second user 504 (e.g., such that the amount of the shared application window 540 having breakthrough effect 541 in the three-dimensional environment 550A corresponds to the size and / or volume of the body of the second user 504, as shown in FIG. 5F).

[0104] It should be understood that the above-described technique for applying breakthrough effect 541 to shared application window 540 also applies in the case of a direct spatial intersection between second user 504 and shared application window 540. For example, in the example of FIG. 5F , when second user 504 moves through physical environment 500 such that at least a portion of second user 504 intersects shared application window 540 relative to the perspective of first electronic device 101a in three-dimensional environment 550A (e.g., when second user 504 walks through and / or into shared application window 540 as indicated by arrow 571 in overhead view 510 of FIG. 5E ), first electronic device 101a applies the above-described breakthrough effect 541 to one or more portions of shared application window 540 that correspond to at least the portion of second user 504 in three-dimensional environment 550A. Additionally, breakthrough effect 541 is optionally provided to a private virtual object (e.g., private application window 330 of FIG. 3 ) pursuant to a determination that second user 504 has a depth / spatial conflict with the private virtual object from the perspective of first electronic device 101 a, as also described above. Further, in the example of FIG. 5F , after second user 504 moves within physical environment 500 as shown in overhead view 510, it should be understood that second electronic device 101 b optionally determines that, relative to the updated perspective of second electronic device 101 b, a depth conflict exists between shared application window 540 and first user 502 (e.g., shared application window 540 at least partially overlaps and / or obscures first user 502 in the three-dimensional environment presented on second electronic device 101 b).Thus, in some such examples, the second electronic device 101b applies a breakthrough effect (e.g., similar to the breakthrough effect 541 described above) to the shared application window 540 displayed on the second electronic device 101b to resolve depth conflicts between the shared application window 540 and the first user 502 relative to the viewpoint of the second electronic device 101b (e.g., using skeletal data associated with the first user 502 in a manner similar to that described above).

[0105] It should be understood that in some examples, in response to detecting a depth / spatial conflict between the second user 504 and the shared application window 540 from the perspective of the first electronic device 101a, the first electronic device 101a alternatively refrains from applying the breakthrough effect 541 to the shared application window 540 as described above. Rather, in some examples, the shared application window 540 may occlude the second user 504 within the three-dimensional environment 550A from the perspective of the first electronic device 101a. In such instances, the first electronic device 101a utilizes skeletal data associated with the second user 504 to determine the portion of the shared application window 540 to render / display as occluding the second user 504 after detecting a depth / spatial conflict between the second user 504 and the shared application window 540 from the perspective of the first electronic device 101a.

[0106] In some examples, skeletal data associated with one or more users participating in a multi-user communication session can be utilized to generate and display one or more virtual objects associated with one or more users within the multi-user communication session. For example, skeletal data associated with individual users can be utilized to display virtual objects on, within, or otherwise associated with portions of the individual users within a shared three-dimensional environment. As an example, in FIG. 5G , a first electronic device 101a (e.g., and a second electronic device 101b) are displaying a shared game board 545 in a three-dimensional environment 550A. In some examples, the shared game board 545 corresponds to a shared virtual object that is visible to and interactive with a first user 502 and a second user 504 within the multi-user communication session. As shown in the overhead view 510 of FIG. 5G , the first user 502 (e.g., the first electronic device 101a) and the second user 504 (e.g., the second electronic device 101b) are positioned on either side of the shared game board 545 within the shared three-dimensional environment.

[0107] In some examples, shared game board 545 may include one or more virtual game pieces and / or may be associated with one or more virtual game pieces. For example, in FIG. 5G , if shared game board 545 corresponds to a virtual chess board in three-dimensional environment 550A, shared game board 545 optionally includes and / or is associated with multiple virtual chess pieces. As another example, if shared game board 545 is associated with a virtual card game in three-dimensional environment 550A, shared game board 545 includes and / or is associated with multiple virtual playing cards. As shown in FIG. 5G , three-dimensional environment 550A optionally includes virtual objects 546 associated with shared game board 545. For example, virtual objects 546 correspond to virtual game pieces (e.g., virtual chess pieces) as described above. In some examples, the virtual object 546 is associated with the shared game board 545, and therefore the virtual object 546 also corresponds to a shared virtual object (e.g., such that the virtual object 546 is visible and / or interactive to the first user 502 and the second user 504). As shown in FIG. 5G , in some examples, the virtual object 546 is displayed to a portion of the second user 504 from the perspective of the first electronic device 101 a within the three-dimensional environment 550A. For example, as shown in FIG. 5G , the virtual object 546 is displayed at a location within the three-dimensional environment 550A that corresponds to the hand 505 of the second user 504, such that from the perspective of the first electronic device 101 a, the virtual object 546 appears to be located on and / or in the hand 505 of the second user 504 (e.g., as if the second user 504 were holding the virtual object 546), as shown in the overhead view 510.

[0108] In some examples, similar to those described herein above, the first electronic device 101a and the second electronic device 101b use skeletal data associated with the second user 504 to display the virtual object 546 relative to the hand 505 of the second user 504. For example, as described above, the skeletal data associated with the second user 504 includes information corresponding to one or more joints of the second user's 504's upper body in space (e.g., joints of the second user's 504's fingers, hand, and / or arm). In some examples, the first electronic device 101a and the second electronic device 101b utilize information corresponding to one or more joints of the hand 505 to display (e.g., position) the virtual object 546 in the hand 505 of the second user 504 relative to the unique viewpoints of the first electronic device 101a and the second electronic device 101b. Additionally, in some examples, the skeletal data associated with the second user 504 allows one or more fingers of the hand 505 to remain visible to the unique viewpoints of the first electronic device 101a and the second electronic device 101b while the virtual object 546 is displayed on and / or in the hand 505. For example, the display of the virtual object 546 is attached to the knuckles of the hand 505, which optionally indicates whether the hand 505 is in an extended orientation (e.g., fingers and palm facing up) as shown in FIG. 5G or whether the hand 505 is in a grasping orientation (e.g., fingers bent as if grasping / holding a cup or glass). It should be understood that the other examples of the above-described methods for displaying the virtual object 546 for a portion of the second user 504 (e.g., the hand 505) based on skeletal data associated with the second user 504 similarly apply to displaying a virtual object (e.g., similar to the virtual object 546) for a portion of the first user 502 based on skeletal data associated with the first user 502.

[0109]

[0010] Thus, as outlined above, providing a system and method for modifying the visual appearance of virtual content in response to user input directed at the virtual content within a shared three-dimensional environment during a multi-user communication session based on skeletal data associated with the users advantageously enables co-located users in the multi-user communication session to experience synchronized interaction with the virtual content and other users, thereby improving user-device interaction. Additionally, as another advantage, modifying the visual appearance of the virtual content based on skeletal data associated with individual participants in the multi-user communication session pursuant to a determination that the individual participant is at least partially occluded by a portion of the virtual content enables the other participants to preserve spatial awareness of the individual participant.

[0110] It is understood that the examples shown and described herein are merely illustrative, and that additional and / or alternative elements may be provided within the three-dimensional environment for interacting with the example content. It should be understood that the appearance, shape, form, and size of each of the various user interface elements and objects shown and described herein are exemplary, and that alternative appearances, shapes, forms, and / or sizes may be provided. For example, virtual objects representing application windows (e.g., virtual objects 330, 540, and 545) may be provided in alternative shapes other than a rectangle, such as circles, triangles, etc. In some examples, the various selectable options described herein (e.g., options 416a-416d and 521) may be selected verbally via user language commands (e.g., a "select option" verbal command). Additionally or alternatively, in some examples, the various options, user interface elements, controls, etc. described herein may be selected and / or manipulated via user input received via one or more separate input devices in communication with the electronic device(s). For example, the selection input may be received via a physical input device, such as a mouse, trackpad, keyboard, etc., that communicates with the electronic device(s).

[0111] FIG. 6 is a flow diagram illustrating an example process for updating the visual appearance of a virtual object using skeletal data associated with one or more participants in a multi-user communication session, according to some examples of the present disclosure. In some examples, process 600 begins at a first electronic device in communication with one or more displays and one or more input devices, where the first electronic device is co-located with a second electronic device in a first physical environment. In some examples, the first electronic device and the second electronic device are, optionally, head-mounted displays similar to or corresponding to devices 260 / 270 of FIG. 2, respectively. As shown in FIG. 6, in some examples, while the first electronic device is in a communication session with the second electronic device, at 602, the first electronic device receives first data provided by the second electronic device, including skeletal data associated with a user of the second electronic device. For example, as described with reference to FIGS. 4A-1 and 4A-2, the first electronic device 101a receives skeletal data associated with the second user 504 from the second electronic device 101b.

[0112] In some examples, after receiving the first data, at 604, the first electronic device detects an indication of a request to share content in the three-dimensional environment. For example, as shown in FIG. 5A-2, the second electronic device 101b detects a selection of an option 521 (e.g., provided by the hand 503 of the second user 504) on the first electronic device 101a corresponding to a request to share content (e.g., movie A) with the first user 502. In some examples, in response to detecting the indication, at 606, the first electronic device presents, via one or more displays, a first object corresponding to the shared content in the three-dimensional environment. For example, as shown in FIG. 5C, the first electronic device 101a displays a shared application window 540 in the three-dimensional environment 550A.

[0113] In some examples, at 608, while presenting the first object corresponding to the shared content, the first electronic device detects a change in one or more spatial properties between the user of the second electronic device and the first object in the three-dimensional environment. For example, as shown in the overhead view 510 of FIG. 5E, the first electronic device 101a detects a movement of the second user 504, indicated by arrow 571, relative to the shared application window 540. In some examples, in response to detecting the change in one or more spatial properties between the user of the second electronic device and the first object at 610, the first electronic device updates, via one or more displays, a display of the visual appearance of the first object in accordance with the change in the one or more spatial properties and based on the first data. For example, as shown in FIG. 5F, movement of second user 504 causes at least a portion of shared application window 540 to overlap second user 504 relative to the viewpoint of first electronic device 101a, causing first electronic device 101a to apply breakthrough effect 541 to at least a portion of shared application window 540 within three-dimensional environment 550A.

[0114] It will be understood that process 600 is an example, and that more, fewer, or different operations may be performed in the same or a different order. Additionally, the operations of process 600 described above are, optionally, implemented by executing one or more functional modules of an information processing device, such as a general-purpose processor (e.g., as described with respect to FIG. 2) or an application-specific chip, and / or by other components of FIG. 2.

[0115] FIG. 7 is a flow diagram illustrating an exemplary process for presenting visual indications of participants in a multi-user communication session based on skeletal data associated with the participants, according to some examples of the present disclosure. In some examples, process 700 begins at a first electronic device in communication with one or more displays and one or more input devices, where the first electronic device is co-located with a second electronic device in a first physical environment. In some examples, the first electronic device and the second electronic device are, optionally, head-mounted displays similar to or corresponding to devices 260 / 270 of FIG. 2, respectively. As shown in FIG. 7, in some examples, at 702, while the first electronic device is in a communication session with the second electronic device, the first electronic device receives first data provided by the second electronic device, including skeletal data associated with a user of the second electronic device. For example, as described with reference to FIGS. 4A-1 and 4A-2, the first electronic device 101a receives skeletal data associated with the second user 504 from the second electronic device 101b.

[0116] In some examples, after receiving the first data at 704, the first electronic device detects movement of a user of a second electronic device in the first physical environment. For example, as shown in the overhead view 410 of FIG. 4F , the first electronic device 101 a detects movement of a third user 406 in the physical environment 400, as indicated by arrow 472. In some examples, in response to detecting movement of the user of the second electronic device at 706 and in accordance with a determination that the movement of the user of the second electronic device causes at least a portion of the user of the second electronic device to be occluded by at least a portion of the first physical environment relative to a viewpoint of the first electronic device, the first electronic device displays, via one or more displays, a visual indication corresponding to at least a portion of the user of the second electronic device at a location in the three-dimensional environment that corresponds to at least a portion of the first physical environment based on the first data. For example, as shown in FIG. 4G-1, movement of the third user 406 causes the third user 406 to be at least partially occluded by a wall 407 in the physical environment 400 relative to the viewpoint of the first electronic device 101a, causing the first electronic device 101a to display an avatar 426 corresponding to the third user, as shown in FIG. 4H, or a user interface object 418, as shown in FIG. 4I, indicating the location of the third user 406 relative to the viewpoint of the first electronic device 101a within the three-dimensional environment 450A.

[0117] It will be understood that process 700 is an example, and that more, fewer, or different operations may be performed in the same or a different order. Furthermore, the operations in process 700 described above are, optionally, implemented by executing one or more functional modules within an information processing device, such as a general-purpose processor (e.g., as described with respect to FIG. 2) or an application-specific chip, and / or by other components of FIG. 2.

[0118]

[0013] Thus, in accordance with the above, some examples of the present disclosure are directed to a method, at a first electronic device in communication with one or more displays and one or more input devices, co-located with a second electronic device in a first physical environment, receiving first data provided by the second electronic device while the first electronic device is in a communication session with the second electronic device, the first data including skeletal data associated with a user of the second electronic device; detecting an indication of a request to share content in a three-dimensional environment after receiving the first data; in response to detecting the indication, presenting, via the one or more displays, a first object corresponding to the shared content in the three-dimensional environment; detecting, while presenting the first object corresponding to the shared content, a change in one or more spatial properties between the user of the second electronic device and the first object in the three-dimensional environment; and in response to detecting the change in the one or more spatial properties between the user of the second electronic device and the first object, updating, via the one or more displays, a display of a visual appearance of the first object in accordance with the change in the one or more spatial properties and based on the first data.

[0119] Additionally or alternatively, in some examples, the method further includes, while the first electronic device is in a communication session with the second electronic device, ceasing to present an avatar corresponding to a user of the second electronic device in the three-dimensional environment based on skeletal data associated with the user of the second electronic device. Additionally or alternatively, in some examples, the method further includes, while the first electronic device is in a communication session with the second electronic device, detecting an indication adding a third electronic device to the communication session, the third electronic device being different from the first electronic device and the second electronic device, the third electronic device not being co-located with the first electronic device and the second electronic device in the first physical environment, and, in response to detecting the indication, entering the communication session with the second electronic device and the third electronic device and presenting, via the one or more displays, an avatar corresponding to the user of the third electronic device in the three-dimensional environment, the avatar being presented based on skeletal data associated with the user of the third electronic device provided by the third electronic device. Additionally or alternatively, in some examples, the first electronic device being co-located with the second electronic device in the physical environment is pursuant to a determination that the second electronic device is within a threshold distance of the first electronic device in the physical environment. Additionally or alternatively, in some examples, the second electronic device being co-located with the first electronic device in the physical environment is pursuant to a determination that the second electronic device is located within a field of view of the first electronic device.

[0120] Additionally or alternatively, in some examples, the second electronic device being co-located with the first electronic device in the first physical environment follows a determination that the second electronic device is located in the same physical room as the first electronic device. Additionally or alternatively, in some examples, the skeletal data associated with the user of the second electronic device indicates poses of multiple body parts of the user of the second electronic device. Additionally or alternatively, in some examples, the first data provided by the second electronic device includes object data corresponding to one or more physical objects associated with the user of the second electronic device. Additionally or alternatively, in some examples, the first data provided by the second electronic device includes information corresponding to at least one of a location of the second user and an orientation of the second user in a three-dimensional environment. Additionally or alternatively, in some examples, the first data provided by the second electronic device includes information indicating sizes of multiple body parts of the user of the second electronic device. Additionally or alternatively, in some examples, presenting a first object corresponding to the shared content in the three-dimensional environment includes presenting the first object at a first location in the three-dimensional environment selected based on a perspective of the first electronic device and the first data. Additionally or alternatively, in some examples, the method further includes transmitting second data to the second electronic device while the first electronic device is in a communication session with the second electronic device, the second data including skeletal data associated with a user of the first electronic device.

[0121] Additionally or alternatively, in some examples, detecting a change in one or more spatial properties between the user of the second electronic device and the first object in the three-dimensional environment includes detecting a change in at least one of a location and an orientation of the first object relative to the user of the second electronic device in the three-dimensional environment determined based on skeletal data associated with the user of the second electronic device. Additionally or alternatively, in some examples, detecting a change in one or more spatial properties between the user of the second electronic device and the first object in the three-dimensional environment includes detecting a request to move the first object in the three-dimensional environment, and updating the display of the visual appearance of the first object in accordance with the change in the one or more spatial properties and based on the first data includes moving the first object in the three-dimensional environment relative to a viewpoint of the first electronic device. Additionally or alternatively, in some examples, updating the display of the visual appearance of the first object in accordance with the change in the one or more spatial properties and based on the first data further includes updating an orientation of the first object relative to a viewpoint of the first user so that the first object is oriented toward an average of the viewpoint of the first user and the location of the second electronic device in the three-dimensional environment. Additionally or alternatively, in some examples, detecting a change in one or more spatial properties between the user of the second electronic device and the first object in the three-dimensional environment includes detecting a movement of the second user such that at least a portion of the second user is overlaid with at least a portion of the first object from a perspective of the first electronic device, and updating the display of the visual appearance of the first object in accordance with the change in the one or more spatial properties and based on the first data includes adjusting at least one of a brightness and an opacity of at least a portion of the first object so that at least a portion of the second user remains visible in the three-dimensional environment from the perspective of the first electronic device. Additionally or alternatively, in some examples, the three-dimensional environment further includes a separate user, distinct from the user of the second electronic device, that is visible from the perspective of the first electronic device, and the separate user is not participating in the communication session.In some examples, the method further includes receiving a selection input via one or more input devices while the first electronic device is in a communication session with the second electronic device; and in response to receiving the selection input, displaying a communication session user interface within the three-dimensional environment via one or more displays in accordance with a determination that the selection input is directed to a user of the second electronic device within the three-dimensional environment; and ceasing to display the communication session user interface within the three-dimensional environment in accordance with a determination that the selection input is directed to an individual user within the three-dimensional environment.

[0122] Additionally or alternatively, in some examples, determining that the selection input is directed toward the user of the second electronic device is responsive to determining that a gaze of the user of the first electronic device is directed toward the user of the second electronic device when the selection input is detected, and determining that the selection input is directed toward the individual user is responsive to determining that a gaze of the user of the first electronic device is directed toward the individual user when the selection input is detected, and a location of the gaze of the user of the first electronic device in the three-dimensional environment is determined based on skeletal data associated with the user of the first electronic device. Additionally or alternatively, in some examples, determining that a gaze of the user of the first electronic device is directed toward the user of the second electronic device when the selection input is detected is in accordance with determining that a location of the gaze of the user of the first electronic device corresponds to a location of the user of the second electronic device, and a location of the user of the second electronic device is determined based on skeletal data associated with the user of the second electronic device. Additionally or alternatively, in some examples, the method further includes: while the first electronic device is in a communication session with the second electronic device, detecting an indication that the second electronic device has dropped out of the communication session; and, in response to detecting the indication, displaying, via one or more displays, a visual indication in the three-dimensional environment that the second electronic device has dropped out of the communication session, the visual indication being displayed at a location in the three-dimensional environment based on skeletal data associated with a user of the second electronic device.

[0123] Additionally or alternatively, in some examples, the method further includes detecting, during a communication session with the second electronic device, movement of a user of the second electronic device in the first physical environment by the first electronic device, and, in response to detecting the movement of the user of the second electronic device, in accordance with a determination that the movement of the user of the second electronic device causes at least a portion of the user of the second electronic device to be occluded by at least a portion of the first physical environment relative to a viewpoint of the first electronic device, displaying, via the one or more displays, a visual indication corresponding to at least a portion of the user of the second electronic device at a location within the three-dimensional environment corresponding to at least a portion of the first physical environment based on skeletal data associated with the user of the second electronic device. Additionally or alternatively, in some examples, the method further includes adding a third electronic device to the communication session, including detecting, while the first electronic device is in a communication session with the second electronic device, an indication of a request to add a third electronic device different from the second electronic device to the communication session, and, in response to detecting the indication, displaying via one or more displays a visual indication that the third electronic device has joined the communication session in the three-dimensional environment, wherein the visual indication is displayed at a location within the three-dimensional environment based on skeletal data associated with a user of the third electronic device provided by the third electronic device.

[0124] Some examples of the present disclosure are directed to a method including: at a first electronic device in communication with one or more displays and one or more input devices, the first electronic device being co-located with a second electronic device in a first physical environment; receiving first data provided by the second electronic device while the first electronic device is in a communication session with the second electronic device, the first data including skeletal data associated with a user of the second electronic device; and, in response to detecting movement of the user of the second electronic device in the first physical environment after receiving the first data, displaying, via the one or more displays, a visual indication corresponding to at least a portion of the user of the second electronic device at a location within a three-dimensional environment corresponding to at least a portion of the first physical environment based on the first data, in accordance with determining that the movement of the user of the second electronic device causes at least a portion of the first physical environment to occlude at least a portion of the user of the second electronic device relative to a viewpoint of the first electronic device.

[0125] Additionally or alternatively, in some examples, the method further includes, while the first electronic device is in a communication session with the second electronic device, ceasing to present an avatar corresponding to a user of the second electronic device in the three-dimensional environment based on skeletal data associated with the user of the second electronic device. Additionally or alternatively, in some examples, the method further includes, while the first electronic device is in a communication session with the second electronic device, detecting an indication adding a third electronic device to the communication session, the third electronic device being different from the first electronic device and the second electronic device, the third electronic device not being co-located with the first electronic device and the second electronic device in the first physical environment, and, in response to detecting the indication, entering the communication session with the second electronic device and the third electronic device and presenting, via the one or more displays, an avatar corresponding to the user of the third electronic device in the three-dimensional environment, the avatar being presented based on skeletal data associated with the user of the third electronic device provided by the third electronic device. Additionally or alternatively, in some examples, the first electronic device being co-located with the second electronic device in the physical environment is pursuant to a determination that the second electronic device is within a threshold distance of the first electronic device in the physical environment. Additionally or alternatively, in some examples, the second electronic device being co-located with the first electronic device in the physical environment is pursuant to a determination that the second electronic device is located within a field of view of the first electronic device. Additionally or alternatively, in some examples, the second electronic device being co-located with the first electronic device in the first physical environment is pursuant to a determination that the second electronic device is located in the same physical room as the first electronic device. Additionally or alternatively, in some examples, the skeletal data associated with the user of the second electronic device indicates poses of multiple body parts of the user of the second electronic device. Additionally or alternatively, in some examples, the first data provided by the second electronic device includes object data corresponding to one or more physical objects associated with the user of the second electronic device.

[0126] Additionally or alternatively, in some examples, the first data provided by the second electronic device includes information corresponding to at least one of a location of the second user and an orientation of the second user in the three-dimensional environment. Additionally or alternatively, in some examples, the first data provided by the second electronic device includes information indicating sizes of multiple body parts of the user of the second electronic device. Additionally or alternatively, in some examples, the visual indication corresponding to at least a portion of the user of the second electronic device includes a visual representation corresponding to at least a portion of the user of the second electronic device. Additionally or alternatively, in some examples, the visual indication corresponding to at least a portion of the user of the second electronic device includes a user interface object indicating a presence of the user of the second electronic device at a location within the three-dimensional environment. Additionally or alternatively, in some examples, the at least a portion of the first physical environment includes one or more physical objects within the first physical environment. Additionally or alternatively, in some examples, the at least a portion of the first physical environment includes a first partition within the first physical environment. Additionally or alternatively, in some examples, the at least a portion of the user of the second electronic device includes one or more hands of the user of the second electronic device. Additionally or alternatively, in some examples, the at least a portion of the user of the second electronic device includes a face of the user of the second electronic device.

[0127] Additionally or alternatively, in some examples, the method further includes, after detecting a movement of the user of the second electronic device, detecting, via the one or more input devices, a movement of a viewpoint of the first electronic device within the three-dimensional environment while displaying a visual indication corresponding to at least a portion of the user of the second electronic device at a location within the three-dimensional environment corresponding to at least a portion of the first physical environment, and, in response to detecting the movement of the viewpoint of the first electronic device, ceasing to display the visual indication corresponding to at least a portion of the user of the second electronic device in the three-dimensional environment in accordance with a determination that at least a portion of the user of the second electronic device relative to the viewpoint of the first electronic device is no longer occluded by at least a portion of the first physical environment. Additionally or alternatively, in some examples, the method further includes, after detecting a movement of the user of the second electronic device, detecting a second movement of the user of the second electronic device in the first physical environment via the one or more input devices while displaying a visual indication corresponding to at least a portion of the user of the second electronic device at a location in the three-dimensional environment corresponding to at least a portion of the first physical environment, and in response to detecting the second movement of the user of the second electronic device, ceasing the display of the visual indication corresponding to at least a portion of the user of the second electronic device in the three-dimensional environment in accordance with a determination that at least a portion of the user of the second electronic device is no longer occluded by at least a portion of the first physical environment relative to a viewpoint of the first electronic device. Additionally or alternatively, in some examples, the method further includes, in response to detecting a second movement of the user of the second electronic device, displaying, via the one or more displays, a second visual indication corresponding to at least a second portion of the user of the second electronic device in the three-dimensional environment based on the first data, in accordance with a determination that the second movement of the user of the second electronic device causes at least a second portion, different from the at least one portion of the user of the second electronic device, to be occluded by at least a portion of the first physical environment relative to a viewpoint of the first electronic device.

[0128] Some examples of the present disclosure are directed to a first electronic device comprising one or more processors, a memory, and one or more programs, the one or more programs being stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described above.

[0129] Some examples of the present disclosure are directed to a non-transitory computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by one or more processors of the first electronic device, cause the first electronic device to perform any of the methods described above.

[0130] Some examples of the present disclosure are directed to a first electronic device comprising one or more processors, a memory, and means for performing any of the above methods.

[0131] Some examples of the present disclosure are directed to an information processing apparatus for use in a first electronic device, the information processing apparatus comprising means for performing any of the above methods.

[0132] The foregoing description has been set forth with reference to specific embodiments for purposes of explanation. However, the illustrative description above is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The examples were chosen and described to best explain the principles of the disclosure and its practical application, thereby enabling those skilled in the art to best utilize the disclosure and various described examples, with various modifications suited to the particular use contemplated.

Claims

1. 1. A first electronic device in communication with one or more displays and one or more input devices, the first electronic device being co-located with a second electronic device in a first physical environment, comprising: receiving first data provided by a second electronic device while the first electronic device is in a communication session with the second electronic device, the first data including skeletal data associated with a user of the second electronic device; detecting an indication of a request to share content in a three-dimensional environment after receiving the first data; In response to detecting the indication, presenting within the three-dimensional environment via the one or more displays a first object corresponding to the shared content; detecting a change in one or more spatial properties between the user of the second electronic device and the first object in the three-dimensional environment while presenting the first object corresponding to the shared content; and in response to detecting the change in the one or more spatial properties between the user of the second electronic device and the first object, updating, via the one or more displays, a display of the visual appearance of the first object in accordance with the change in the one or more spatial properties and based on the first data.

2. detecting, while the first electronic device is in the communication session with the second electronic device, an indication of adding a third electronic device to the communication session, the third electronic device being different from the first electronic device and the second electronic device, the third electronic device not being co-located with the first electronic device and the second electronic device in the first physical environment; In response to detecting the indication, entering into a communication session with the second electronic device and the third electronic device; presenting, via the one or more displays, an avatar corresponding to a user of the third electronic device in the three-dimensional environment, the avatar being presented based on skeletal data associated with the user of the third electronic device provided by the third electronic device; The method of claim 1 further comprising:

3. 10. The method of claim 1, wherein the first electronic device being co-located with the second electronic device in the first physical environment is pursuant to a determination that the second electronic device is within a threshold distance of the first electronic device in the first physical environment.

4. 10. The method of claim 1, wherein the second electronic device being co-located with the first electronic device in the first physical environment is pursuant to a determination that the second electronic device is located within a field of view of the first electronic device.

5. 10. The method of claim 1, wherein the second electronic device being co-located with the first electronic device in the first physical environment is pursuant to a determination that the second electronic device is located in the same physical room as the first electronic device.

6. The method of claim 1 , wherein the skeletal data associated with the user of the second electronic device indicates poses of multiple body parts of the user of the second electronic device.

7. The method of claim 1 , wherein the first data provided by the second electronic device includes object data corresponding to one or more physical objects associated with the user of the second electronic device.

8. 10. The method of claim 1, wherein the first data provided by the second electronic device includes information corresponding to at least one of a location of the user of the second electronic device and an orientation of the user of the second electronic device in the three-dimensional environment.

9. The method of claim 1 , wherein the first data provided by the second electronic device includes information indicative of sizes of a plurality of body parts of the user of the second electronic device.

10. 2. The method of claim 1, wherein presenting the first object corresponding to the shared content in the three-dimensional environment includes presenting the first object at a first location in the three-dimensional environment selected based on a viewpoint of the first electronic device and the first data.

11. transmitting second data to the second electronic device while the first electronic device is in a communication session with the second electronic device, the second data including skeletal data associated with a user of the first electronic device; The method of claim 10 further comprising:

12. 2. The method of claim 1 , wherein detecting the change in the one or more spatial properties between the user of the second electronic device and the first object in the three-dimensional environment comprises detecting a change in at least one of a location and an orientation of the first object relative to the user of the second electronic device in the three-dimensional environment determined based on the skeletal data associated with the user of the second electronic device.

13. detecting the change in the one or more spatial properties between the user of the second electronic device and the first object in the three-dimensional environment includes detecting a request to move the first object in the three-dimensional environment; updating the display of the visual appearance of the first object in accordance with the change in the one or more spatial properties and based on the first data includes moving the first object within the three-dimensional environment relative to a viewpoint of the first electronic device. The method of claim 1.

14. 14. The method of claim 13, wherein updating the display of the visual appearance of the first object in accordance with the change in the one or more spatial properties and based on the first data further comprises updating an orientation of the first object relative to the viewpoint of the first electronic device such that the first object is oriented toward an average of locations of the viewpoints of the first electronic device and the second electronic device within the three-dimensional environment.

15. detecting the change in the one or more spatial properties between the user of the second electronic device and the first object in the three-dimensional environment includes detecting a movement of the user of the second electronic device such that at least a portion of the user of the second electronic device is superimposed with at least a portion of the first object from a perspective of the first electronic device; updating the display of the visual appearance of the first object in accordance with the change in the one or more spatial properties and based on the first data includes adjusting at least one of a brightness and an opacity of the at least part of the first object so that the at least part of the user of the second electronic device remains visible in the three-dimensional environment from the viewpoint of the first electronic device. The method of claim 1.

16. the three-dimensional environment is visible from a viewpoint of the first electronic device, the three-dimensional environment further comprising a separate user distinct from the user of the second electronic device, the separate user not participating in the communication session, the method further comprising: receiving a selection input via the one or more input devices while the first electronic device is in the communication session with the second electronic device; In response to receiving the selection input, displaying, via the one or more displays, a communication session user interface in the three-dimensional environment in accordance with determining that the selection input is directed to the user of the second electronic device within the three-dimensional environment; and 2. The method of claim 1, further comprising: ceasing to display the communication session user interface within the three-dimensional environment in accordance with a determination that the selection input is directed toward the individual user within the three-dimensional environment.

17. the determining that the selection input is directed toward the user of the second electronic device is in accordance with a determination that a gaze of the user of the first electronic device is directed toward the user of the second electronic device when the selection input is detected; the determining that the selection input is directed toward the individual user is in accordance with a determination that the gaze of the user of the first electronic device is directed toward the individual user when the selection input is detected; a location of the line of sight of the user of the first electronic device in the three-dimensional environment is determined based on skeletal data associated with the user of the first electronic device; 17. The method of claim 16.

18. detecting, while the first electronic device is in the communication session with the second electronic device, an indication of a request to add a third electronic device different from the second electronic device to the communication session; In response to detecting the indication, adding the third electronic device to the communication session, the third electronic device including displaying, via the one or more displays, a visual indication in the three-dimensional environment that the third electronic device has joined the communication session, the visual indication being displayed at a location in the three-dimensional environment based on skeletal data associated with a user of the third electronic device provided by the third electronic device; The method of claim 1 further comprising:

19. a first electronic device, one or more processors; Memory and and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing the method of any one of claims 1 to 18.

Images