Hybrid space groups in multi-user communication session

Abstract

The invention relates to a hybrid space group in a multi-user communication session. Some examples of the present disclosure relate to systems and methods for determining placement locations of avatars corresponding to remote users within a multi-user communication session when initiating the multi-user communication session including a set of collocated users. In some examples, a first electronic device detects an indication of a request to enter a communication session with a third electronic device, where the third electronic device is not juxtaposed in a first physical environment of the first electronic device. In some examples, in response to detecting the indication, the first electronic device enters the communication session including the first electronic device, a second electronic device and the third electronic device, where the second electronic device is disposed in the first physical environment. In some examples, the first electronic device displays a visual representation of a user of the third electronic device in a computer-generated environment.

Description

Technical Field

[0001] The present disclosure generally relates to systems and methods for establishing multi-user communication sessions, where at least one subset of the participants within the multi-user communication session is co-located in a physical environment. Background Art

[0002] Some computer graphics environments provide two-dimensional and / or three-dimensional environments in which at least some of the objects displayed for a user to view are virtual and computer-generated. In some examples, the three-dimensional environment is presented by multiple devices communicating in a multi-user communication session. In some examples, the avatar (e.g., representation) of each non-co-located user participating in a multi-user communication session (e.g., via a computing device) is displayed within the three-dimensional environment of the multi-user communication session. In some examples, content can be shared within the three-dimensional environment for multiple users participating in the multi-user communication session to view and interact with. Summary of the Invention

[0003] Some examples of the present disclosure relate to systems and methods for determining the placement of an avatar corresponding to a remote user within a multi-user communication session when initiating a multi-user communication session including a group of co-located users. In some examples, the method is executed at a first electronic device communicating with one or more displays and one or more input devices, where the first electronic device is co-located with a second electronic device in a first physical environment. In some examples, the first electronic device detects an indication of a request to enter a communication session with a third electronic device, where the third electronic device is not co-located in the first physical environment. In some examples, in response to detecting the indication, the first electronic device enters a communication session including the first electronic device, the second electronic device, and the third electronic device. In some examples, the first electronic device obtains first data corresponding to the position of the user of the second electronic device relative to the viewpoint of the first electronic device in the first physical environment. In some examples, the first electronic device obtains second data corresponding to the orientation of the second electronic device relative to the viewpoint of the first electronic device in the first physical environment. In some examples, the first electronic device displays a visual representation of the user of the third electronic device at a second position within a computer-generated environment via one or more displays, the second position being based on the first data and the second data.

[0004] A full description of these examples is provided in the drawings and the detailed description, and it should be understood that the summary of the invention does not limit the scope of the present disclosure in any way. Brief Description of the Drawings

[0005] To better understand the various examples described herein, reference should be made to the following detailed description and the following drawings. Throughout the drawings, like reference numerals generally refer to corresponding components.

[0006] Figure 1An electronic device presenting an extended reality environment according to some examples of the present disclosure is shown.

[0007] Figure 2 A block diagram of an example architecture of a system according to some examples of the present disclosure is shown.

[0008] Figure 3 An example of a spatial group in a multi-user communication session including a first electronic device and a second electronic device according to some examples of the present disclosure is shown.

[0009] Figures 4A to 4J An example of initiating a multi-user communication session including co-located and non-co-located users according to some examples of the present disclosure is shown.

[0010] Figures 5A to 5E An example of presenting content within a multi-user communication session including co-located and non-co-located users according to some examples of the present disclosure is shown.

[0011] Figure 6 A flowchart showing an example process for establishing a multi-user communication session between multiple electronic devices according to some examples of the present disclosure, where at least one subset of the multiple electronic devices is non-co-located. Detailed Description

[0012] Some examples of the present disclosure relate to systems and methods for determining the placement of an avatar corresponding to a remote user within a multi-user communication session when initiating a multi-user communication session including a group of co-located users. In some examples, the method is executed at a first electronic device communicatively coupled to one or more displays and one or more input devices, where the first electronic device is co-located with a second electronic device in a first physical environment. In some examples, the first electronic device detects an indication of a request to enter a communication session with a third electronic device, where the third electronic device is not co-located with the first physical environment. In some examples, in response to detecting the indication, the first electronic device enters a communication session including the first electronic device, the second electronic device, and the third electronic device. In some examples, the first electronic device obtains first data corresponding to the position of the user of the second electronic device relative to the viewpoint of the first electronic device in the first physical environment. In some examples, the first electronic device obtains second data corresponding to the orientation of the second electronic device relative to the viewpoint of the first electronic device in the first physical environment. In some examples, the first electronic device displays a visual representation of the user of the third electronic device at a second position in a computer-generated environment via one or more displays, the second position being based on the first data and the second data.

[0013] As used herein, a spatial group corresponds to one or more participants (e.g., users) in a multi-user communication session. In some examples, a spatial group in a multi-user communication session has a spatial arrangement indicating the locations of the users and content within the spatial group. In some examples, users in the same spatial group within a multi-user communication session experience a spatial reality according to the spatial arrangement of the spatial group. In some examples, when a user of a first electronic device is in a first spatial group in a multi-user communication session and a user of a second electronic device is in a second spatial group in the multi-user communication session, the users experience a spatial reality localized to their respective spatial groups. In some examples, when a user of a first electronic device and a user of a second electronic device are grouped into separate spatial groups within a multi-user communication session, if the first electronic device and the second electronic device return to the same operating state, the user of the first electronic device and the user of the second electronic device are regrouped into the same spatial group within the multi-user communication session.

[0014] As used herein, a mixed spatial group corresponds to one or more participants (e.g., users) in a multi-user communication session in which at least one subset of the participants is non-collocated in a physical environment. For example, as described by way of one or more examples in the present disclosure, a mixed spatial group includes at least two participants collocated in a first physical environment and at least one participant not collocated with the at least two participants in the first physical environment (e.g., the at least one participant is located in a second physical environment different from the first physical environment). In some examples, a mixed spatial group in a multi-user communication session has a spatial arrangement that determines the locations of the users and content within the spatial group. In some examples, users in the same mixed spatial group within a multi-user communication session experience a spatial reality according to the spatial arrangement of the spatial group, as similarly discussed above.

[0015] In some examples, initiating a multi-user communication session can include interacting with one or more user interface elements. In some examples, an electronic device tracks a user's gaze as an input for targeting selectable options / affordances within a corresponding user interface element displayed in a three-dimensional environment. For example, gaze can be used to identify one or more options / affordances targeted for selection using another selection input. In some examples, a hand tracking input detected via an input device communicatively coupled to the electronic device can be used to select a corresponding option / affordance. In some examples, an object displayed in a three-dimensional environment can be moved and / or reoriented in the three-dimensional environment according to a movement input detected via the input device.

[0016] Figure 1 An electronic device 101 is shown presenting an extended reality (XR) environment (e.g., a computer-generated environment optionally including representations of physical and / or virtual objects) according to some examples of the present disclosure. In some examples, asFigure 1 As shown, the electronic device 101 is a head-mounted display or other head-mounted device configured to be worn on the head of a user of the electronic device 101. The following describes an example of the electronic device 101 with reference to Figure 2 the architecture block diagram. As Figure 1 shown, the electronic device 101 and the table 106 are located in a physical environment. The physical environment may include physical features such as physical surfaces (e.g., floors, walls) or physical objects (e.g., tables, lights, etc.). In some examples, the electronic device 101 may be configured to detect and / or capture an image of the physical environment including the table 106 (shown in the field of view of the electronic device 101).

[0017] In some examples, as Figure 1 shown, the electronic device 101 includes one or more internal image sensors 114a oriented towards the user's face (e.g., the eye-tracking camera described below with reference to Figure 2 ). In some examples, the internal image sensor 114a is used for eye tracking (e.g., detecting the user's gaze). The internal image sensor 114a is optionally arranged on the left and right portions of the display 120 to enable eye tracking of the user's left and right eyes. In some examples, the electronic device 101 further includes external image sensors 114b and 114c facing outward from the user to detect and / or capture the physical environment of the electronic device 101 and / or the movement of the user's hand or other body parts.

[0018] In some examples, the display 120 has a field of view visible to the user (e.g., it may or may not correspond to the field of view of the external image sensors 114b and 114c). Since the display 120 is optionally part of a head-mounted device, the field of view of the display 120 is optionally the same as or similar to the field of view of the user's eyes. In other examples, the field of view of the display 120 may be smaller than the field of view of the user's eyes. In some examples, the electronic device 101 may be an optical see-through device, where the display 120 is a transparent or semi-transparent display through which parts of the physical environment can be directly viewed. In some examples, the display 120 may be included within a transparent lens and may overlap all or only a portion of the transparent lens. In other examples, the electronic device may be a video see-through device, where the display 120 is an opaque display configured to display an image of the physical environment captured by the external image sensors 114b and 114c. Although a single display 120 is shown, it should be understood that the display 120 may include a stereoscopic display pair.

[0019] In some examples, in response to a trigger, the electronic device 101 may be configured to display in an XR environment by Figure 1The virtual object 104 represented by the cube shown is not present in the physical environment but is displayed in an XR environment located on top of a real-world table 106 (or its representation). Optionally, in response to detecting the plane of the table 106 in the physical environment 100, the virtual object 104 may be displayed on the surface of the table 106 in the XR environment displayed via the display 120 of the electronic device 101.

[0020] It should be understood that the virtual object 104 is a representative virtual object and may include and render one or more different virtual objects (e.g., virtual objects having various dimensions such as two-dimensional or other three-dimensional virtual objects) in a three-dimensional XR environment. For example, the virtual object may represent an application or a user interface displayed in the XR environment. In some examples, the virtual object may represent an application and / or content displayed via the user interface in the XR environment. In some examples, the virtual object 104 is optionally configured to be interactive and responsive to user input (e.g., air gestures such as air pinch gestures, air tap gestures, and / or air touch gestures) such that the user can virtually touch, tap, move, rotate, or otherwise interact with the virtual object 104.

[0021] In some examples, displaying an object in a three-dimensional environment may include interacting with one or more user interface objects in the three-dimensional environment. For example, initiating the display of an object in a three-dimensional environment may include interacting with one or more virtual options / affordance representations displayed in the three-dimensional environment. In some examples, when initiating the display of an object in a three-dimensional environment, the electronic device may track the user's gaze as an input for identifying one or more virtual options / affordance representations as targets for selection. For example, the gaze may be used to identify one or more virtual options / affordance representations as targets for selection using another selection input. In some examples, hand tracking input detected via an input device communicatively coupled to the electronic device may be used to select virtual options / affordance representations. In some examples, an object displayed in a three-dimensional environment may move and / or reorient in the three-dimensional environment in accordance with movement input detected via the input device.

[0022] In the following discussion, an electronic device that communicates with a display generation component and one or more input devices is described. It should be understood that the electronic device optionally communicates with one or more other physical user interface devices, such as a touch-sensitive surface, a physical keyboard, a mouse, a joystick, a hand-tracking device, an eye-tracking device, a stylus, etc. Additionally, as described above, it should be understood that the described electronic device, display, and touch-sensitive surface are optionally distributed among two or more devices. Thus, as used in this disclosure, information displayed on or by the electronic device is optionally used to describe information output by the electronic device for display on a separate display device (touch-sensitive or non-touch-sensitive). Similarly, as used in this disclosure, an input received on the electronic device (e.g., a touch input received on the touch-sensitive surface of the electronic device, or a touch input received on the surface of a stylus) is optionally used to describe an input received on a separate input device from which the electronic device receives input information.

[0023] The device generally supports a variety of applications, such as one or more of the following: a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a gaming application, a telephone application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, a television channel browsing application, and / or a digital video player application.

[0024] Figure 2 A block diagram of an example architecture of a system 201 in accordance with some examples of the present disclosure is shown. In some examples, the system 201 includes a plurality of devices. For example, the system 201 includes a first electronic device 260 and a second electronic device 270, where the first electronic device 260 and the second electronic device 270 communicate with each other. In some examples, the first electronic device 260 and the second electronic device 270 are each a portable device, such as a mobile phone, a smartphone, a tablet computer, a laptop computer, an accessory device that communicates with another device, a head-mounted display, etc. In some examples, the first electronic device 260 and the second electronic device 270 correspond to the electronic device 101 described above with reference to Figure 1 the electronic device described.

[0025] As Figure 2As shown, the first device 260 optionally includes various sensors (e.g., one or more hand tracking sensors 202A, one or more position sensors 204A, one or more image sensors 206A, one or more touch-sensitive surfaces 209A, one or more motion and / or orientation sensors 210A, one or more eye tracking sensors 212A, one or more microphones 213A or other audio sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors), one or more display generation components 214A, one or more speakers 216A, one or more processors 218A, one or more memories 220A, and / or communication circuitry 222A. In some examples, the second device 270 optionally includes various sensors (e.g., one or more hand tracking sensors 202B, one or more position sensors 204B, one or more image sensors 206B, one or more touch-sensitive surfaces 209B, one or more motion and / or orientation sensors 210B, one or more eye tracking sensors 212B, one or more microphones 213B or other audio sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors), one or more display generation components 214B, one or more speakers 216, one or more processors 218B, one or more memories 220B, and / or communication circuitry 222B. In some examples, one or more display generation components 214A, 214B correspond to Figure 1 the display 120 in

[0026] One or more communication buses 208A and 208B are optionally used for communication between the above components of the electronic devices 260 and 270, respectively. The first electronic device 260 and the second electronic device 270 optionally communicate via a wired or wireless connection between the two devices (e.g., via communication circuitry 222A, 222B). The communication circuitry 222A, 222B optionally includes circuitry for communicating with electronic devices, networks (such as the Internet, intranet, wired network, and / or wireless network, cellular network, and wireless local area network (LAN)). The communication circuitry 222A, 222B optionally includes circuitry for communicating using near field communication (NFC) and / or short-range communication such as

[0027] The processors 218A, 218B include one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, the memories 220A, 220B are non-transitory computer-readable storage media (e.g., flash memory, random access memory, or other volatile or non-volatile memory or storage devices) storing computer-readable instructions configured to be executed by the processors 218A, 218B to perform the techniques, processes, and / or methods described below. In some examples, the memories 220A, 220B may include more than one non-transitory computer-readable storage media. A non-transitory computer-readable storage medium can be any medium (e.g., excluding signals) that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, and device. In some examples, the storage medium is a transitory computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media can include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, compact disc (CD)-based discs, digital versatile discs (DVDs), or Blu-ray technology-based optical discs, and persistent solid-state memories (such as flash memory, solid-state drives, etc.).

[0028] In some examples, the display generation components 214A, 214B include a single display (e.g., a liquid crystal display (LCD), an organic light-emitting diode (OLED), or other type of display). In some examples, the display generation components 214A, 214B include multiple displays. In some examples, the display generation components 214A, 214B may include a display with touch capabilities (e.g., a touch screen), a projector, a holographic projector, a retinal projector, a transparent or semi-transparent display, etc. In some examples, the electronic devices 260 and 270 respectively include touch-sensitive surfaces 209A and 209B for receiving user inputs such as tap inputs and swipe inputs or other gestures. In some examples, the display generation components 214A, 214B and the touch-sensitive surfaces 209A, 209B form a touch-sensitive display (e.g., a touch screen integrated with the electronic devices 260 and 270 respectively or a touch screen external to the electronic devices 260 and 270 and communicating with the electronic devices 260 and 270).

[0029] The electronic devices 260 and 270 optionally include image sensors 206A and 206B, respectively. The image sensors 206A / 206B optionally include one or more visible light image sensors (such as charge-coupled device (CCD) sensors) and / or complementary metal oxide semiconductor (CMOS) sensors operable to obtain an image of a physical object from a real-world environment. The image sensors 206A / 206B also optionally include one or more infrared (IR) sensors, such as passive IR sensors or active IR sensors, for detecting infrared light from a real-world environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real-world environment. The image sensors 206A / 206B also optionally include one or more cameras configured to capture the movement of a physical object in a real-world environment. The image sensors 206A / 206B also optionally include one or more depth sensors configured to detect the distance of a physical object from the electronic devices 260 / 270. In some examples, information from one or more depth sensors may allow the device to identify an object in the real-world environment and distinguish it from other objects in the real-world environment. In some examples, one or more depth sensors may allow the device to determine the texture and / or topography of an object in the real-world environment.

[0030] In some examples, the electronic devices 260 and 270 use a CCD sensor, an event camera, and a depth sensor in combination to detect the physical environment around the electronic devices 260 and 270. In some examples, the image sensors 206A / 206B include a first image sensor and a second image sensor. The first image sensor and the second image sensor work together and are optionally configured to capture different information about a physical object in a real-world environment. In some examples, the first image sensor is a visible light image sensor and the second image sensor is a depth sensor. In some examples, the electronic devices 260 / 270 use the image sensors 206A / 206B to detect the position and orientation of the electronic devices 260 / 270 and / or the display generation components 214A / 214B in a real-world environment. For example, the electronic devices 260 / 270 use the image sensors 206A / 206B to track the position and orientation of the display generation components 214A / 214B relative to one or more fixed objects in a real-world environment.

[0031] In some examples, the electronic device 260 / 270 includes a microphone 213A / 213B or other audio sensors. The device 260 / 270 uses the microphone 213A / 213B to detect sounds from the user and / or the user's real-world environment. In some examples, the microphone 213A / 213B includes a microphone array (multiple microphones) that optionally operate cooperatively to identify ambient noise or to locate a sound source in the space of the real-world environment.

[0032] In some examples, the device 260 / 270 includes a position sensor 204A / 204B for detecting the position of the device 260 / 270 and / or the display generation component 214A / 214B. For example, the position sensor 204A / 204B may include a Global Positioning System (GPS) receiver that receives data from one or more satellites and allows the electronic device 260 / 270 to determine the absolute position of the device in the physical world.

[0033] In some examples, the electronic device 260 / 270 includes an orientation sensor 210A / 210B for detecting the orientation and / or movement of the electronic device 260 / 270 and / or the display generation component 214A / 214B. For example, the electronic device 260 / 270 uses the orientation sensor 210A / 210B to track changes in the position and / or orientation of the electronic device 260 / 270 and / or the display generation component 214A / 214B, such as changes relative to physical objects in the real-world environment. The orientation sensor 210A / 210B optionally includes one or more gyroscopes and / or one or more accelerometers.

[0034] In some examples, the electronic device 260 / 270 includes a hand-tracking sensor 202A / 202B and / or an eye-tracking sensor 212A / 212B (and / or other body-tracking sensors, such as leg, torso, and / or head-tracking sensors). The hand-tracking sensor 202A / 202B is configured to track the position / location of one or more parts of the user's hand, and / or the movement of one or more parts of the user's hand relative to the extended reality environment, relative to the display generation component 214A / 214B, and / or relative to another defined coordinate system. The eye-tracking sensor 212A / 212B is configured to track the position and movement of the user's gaze (more generally, the eyes, face, or head) relative to the real world or the extended reality environment and / or relative to the display generation component 214A / 214B. In some examples, the hand-tracking sensor 202A / 202B and / or the eye-tracking sensor 212A / 212B are implemented together with the display generation component 214A / 214B. In some examples, the hand-tracking sensor 202A / 202B and / or the eye-tracking sensor 212A / 212B are implemented separately from the display generation component 214A / 214B.

[0035] In some examples, the hand tracking sensors 202A / 202B (and / or other body tracking sensors such as leg, torso, and / or head tracking sensors) may use image sensors 206A / 206B (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that capture three-dimensional information from the real world including one or more body parts (e.g., the hands, legs, torso of a human user). In some examples, there is sufficient resolution to resolve the hand to distinguish the fingers and their respective positions. In some examples, one or more image sensors 206A / 206B are positioned relative to the user to define the field of view and interaction space of the image sensors 206A / 206B, in which the finger / hand positions, orientations, and / or movements captured by the image sensors are used as input (e.g., to distinguish from the user's idle hand or the hands of others in the real-world environment). Tracking the finger / hand used for input (e.g., gestures, touches, taps, etc.) may be advantageous because this does not require the user to touch, hold, or wear any type of beacon, sensor, or other marker.

[0036] In some examples, the eye tracking sensors 212A / 212B include at least one eye tracking camera (e.g., an infrared (IR) camera) and / or illumination source (e.g., an IR light source such as an LED) that emits light towards the user's eyes. The eye tracking camera may be pointed at the user's eyes to directly or indirectly receive reflected IR light from the light source from the eyes. In some examples, both eyes are tracked separately by respective eye tracking cameras and illumination sources, and the focus / gaze may be determined by tracking both eyes. In some examples, one eye (e.g., the dominant eye) is tracked by one or more respective eye tracking cameras / illumination sources.

[0037] The electronic devices 260 / 270 and the system 201 are not limited to Figure 2Rather than the components and configurations thereof, it may include fewer components, other components, or additional components in various configurations. In some examples, system 201 may be implemented in a single device. One or more persons using system 201 are optionally referred to herein as one or more users of the device. Attention is now turned to an exemplary simultaneous display of a three-dimensional environment on a first electronic device (e.g., corresponding to electronic device 260) and a second electronic device (e.g., corresponding to electronic device 270). As described below, the first electronic device may communicate with the second electronic device in a multi-user communication session. In some examples, an avatar of a user of the first electronic device (e.g., representation) may be displayed in the three-dimensional environment at the second electronic device, and an avatar of a user of the second electronic device may be displayed in the three-dimensional environment at the first electronic device. In some examples, a user of the first electronic device and a user of the second electronic device may be associated with a spatial group in a multi-user communication session. In some examples, interactions with the content in the three-dimensional environment when the first electronic device and the second electronic device are in a multi-user communication session may cause the user of the first electronic device and the user of the second electronic device to become associated with different spatial groups in the multi-user communication session.

[0038] Figure 3 An example of a spatial group 340 in a multi-user communication session including a first electronic device 360 and a second electronic device 370 is shown in accordance with some examples of the present disclosure. In some examples, the first electronic device 360 may present a three-dimensional environment 350A, and the second electronic device 370 may present a three-dimensional environment 350B. The first electronic device 360 and the second electronic device 370 may be similar to electronic device 101 or 260 / 270, and / or may be head-mounted systems / devices and / or projection-based systems / devices (including hologram-based systems / devices) configured to generate and present a three-dimensional environment, such as a head-up display (HUD), a head-mounted display (HMD), a window with integrated display capabilities, a display formed as a lens designed to be placed on a person's eye (e.g., similar to a contact lens), respectively. In Figure 3 the example, the first user optionally wears the first electronic device 360 and the second user optionally wears the second electronic device 370 such that the three-dimensional environment 350A / 350B may be defined by the X, Y, and Z axes viewed from the perspective of the electronic device (e.g., the viewpoint associated with the electronic device 360 / 370, which may be a head-mounted display, for example).

[0039] As Figure 3As shown, the first electronic device 360 may be in a first physical environment including a table 306 and a window 309. Accordingly, the three-dimensional environment 350A presented using the first electronic device 360 optionally includes a captured portion of the physical environment surrounding the first electronic device 360, such as a representation 306' of the table and a representation 309' of the window. Similarly, the second electronic device 370 may be in a second physical environment different from (e.g., separate from) the first physical environment, the second physical environment including a floor lamp 307 and a coffee table 308. Accordingly, the three-dimensional environment 350B presented using the second electronic device 370 optionally includes a captured portion of the physical environment surrounding the second electronic device 370, such as a representation 307' of the floor lamp and a representation 308' of the coffee table. Additionally, the three-dimensional environments 350A and 350B may include representations of the floor, ceiling, and walls of the room in which the first electronic device 360 and the second electronic device 370 are respectively located.

[0040] As described above, in some examples, the first electronic device 360 is optionally in a multi-user communication session with the second electronic device 370. For example, the first electronic device 360 and the second electronic device 370 (e.g., via communication circuits 222A / 222B) are configured to present a shared three-dimensional environment 350A / 350B including one or more shared virtual objects (e.g., content such as images, videos, audio, etc., representations of user interfaces of applications, etc.). As used herein, the term "shared three-dimensional environment" refers to a three-dimensional environment that is independently presented, displayed, and / or visible at two or more electronic devices, via which content, applications, data, etc. can be shared and / or presented to users of the two or more electronic devices. In some examples, when the first electronic device 360 is in a multi-user communication session with the second electronic device 370, an avatar corresponding to a user of one electronic device is optionally displayed in the three-dimensional environment displayed via the other electronic device. For example, as Figure 3 shown, at the first electronic device 360, an avatar 315 corresponding to the user of the second electronic device 370 is displayed in the three-dimensional environment 350A. Similarly, at the second electronic device 370, an avatar 317 corresponding to the user of the first electronic device 360 is displayed in the three-dimensional environment 350B.

[0041] In some examples, the rendering of the avatars 315 / 317 as part of a shared three-dimensional environment is optionally accompanied by audio effects corresponding to the speech of the user of the electronic device 370 / 360. For example, the avatar 315 displayed in the three-dimensional environment 350A using the first electronic device 360 is optionally accompanied by audio effects corresponding to the speech of the user of the second electronic device 370. In some such examples, when the user of the second electronic device 370 speaks, the user's speech can be detected by the second electronic device 370 (e.g., via the microphone 213B) and transmitted to the first electronic device 360 (e.g., via the communication circuits 222B / 222A), such that the detected speech of the user of the second electronic device 370 can be presented as audio (e.g., using the speaker 216A) to the user of the first electronic device 360 in the three-dimensional environment 350A. In some examples, the audio effects corresponding to the speech of the user of the second electronic device 370 can be spatialized such that they appear to the user of the first electronic device 360 to emanate from the position of the avatar 315 in the shared three-dimensional environment 350A (e.g., even though output from the speaker of the first electronic device 360). Similarly, the avatar 317 displayed in the three-dimensional environment 350B using the second electronic device 370 is optionally accompanied by audio effects corresponding to the speech of the user of the first electronic device 360. In some such examples, when the user of the first electronic device 360 speaks, the user's speech can be detected by the first electronic device 360 (e.g., via the microphone 213A) and transmitted to the second electronic device 370 (e.g., via the communication circuits 222A / 222B), such that the detected speech of the user of the first electronic device 360 can be presented as audio (e.g., using the speaker 216B) to the user of the second electronic device 370 in the three-dimensional environment 350B. In some examples, the audio effects corresponding to the speech of the user of the first electronic device 360 can be spatialized such that they appear to the user of the second electronic device 370 to emanate from the position of the avatar 317 in the shared three-dimensional environment 350B (e.g., even though output from the speaker of the first electronic device 360).

[0042] In some examples, when in a multi-user communication session, the avatars 315 / 317 are displayed in the three-dimensional environments 350A / 350B in respective orientations corresponding to and / or based on the orientation of the electronic device 360 / 370 (and / or the user of the electronic device 360 / 370) in the physical environment surrounding the electronic device 360 / 370. For example, as Figure 3As shown, in the three-dimensional environment 350A, the avatar 315 optionally faces the viewpoint of the user of the first electronic device 360, and in the three-dimensional environment 350B, the avatar 317 optionally faces the viewpoint of the user of the second electronic device 370. When a particular user moves the electronic device (and / or themselves) in the physical environment, the user's viewpoint changes according to the movement, which can thus also change the orientation of the user's avatar in the three-dimensional environment. For example, referring to Figure 3 , if the user of the first electronic device 360 looks left in the three-dimensional environment 350A such that the first electronic device 360 rotates left (e.g., counterclockwise) (e.g., by a corresponding amount), then the user of the second electronic device 370 will see the avatar 317 corresponding to the user of the first electronic device 360 rotate right (e.g., clockwise) relative to the viewpoint of the user of the second electronic device 370 according to the movement of the first electronic device 360.

[0043] Additionally, in some examples, when in a multi-user communication session, the viewpoint of the three-dimensional environment 350A / 350B and / or the position of the viewpoint of the three-dimensional environment 350A / 350B optionally change according to the movement of the electronic devices 360 / 370 (e.g., made by the users of the electronic devices 360 / 370). For example, when in a communication session, if the first electronic device 360 moves closer to the representation 306' of the table and / or the avatar 315 (e.g., because the user of the first electronic device 360 moves forward in the physical environment around the first electronic device 360), then the viewpoint of the three-dimensional environment 350A will change accordingly such that the representation 306' of the table, the representation 309' of the window, and the avatar 315 appear larger in the field of view. In some examples, each user can independently interact with the three-dimensional environment 350A / 350B such that a change in the viewpoint of the three-dimensional environment 350A and / or the interaction of the first electronic device 360 with virtual objects in the three-dimensional environment 350A optionally do not affect the content shown in the three-dimensional environment 350B at the second electronic device 370, and vice versa.

[0044] In some examples, the avatars 315 / 317 are representations of the users of the electronic devices 370 / 360 (e.g., full-body renderings). In some examples, the avatars 315 / 317 are representations of a part of the users of the electronic devices 370 / 360 (e.g., renderings of the head, face, head and torso, etc.). In some examples, the avatars 315 / 317 are user-personalized, user-selected, and / or user-created representations displayed in the three-dimensional environment 350A / 350B that represent the users of the electronic devices 370 / 360. It should be understood that although Figure 3 the avatars 315 / 317 shown therein respectively correspond to full-body representations of the users of the electronic devices 370 / 360, alternative avatars such as those described above may be provided.

[0045] As described above, when the first electronic device 360 and the second electronic device 370 are in a multi-user communication session, the three-dimensional environment 350A / 350B can be a shared three-dimensional environment presented using the electronic devices 360 / 370. In some examples, during a multi-user communication session, content viewed by a user at one electronic device can be shared with another user at another electronic device. In some such examples, the content can be experienced (e.g., viewed and / or interacted with) by two users (e.g., via their respective electronic devices) in the shared three-dimensional environment. For example, as Figure 3 shown, the three-dimensional environment 350A / 350B includes a shared virtual object 310 (e.g., which is optionally a three-dimensional virtual sculpture) that can be viewed and interacted with by two users. As Figure 3 shown, the shared virtual object 310 can be displayed together with a grabber affordance (e.g., a manipulation bar) 335 that can be selected to initiate movement of the shared virtual object 310 within the three-dimensional environment 350A / 350B.

[0046] In some examples, the three-dimensional environment 350A / 350B includes non-shared content that is private to a user in the multi-user communication session. For example, in Figure 3 , the first electronic device 360 is displaying a private application window 330 in the three-dimensional environment 350A, which is optionally an object that is not shared between the first electronic device 360 and the second electronic device 370 during the multi-user communication session. In some examples, the private application window 330 can be associated with a corresponding application operating on the first electronic device 360 (e.g., such as a media player application, a web browsing application, a messaging application, etc.). Since the private application window 330 is not shared with the second electronic device 370, the second electronic device 370 optionally displays a representation 330” of the private application window in the three-dimensional environment 350B. As Figure 3 shown, in some examples, the representation 330” of the private application window can be a faded, occluded, color-changed, and / or translucent representation of the private application window 330 that prevents a user of the second electronic device 370 from viewing the content of the private application window 330.

[0047] As previously mentioned above, in some examples, the user of the first electronic device 360 and the user of the second electronic device 370 are in the spatial group 340 within the multi-user communication session. In some examples, the spatial group 340 can be the baseline (e.g., the first or default) spatial group within the multi-user communication session. For example, when the user of the first electronic device 360 and the user of the second electronic device 370 initially join the multi-user communication session, the user of the first electronic device 360 and the user of the second electronic device 370 are automatically (and initially, as discussed in more detail below) associated with (e.g., grouped into) the spatial group 340 within the multi-user communication session. In some examples, when the user is in the spatial group 340 as Figure 3 shown, the user of the first electronic device 360 and the user of the second electronic device 370 have a first spatial arrangement (e.g., a first spatial template) within the shared three-dimensional environment. For example, the user of the first electronic device 360 and the user of the second electronic device 370 (including the objects displayed in the shared three-dimensional environment) have a spatial live within the spatial group 340. In some examples, the spatial live requires a consistent spatial arrangement between the user (or their representation) and the virtual object. For example, the distance between the viewpoint of the user of the first electronic device 360 and the avatar 315 corresponding to the user of the second electronic device 360 can be the same as the distance between the viewpoint of the user of the second electronic device 370 and the avatar 317 corresponding to the user of the first electronic device 370. As described herein, if the position of the viewpoint of the user of the first electronic device 360 moves, the avatar 317 corresponding to the user of the first electronic device 360 moves in the three-dimensional environment 350B according to the movement of the position of the user's viewpoint relative to the viewpoint of the user of the second electronic device 370. Additionally, if the user of the first electronic device 360 performs an interaction with the shared virtual object 310 (e.g., moves the virtual object 310 in the three-dimensional environment 350A), the second electronic device 370 changes the display of the shared virtual object 310 in the three-dimensional environment 350B according to the interaction (e.g., moves the virtual object 310 in the three-dimensional environment 350B).

[0048] It should be understood that in some examples, more than two electronic devices can be communicatively linked in the multi-user communication session. For example, in a case where three electronic devices are communicatively linked in the multi-user communication session, the first electronic device will display two avatars corresponding to the users of the other two electronic devices, rather than just one avatar. Therefore, it should be understood that the various processes and exemplary interactions described herein with reference to the first electronic device 360 and the second electronic device 370 in the multi-user communication session optionally apply to cases where more than two electronic devices are communicatively linked in the multi-user communication session.

[0049] In some examples, it may be advantageous to provide mechanisms for facilitating multi-user communication sessions that include co-located and non-co-located users (e.g., co-located and non-co-located electronic devices associated with users). For example, it may be desirable to enable users co-located in a first physical environment to establish a multi-user communication session with one or more users not co-located in the first physical environment such that virtual content can be shared and presented in a three-dimensional environment, which optionally is viewable and / or interactive by co-located and non-co-located users in the multi-user communication session. As used herein, a co-located user corresponds to a local user relative to a first electronic device, and a non-co-located user corresponds to a remote user. As similarly discussed above, the three-dimensional environment optionally includes avatars corresponding to users of non-co-located electronic devices in the multi-user communication session. In some examples, as discussed below, the presentation of virtual objects (e.g., avatars and shared virtual content) in a three-dimensional environment during a multi-user communication session that includes co-located and non-co-located users (e.g., relative to a first electronic device) is based on the position and / or orientation of the co-located user in the physical environment of the first electronic device.

[0050] Figures 4A to 4J An example of initiating a multi-user communication session that includes co-located and non-co-located users in accordance with some examples of the present disclosure is shown. In some examples, when the first electronic device 101a is in a multi-user communication session with the second electronic device 101b, a three-dimensional environment 450A is presented using the first electronic device 101a (e.g., via the display 120a), and a three-dimensional environment 450B is presented using the second electronic device 101b (e.g., via the display 120b). In some examples, the electronic devices 101a / 101b optionally correspond to or are similar to the electronic devices 360 / 370 and / or Figure 2 the electronic devices 260 / 270 discussed above. In some examples, as Figure 4A shown, the first electronic device 101a is being used by a first user 402 (e.g., worn on their head), and the second electronic device 101b is being used by a second user 404 (e.g., worn on their head).

[0051] In Figure 4A as indicated by the top view 410, the first electronic device 101a and the second electronic device 101b are co-located in the physical environment 400. For example, both the first electronic device 101a and the second electronic device 101b are located in the same room that includes a stand 408 and a houseplant 409. In some examples, the determination that the first electronic device 101a and the second electronic device 101b are co-located in the physical environment 400 is based on the distance between the first electronic device 101a and the second electronic device 101b. For example, in Figure 4AIn this case, since the first electronic device 101a is within the threshold distance (e.g., 0.1 meter, 0.5 meter, 1 meter, 2 meters, 3 meters, 5 meters, 10 meters, 15 meters, 20 meters, etc.) of the second electronic device 101b, the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400. In some examples, the determination that the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400 is based on the communication between the first electronic device 101a and the second electronic device 101b. For example, in Figure 4A the first electronic device 101a and the second electronic device 101b are configured to communicate (e.g., wirelessly, such as via Bluetooth, Wi-Fi, or a server (e.g., a wireless communication terminal)). In some examples, the first electronic device 101a and the second electronic device 101b are connected to the same wireless network in the physical environment 400. In some examples, the determination that the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400 is based on the strength of the wireless signals transmitted between the electronic devices 101a and 101b. For example, in Figure 4A since the strength of the Bluetooth signal (or other wireless signal) transmitted between the electronic devices 101a and 101b is greater than the threshold strength, the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400. In some examples, the determination that the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400 is based on the visual detection of the electronic devices 101a and 101b in the physical environment 400. For example, as Figure 4A shown, the second electronic device 101b is positioned within the field of view of the first electronic device 101a (e.g., because the second user 404 is standing within the field of view of the first electronic device 101a), which enables the first electronic device 101a to visually detect (e.g., identify or scan, such as via object detection or other image processing techniques) the second electronic device 101b (e.g., in one or more images captured by the first electronic device 101a via external image sensors 114b-i and 114c-i). Similarly, as Figure 4A shown, the first electronic device 101a is optionally positioned within the field of view of the second electronic device 101b (e.g., because the first user 402 is standing within the field of view of the second electronic device 101b), which enables the second electronic device 101b to visually detect the first electronic device 101a (e.g., in one or more images captured by the second electronic device 101b via external image sensors 11b-ii and 114c-ii).

[0052] In some examples, the three-dimensional environment 450A / 450B includes a captured portion of the physical environment 400 in which the electronic devices 460 / 470 are located. For example, because the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400, the three-dimensional environments 450A and 450B include the stand 408 (e.g., a representation of the stand) and the indoor plant 409 (e.g., a representation of the indoor plant), as viewed from the viewpoints of the first electronic device 101a and the second electronic device 101b, as Figure 4A shown. In some examples, these representations may include portions of the physical environment 400 viewed through the transparent or translucent displays of the electronic devices 101a and 101b. In some examples, the three-dimensional environment 450A / 450B has one or more characteristics of the three-dimensional environments 350A / 350B described above with reference to Figure 3 .

[0053] As described above with reference to Figure 3 , when the electronic devices are communicatively linked in a multi-user communication session, the users may be represented by avatars corresponding to the users of the electronic devices. In Figure 4A , because the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400, the users of the electronic devices 101a and 101b are represented in the multi-user communication session by their physical figures (e.g., bodies) that are visible in the passthrough of the physical environment 400 (e.g., rather than via virtual avatars). For example, as Figure 4A shown, when the first electronic device 101a and the second electronic device 101b are in a multi-user communication session, the second user 404 is visible in the field of view of the first electronic device 101a, and the first user 402 is visible in the field of view of the second electronic device 101b. As discussed in more detail below, if a third user (e.g., a remote user) who is not juxtaposed in the physical environment 400 joins the multi-user communication session, the third user is represented in the three-dimensional environments 450A and 450B via an avatar.

[0054] As described above with reference to Figure 3 , when the first user 402 of the first electronic device 101a and the second user 404 of the second electronic device 101b are juxtaposed in the physical environment 400 and when the first electronic device 101a is in a multi-user communication session with the second electronic device 101b, the first user 402 and the second user 404 may be in a first spatial group within the multi-user communication session. In some examples, the first spatial group has one or more characteristics of the spatial group 340 discussed above with reference to Figure 3 . As described above similarly, when the first user 402 and the second user 404 are in the first spatial group within the multi-user communication session, the users have a first spatial arrangement in the shared three-dimensional environment (e.g., byFigure 4A (represented by the positions of users 402 and 404 and / or the distance between the users in the top view 410 in), the first spatial arrangement is determined by the physical positions of the electronic devices 101a and 101b in the physical environment 440. In particular, the first electronic device 101a and the second electronic device 101b experience spatial reality within the first spatial group, as determined by the physical positions and / or orientations of the first user 402 and the second user 404, respectively.

[0055] In Figure 4B , when the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400 (e.g., and optionally when the first electronic device 101a is in a multi-user communication session with the second electronic device 110b), the first electronic device 101a and the second electronic device 101b detect an indication of a request to join a multi-user communication session with a third electronic device. In some examples, the indication corresponds to an indication of a request to add the third electronic device to the current multi-user communication session between the first electronic device 101a and the second electronic device 101b.

[0056] In some examples, the third electronic device is not juxtaposed with the first electronic device 101a and the second electronic device 101b. For example, as Figure 4B shown in the top view 412 in, the third electronic device 101c (e.g., together with the third user 406) is located in the physical environment 440 (e.g., including the table 442), which is different from the physical environment 400 where both the first electronic device 101a and the second electronic device 101b are located. In some examples, when the third electronic device 101c is in the physical environment 440, the third electronic device 101c is outside the threshold distance (e.g., as discussed above) of the first electronic device 101a and the second electronic device 101b. Additionally, in some examples, as Figure 4B shown, the third electronic device 101c is not within the field of view of the first electronic device 101a or the second electronic device 101b.

[0057] In some examples, when the first electronic device 101a and the second electronic device 101b detect the indication discussed above, the first electronic device 101a and the second electronic device 101b display a message element 420 (e.g., a notification) corresponding to the request to join a multi-user communication session with the third electronic device 101c. In some examples, as Figure 4B shown, the message element 420 includes a first option 421 that can be selected to accept the request (e.g., and join the multi-user communication session with the third electronic device 101c) and a second option 422 that can be selected to reject the request (e.g., and forego joining the multi-user communication session with the third electronic device 101c).

[0058] In Figure 4B , the first electronic device 101a and the second electronic device 101b detect one or more inputs accepting a request to join a multi-user communication session with a third electronic device 101c. For example, in Figure 4B , the first electronic device 101a and the second electronic device 101b detect a selection of a first option 421 in a message element 420. As an example, the first electronic device 101a and the second electronic device 101b detect an air pinching gesture pointing to the first option 421. For example, as Figure 4B shown, with respect to the first electronic device 101a, the first electronic device 101a and the second electronic device 101b respectively detect pinches performed by the hand of a first user 402 (e.g., hand 403) and the hand of a second user 404, optionally when the gazes of the first user 402 (e.g., gaze point 425) and the second user 404 respectively point to the first option 421 at the first electronic device 101a and the second electronic device 101b. It should be understood that additional or alternative inputs are possible, such as air tapping gestures, gaze and dwell inputs, verbal commands, etc.

[0059] In some examples, in response to detecting an input accepting a request to join a multi-user communication session with a third electronic device 101c, the first electronic device 101a and the second electronic device 101b initiate a process for presenting an avatar corresponding to a third user 406 of the third electronic device 101c in three-dimensional environments 450A and 450B, which indicates entering a multi-user communication session with the third electronic device 101c. For example, as described above, since the third user 406 is not co-located with the first user 402 and the second user 404 in the physical environment 400, the third user 406 is represented via an avatar (or other virtual representation) in the three-dimensional environments 450A / 450B when in the multi-user communication session. In some examples, as discussed below, initiating the process for presenting an avatar corresponding to the third user 406 in the three-dimensional environments 450A / 450B includes: identifying the placement location of the avatar within a first spatial group of the first user 402 and the second user 404.

[0060] In some examples, as Figure 4C shown, when the first electronic device 101a and the second electronic device 101b identify the placement location of the avatar corresponding to the third user 406 within the first spatial group, as shown in the top view 410, the first electronic device 101a and the second electronic device 101b analyze / identify the physical positions of the first electronic device 101a and the second electronic device 101b within a shared (e.g., synchronized) coordinate space / coordinate system of the first spatial group. For example, as Figure 4CAs indicated by the top view 410 in [description], the first electronic device 101a is located at a first position relative to the origin 431 (e.g., the geometric center) of the first spatial group, and the second electronic device 101b is located at a second position different from the first position relative to the origin 431. Additionally, the first electronic device 101a is located at a first distance from the origin 431, and the second electronic device 101b is located at a second distance from the origin 431 (e.g., different from or equal to the first distance). Additionally, in some examples, the origin 431 enables virtual content (e.g., shared applications, user interfaces, 3D objects / models, etc.) presented in a shared 3D environment to be positioned at the same position within the first spatial group for all local users (e.g., by positioning the virtual content relative to the origin 431).

[0061] In some examples, the origin 431 (e.g., and thus the shared coordinate system) discussed above is defined based on the physical environment 400 (e.g., the physical room in which the first electronic device 101a and the second electronic device 101b are located). In some examples, the first electronic device 101a and the second electronic device 101b are each configured to analyze the physical environment 400 to determine the origin 431 (e.g., and the shared coordinate system) based on simultaneous localization and mapping (SLAM) data exchanged between the first electronic device 101a and the second electronic device 101b (e.g., SLAM data stored separately on the electronic devices 101a and 101b, or SLAM data stored on one of the electronic devices 101a and 101b). For example, the first electronic device 101a and the second electronic device 101b use the SLAM data to facilitate a shared understanding of one or more physical properties of the physical environment 400, such as the dimensions of the physical environment, physical objects within the physical environment, the visual appearance of the physical environment (e.g., color and lighting characteristics), etc., based on which the origin 431 can be defined in the first spatial group. In some examples, the first electronic device 101a and the second electronic device 101b are each configured to analyze the physical environment 400 to determine the origin 431 based on one or more characteristics of the other electronic device as sensed by the electronic device individually. For example, based on one or more images captured via the external image sensors 114b-i and 114c-i, the first electronic device 101a analyzes the position of the second electronic device 101b relative to the viewpoint of the first electronic device 101a in the physical environment, and based on one or more images captured via the external image sensors 114b-ii and 114c-ii, the second electronic device 101b analyzes the position of the first electronic device 101a relative to the viewpoint of the second electronic device 101b in the physical environment 400 to establish a spatial reality within the first spatial group, and thus define the origin 431.

[0062] In some examples, when the first electronic device 101a and the second electronic device 101b identify the placement location of the avatar corresponding to the third user 406 in the first space group, as shown in the top view 410, the first electronic device 101a and the second electronic device 101b analyze / identify one or more physical properties of the physical environment 400. For example, as discussed above, the physical environment 400 includes a bracket 408 (e.g., including a houseplant 409). In some examples, the first electronic device 101a and the second electronic device 101b select the placement location of the avatar corresponding to the user of the third electronic device based on the position of the bracket 408 in the physical environment 400. For example, the location where the avatar corresponding to the third user is positioned in the shared three-dimensional environment is selected to not correspond to the position of the bracket 408 in the physical environment 400.

[0063] In some examples, as Figure 4C shown, when the first electronic device 101a and the second electronic device 101b identify the placement location of the avatar corresponding to the third user 406 in the first space group, as shown in the top view 410, the first electronic device 101a and the second electronic device 101b analyze / identify the orientation of the first electronic device 101a and the second electronic device 101b within the first space group. For example, the orientation of the first electronic device 101a defines the forward direction of the first electronic device 101a (e.g., the forward head direction of the first user 402), and the orientation of the second electronic device 101b defines the forward direction of the second electronic device 101b (e.g., the forward head direction of the second user 404). In Figure 4C this, as an example, in the top view 410, the forward direction of the first electronic device 101a and the forward direction of the second electronic device 101b are respectively indicated by arrows extending from the first electronic device 101a and the second electronic device 101b. In some examples, the first electronic device 101a and the second electronic device 101b use the forward directions of the electronic devices 101a and 101b to determine the average forward direction of the electronic devices 101a and 101b within the first space group (e.g., the average forward head direction of users 402 and 404). For example, as indicated in the top view 410, the first electronic device 101a and the second electronic device 101b determine the average forward direction 432 within the first space group based on averaging the forward directions of the first electronic device 101a and the second electronic device 101b.

[0064] In some examples, when the first electronic device 101a and the second electronic device 101b identify the placement location of the avatar corresponding to the third user 406 in the first space group, as shown in the top view 410, the first electronic device 101a and the second electronic device 101b analyze / identify one or more properties of the shared three-dimensional environment. In some examples, one or more properties of the shared three-dimensional environment correspond to the presence of virtual objects (e.g., shared content) currently displayed in the shared three-dimensional environment (e.g., in Figure 4C the current example, there are no virtual objects). In some examples, one or more properties of the shared three-dimensional environment include a spatial template associated with the first space group. For example, as shown in the top view 410 in Figure 4C , the spatial template indicates multiple or several seats that the participants in the multi-user communication session can occupy. As used herein, a space group within a multi-user communication session can be associated with multiple seats that determine the spatial layout of the space group. For example, the space group is configured to accommodate multiple users (e.g., from two users up to "n" users), and each of the multiple users is assigned (e.g., occupies) one of the multiple seats within the space group. In Figure 4C the example, the first user 402 is occupying the first seat among the multiple seats 430 in the first space group, and the second user 404 is occupying the second seat. Thus, if the first space group is associated with a spatial template, as indicated by the top view 410, the unoccupied seats (e.g., labeled A, B, C, and D) among the multiple seats 430 correspond to the positions where the avatar corresponding to the third user within the first space group can be placed.

[0065] In some examples, the first electronic device 101a and the second electronic device 101b select / coordinate the placement location of the avatar corresponding to the third user based on any one or combination of the factors described above. In Figure 4D , after selecting the placement location of the avatar corresponding to the third user, the first electronic device 101a and the second electronic device 101b present the avatar 405 corresponding to the third user 406 of the third electronic device 101c. In some examples, the avatar 405 has one or more characteristics of the avatars 315 and 317 described above with reference to Figure 3 .

[0066] As Figure 4D indicated by the top view 410 in Figure 4CSeat D among the multiple seats 430. Additionally, in some examples, the avatar 405 is positioned at seat D within the first spatial group based on the average forward direction 432 described above. For example, seat D (e.g., substantially) lies in the direction of the average forward direction 432. Alternatively, in Figure 4D , without using a spatial template, the placement of the avatar 405 within the first spatial group is at a predetermined distance from the origin 431 and in the direction of the average forward direction 432.

[0067] In some examples, as Figure 4D shown in the top view 412 in Figure 4D , when the third electronic device 101c enters a multi-user communication session with the first electronic device 101a and the second electronic device 101b, the third electronic device 101c presents avatars corresponding to the users of the first electronic device 101a and the second electronic device 101b. For example, in Figure 3 the top view 412 in Figure 4D , the third electronic device 101c (e.g., in a three-dimensional environment presented at the third electronic device 101c (similar to the three-dimensional environments 450A / 450B)) displays the avatar 411 corresponding to the second user 404 of the second electronic device 101b and the avatar 413 corresponding to the first user 402 of the first electronic device 101a. In some examples, the avatars 411 and 413 have one or more characteristics of the avatars 315 and 317 described above with reference to Figure 3 . In some examples, as Figure 4D indicated by the top view 412 in

[0068] , the third electronic device 101c displays the avatars 411 and 413 in corresponding orientations respectively based on the orientations of the second electronic device 101b and the first electronic device 101a, and displays the avatars 411 and 413 at corresponding positions in the three-dimensional environment presented at the third electronic device 101c based on the spatial arrangement of the second electronic device 101b and the first electronic device 101a within the first spatial group (e.g., shown in the top view 410), as similarly discussed above in this document.

[0068] In some examples, the above method for selecting the placement position of the avatar corresponding to the third user 406 can be similarly used to select the placement position of virtual content shared within a multi-user communication session. For example, in Figure 4E , instead of detecting an indication of a request to enter a multi-user communication session with the third electronic device 101c, the first electronic device 101a (e.g., or the second electronic device 101b) detects an input corresponding to a request to display shared content in a shared three-dimensional environment. As Figure 4EAs shown, the first electronic device 101a optionally displays user interface elements 424 associated with a media player application (e.g., a movie player application). In some examples, the user interface elements 424 include one or more selectable options for sharing content (e.g., Movie A) in a multi-user communication session. For example, as Figure 4E shown, the user interface element 424 includes a selectable option 426 that can be selected to share Movie A with all participants (e.g., everyone) in the multi-user communication session.

[0069] In Figure 4E , when the user interface element 424 is displayed, the first electronic device 101a detects an input corresponding to the selection of the selectable option 426. For example, as Figure 4E shown, the first electronic device 101a detects an air pinching gesture performed by the hand 403 of the first user 402, optionally when the gaze 425 of the first user 402 is directed at the selectable option 426.

[0070] In some examples, in response to detecting the selection of the selectable option 426, the first electronic device 101a initiates a process of displaying a shared virtual object in a shared three-dimensional environment. In some examples, as described above, the first electronic device 101a and the second electronic device 101b coordinate to select a placement location of the shared virtual object within the shared three-dimensional environment (e.g., based on the spatial arrangement of the first electronic device 101a and the second electronic device 101b in the first spatial group). Specifically, as described above, the first electronic device 101a and the second electronic device 101b select the placement location of the shared virtual object based on: the positions of the first electronic device 101a and the second electronic device 101b relative to the origin 431, the average forward directions of the first electronic device 101a and the second electronic device 101b, the positions of physical objects in the physical environment 400 (e.g., the position of a bracket 408), the seat positions within the spatial template of the first spatial group, and / or the positions of other virtual objects currently displayed in the shared three-dimensional environment (e.g., avatars or other application windows). Additionally, in some examples, the first electronic device 101a and the second electronic device 101b select the placement location of the shared virtual object based on the object type. For example, the object type is based on the orientation of the shared virtual object, such as whether the object is vertically oriented or horizontally oriented, as discussed in more detail later in this document.

[0071] In some examples, when the first electronic device 101a and the second electronic device 101b select the placement location of the shared virtual object in the shared three-dimensional environment (e.g., within the first spatial group), the first electronic device 101a and the second electronic device 101b display the shared virtual object at the selected placement location. For example, as Figure 4FAs shown, the shared virtual object is displayed as a shared application window 435 (e.g., a media player user interface that is displaying Movie A) in the shared three-dimensional environment. In some examples, as Figure 4F shown in the top view 410 in Figure 4D , because the same method used above to identify the placement location of the avatar is used to identify the placement location of the shared virtual object, the position of the shared application window 435 in the first spatial group is the same as the Figure 3 position of the avatar 405 in

[0072] In some examples, the above method for selecting the placement location of the avatar or the shared application window 435 corresponding to the third user 406 can be similarly used to select the placement location of the avatars corresponding to a group of remote users joining a multi-user communication session. In Figure 4G , the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c are juxtaposed in the physical environment 400, as similarly discussed above. For example, as Figure 4G shown, the first user 402, the second user 404, and the third user 406 are physically located in the same room. In some examples, the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c are communicatively linked in a multi-user communication session. In Figure 4G 's example, the first user 402, the second user 404, and the third user 406 form a group of local users (e.g., local to the physical environment 400).

[0073] In Figure 4H , the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c detect an indication of a request to join a multi-user communication session with a group of remote users. For example, as Figure 4H shown in the top view 412 in Figure 4HIn the example, the fourth user 414, the fifth user 416, and the sixth user 418 constitute a group of remote users relative to the group of local users discussed above (e.g., remote relative to the physical environment 400). In some examples, as similarly discussed above, when the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c detect an indication of a request to enter a multi-user communication session with the fourth electronic device 101d, the fifth electronic device 101e, and the sixth electronic device 101f, a message element 420 (e.g., including a first option 421 and a second option 422) is displayed at each electronic device.

[0074] In Figure 4H , the first electronic device 101a (e.g., as well as the second electronic device 101b and the third electronic device 101c) detects an input indicating acceptance of a request to enter a multi-user communication session with a group of remote users. For example, as similarly discussed above, the first electronic device 101a detects a selection of the first option 421 in the message element 420 displayed in the three-dimensional environment 450A.

[0075] In some examples, in response to detecting an input indicating acceptance of a request to enter a multi-user communication session with a group of remote users, the first electronic device 101a (e.g., as well as the second electronic device 101b and the third electronic device 101c) initiates a process of displaying a plurality of avatars corresponding to the group of remote users in a shared three-dimensional environment. In some examples, because a group of remote users rather than a single remote user is entering the multi-user communication session with the first user 402, the second user 404, and the third user 406, in addition to being determined based on the spatial arrangement of at least the first user 402, the second user 404, and the third user 406 (e.g., as previously discussed above with reference to Figure 4C ), the placement position of the avatars corresponding to the group of remote users is also determined based on the spatial arrangement of the remote users. For example, as Figure 4IAs shown, in the physical environment 440 where the fourth user 414, the fifth user 416, and the sixth user 418 are located, the fourth electronic device 101d, the fifth electronic device 101e, and the sixth electronic device 101f are associated with a second spatial group 445, which is different from the first spatial group of the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c. Additionally, in some examples, the fourth electronic device 101d, the fifth electronic device 101e, and the sixth electronic device 101f have a unique spatial arrangement within the second spatial group 445. For example, the second spatial group 445 is associated with a shared coordinate system / coordinate space that has an origin 431b (e.g., a center) synchronized among the fourth electronic device 101d, the fifth electronic device 101e, and the sixth electronic device 101f, as similarly discussed previously herein. Thus, the fourth electronic device 101d is located at a first position within the second spatial group 445 (e.g., relative to the origin 431b), the fifth electronic device 101e is located at a second position within the second spatial group 445, and the sixth electronic device 101f is located at a third position within the second spatial group 445. Additionally, in some examples, within the second spatial group 445, the fourth electronic device 101d is separated from the fifth electronic device 101e by a first distance 444a, the fifth electronic device 101e is separated from the sixth electronic device 101f by a second distance 444b, and the sixth electronic device 101f is separated from the fourth electronic device 101d by a third distance 444c.

[0076] Additionally, in some examples, the placement of an avatar corresponding to a group of remote users in the shared environment of a group of local users (e.g., the first user 402, the second user 404, and the third user 406) is determined based on the average forward direction of the electronic devices associated with the remote users. For example, as Figure 4I shown in the top view 412 in, the fourth electronic device 101d has a first orientation in the physical environment 440, the fifth electronic device 101e has a second orientation in the physical environment 440, and the sixth electronic device 101f has a third orientation in the physical environment 440. Thus, as similarly discussed above, the fourth electronic device 101d, the fifth electronic device 101e, and the sixth electronic device 101f determine the average forward direction 432b of the second spatial group 445.

[0077] In some examples, in accordance with the methods discussed previously above, the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c determine the average forward direction 432a of the first spatial group based on the individual orientations of the electronic devices within the physical environment 400. Additionally, as discussed previously herein, the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c identify the positions of the electronic devices and / or the distances between the electronic devices relative to the origin 431a within the first spatial group.

[0078] In some examples, data corresponding to the positions of the electronic devices associated with a group of remote users within the second spatial group 445 and / or the distances between the electronic devices is transmitted to the first electronic device 101a, the second electronic device 101b, and / or the third electronic device 101c (e.g., directly from each electronic device, or via a wireless server). Additionally or alternatively, in some examples, data corresponding to the orientations and / or average forward directions of the electronic devices associated with a group of remote users within the second spatial group 445 is transmitted to the first electronic device 101a, the second electronic device 101b, and / or the third electronic device 101c (e.g., directly from each electronic device, or via a wireless server). In some examples, the first electronic device 101a, the second electronic device 101b, and / or the third electronic device 101c utilize the above data to determine the placement location of an avatar corresponding to a group of remote users in coordination with the spatial arrangement of the first spatial group and / or the average forward direction of the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c.

[0079] In some examples, as Figure 4J shown, when the first electronic device 101a, the second electronic device 101b, and / or the third electronic device 101c determine the placement location of an avatar corresponding to a group of remote users, the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c display multiple avatars corresponding to a group of remote users within their respective three-dimensional environments. For example, as Figure 4J shown, the first electronic device 101a displays an avatar 415 corresponding to a fourth user 414, an avatar 417 corresponding to a fifth user 416, and an avatar 419 corresponding to a sixth user 418 within the three-dimensional environment 450A. In some examples, the avatars 415, 417, and 419 have one or more characteristics of the avatars 317 and 319 discussed above with reference to Figure 3 .

[0080] In some examples, when avatars 415, 417, and 419 corresponding to a group of remote users are displayed in the three-dimensional environment 450A at the first electronic device 101a (e.g., and in corresponding three-dimensional environments at the second electronic device 101b and the third electronic device 101c), the spatial arrangement of avatars 415, 417, and 419 within the three-dimensional environment 450A corresponds to Figure 4I the spatial arrangement of the fourth user 414, the fifth user 416, and the sixth user 418 within the second spatial group 445 in Figure 4J For example, as shown in the top view 410 in Figure 4I the avatar 415 corresponding to the fourth user 414 and the avatar 417 corresponding to the fifth user 416 are separated by a first distance 444a (e.g., corresponding to Figure 3 the first distance 444a between the fourth electronic device 101d and the fifth electronic device 101e in the second spatial group 445 in Figure 4J the avatar 417 and the avatar 419 corresponding to the sixth user 418 are separated by a second distance 444b (e.g., corresponding to the second distance 444b between the fifth electronic device 101e and the sixth electronic device 101f in the second spatial group 445), and the avatar 419 and the avatar 415 are separated by a third distance 444c (e.g., corresponding to the third distance 444c between the sixth electronic device 101f and the fourth electronic device 101d in the second spatial group 445). Additionally, in some examples, as shown in the top view 410 and as discussed above with reference to

[0081] When avatars 415, 417, and 419 corresponding to a group of remote users are displayed in the three-dimensional environment 450A at the first electronic device 101a (e.g., and in corresponding three-dimensional environments at the second electronic device 101b and the third electronic device 101c), the positions of avatars 415, 417, and 419 within the three-dimensional environment 450A are selected based on the average forward direction 432b of the electronic devices associated with the group of remote users. Specifically, as Figure 4JAs shown in the top view 410, the average forward direction 432b is aligned (e.g., parallel) with the average forward direction 432a of the electronic devices associated with a group of local users. In some examples, the avatars 415, 417, and 419 corresponding to a group of remote users are positioned at a predetermined distance from the viewpoints of the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c (e.g., relative to the origin 431a of the first spatial group).

[0082] Additionally, in some examples, when the fourth electronic device 101d, the fifth electronic device 101e, and the sixth electronic device 101f enter a multi-user communication session with the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c, the fourth electronic device 101d, the fifth electronic device 101e, and the sixth electronic device 101f generate avatars corresponding to the first user 402, the second user 404, and the third user 406, and display these avatars in the corresponding three-dimensional environments at the fourth electronic device 101d, the fifth electronic device 101e, and the sixth electronic device 101f. For example, as Figure 4J shown in the top view 412, the fourth electronic device 101d, the fifth electronic device 101e, and the sixth electronic device 101f display the avatar 452 corresponding to the first user 402, the avatar 454 corresponding to the second user 404, and the avatar 456 corresponding to the third user 406. In some examples, the avatars 452, 454, and 456 have one or more characteristics of the avatars 317 and 319 discussed above with reference to Figure 3 .

[0083] In some examples, as similarly described above, when the avatars 452, 454, and 456 are displayed in the corresponding three-dimensional environments at the fourth electronic device 101d, the fifth electronic device 101e, and the sixth electronic device 101f, the avatars 452, 454, and 456 are displayed in a spatial arrangement that is based on the spatial arrangement of the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c in the first spatial group discussed previously. For example, as Figure 4JAs shown in the top view 412, the avatar 452 corresponding to the first user 402 and the avatar 454 corresponding to the second user 404 are separated by a fourth distance 444d (e.g., corresponding to the distance between the first electronic device 101a and the second electronic device 101b within the first spatial group), the avatar 452 and the avatar 456 corresponding to the third user 406 are separated by a fifth distance 444e (e.g., corresponding to the distance between the first electronic device 101a and the third electronic device 101c), and the avatar 456 and the avatar 454 are separated by a sixth distance 444f (e.g., corresponding to the distance between the second electronic device 101b and the third electronic device 101c within the first spatial group). Additionally, as shown in the top view 412 and discussed similarly above, the avatars 452, 454, and 456 are displayed at positions with respect to the viewpoints of the fourth electronic device 101d, the fifth electronic device 101e, and the sixth electronic device 101f (e.g., with respect to the origin 431b of the second spatial group), and these positions are based on the average forward direction 432a of the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c (e.g., by aligning the average forward direction 432a with the average forward direction 432b of the fourth electronic device 101d, the fifth electronic device 101e, and the sixth electronic device 101f). In some examples, as discussed similarly above, the fourth electronic device 101d, the fifth electronic device 101e, and the sixth electronic device 101f determine the placement positions of the avatars 452, 454, and 456 based on data provided by the first electronic device 101a, the second electronic device 101b, and / or the third electronic device 101c (e.g., related to the position and / or orientation of the electronic devices).

[0084] Accordingly, as outlined above, systems and methods for displaying virtual objects (e.g., avatars and / or virtual content) in a shared three-dimensional environment while in a multi-user communication session advantageously enable collocated and non-collocated users to participate in the multi-user communication session and experience synchronized interactions with the content and other users, thereby improving user-device interactions. Additionally, automatically determining the positions of virtual objects (e.g., avatars and / or virtual content) to be displayed in the shared three-dimensional environment reduces and / or helps avoid user input for manually selecting positions in the shared three-dimensional environment, which, as another benefit, helps save computational resources that would otherwise be consumed in response to such user input. Attention will now be turned to additional examples of displaying virtual objects (e.g., avatars and / or virtual content) within a multi-user communication session including collocated and non-collocated users and electronic devices.

[0085] Figures 5A to 5E An example of presenting content within a multi-user communication session including collocated and non-collocated users is shown. In Figure 5AIn this case, a first electronic device 101a (e.g., associated with a first user 502), a second electronic device 101b (e.g., associated with a second user 504), and a third electronic device 101c (e.g., associated with a third user 506) are juxtaposed in a physical environment 500, as similarly discussed above. In some examples, the first user 502, the second user 504, and the third user 506 respectively correspond to Figures 4A to 4J the first user 402, the second user 404, and the third user 406.

[0086] As Figure 5A shown, the first electronic device 101a is presenting a three-dimensional environment 550A (e.g., via a display 120a). In Figure 5A this case, as similarly discussed above, the three-dimensional environment 550A includes a representation (e.g., a passthrough representation or a computer-generated representation) of the physical environment 500 of the first electronic device 101a. For example, as Figure 5A shown in the top view 510, the physical environment 500 corresponds to a room (e.g., a meeting room) including a table 508 and a plurality of chairs 547. Thus, as Figure 5A shown, the three-dimensional environment 550a presented using the first electronic device 101a includes representations of the table 508 and the plurality of chairs 547 (e.g., the table 508 and the plurality of chairs 547 are visible in the field of view of the first electronic device 101a). Additionally, as Figure 5A shown, the second user 504 (e.g., and the second electronic device 101b) and the third user 506 (e.g., and the third electronic device 101c) are currently visible in the three-dimensional environment 550A as viewed from the current viewpoint of the first electronic device 101a. In some examples, the three-dimensional environment 550A has one or more characteristics of the three-dimensional environment 450A discussed above.

[0087] In Figure 5A this example, the first electronic device 101a (e.g., and / or the second electronic device 101b and the third electronic device 101c) has received an indication / input corresponding to a request to enter a multi-user communication session with a plurality of electronic devices (including the second electronic device 101b and the third electronic device 101c). In some examples, the indication / input has one or more characteristics of the indication / input discussed above with reference to Figures 4A to 4J .

[0088] In some examples, as previously discussed herein, when the first electronic device 101a (e.g., and / or the second electronic device 101b and the third electronic device 101c) receives an indication of a request to enter a multi-user communication session, the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c establish / form a spatial group corresponding to a shared three-dimensional environment within which virtual objects (e.g., avatars and / or virtual content) will be displayed (e.g., according to a shared / synchronized coordinate space, such as relative to an origin 531 (e.g., centered at the origin)). In some examples, as previously mentioned above, the spatial group may be associated with a spatial template indicating multiple or several seats that participants in the multi-user communication session may occupy. In some examples, the spatial template is determined based on one or more physical properties of the physical environment 500 in which the electronic devices 101a, 101b, and 101c are located. For example, the first electronic device 101a (e.g., and the second electronic device 101b and the third electronic device 101c) defines the spatial template of the spatial group based on the size of the physical environment 500 and / or physical objects in the physical environment 500, such as the table 508 and the multiple chairs 547. In Figure 5A the example, the spatial template is centered on the table 508 in the physical environment 500 (e.g., the origin 531 is positioned at the center of the table 508). Additionally, as Figure 5A indicated by the top view 510 in

[0089] In some examples, when using a spatial template based on the physical space around the first electronic device 101a (e.g., and the second electronic device 101b and the third electronic device 101c) to arrange the participants (e.g., remote and local users) in a multi-user communication session, the first electronic device 101a (e.g., and the second electronic device 101b and the third electronic device 101c) provides a visual indication of the seat 530 within the spatial template. For example, since the seat 530 is defined as corresponding to the end of the chair 547 and the table 508, as indicated in the top view 510, the first electronic device 101a displays a visual indication 536 at the position corresponding to the seat 530 in the three-dimensional environment 550A. In some examples, the visual indication 536 corresponds to a highlighting effect, a glowing effect, a flashing effect, or other animation effects. Thus, the visual indication 536 optionally provides guidance and / or suggestions to the first user 502, the second user 504, and the third user 506 for arranging them at the positions corresponding to the seat 530 within the spatial template of the spatial group of the multi-user communication session in the physical environment 500.

[0090] In Figure 5A it, the spatial group established among the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c is a hybrid spatial group. For example, as previously discussed above, the multi-user communication session includes collocated and non-collocated electronic devices. For example, in Figure 5B it, a fourth electronic device (not shown) and a fifth electronic device (not shown) that are not collocated with the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c in the physical environment 500 have joined the multi-user communication session. Thus, an avatar 515 corresponding to the fourth user of the fourth electronic device and an avatar 517 corresponding to the fifth user of the fifth electronic device are displayed in the shared three-dimensional environment. In some examples, the avatars 515 and 517 have one or more characteristics of the avatars 315 and 317 described with reference to Figure 3 .

[0091] As Figure 5B shown in the top view 510 in it, since the spatial template is being used to position the participants (e.g., users) in the multi-user communication session, the avatars 515 and 517 are displayed at the positions corresponding to the unoccupied seats (e.g., Figure 5A the seat 530 in it) in the shared three-dimensional environment. As Figure 5BAs shown in the top view 510, the first user 502, the second user 504, and the third user 506 have positioned themselves in the physical environment 500 at positions corresponding to the seats 530 within the spatial template, specifically within three of the chairs 547 of the table 508 in the physical environment. Thus, the avatars 515 and 517 are positioned at the seats 530 not occupied by one of the first user 502, the second user 504, and the third user 506, as Figure 5B shown. For example, the avatars 515 and 517 are positioned at the seats 530 on opposite ends of the table 508 within the spatial group, as shown in the top view 510. Thus, the first user 502, the second user 504, the third user 506, the fourth user (e.g., represented by the avatar 515), and the fifth user (e.g., represented by the avatar 517) can communicate within a multi-user communication session (e.g., via their respective electronic devices), and each has a unique viewpoint within the shared three-dimensional environment.

[0092] In some examples, as similarly discussed, when users are participating in a multi-user communication session, content can be shared within the shared three-dimensional environment, where the content is viewable and / or interactable by the users in the multi-user communication session. For example, in Figure 5C one, an electronic device among the electronic devices in the multi-user communication session detects an indication (e.g., an input) corresponding to a request to share content within the shared three-dimensional environment. In some examples, the electronic devices within the multi-user communication session utilize the methods previously discussed above with reference to Figures 4A to 4J to identify the placement location of the shared content within the shared three-dimensional environment. For example, as shown in the top view 510, the average forward direction 532 of the electronic device and / or the user is determined based on the individual orientations of the electronic device and / or the user, as previously discussed above. In Figure 5C this, because the multi-user communication session includes collocated and non-collocated electronic devices, the average forward direction 532 is determined based on both the orientations of the electronic devices collocated in the physical environment 500 (e.g., the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c) and the orientations of the electronic devices not collocated in the physical environment 500 (e.g., the fourth electronic device (whose orientation is represented by the avatar 515) and the fifth electronic device (whose orientation is represented by the avatar 517)). As previously discussed above, data corresponding to the orientations of the electronic devices is communicated between the electronic devices to coordinate the average forward direction 532 of the users within the spatial group in the multi-user communication session.

[0093] In some examples, the placement location of the shared content is selected to be in the direction of the average forward direction 532, as previously discussed above. For example, as Figure 5DAs shown in the top view 510 in [description], a virtual object 535 (e.g., an application window) is displayed at a position in the direction of the average forward direction 532 (e.g., a position intersecting the average forward direction) in the shared three-dimensional environment. Additionally, in some examples, the virtual object 535 is displayed at a position that is a predetermined distance from the origin 531 in the shared three-dimensional environment. Additionally or alternatively, in some examples, the virtual object 535 is displayed based on one or more physical properties of the juxtaposed physical environment 500 of the user. For example, as a supplement to or an alternative to being displayed at a position in the average forward direction 532, the virtual object 535 is displayed based on the physical surface of the physical environment 500 (e.g., in the top view 510, the right wall of the meeting room where the first user 502, the second user 504, and the third user 506 are located). As Figure 5D shown in the top view 510 in [description], the virtual object 535 is displayed relative to the right wall (such as at a predetermined distance from the right wall). In some examples, the virtual object 535 is displayed as being aligned with the right wall (e.g., the orientation of the virtual object 535 corresponds to the orientation of the right wall, and / or the virtual object 535 is anchored to the surface of the right wall, although not explicitly shown in Figure 5D [description].

[0094] In some examples, in addition to selecting the placement location of the shared content based on one or more physical properties of the physical environment 500, the placement location of the shared content can also be selected based on one or more properties of the shared content itself. For example, as previously mentioned above, one or more properties of the shared content include the orientation associated with the shared content. In Figure 5D [description], the virtual object 535 is optionally an object with a vertical orientation. For example, the forward-facing surface of the virtual object 535 (e.g., on which or in which the content is displayed) is two-dimensional. In some examples, if the shared content is alternatively associated with an object with a horizontal orientation, an alternative physical surface in the physical environment 500 can be used to anchor and / or display the shared content. For example, as Figure 5E shown in the top view 510 in [description], alternatively, a virtual object 537 corresponding to the shared content is displayed in the shared three-dimensional environment. In Figure 5E the example of [description], the virtual object 537 is an object with a horizontal orientation (e.g., the virtual object 537 corresponds to the user interface of a game application (such as a virtual board game)). Thus, as Figure 5E shown in the top view 510 in [description], the virtual object 537 is displayed based on the physical surface of the table 508 in the physical environment 500 (e.g., the horizontal orientation of the virtual object 537 corresponds to the horizontal (e.g., flat) surface of the table 508). In some examples, as Figure 5E indicated by the top view 510 in [description], the virtual object 537 can be centered at the origin 531 and does not have to be positioned to be in Figure 5Cin the direction of the indicated average forward direction 532.

[0095] Accordingly, as outlined above, a system and method for displaying virtual content in a shared three-dimensional environment when in a multi-user communication session advantageously enables co-located and non-co-located users to participate in a multi-user communication session and experience synchronized interactions with the virtual content, thereby improving user-device interaction. Additionally, automatically determining the location for displaying virtual content in the shared three-dimensional environment (e.g., based on one or more properties of the user's physical environment or one or more properties of the virtual content) reduces and / or helps avoid user input for manually selecting a location in the shared three-dimensional environment, which, as another benefit, helps save computational resources that would otherwise be consumed in response to such user input.

[0096] It should be understood that the examples shown and described herein are merely exemplary, and additional and / or alternative elements for interacting with the exemplary content may be provided within the three-dimensional environment. It should be understood that the appearance, shape, form, and size of each of the various user interface elements and objects shown and described herein are exemplary, and alternative appearances, shapes, forms, and / or sizes may be provided. For example, virtual objects representing application windows (e.g., virtual objects 330, 435, 535, and 537) may be provided in alternative shapes other than a rectangular shape (such as a circular shape, a triangular shape, etc.). In some examples, the various selectable options (e.g., options 421 and 422), user interface elements (e.g., message element 420 or user interface element 424), etc. described herein may be selected orally via a user verbal command (e.g., "select option" verbal command). Additionally or alternatively, in some examples, the various options, user interface elements, control elements, etc. described herein may be selected and / or manipulated via user input received through one or more separate input devices communicatively coupled to the electronic device. For example, a selection input may be received via a physical input device (such as a mouse, a touchpad, a keyboard, etc.) communicatively coupled to the electronic device.

[0097] Figure 6 A flowchart showing an example process for establishing a multi-user communication session between multiple electronic devices, at least a subset of which are non-co-located, in accordance with some examples of the present disclosure is shown. In some examples, process 600 begins at a first electronic device communicatively coupled to one or more displays and one or more input devices, where the first electronic device is co-located with a second electronic device in a first physical environment. In some examples, the first electronic device and the second electronic device are optionally similar to or correspond to Figure 2 the head-mounted display of device 200. As Figure 6As shown, in some examples, at 602, a first electronic device detects an indication of a request to enter a communication session with a third electronic device, where the third electronic device is not co-located in the first physical environment. For example, in Figure 4B the first electronic device 101a is displaying a message element 420 that includes a first option 421 that can be selected to enter a multi-user communication session with a second electronic device 101b and a third electronic device 101c.

[0098] In some examples, at 604, in response to detecting the indication, the first electronic device enters a communication session that includes the first electronic device, the second electronic device, and the third electronic device. For example, as described with reference to Figure 4C the first electronic device 101a initiates a process to determine the placement location of an avatar corresponding to the user of the third electronic device 101c. In some examples, at 606, the first electronic device obtains first data corresponding to the position of the user of the second electronic device relative to the viewing point of the first electronic device in the first physical environment. For example, in Figure 4C the top view 410 in, the first electronic device 101a identifies or determines the position of the second electronic device 101b relative to the viewing point of the first electronic device 101a.

[0099] In some examples, at 608, the first electronic device obtains second data corresponding to the orientation of the second electronic device relative to the viewing point of the first electronic device in the first physical environment. For example, in Figure 4C the top view 410 in, the first electronic device 101a identifies or determines the orientation of the second electronic device 101b in order to determine the average forward direction 432 of the first electronic device 101a and the second electronic device 101b. In some examples, at 610, the first electronic device displays a visual representation of the user of the third electronic device at a second location in a computer-generated environment via one or more displays, the second location being based on the first data and the second data. For example, as Figure 4D shown, the first electronic device 101a displays an avatar 405 corresponding to the user of the third electronic device 101c in a three-dimensional environment 450A.

[0100] It should be understood that process 600 is an example, and more, fewer, or different operations may be performed in the same or a different order. Additionally, the operations in the above process 600 are optionally implemented by running an information processing device such as a general-purpose processor (e.g., as described with respect to Figure 2 or one or more functional modules in a dedicated chip and / or by Figure 2 other components of.

[0101] Accordingly, in accordance with the foregoing, some examples of the present disclosure relate to a method that includes: at a first electronic device in communication with one or more displays and one or more input devices, detecting an indication of a request to enter a communication session with a third electronic device, wherein the first electronic device is co-located with a second electronic device in a first physical environment, and in response to detecting the indication, entering a communication session that includes the first electronic device, the second electronic device, and the third electronic device, the entering including: obtaining first data corresponding to a position of a viewpoint of a user of the second electronic device relative to the first electronic device in the first physical environment; obtaining second data corresponding to an orientation of the viewpoint of the second electronic device relative to the first electronic device in the first physical environment; and displaying a visual representation of a user of the third electronic device at a second position in a computer-generated environment via the one or more displays, the second position being determined based on the first data and the second data.

[0102] Additionally or alternatively, in some examples, the first electronic device being co-located with the second electronic device in the first physical environment is based on a determination that the second electronic device is within a threshold distance of the first electronic device in the first physical environment. Additionally or alternatively, in some examples, the third electronic device not being co-located with the first electronic device in the first physical environment is based on a determination that the third electronic device is outside a threshold distance of the first electronic device in the first physical environment. Additionally or alternatively, in some examples, the second electronic device being co-located with the first electronic device in the first physical environment is based on a determination that the second electronic device is within the field of view of the first electronic device. Additionally or alternatively, in some examples, the third electronic device not being co-located with the first electronic device in the first physical environment is based on a determination that the third electronic device is in a second physical environment different from the first physical environment. Additionally or alternatively, in some examples, the first data corresponding to the position of the viewpoint of the user of the second electronic device relative to the first electronic device in the first physical environment and the second data corresponding to the orientation of the viewpoint of the second electronic device relative to the first electronic device in the first physical environment are obtained based on one or more images of the first physical environment captured via one or more cameras of the first electronic device. Additionally or alternatively, in some examples, obtaining the first data corresponding to the position of the viewpoint of the user of the second electronic device relative to the first electronic device in the first physical environment includes receiving the first data from the second electronic device, and obtaining the second data corresponding to the orientation of the viewpoint of the second electronic device relative to the first electronic device in the first physical environment includes receiving the second data from the second electronic device.

[0103] Additionally or alternatively, in some examples, after entering a communication session that includes a first electronic device, a second electronic device, and a third electronic device, a user of the second electronic device is visible within the field of view of the first electronic device. Additionally or alternatively, in some examples, entering a communication session that includes a first electronic device, a second electronic device, and a third electronic device further includes: determining an average orientation of the electronic devices in the communication session based on the orientation of the second electronic device and the orientation of the first electronic device within a first physical environment. Additionally or alternatively, in some examples, the second location is further determined based on the average orientation of the electronic devices. Additionally or alternatively, in some examples, the second location is further determined based on one or more physical characteristics of the first physical environment. Additionally or alternatively, in some examples, one or more physical characteristics of the first physical environment include one or more locations of one or more physical objects within the first physical environment. Additionally or alternatively, in some examples, the second location is further determined based on one or more characteristics of a computer-generated environment. Additionally or alternatively, in some examples, one or more characteristics of the computer-generated environment include one or more locations of one or more virtual objects within the computer-generated environment.

[0104] Additionally or alternatively, in some examples, the first electronic device communicates with the fourth electronic device before detecting an indication of a request to enter a communication session, the first electronic device and the fourth electronic device are juxtaposed in a first physical environment, and after entering the communication session, the communication session includes the first electronic device, the second electronic device, the third electronic device, and the fourth electronic device, and entering the communication session further includes: obtaining third data corresponding to the position of the user of the fourth electronic device relative to the viewpoint of the first electronic device in the first physical environment; and obtaining fourth data corresponding to the orientation of the fourth electronic device relative to the viewpoint of the first electronic device in the first physical environment, wherein the second position is determined based on the first data, the second data, the third data, and the fourth data. Additionally or alternatively, in some examples, the method further includes: detecting a second indication of a request to add the fourth electronic device to the communication session, wherein the fourth electronic device is not juxtaposed in the first physical environment; and in response to detecting the second indication, displaying a visual representation of the user of the fourth electronic device at a third position different from the second position in a computer-generated environment via one or more displays, the third position being based on the first data, the second data, and the second position at which the visual representation of the user of the third electronic device is displayed. Additionally or alternatively, in some examples, the method further includes: after entering the communication session, detecting a third indication of a request to display shared content in a computer-generated environment; and in response to detecting the third indication, displaying a first object corresponding to the shared content at a fourth position different from the second position in a computer-generated environment via one or more displays, the fourth position being based on the first data, the second data, and the second position at which the visual representation of the user of the third electronic device is displayed. Additionally or alternatively, in some examples, depending on the determination that the shared content is of a first type of content, the fourth position is a first corresponding position in the computer-generated environment, and depending on the determination that the shared content is of a second type of content different from the first type of content, the fourth position is a second corresponding position different from the first corresponding position in the computer-generated environment.

[0105] Additionally or alternatively, in some examples, an indication of a request to enter a communication session with a third electronic device corresponds to an indication of a request to enter a communication session with a third electronic device and a fifth electronic device, where the fifth electronic device is co-located with the third electronic device in a second physical environment different from the first physical environment. Additionally or alternatively, in some examples, the method further comprises: in response to detecting the indication, displaying a visual representation of a user of the fifth electronic device at a fifth location different from a second location in a computer-generated environment via one or more displays, the fifth location being based on first data and second data. Additionally or alternatively, in some examples, in the second physical environment the third electronic device is separated from the fifth electronic device by a first distance, and in the computer-generated environment the fifth location is separated from the second location by the first distance. Additionally or alternatively, in some examples, the method further comprises: in response to detecting the indication: determining a first average orientation of the electronic devices in the first physical environment based on the orientation of the second electronic device and the orientation of the first electronic device; and determining a second average orientation of the electronic devices in the second physical environment based on the orientation of the third electronic device and the orientation of the fifth electronic device; wherein the second location and the fifth location are further determined based on aligning the first average orientation and the second average orientation. Additionally or alternatively, in some examples, detecting an indication of a request to enter a communication session with a third electronic device comprises: detecting, via one or more input devices, an input corresponding to a request to initiate a communication session with the second electronic device and the third electronic device. Additionally or alternatively, in some examples, an indication of a request to enter a communication session with a third electronic device corresponds to a user input detected by the second electronic device or the third electronic device to initiate a communication session with the first electronic device and the third electronic device or the first electronic device and the second electronic device.

[0106] Some examples of the present disclosure relate to a first electronic device comprising: one or more processors; a memory; and one or more programs, the one or more programs being stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described above.

[0107] Some examples of the present disclosure relate to a non-transitory computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by one or more processors of a first electronic device, cause the first electronic device to perform any of the methods described above.

[0108] Some examples of the present disclosure relate to a first electronic device comprising one or more processors, a memory, and means for performing any of the methods described above.

[0109] Some examples of the present disclosure relate to an information processing apparatus for use in a first electronic device, the information processing apparatus including components for performing any of the above methods.

[0110] For purposes of explanation, the foregoing description has been presented by reference to specific examples. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The examples were chosen and described in order to best illustrate the principles of the present disclosure and its practical application, thereby enabling others skilled in the art to best utilize the present disclosure with various modifications as are suited to the particular use contemplated and the various examples described.

[0111] Cross - reference to related applications

[0112] This application claims the benefit of U.S. Provisional Application No. 63 / 614,486, filed on December 22, 2023, and U.S. Patent Application No. 18 / 978,411, filed on December 12, 2024, the entire contents of both of which are incorporated herein by reference for all purposes.

Claims

1. A method, comprising: At a first electronic device in communication with one or more displays and one or more input devices, wherein the first electronic device is co-located with the second electronic device in a first physical environment: detecting an indication of a request to enter into a communication session with a third electronic device, wherein the third electronic device is not collocated in the first physical environment; as well as In response to detecting the indication, entering into the communication session including the first electronic device, the second electronic device, and the third electronic device, the entering comprising: obtaining first data corresponding to a position of a user of the second electronic device relative to a viewpoint of the first electronic device in the first physical environment; obtaining second data corresponding to an orientation of the second electronic device relative to the viewpoint of the first electronic device in the first physical environment; and A visual representation of a user of the third electronic device is displayed via the one or more displays at a second location in the three-dimensional environment, the second location being determined based on the first data and the second data.

2. The method according to claim 1, wherein: The first electronic device and the second electronic device are co-located in the first physical environment based on a determination that the second electronic device is within a threshold distance of the first electronic device in the first physical environment; and The third electronic device is not co-located with the first electronic device in the first physical environment based on a determination that the third electronic device is outside the threshold distance of the first electronic device in the first physical environment. 3 . The method of claim 1 , wherein the second electronic device is juxtaposed with the first electronic device in the first physical environment is determined based on a determination that the second electronic device is within a field of view of the first electronic device.

4. The method according to claim 1, wherein: The first data corresponding to the position of the user of the second electronic device relative to the viewpoint of the first electronic device in the first physical environment and the second data corresponding to the orientation of the second electronic device relative to the viewpoint of the first electronic device in the first physical environment are obtained based on one or more images of the first physical environment captured via one or more cameras of the first electronic device.

5. The method according to claim 1, wherein: Obtaining the first data corresponding to the position of the user of the second electronic device relative to the viewpoint of the first electronic device in the first physical environment includes: receiving the first data from the second electronic device; and Obtaining second data corresponding to the orientation of the second electronic device relative to the viewpoint of the first electronic device in the first physical environment includes receiving the second data from the second electronic device. The method of claim 1 , wherein the second location is determined further based on one or more physical characteristics of the first physical environment. 7 . The method of claim 6 , wherein the one or more physical characteristics of the first physical environment include one or more positions of one or more physical objects in the first physical environment.

8. The method of claim 1, wherein the second location is determined further based on one or more characteristics of the three-dimensional environment. 9 . The method of claim 8 , wherein the one or more characteristics of the three-dimensional environment include one or more positions of one or more virtual objects in the three-dimensional environment.

10. The method of claim 1, wherein: prior to detecting the indication of the request to enter the communication session, the first electronic device communicating with a fourth electronic device, the first electronic device and the fourth electronic device being co-located in the first physical environment; After entering the communication session, the communication session includes the first electronic device, the second electronic device, the third electronic device, and the fourth electronic device; and Entering into the communication session further comprises: obtaining third data corresponding to a position of a user of the fourth electronic device relative to the viewpoint of the first electronic device in the first physical environment; as well as obtaining fourth data corresponding to an orientation of the fourth electronic device relative to the viewpoint of the first electronic device in the first physical environment; and The second position is determined based on the first data, the second data, the third data, and the fourth data.

11. The method according to claim 1, further comprising: detecting a second indication of a request to add a fourth electronic device to the communication session, wherein the fourth electronic device is not co-located in the first physical environment; as well as In response to detecting the second indication, a visual representation of a user of the fourth electronic device is displayed via the one or more displays at a third position in the three-dimensional environment that is different from the second position, and the third position is based on the first data, the second data, and the second position of the visual representation of the user of the third electronic device.

12. The method according to claim 1, further comprising: after entering the communication session, detecting a third indication of a request to display shared content in the three-dimensional environment; as well as In response to detecting the third indication: Displaying a first object corresponding to the shared content at a fourth position in the three-dimensional environment that is different from the second position via the one or more displays, the fourth position being based on the first data, the second data, and the second position of the visual representation of the user of the third electronic device.

13. The method according to claim 12, wherein: Based on a determination that the shared content is a first type of content, the fourth position is a first corresponding position in the three-dimensional environment; and Based on a determination that the shared content is a second type of content different from the first type of content, the fourth location is a second corresponding location in the three-dimensional environment different from the first corresponding location.

14. The method of claim 1 , wherein the indication of the request to enter into the communication session with the third electronic device corresponds to an indication of a request to enter into the communication session with the third electronic device and a fifth electronic device, wherein the fifth electronic device is co-located with the third electronic device in a second physical environment different from the first physical environment, the method further comprising: In response to detecting the indication: A visual representation of a user of the fifth electronic device is displayed via the one or more displays at a fifth location in the three-dimensional environment that is different from the second location, the fifth location being based on the first data and the second data.

15. The method according to claim 14, wherein: The third electronic device is separated from the fifth electronic device by a first distance in the second physical environment; and The fifth position is separated from the second position by the first distance in the three-dimensional environment.

16. The method according to claim 14, further comprising: In response to detecting the indication: determining a first average orientation of electronic devices in the first physical environment based on the orientation of the second electronic device and the orientation of the first electronic device; as well as determining a second average orientation of electronic devices in the second physical environment based on the orientation of the third electronic device and the orientation of the fifth electronic device; Wherein the second position and the fifth position are further determined based on aligning the first average orientation and the second average orientation.

17. The method of claim 1, wherein detecting the indication of the request to enter into the communication session with the third electronic device comprises: An input corresponding to a request to initiate a communication session with the second electronic device and the third electronic device is detected via the one or more input devices.

18. A method according to claim 1, wherein the indication of the request to enter into the communication session with the third electronic device corresponds to user input detected by the second electronic device or the third electronic device for initiating a communication session with the first electronic device and the third electronic device or the first electronic device and the second electronic device.

19. A first electronic device, comprising: one or more processors; Memory; and One or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for executing the method according to any one of claims 1 to 18.

20. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of a first electronic device, cause the first electronic device to perform the method according to any one of claims 1 to 18.

Images