Maintaining visual consistency of visual representations of remote users in a three-dimensional environment within a multi-user communication session

Abstract

Some examples of the disclosure relate to systems and methods for maintaining visual consistency of visual representations of one or more remote users in a three-dimensional environment within a multi-user communication session comprising a group of co-located users. Some examples of the disclosure relate to systems and methods for transitioning display of visual representations of remote users in a three-dimensional environment from virtual tiles to three-dimensional avatars within a multi-user communication session comprising a group of co-located users.

Description

Cross-references to related applications

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 760,002, filed February 18, 2025, and U.S. Patent Application No. 19 / ---, filed January 1, 2026, the contents of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0002] This disclosure relates in its entirety to systems and methods for managing the visual representation of users in a multi-user communication session, wherein at least one subgroup of participants within the multi-user communication session are juxtaposed in a physical environment. Background Technology

[0003] Some computer graphics environments provide two-dimensional and / or three-dimensional environments, in which at least some objects displayed for user viewing are virtual and computer-generated. In some examples, the three-dimensional environment is presented by multiple devices communicating in a multi-user communication session. In some examples, an avatar (e.g., a representation) of each non-co-located user participating in the multi-user communication session (e.g., via a computing device) is displayed in the three-dimensional environment of the multi-user communication session. In some examples, content can be shared within the three-dimensional environment for viewing and interaction by multiple users participating in the multi-user communication session. Summary of the Invention

[0004] Some examples of this disclosure relate to systems and methods for maintaining visual consistency of visual representations of one or more remote users in a three-dimensional environment within a multi-user communication session comprising a set of juxtaposed users. In some examples, the method is performed at a first electronic device communicating with one or more displays and one or more input devices, wherein the first electronic device is juxtaposed with a second electronic device in a physical environment. In some examples, when in a multi-user communication session with the second electronic device and a plurality of corresponding electronic devices, wherein the plurality of corresponding electronic devices are not juxtaposed with the first and second electronic devices in the physical environment, the first electronic device presents multiple virtual representations of multiple users of the plurality of corresponding electronic devices in a three-dimensional environment via the one or more displays, wherein, from the viewpoint of the first electronic device, the plurality of virtual representations have a first spatial arrangement in the three-dimensional environment, and wherein the first spatial arrangement is based on first priority ordering data. In some examples, when presenting the plurality of virtual representations of the plurality of users of the plurality of corresponding electronic devices in the three-dimensional environment, the first electronic device detects changes or updates to the first priority ordering data. In some examples, in response to detecting a change in the first priority sorting data, and determining, based on the change in the first priority sorting data, that the first priority sorting data meets one or more criteria, the first electronic device updates the rendering of the plurality of virtual representations via the one or more displays to have a second spatial arrangement different from the first spatial arrangement, and sends corresponding display data to the second electronic device, which causes the second electronic device to update the rendering of the plurality of virtual representations via the one or more second displays of the second electronic device to have the second spatial arrangement. In some examples, if, based on the change in the first priority sorting data, it is determined that the first priority sorting data fails to meet the one or more criteria, the first electronic device maintains the rendering of the plurality of virtual representations with the first spatial arrangement in the 3D environment, and abandons sending the corresponding display data to the second electronic device.

[0005] Some examples of this disclosure relate to systems and methods for transforming a display of a remote user's visual representation in a three-dimensional environment from a virtual canvas into a three-dimensional avatar within a multi-user communication session comprising a group of juxtaposed users. In some examples, the method is performed at a first electronic device communicating with one or more displays and one or more input devices, wherein the first electronic device is juxtaposed with a second electronic device in a physical environment. In some examples, when in a multi-user communication session with the second electronic device and a plurality of corresponding electronic devices including a third electronic device, wherein the plurality of corresponding electronic devices are not juxtaposed with the first and second electronic devices in the physical environment, the first electronic device presents multiple virtual representations of the plurality of users of the plurality of corresponding electronic devices in a three-dimensional environment via the one or more displays, wherein, from the viewpoint of the first electronic device, the plurality of virtual representations have a first spatial arrangement in the three-dimensional environment. In some examples, when presenting multiple virtual representations of multiple users of the plurality of respective electronic devices in the three-dimensional environment, the first electronic device detects an indication of input received at the third electronic device among the plurality of respective electronic devices, the input corresponding to a request to represent the user of the third electronic device as a three-dimensional representation in the three-dimensional environment different from the first virtual representation of the user of the third electronic device, wherein the first virtual representation of the user of the third electronic device occupies a first position within the first spatial arrangement. In some examples, in response to detecting the indication of the input, the first electronic device stops presenting the first virtual representation of the plurality of virtual representations in the three-dimensional environment via the one or more displays. In some examples, the first electronic device presents the three-dimensional representation of the user of the third electronic device in the three-dimensional environment at a placement position relative to the plurality of virtual representations via the one or more displays, wherein the placement position in the three-dimensional environment is based on the first position of the first virtual representation within the first spatial arrangement.

[0006] A full description of these examples is provided in the accompanying drawings and detailed embodiments, and it should be understood that the content of this invention does not limit the scope of this disclosure in any way. Attached Figure Description

[0007] To better understand the various examples described herein, reference should be made to the following detailed embodiments and accompanying drawings. Throughout the drawings, similar reference numerals generally refer to corresponding parts.

[0008] Figure 1 Examples of electronic devices that present extended real-world environments according to some examples of this disclosure are illustrated.

[0009] Figure 2 A block diagram illustrating an example architecture of a system according to some examples of this disclosure is shown.

[0010] Figure 3 Examples of spatial groups in a multi-user communication session including a first electronic device and a second electronic device are illustrated according to some examples of this disclosure.

[0011] Figures 4A to 4J Examples of maintaining visual consistency of the visual representation of a remote user in a three-dimensional environment within a multi-user communication session that includes co-located and non-co-located users, according to some examples of this disclosure, are illustrated.

[0012] Figures 5A to 5K Examples of how, within a multi-user communication session including co-located and non-co-located users, the display of a remote user's visual representation in a three-dimensional environment is transformed from a virtual canvas into a three-dimensional avatar are illustrated according to some examples of this disclosure.

[0013] Figure 6 This is a flowchart illustrating some examples of a process for maintaining visual consistency of the visual representation of a remote user in a three-dimensional environment within a multi-user communication session, according to some examples of this disclosure.

[0014] Figure 7 This is a flowchart illustrating an example process, according to some examples of the present disclosure, for transforming the display of a remote user's visual representation in a three-dimensional environment from a virtual canvas into a three-dimensional avatar within a multi-user communication session. Detailed Implementation

[0015] Some examples of this disclosure relate to systems and methods for maintaining visual consistency of the visual representation of remote users in a three-dimensional environment within a multi-user communication session comprising a set of juxtaposed users. In some examples, the method is performed at a first electronic device communicating with one or more displays and one or more input devices, wherein the first electronic device is juxtaposed with a second electronic device in a physical environment. In some examples, when in a multi-user communication session with the second electronic device and a plurality of corresponding electronic devices, wherein the plurality of corresponding electronic devices are not juxtaposed with the first and second electronic devices in the physical environment, the first electronic device presents multiple virtual representations of multiple users of the plurality of corresponding electronic devices in a three-dimensional environment via the one or more displays, wherein, from the viewpoint of the first electronic device, the plurality of virtual representations have a first spatial arrangement in the three-dimensional environment, and wherein the first spatial arrangement is based on first priority ordering data. In some examples, when presenting the plurality of virtual representations of the plurality of users of the plurality of corresponding electronic devices in the three-dimensional environment, the first electronic device detects changes or updates to the first priority ordering data. In some examples, in response to detecting a change in the first priority sorting data, and determining, based on the change in the first priority sorting data, that the first priority sorting data meets one or more criteria, the first electronic device updates the rendering of the plurality of virtual representations via the one or more displays to have a second spatial arrangement different from the first spatial arrangement, and sends corresponding display data to the second electronic device, which causes the second electronic device to update the rendering of the plurality of virtual representations via the one or more second displays of the second electronic device to have the second spatial arrangement. In some examples, if, based on the change in the first priority sorting data, it is determined that the first priority sorting data fails to meet the one or more criteria, the first electronic device maintains the rendering of the plurality of virtual representations with the first spatial arrangement in the 3D environment, and abandons sending the corresponding display data to the second electronic device.

[0016] Some examples of this disclosure relate to systems and methods for transforming a display of a remote user's visual representation in a three-dimensional environment from virtual tiles into a three-dimensional avatar within a multi-user communication session comprising a group of juxtaposed users. In some examples, the method is performed at a first electronic device communicating with one or more displays and one or more input devices, wherein the first electronic device is juxtaposed with a second electronic device in a physical environment. In some examples, when in a multi-user communication session with the second electronic device and a plurality of corresponding electronic devices including a third electronic device, wherein the plurality of corresponding electronic devices are not juxtaposed with the first and second electronic devices in the physical environment, the first electronic device presents multiple virtual representations of the plurality of users of the plurality of corresponding electronic devices in a three-dimensional environment via the one or more displays, wherein, from the viewpoint of the first electronic device, the plurality of virtual representations have a first spatial arrangement in the three-dimensional environment. In some examples, when presenting multiple virtual representations of multiple users of the plurality of respective electronic devices in the three-dimensional environment, the first electronic device detects an indication of input received at the third electronic device among the plurality of respective electronic devices, the input corresponding to a request to represent the user of the third electronic device as a three-dimensional representation in the three-dimensional environment different from the first virtual representation of the user of the third electronic device, wherein the first virtual representation of the user of the third electronic device occupies a first position within the first spatial arrangement. In some examples, in response to detecting the indication of the input, the first electronic device stops presenting the first virtual representation of the plurality of virtual representations in the three-dimensional environment via the one or more displays. In some examples, the first electronic device presents the three-dimensional representation of the user of the third electronic device in the three-dimensional environment at a placement position relative to the plurality of virtual representations via the one or more displays, wherein the placement position in the three-dimensional environment is based on the first position of the first virtual representation within the first spatial arrangement.

[0017] In some examples, a spatial group or state in a multi-user communication session indicates a spatial arrangement or template indicating the location of users and content located within or otherwise associated with that spatial group. In some examples, users in the same spatial group within a multi-user communication session experience spatial realism based on the spatial arrangement of the spatial group. In some examples, when a user of a first electronic device is in a first spatial group in a multi-user communication session and a user of a second electronic device is in a second spatial group in a multi-user communication session, the user experiences spatial realism localized to their respective spatial group. In some examples, when users of a first electronic device and users of a second electronic device are grouped into separate spatial groups or states within a multi-user communication session, if the first and second electronic devices return to the same operating state, the users of the first and second electronic devices are regrouped into the same spatial group within the multi-user communication session.

[0018] As used herein, a hybrid space group corresponds to a group or number of participants (e.g., users) in a multi-user communication session in which at least a subset of the participants are not juxtaposed in a physical environment. For example, as described via one or more examples in this disclosure, a hybrid space group includes at least two participants juxtaposed in a first physical environment and at least one participant not juxtaposed with the at least two participants in the first physical environment (e.g., the at least one participant is located in a second physical environment different from the first physical environment). In some examples, the hybrid space group in the multi-user communication session has a spatial arrangement that determines the location of users and content within the space group. In some examples, users in the same hybrid space group within the multi-user communication session experience spatial realism based on the spatial arrangement of the space group, as discussed similarly above.

[0019] In some examples, initiating a multi-user communication session may include interaction with one or more user interface elements. In some examples, the electronic device tracks a user's gaze as input for aiming at selectable options / enabled representations within a corresponding user interface element displayed in a 3D environment. For example, a gaze can be used to identify one or more options / enabled representations selected as a target using another selection input. In some examples, a corresponding option / enabled representation can be selected using hand-tracking input detected via an input device communicating with the electronic device. In some examples, objects displayed in a 3D environment may be moved and / or reoriented in the 3D environment based on movement input detected via an input device.

[0020] Figure 1 An electronic device 101 is illustrated according to some examples of this disclosure, which presents an extended reality (XR) environment (e.g., a computer-generated environment that optionally includes representations of physical and / or virtual objects). In some examples, such as Figure 1 As shown, electronic device 101 is a head-mounted display or other head-mountable device configured to be worn on the head of a user of electronic device 101. An example of electronic device 101 is referenced below. Figure 2 Describe the architecture using a block diagram. For example... Figure 1 As shown, electronic device 101 and table 106 are located in a physical environment. The physical environment may include physical features such as physical surfaces (e.g., floor, wall) or physical objects (e.g., table, lamp, etc.). In some examples, electronic device 101 may be configured to detect and / or capture images of the physical environment including table 106 (exemplified in the field of view of electronic device 101).

[0021] In some examples, such as Figure 1As shown, the electronic device 101 includes one or more internal image sensors 114a oriented toward the user's face (e.g., referred to below). Figure 2 (The described eye-tracking camera). In some examples, an internal image sensor 114a is used for eye tracking (e.g., detecting the user's gaze). The internal image sensor 114a is optionally arranged on the left and right portions of the display 120 to enable eye tracking of the user's left and right eyes. In some examples, the electronic device 101 also includes external image sensors 114b and 114c facing outwards from the user to detect and / or capture the physical environment of the electronic device 101 and / or movement of the user's hands or other body parts.

[0022] In some examples, display 120 has a field of view visible to the user (e.g., it may or may not correspond to the fields of view of external image sensors 114b and 114c). Because display 120 is optionally part of a head-mounted device, the field of view of display 120 is optionally the same as or similar to the field of view of the user's eyes. In other examples, the field of view of display 120 may be smaller than the field of view of the user's eyes. In some examples, electronics 101 may be an optical pass-through device, through which display 120 is a transparent or translucent display through which parts of the physical environment can be directly viewed. In some examples, display 120 may be included within a transparent lens and may overlap with all or only a portion of the transparent lens. In other examples, electronics may be a video pass-through device, through which display 120 is an opaque display configured to display images of the physical environment captured by external image sensors 114b and 114c. Although a single display 120 is shown, it should be understood that display 120 may include a pair of stereoscopic displays.

[0023] In some examples, in response to a trigger, electronic device 101 can be configured to display in an XR environment... Figure 1 The illustrated cube represents a virtual object 104 that does not exist in the physical environment but is displayed in the XR environment as positioned on top of a table 106 (a real-world table or its representation). Optionally, in response to detecting a flat surface of the table 106 in the physical environment 100, the virtual object 104 may be displayed on the surface of the table 106 in the XR environment as displayed via the display 120 of the electronic device 101.

[0024] It should be understood that virtual object 104 is a representative virtual object and may include and render one or more different virtual objects (e.g., virtual objects with various dimensions, such as two-dimensional or other three-dimensional virtual objects) in a three-dimensional XR environment. For example, a virtual object may represent an application or user interface displayed in an XR environment. In some examples, a virtual object may represent content corresponding to an application and / or displayed via a user interface in an XR environment. In some examples, virtual object 104 is optionally configured to be interactive and responsive to user input (e.g., air gestures, such as air pinch gestures, air tap gestures, and / or air touch gestures), allowing the user to virtually touch, tap, move, rotate, or otherwise interact with virtual object 104.

[0025] In some examples, displaying an object in a 3D environment may include interaction with one or more user interface objects in the 3D environment. For example, initiating the display of an object in a 3D environment may include interaction with one or more virtual option / power representations displayed in the 3D environment. In some examples, when initiating the display of an object in a 3D environment, the electronic device may track the user's gaze as input for identifying one or more virtual option / power representations as a target for selection. For example, a gaze may be used to identify one or more virtual option / power representations as a target for selection using another selection input. In some examples, a virtual option / power representation may be selected using hand-tracking input detected via an input device communicating with the electronic device. In some examples, an object displayed in a 3D environment may move and / or reorient itself in the 3D environment based on movement input detected via an input device.

[0026] In the following discussion, an electronic device communicating with a display generation component and one or more input devices is described. It should be understood that the electronic device optionally communicates with one or more other physical user interface devices, such as a touch-sensitive surface, physical keyboard, mouse, joystick, hand-tracking device, eye-tracking device, stylus, etc. Furthermore, as described above, it should be understood that the described electronic device, display, and touch-sensitive surface are optionally distributed among two or more devices. Therefore, as used in this disclosure, information on or displayed by the electronic device is optionally used to describe information output by the electronic device for display on a separate display device (touch-sensitive or non-touch-sensitive). Similarly, as used in this disclosure, input received on the electronic device (e.g., touch input received on a touch-sensitive surface of the electronic device, or touch input received on the surface of a stylus) is optionally used to describe input received on a separate input device from which the electronic device receives input information.

[0027] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, phone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, TV channel browsing applications, and / or digital video player applications.

[0028] Figure 2 A block diagram illustrating an example architecture of a system 201 according to some examples of this disclosure is provided. In some examples, system 201 includes multiple devices. For example, system 201 includes a first electronic device 260 and a second electronic device 270, wherein the first electronic device 260 and the second electronic device 270 communicate with each other. In some examples, the first electronic device 260 and the second electronic device 270 are respectively portable devices, such as mobile phones, smartphones, tablet computers, laptop computers, auxiliary devices for communicating with another device, head-mounted displays, etc. In some examples, the first electronic device 260 and the second electronic device 270 are related to the above references. Figure 1 The described electronic device 101 corresponds to this.

[0029] like Figure 2As illustrated, the first electronic device 260 optionally includes various sensors (e.g., one or more hand tracking sensors 202A, one or more position sensors 204A, one or more image sensors 206A, one or more touch-sensitive surfaces 209A, one or more motion and / or orientation sensors 210A, one or more eye tracking sensors 212A, one or more microphones 213A or other audio sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors), one or more display generation components 214A, one or more speakers 216A, one or more processors 218A, one or more memories 220A, and / or communication circuitry 222A). In some examples, the second electronic device 27 Optionally, it includes various sensors (e.g., one or more hand tracking sensors 202B, one or more position sensors 204B, one or more image sensors 206B, one or more touch-sensitive surfaces 209B, one or more motion and / or orientation sensors 210B, one or more eye tracking sensors 212B, one or more microphones 213B or other audio sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors), one or more display generation components 214B, one or more speakers 216, one or more processors 218B, one or more memories 220B, and / or communication circuitry 222B. In some examples, one or more display generation components 214A, 214B and... Figure 1 Corresponding to display 120. One or more communication buses 208A and 208B are optionally used for communication between the aforementioned components of electronic device 260 and electronic device 270, respectively. The first electronic device 260 and the second electronic device 270 optionally communicate via a wired or wireless connection between the two devices (e.g., via communication circuits 222A, 222B).

[0030] Communication circuits 222A and 222B optionally include circuitry for communicating with electronic devices and networks such as the Internet, intranets, wired and / or wireless networks, cellular networks, and wireless local area networks (LANs). Communication circuits 222A and 222B optionally include circuitry for using near-field communication (NFC) and / or short-range communication such as Bluetooth. ® The circuit used for communication.

[0031] Processors 218A and 218B include one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, memories 220A and 220B are non-transitory computer-readable storage media (e.g., flash memory, random access memory, or other volatile or non-volatile memory or storage device) storing computer-readable instructions configured to be executed by processors 218A and 218B to perform the techniques, processes, and / or methods described below. In some examples, memories 220A and 220B may include more than one non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium can be any medium (e.g., excluding signals) that can tangibly contain or store computer-executable instructions for use by or in connection with instruction execution systems, apparatuses, and devices. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include hard disks, optical discs based on compressed disc (CD), digital versatile optical disc (DVD), or Blu-ray technology, as well as persistent solid-state storage (such as flash memory, solid-state drives, etc.).

[0032] In some examples, display generating components 214A, 214B include a single display (e.g., a liquid crystal display (LCD), an organic light-emitting diode (OLED), or other types of display). In some examples, display generating components 214A, 214B include multiple displays. In some examples, display generating components 214A, 214B may include a touch-enabled display (e.g., a touchscreen), a projector, a holographic projector, a retinal projector, a transparent or translucent display, etc. In some examples, electronic devices 260 and 270 respectively include touch-sensitive surfaces 209A and 209B for receiving user input such as tap input and swipe input or other gestures. In some examples, display generating components 214A, 214B and touch-sensitive surfaces 209A, 209B form a touch-sensitive display (e.g., a touchscreen integrated with electronic devices 260 and 270 respectively, or a touchscreen external to electronic devices 260 and 270 and communicating with electronic devices 260 and 270 respectively).

[0033] Electronic devices 260 and 270 optionally include image sensor 206A and image sensor 206B, respectively. Image sensor 206A / 206B optionally includes one or more visible light image sensors (such as charge-coupled device (CCD) sensors) and / or complementary metal-oxide-semiconductor (CMOS) sensors operable to acquire images of physical objects from a real-world environment. Image sensor 206A / 206B also optionally includes one or more infrared (IR) sensors, such as passive or active IR sensors, for detecting infrared light from the real-world environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real-world environment. Image sensor 206A / 206B also optionally includes one or more cameras configured to capture movement of a physical object in the real-world environment. Image sensor 206A / 206B also optionally includes one or more depth sensors configured to detect the distance between the physical object and electronic devices 260 / 270. In some examples, information from one or more depth sensors allows a device to identify objects in a real-world environment and distinguish them from other objects in the real-world environment. In some examples, one or more depth sensors allow a device to determine the texture and / or shape of objects in a real-world environment.

[0034] In some examples, electronic devices 260 and 270 use a combination of CCD sensors, event cameras, and depth sensors to detect the physical environment surrounding them. In some examples, image sensors 206A / 206B include a first image sensor and a second image sensor. The first and second image sensors work collaboratively and are optionally configured to capture different information about physical objects in the real-world environment. In some examples, the first image sensor is a visible light image sensor, and the second image sensor is a depth sensor. In some examples, electronic devices 260 / 270 use image sensors 206A / 206B to detect the location and orientation of electronic devices 260 / 270 and / or display generation components 214A / 214B in the real-world environment. For example, electronic devices 260 / 270 use image sensors 206A / 206B to track the location and orientation of display generation components 214A / 214B relative to one or more stationary objects in the real-world environment.

[0035] In some examples, electronic device 260 / 270 includes microphone 213A / 213B or other audio sensors. Device 260 / 270 uses microphone 213A / 213B to detect sound from the user and / or the user's real-world environment. In some examples, microphone 213A / 213B includes an array of microphones (multiple microphones) that optionally operate cooperatively to identify ambient noise or locate sound sources in the space of the real-world environment.

[0036] In some examples, device 260 / 270 includes a position sensor 204A / 204B for detecting the position of device 260 / 270 and / or display generating components 214A / 214B. For example, position sensor 204A / 204B may include a Global Positioning System (GPS) receiver that receives data from one or more satellites and allows electronic device 260 / 270 to determine the absolute location of the device in the physical world.

[0037] In some examples, electronic device 260 / 270 includes orientation sensors 210A / 210B for detecting the orientation and / or movement of electronic device 260 / 270 and / or display generation components 214A / 214B. For example, electronic device 260 / 270 uses orientation sensors 210A / 210B to track changes in the positioning and / or orientation of electronic device 260 / 270 and / or display generation components 214A / 214B, such as changes relative to physical objects in a real-world environment. Orientation sensors 210A / 210B optionally include one or more gyroscopes and / or one or more accelerometers.

[0038] In some examples, electronic device 260 / 270 includes hand tracking sensors 202A / 202B and / or eye tracking sensors 212A / 212B (and / or other body tracking sensors, such as leg, torso, and / or head tracking sensors). Hand tracking sensors 202A / 202B are configured to track the localization / position of one or more portions of a user's hand, and / or the movement of one or more portions of the user's hand relative to the extended reality environment, relative to display generation components 214A / 214B, and / or relative to another defined coordinate system. Eye tracking sensors 212A / 212B are configured to track the localization and movement of a user's gaze (more generally, eyes, face, or head) relative to the real world or extended reality environment and / or relative to display generation components 214A / 214B. In some examples, hand tracking sensors 202A / 202B and / or eye tracking sensors 212A / 212B are implemented together with display generation components 214A / 214B. In some examples, the hand tracking sensors 202A / 202B and / or eye tracking sensors 212A / 212B are implemented separately from the display generation components 214A / 214B.

[0039] In some examples, hand-tracking sensors 202A / 202B (and / or other body-tracking sensors, such as leg, torso, and / or head-tracking sensors) may use image sensors 206A / 206B (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that capture 3D information from the real world, including one or more body parts (e.g., a human user's hand, leg, or torso). In some examples, sufficient resolution is available to distinguish the hand to differentiate the fingers and their corresponding positions. In some examples, one or more image sensors 206A / 206B are positioned relative to the user to define the field of view and interaction space of the image sensors 206A / 206B, in which the finger / hand positions, orientations, and / or movements captured by the image sensors are used as input (e.g., to differentiate from the user's idle hand or other hands of other people in the real-world environment). Tracking the fingers / hands used for input (e.g., gestures, touches, taps, etc.) can be advantageous because it does not require the user to touch, hold, or wear any type of beacon, sensor, or other marker.

[0040] In some examples, the eye-tracking sensors 212A / 212B include at least one eye-tracking camera (e.g., an infrared (IR) camera) and / or an illumination source (e.g., an IR light source, such as an LED) that emits light toward the user's eyes. The eye-tracking camera may be pointed at the user's eyes to receive reflected IR light from the light source directly or indirectly from the eyes. In some examples, both eyes are tracked separately by the respective eye-tracking cameras and illumination sources, and focus / gaze can be determined by tracking both eyes. In some examples, one eye (e.g., the dominant eye) is tracked by one or more respective eye-tracking cameras / illumination sources.

[0041] Electronic devices 260 / 270 and systems 201 are not limited to Figure 2The components and configurations may include fewer components, other components, or additional components in multiple configurations. In some examples, system 201 may be implemented in a single device. One or more persons using system 201 are optionally referred to herein as one or more users of the device. Attention now turns to the exemplary simultaneous display of a three-dimensional environment on a first electronic device (e.g., corresponding to electronic device 260) and a second electronic device (e.g., corresponding to electronic device 270). As discussed below, the first electronic device may communicate with the second electronic device in a multi-user communication session. In some examples, an avatar (e.g., a representation) of a user of the first electronic device may be displayed in the three-dimensional environment at the second electronic device, and an avatar of a user of the second electronic device may be displayed in the three-dimensional environment at the first electronic device. In some examples, the user of the first electronic device and the user of the second electronic device may be associated with spatial groups in the multi-user communication session. In some examples, interaction with content in the three-dimensional environment when the first and second electronic devices are in a multi-user communication session may cause the user of the first electronic device and the user of the second electronic device to become associated with different spatial groups in the multi-user communication session.

[0042] Figure 3 Examples of spatial groups 340 in a multi-user communication session including a first electronic device 360 ​​and a second electronic device 370 according to some examples of this disclosure are illustrated. In some examples, the first electronic device 360 ​​may present a three-dimensional environment 350A, and the second electronic device 370 may present a three-dimensional environment 350B. The first electronic device 360 ​​and the second electronic device 370 may be similar to electronic devices 101 or 260 / 270, and / or may be head-mounted systems / devices and / or projection-based systems / devices (including hologram-based systems / devices) configured to generate and present three-dimensional environments, such as head-up displays (HUDs), head-mounted displays (HMDs), windows with integrated display capabilities, and displays formed as lenses (e.g., similar to contact lenses) designed to be placed on a person's eyes. Figure 3 In the example, a first user optionally wears a first electronic device 360 ​​and a second user optionally wears a second electronic device 370, such that the three-dimensional environment 350A / 350B can be defined by the X-axis, Y-axis, and Z-axis viewed from the perspective of the electronic device (e.g., the viewpoint associated with the electronic device 360 / 370, which may be, for example, a head-mounted display).

[0043] like Figure 3As shown, the first electronic device 360 ​​may be situated in a first physical environment including a table 306 and a window 309. Therefore, the three-dimensional environment 350A presented using the first electronic device 360 ​​optionally includes captured portions of the physical environment surrounding the first electronic device 360, such as a representation of the table 306' and a representation of the window 309'. Similarly, the second electronic device 370 may be situated in a second physical environment different from the first physical environment (e.g., separate from the first physical environment), which includes a floor lamp 307 and a coffee table 308. Therefore, the three-dimensional environment 350B presented using the second electronic device 370 optionally includes captured portions of the physical environment surrounding the second electronic device 370, such as a representation of the floor lamp 307' and a representation of the coffee table 308'. Additionally, the three-dimensional environments 350A and 350B may include representations of the floor, ceiling, and walls of the rooms where the first electronic device 360 ​​and the second electronic device 370 are respectively located.

[0044] As described above, in some examples, the first electronic device 360 ​​is optionally in a multi-user communication session with the second electronic device 370. For example, the first electronic device 360 ​​and the second electronic device 370 (e.g., via communication circuits 222A / 222B) are configured to present a shared three-dimensional environment 350A / 350B including one or more shared virtual objects (e.g., content such as images, videos, audio, etc., representations of the user interface of an application, etc.). As used herein, the term "shared three-dimensional environment" refers to a three-dimensional environment independently presented, displayed, and / or visible at two or more electronic devices, through which content, applications, data, etc., can be shared and / or presented to users of the two or more electronic devices. In some examples, when the first electronic device 360 ​​is in a multi-user communication session with the second electronic device 370, an avatar corresponding to a user of one electronic device is optionally displayed in the three-dimensional environment displayed via the other electronic device. For example, as Figure 3 As shown, at the first electronic device 360, an avatar 315 corresponding to the user of the second electronic device 370 is displayed in the three-dimensional environment 350A. Similarly, at the second electronic device 370, an avatar 317 corresponding to the user of the first electronic device 360 ​​is displayed in the three-dimensional environment 350B.

[0045] In some examples, the presentation of avatars 315 / 317 as part of a shared 3D environment is optionally accompanied by audio effects corresponding to the speech of a user of electronic devices 370 / 360. For example, avatar 315 displayed in 3D environment 350A using first electronic device 360 ​​is optionally accompanied by audio effects corresponding to the speech of a user of second electronic device 370. In some such examples, when a user of second electronic device 370 speaks, the user's speech can be detected by second electronic device 370 (e.g., via microphone 213B) and transmitted to first electronic device 360 ​​(e.g., via communication circuits 222B / 222A), such that the detected speech of the user of second electronic device 370 can be presented as audio (e.g., using speaker 216A) to the user of first electronic device 360 ​​in 3D environment 350A. In some examples, the audio effects corresponding to the speech of the user of second electronic device 370 can be spatialized so that they appear to the user of first electronic device 360 ​​to originate from the location of avatar 315 in shared 3D environment 350A (e.g., although output from a speaker of first electronic device 360). Similarly, the avatar 317 displayed in the three-dimensional environment 350B using the second electronic device 370 is optionally accompanied by audio effects corresponding to the voice of the user of the first electronic device 360. In some such examples, when the user of the first electronic device 360 ​​speaks, the user's voice can be detected by the first electronic device 360 ​​(e.g., via microphone 213A) and transmitted to the second electronic device 370 (e.g., via communication circuits 222A / 222B), so that the detected voice of the user of the first electronic device 360 ​​can be presented as audio (e.g., using speaker 216B) to the user of the second electronic device 370 in the three-dimensional environment 350B. In some examples, the audio effects corresponding to the voice of the user of the first electronic device 360 ​​can be spatialized so that they appear to the user of the second electronic device 370 to originate from the location of the avatar 317 in the shared three-dimensional environment 350B (e.g., although output from the speaker of the first electronic device 360).

[0046] In some examples, when in a multi-user communication session, avatars 315 / 317 are displayed in a three-dimensional environment 350A / 350B to correspond to the orientation of electronic devices 360 / 370 (and / or the users of electronic devices 360 / 370) in the physical environment surrounding electronic devices 360 / 370 and / or corresponding orientations based on these orientations. For example, as Figure 3As shown, in three-dimensional environment 350A, avatar 315 arbitrarily selects the user's viewpoint from the ground facing the first electronic device 360, and in three-dimensional environment 350B, avatar 317 arbitrarily selects the user's viewpoint from the ground facing the second electronic device 370. When a particular user moves the electronic device (and / or itself) in the physical environment, the user's viewpoint changes according to the movement, which can therefore also change the orientation of the user's avatar in the three-dimensional environment. For example, refer to... Figure 3 If the user of the first electronic device 360 ​​looks to the left in the three-dimensional environment 350A, causing the first electronic device 360 ​​to rotate to the left (e.g., counterclockwise) (e.g., by a corresponding amount), then the user of the second electronic device 370 will see the avatar 317 corresponding to the user of the first electronic device 360 ​​rotate to the right (e.g., clockwise) relative to the viewpoint of the user of the second electronic device 370 according to the movement of the first electronic device 360.

[0047] Additionally, in some examples, when in a multi-user communication session, the viewpoint of the 3D environment 350A / 350B and / or the position of the viewpoint of the 3D environment 350A / 350B may optionally change according to the movement of the electronic device 360 / 370 (e.g., by the user of the electronic device 360 / 370). For example, when in a communication session, if the first electronic device 360 ​​moves closer toward the table representation 306' and / or avatar 315 (e.g., because the user of the first electronic device 360 ​​moves forward in the physical environment surrounding the first electronic device 360), the viewpoint of the 3D environment 350A will change accordingly, making the table representation 306', the window representation 309', and the avatar 315 appear larger in the field of view. In some examples, each user can interact independently with the three-dimensional environment 350A / 350B, such that changes in the viewpoint of the three-dimensional environment 350A and / or interactions between the first electronic device 360 ​​and virtual objects in the three-dimensional environment 350A optionally do not affect the content displayed in the three-dimensional environment 350B at the second electronic device 370, and vice versa.

[0048] In some examples, avatar 315 / 317 is a representation of the user of electronic device 370 / 360 (e.g., full-body rendering). In some examples, avatar 315 / 317 is a representation of a portion of the user of electronic device 370 / 360 (e.g., rendering of head, face, head and torso, etc.). In some examples, avatar 315 / 317 is a representation of user personalization, user selection, and / or user creation displayed in 3D environment 350A / 350B, representing the user of electronic device 370 / 360. It should be understood that, although Figure 3 The avatars 315 / 317 illustrated correspond to the full-body representation of the user of the electronic devices 370 / 360, respectively, but alternative avatars, such as those described above, may be provided.

[0049] As described above, when the first electronic device 360 ​​and the second electronic device 370 are in a multi-user communication session, the three-dimensional environment 350A / 350B can be a shared three-dimensional environment presented using electronic devices 360 / 370. In some examples, in a multi-user communication session, content viewed by a user at one electronic device can be shared with another user at another electronic device. In some such examples, content can be experienced (e.g., viewed and / or interacted with) by two users (e.g., via their respective electronic devices) in a shared three-dimensional environment. For example, as... Figure 3 As shown, the 3D environment 350A / 350B includes a shared virtual object 310 (e.g., optionally a 3D virtual sculpture) that can be viewed and interacted with by two users. Figure 3 As shown, the shared virtual object 310 can be displayed together with a grabber indication (e.g., a control bar) 335 that can be selected to initiate movement of the shared virtual object 310 within the three-dimensional environment 350A / 350B.

[0050] In some examples, the 3D environment 350A / 350B includes non-shared content that is private to one user in a multi-user communication session. For example, in Figure 3 In this context, the first electronic device 360 ​​is displaying a private application window 330 in a three-dimensional environment 350A, which is optionally an object in a multi-user communication session that is not shared between the first electronic device 360 ​​and the second electronic device 370. In some examples, the private application window 330 may be associated with a corresponding application operating on the first electronic device 360 ​​(e.g., a media player application, a web browsing application, a messaging application, etc.). Because the private application window 330 is not shared with the second electronic device 370, the second electronic device 370 optionally displays a representation 330'' of the private application window in a three-dimensional environment 350B. Figure 3 As shown, in some examples, the representation 330'' of the private application window may be a faded, obscured, discolored, and / or semi-transparent representation of the private application window 330, which prevents the user of the second electronic device 370 from viewing the content of the private application window 330.

[0051] As previously mentioned above, in some examples, users of the first electronic device 360 ​​and users of the second electronic device 370 are in space group 340 within a multi-user communication session. In some examples, space group 340 may be a baseline (e.g., first or default) space group within the multi-user communication session. For example, when users of the first electronic device 360 ​​and users of the second electronic device 370 initially join the multi-user communication session, users of the first electronic device 360 ​​and users of the second electronic device 370 are automatically (and initially, as discussed in more detail below) associated with (e.g., grouped into) space group 340 within the multi-user communication session. In some examples, when users in such... Figure 3 In the spatial group 340 shown, the user of the first electronic device 360 ​​and the user of the second electronic device 370 have a first spatial arrangement (e.g., a first spatial template) within a shared three-dimensional environment. For example, the user of the first electronic device 360 ​​and the user of the second electronic device 370 (including objects displayed in the shared three-dimensional environment) have spatial realism within the spatial group 340. In some examples, spatial realism requires a consistent spatial arrangement between the user (or its representation) and the virtual objects. For example, the distance between the viewpoint of the user of the first electronic device 360 ​​and the avatar 315 corresponding to the user of the second electronic device 360 ​​may be the same as the distance between the viewpoint of the user of the second electronic device 370 and the avatar 317 corresponding to the user of the first electronic device 370. As described herein, if the position of the viewpoint of the user of the first electronic device 360 ​​moves, the avatar 317 corresponding to the user of the first electronic device 360 ​​moves within the three-dimensional environment 350B according to the movement of the user's viewpoint relative to the viewpoint of the user of the second electronic device 370. Additionally, if a user of the first electronic device 360 ​​interacts with the shared virtual object 310 (e.g., moves the virtual object 310 in the three-dimensional environment 350A), the second electronic device 370 changes the display of the shared virtual object 310 in the three-dimensional environment 350B based on the interaction (e.g., moves the virtual object 310 in the three-dimensional environment 350B).

[0052] It should be understood that in some examples, more than two electronic devices may be communicatively linked in a multi-user communication session. For example, in the case where three electronic devices are communicatively linked in a multi-user communication session, the first electronic device will display two avatars corresponding to the users of the other two electronic devices, instead of just one avatar. Therefore, it should be understood that the various processes and exemplary interactions described herein with reference to the first electronic device 360 ​​and the second electronic device 370 in a multi-user communication session are optionally applicable to cases where more than two electronic devices are communicatively linked in a multi-user communication session.

[0053] In some examples, it may be advantageous to provide mechanisms for maintaining visual and / or spatial consistency of the visual representations of non-co-located users in a multi-user communication session when facilitating such sessions, which include co-located and non-co-located users (e.g., co-located and non-co-located electronic devices associated with users). For example, it might be desirable to enable users co-located in a first physical environment to visually see the visual representation of a remote user from the same location in a three-dimensional environment while in a multi-user communication session, in order to improve and / or increase interaction with the visual representation in the three-dimensional environment. As used herein, co-located users correspond to local users relative to a first electronic device, while non-co-located users correspond to remote users. As discussed similarly above, the three-dimensional environment optionally includes visual representations (e.g., avatars or virtual canvases) corresponding to remote users of non-co-located electronic devices in the multi-user communication session. In some examples, as described below, within a multi-user communication session that includes co-located users and non-co-located users (e.g., relative to a first electronic device), the visual representation of the non-co-located user in a three-dimensional environment (e.g., an avatar and / or a virtual canvas) is presented based on data and / or other instructions sent between electronic devices associated with the co-located users within the multi-user communication session.

[0054] Figures 4A to 4J Examples of maintaining visual consistency in the visual representation of a remote user in a three-dimensional environment within a multi-user communication session including co-located and non-co-located users, according to some examples of this disclosure, are illustrated. In some examples, when a first electronic device 101a is in a multi-user communication session with a second electronic device 101b, the first electronic device 101a (e.g., via display 120a) is used to present the three-dimensional environment 450A, and the second electronic device 101b (e.g., via display 120b) is used to present the three-dimensional environment 450B. In some examples, electronic devices 101a / 101b optionally correspond to or resemble electronic devices 360 / 370 discussed above and / or Figure 2 Electronic devices 260 / 270. In some examples, such as Figure 4A As shown in the top view 410, the first electronic device 101a is being used by the first user 402 (e.g., Bella) (e.g., worn on their head), and the second electronic device 101b is being used by the second user 404 (e.g., Charlie) (e.g., worn on their head).

[0055] exist Figure 4AAs indicated in top view 410, a first electronic device 101a and a second electronic device 101b are juxtaposed in physical environment 400. For example, both the first electronic device 101a and the second electronic device 101b are located in the same room, including a window 409, a bracket 407, and an indoor plant 408. In some examples, the juxtaposition of the first electronic device 101a and the second electronic device 101b in physical environment 400 is determined based on the distance between them. For example, in... Figure 4A In this context, the first electronic device 101a and the second electronic device 101b are juxtaposed in physical environment 400 because the first electronic device 101a is within a threshold distance (e.g., 0.1 m, 0.5 m, 1 m, 2 m, 3 m, 5 m, 10 m, 15 m, 20 m, etc.) of the second electronic device 101b. In some examples, the juxtaposition of the first electronic device 101a and the second electronic device 101b in physical environment 400 is determined based on communication between them. For example, in... Figure 4A In this configuration, a first electronic device 101a and a second electronic device 101b are configured to communicate (e.g., wirelessly, such as via Bluetooth, Wi-Fi, or a server (e.g., a wireless communication terminal)). In some examples, the first electronic device 101a and the second electronic device 101b are connected to the same wireless network in physical environment 400. In some examples, the juxtaposition of the first electronic device 101a and the second electronic device 101b in physical environment 400 is determined based on the strength of the wireless signals transmitted between electronic devices 101a and 101b. For example, in Figure 4A In this context, because the strength of the Bluetooth signal (or other wireless signal) transmitted between electronic devices 101a and 101b is greater than a threshold strength, the first electronic device 101a and the second electronic device 101b are placed side-by-side in the physical environment 400. In some examples, the determination that the first electronic device 101a and the second electronic device 101b are placed side-by-side in the physical environment 400 is based on visual detection of electronic devices 101a and 101b in the physical environment 400. For example, in Figure 4ABefore and / or when the multi-user communication session is initialized in physical environment 400, the second electronic device 101b is positioned in the field of view of the first electronic device 101a (e.g., because the second user 404 is standing in the field of view of the first electronic device 101a), which enables the first electronic device 101a to visually detect (e.g., identify or scan, such as via object detection or other image processing techniques) the second electronic device 101b (e.g., in one or more images captured by the first electronic device 101a, such as via external image sensors 114b-i and 114c-i). Additionally or alternatively, before and / or when initializing a multi-user communication session, the first electronic device 101a may optionally be positioned within the field of view of the second electronic device 101b (e.g., because the first user 402 is standing within the field of view of the second electronic device 101b), enabling the second electronic device 101b to visually detect the first electronic device 101a (e.g., in one or more images captured by the second electronic device 101b, such as via external image sensors 11b-ii and 114c-ii). In some examples, the juxtaposition of the first electronic device 101a and the second electronic device 101b in the physical environment 400 is determined based on the visual detection of one or more identical (e.g., overlapping) features or portions of the physical environment 400. For example, as mentioned above, the physical environment 400 includes one or more physical objects (such as window 409, bracket 407, and indoor plants 408) and / or one or more physical surfaces (such as the walls, floor, and ceiling of the room where the electronic devices 101a / 101b are located). Figure 4A In the example, the first electronic device 101a and the second electronic device 101b determine the juxtaposition of the electronic devices in the physical environment 400 based on visual detection of windows 409, brackets 407, indoor plants 408 and / or the walls and floors of the room that are visible in the field of view of both electronic devices 101a / 101b (e.g., in one or more images captured by the first electronic device 101a and the second electronic device 101b).

[0056] In some examples, such as Figure 4A As shown, the three-dimensional environments 450A / 450B include the capture portion of the physical environment 400 where the electronic devices 101a / 101b are located. For example, since the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400, the three-dimensional environments 450A and 450B include a window 409 (e.g., a representation of a window), a bracket 407 (e.g., a representation of a bracket), and a houseplant 408 (e.g., a representation of a houseplant), but viewed from the perspective of the first electronic device 101a and the second electronic device 101b, as... Figure 4AAs shown. In some examples, these represent portions of the physical environment 400 viewed through transparent or semi-transparent displays of electronic devices 101a and 101b. In some examples, the three-dimensional environment 450A / 450B has the features described above. Figure 3 One or more characteristics of the described three-dimensional environment 350A / 350B.

[0057] As referenced above Figure 3 As described, when electronic devices are communicatively linked in a multi-user communication session, a user can be represented by a visual representation (e.g., a three-dimensional representation and / or a two-dimensional representation) corresponding to the user of the electronic device. Figure 4A In this context, since the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400, the users of electronic devices 101a and 101b are represented in a multi-user communication session via a physical person (e.g., a body) that may be visible in the transparency of the physical environment 400 (e.g., rather than via a virtual avatar). For example, as... Figure 4A As indicated by the top view 410, based on the physical locations of users 404 / 402 in physical environment 400, the second user 404 is currently not visible in the field of view of the first electronic device 101a, and the first user 402 is currently not visible in the field of view of the second electronic device 101b. However, since users 404 / 402 are juxtaposed in physical environment 400, the first electronic device 101a and the second electronic device 101b refrain from displaying a visual representation (e.g., an avatar) of users 404 / 402. As discussed in more detail below, users not juxtaposed in physical environment 400 (e.g., remote users) and users joining and / or participating in multi-user communication sessions are optionally represented in three-dimensional environments 450A and 450B via visual representations (e.g., avatars or virtual canvases).

[0058] As referenced above Figure 3 Similarly, when a first user 402 of the first electronic device 101a and a second user 404 of the second electronic device 101b are placed together in a physical environment 400, and when the first electronic device 101a is in a multi-user communication session with the second electronic device 101b, the first user 402 and the second user 404 may be in a first spatial group within the multi-user communication session. In some examples, the first spatial group has the above reference. Figure 3 One or more characteristics of the spatial group 340 under discussion. As described similarly above, when the first user 402 and the second user 404 are in the first spatial group within a multi-user communication session, the users have a first spatial arrangement in a shared three-dimensional environment (e.g., by...). Figure 4AThe first spatial arrangement (represented by the positions of users 402 and 404 and / or the distances between these users in the top view 410) is determined by the physical positions of electronic devices 101a and 101b in the physical environment 400. Specifically, the first electronic device 101a and the second electronic device 101b experience spatial realism within the first spatial group, as determined by the physical positions and / or orientations of the first user 402 and the second user 404, respectively.

[0059] In some examples, as referenced above Figure 3 Similarly, when the first electronic device 101a is in a multi-user communication session with the second electronic device 101b, virtual content can be shared and / or interacted between the first electronic device 101a and the second electronic device 101b in the three-dimensional environment 450A / 450B. For example, as Figure 4A As shown, a first electronic device 101a and a second electronic device 101b are displaying a virtual object 420 in a three-dimensional environment 450A / 450B. In some examples, the virtual object 420 corresponds to an application window (e.g., a user interface), such as a media player (e.g., a music player) application running on the first electronic device 101a and / or the second electronic device 101b. In some examples, the virtual object 420 is shared between the first electronic device 101a and the second electronic device 101b in a multi-user communication session, such that the content of the virtual object 420 (e.g., a music player user interface) is visible to and / or interacted with by the respective electronic devices 101a / 101b of the first user 402 and the second user 404, as indicated by a capsule button 422 displayed with the virtual object 420. In some examples, the capsule button 422 can be selected to initiate one or more sharing operations associated with the virtual object 420. For example, the capsule button 422 can be selected (e.g., via air gestures, such as air pinch or tap gestures) to share (e.g., and / or stop sharing) the contents of the virtual object 420 in a multi-user communication session. Additionally, in some examples, such as Figure 4A As shown, the virtual object 420 is displayed together with and / or includes the grab bar 435, which can optionally be selected to initiate movement of the virtual object 420 in the three-dimensional environment 450A / 450B.

[0060] exist Figure 4A In this context, a first user 402 of the first electronic device 101a and a second user 404 of the second electronic device 101b are in a multi-user communication session with multiple remote users (e.g., non-co-located users), which are virtually represented by multiple visual representations in a three-dimensional environment 450A / 450B. For example, as Figure 4AAs shown, the three-dimensional environment 450A / 450B includes a first virtual canvas 432a (optionally, two-dimensional representation) corresponding to a third user of a third electronic device (not shown) and a second virtual canvas 432b corresponding to a fourth user of a fourth electronic device (not shown). In some examples, the first virtual canvas 432a and the second virtual canvas 432b include images or other visual representations of their respective users, such as photographs of the respective users, drawings, cartoons, icons or other images of the respective users, (e.g., real-time) camera feeds of the respective users (e.g., captured via one or more cameras of their electronic devices), and / or two-dimensional avatars of the respective users. In some examples, although in Figure 4A Not illustrated, but the first virtual canvas 432a and the second virtual canvas 432b include names, labels or other textual indications that identify the third and fourth users in the three-dimensional environment 450A / 450B.

[0061] exist Figure 4A In this context, the first virtual canvas 432a and the second virtual canvas 432b have a corresponding spatial arrangement relative to the virtual object 420 within the three-dimensional environment 450A / 450B. For example, as... Figure 4A As shown, since the virtual object 420 corresponds to a shared object in a multi-user communication session (e.g., as described above and indicated by the capsule button 422), the first electronic device 101a and the second electronic device 101b provide a vertical arrangement of the first virtual canvas 432a and the second virtual canvas 432b relative to the virtual object 420 in the three-dimensional environment 450A / 450B. Specifically, in some examples, such as Figure 4AAs shown, from the viewpoints of the first electronic device 101a and the second electronic device 101b, the first virtual canvas 432a and the second virtual canvas 432b are arranged in a column adjacent to (e.g., to the right or edge of) the virtual object 420 in the three-dimensional environment 450A / 450B. In some examples, when the first virtual canvas 432a and the second virtual canvas 432b are arranged vertically relative to the virtual object 420, the first virtual canvas 432a and the second virtual canvas 432b are docked onto the virtual object 420. For example, if a movement input pointing towards virtual object 420 (e.g., an air pinch and drag gesture such as pointing towards grip bar 435) is detected and this movement input causes virtual object 420 to be repositioned in the three-dimensional environment 450A / 450B, then from the viewpoint of the first electronic device 101a and the second electronic device 101b, the first virtual canvas 432a and the second virtual canvas 432b move together with virtual object 420 in the three-dimensional environment 450A / 450B. It should be understood that in some examples, from the viewpoint of the first electronic device 101a and the second electronic device 101b, the first virtual canvas 432a and the second virtual canvas 432b are alternatively arranged in a column to the left or edge of virtual object 420 in the three-dimensional environment 450A / 450B. In some examples, the specific sides or edges of the virtual objects 420 aligned in the three-dimensional environment 450A / 450B by the first virtual canvas 432a and the second virtual canvas 432b are determined by predefined (e.g., as a default) and / or based on the content or content type of the virtual objects 420 by the first electronic device 101a and / or the second electronic device 101b. Alternatively, in some examples, the specific sides or edges of the virtual objects 420 aligned in the three-dimensional environment 450A / 450B by the first virtual canvas 432a and the second virtual canvas 432b are determined based on user input and / or user preferences (e.g., in the user settings of the first electronic device 101a and / or the second electronic device 101b).

[0062] In some examples, when sharing virtual object 420 in a multi-user communication session, the first virtual canvas 432a and the second virtual canvas 432b are... Figure 4A The illustrated vertical arrangement remains anchored to the virtual object 420. For example, if the virtual object 420 is not shared in a multi-user communication session and / or stops displaying in the 3D environment 450A / 450B, the first virtual canvas 432a and the second virtual canvas 432b are re-displayed in an alternative spatial arrangement, as described below. Figure 4B A more detailed discussion follows. Additionally, in some examples, the first virtual canvas 432a and the second virtual canvas 432b are... Figure 4A The order or spatial distribution within the illustrated vertical arrangement is based on and / or corresponds to the priority order of remote users in a multi-user communication session. For example, as discussed in more detail below, Figure 4A The positioning of the first virtual canvas 432a at the top of the virtual canvas column indicates that a third user (e.g., represented by the first virtual canvas 432a) has or is associated with a first priority sorting value or indicator that is higher than or greater than a second priority sorting value or indicator associated with a fourth user (e.g., represented by the second virtual canvas 432b at the bottom of the virtual canvas column). In some examples, the priority sorting order of remote users in a multi-user communication session is determined based on priority sorting data compiled by the first electronic device 101a and the second electronic device 101b (e.g., local users in the multi-user communication session). Additional details regarding the determination of the priority sorting data are provided below. Additionally, in some examples, although two virtual canvases are displayed at once in the virtual tile column when sharing virtual object 420 in a multi-user communication session, an additional virtual canvas (e.g., representing an additional remote user in the multi-user communication session) is associated with the virtual tile column (e.g., although not explicitly visible or displayed within the column in the 3D environment 450A / 450B). For example, as described in more detail below, if the priority sorting data changes or is updated, which would change or update the priority sorting order of remote users, the order or spatial distribution of the virtual canvases in the column may change, optionally causing the third virtual canvas (e.g., representing the fifth user of the fifth electronic device) to be displayed and visible in the column in the three-dimensional environment 450A / 450B (e.g., and thus causing the first virtual canvas 432a or the second virtual canvas 432b to no longer be displayed or visible in the column).

[0063] As described above, in some examples, if there is no shared content in a multi-user communication session, it means that the spatial arrangement of the virtual canvas for remote users in the multi-user communication session is changed and / or alternatively provided. For example, in Figure 4B In the 3D environments 450A / 450B, shared objects, such as the aforementioned virtual object 420, are not included. Therefore, as Figure 4B As shown, in some examples, a virtual canvas representing a remote user in a multi-user communication session is provided in an alternative spatial arrangement within a 3D environment 450A / 450B. Specifically, as Figure 4B As illustrated, the virtual canvas is optionally displayed within a horizontal array of virtual canvases in the 3D environment 450A / 450B. For example, as Figure 4B As shown, since the 3D environment 450A / 450B does not include shared objects, the virtual canvas is arranged within the two rows 433a and 433b of the virtual canvas from the viewpoints of the first electronic device 101a and the second electronic device 101b. In some examples, as discussed similarly above, each virtual canvas corresponds to and / or visually represents a remote user within a multi-user communication session. For example, as... Figure 4BAs shown, in addition to the first virtual canvas 432a and the second virtual canvas 432b discussed above, the three-dimensional environment 450A / 450B also includes a third virtual canvas 432c corresponding to the fifth user of the fifth electronic device, a fourth virtual canvas 432d corresponding to the sixth user of the sixth electronic device, a fifth virtual canvas 432e corresponding to the seventh user of the seventh electronic device, and a sixth virtual canvas 432f corresponding to the eighth user of the eighth electronic device. In some examples, such as Figure 4B As shown, although virtual canvases 432a-432f are arranged within two rows 433a and 433b in the 3D environment 450A / 450B, the virtual canvases in the first row 433a are not axis-aligned like those in the second row 433b. For example, as... Figure 4B As shown, from the viewpoints of the first electronic device 101a and the second electronic device 101b, the virtual canvases in the first row 433a of the three-dimensional environment 450A / 450B have staggered spatial alignment, while the virtual canvases in the second row 433b are axis-aligned (e.g., arranged horizontally on a straight line). Additionally, in some examples, such as... Figure 4B As shown, the virtual canvas in the first row 433a has a different size than (for example, larger than) the virtual canvas in the second row 433b, as discussed in more detail below.

[0064] In some examples, such as Figure 4B As shown, when virtual canvases 432a-432f are displayed in the 3D environment 450A / 450B while shared content is not displayed in the 3D environment 450A / 450B, virtual canvases 432a-432f display a grab bar 437. In some examples, grab bar 437 has one or more characteristics of grab bar 435 discussed above. For example, grab bar 437 can be selected (e.g., via an air pinch gesture) to initiate movement of virtual canvases 432a-432f in the 3D environment 450A / 450B. In some examples, in response to receiving such input, first electronic device 101a and second electronic device 101b move and / or reposition virtual canvases 432a-432f in unison within the 3D environment 450A / 450B.

[0065] As described above, it may be advantageous to provide a mechanism for maintaining visual and / or spatial consistency of the virtual canvases 432a-432f across the first electronic device 101a and the second electronic device 101b when the first user 402 and the second user 404 are in a multi-user communication session with multiple remote users represented by virtual canvases 432a-432f. Specifically, it may be desirable to provide the virtual canvases 432a-432f in the same order and with the same spatial distribution at both the first electronic device 101a and the second electronic device 101b. For example, in Figure 4B In this context, the order and spatial distribution of virtual canvases 432a-432f differ between the first electronic device 101a and the second electronic device 101b, as illustrated by the following scenario: the second virtual canvas 432b is displayed at the leftmost position within the first row 433a in the three-dimensional environment 450A, but at the rightmost position within the first row 433a in the three-dimensional environment 450B. In this case, because the second virtual canvas 432b, representing the fourth user in a multi-user communication session, is spatially located at different positions within the first row 433a in the three-dimensional environment 450A of the first electronic device 101a and the three-dimensional environment 450B of the second electronic device 101b, the spatial understanding and / or perception of the virtual canvases 432a-432f between the first user 402 and the second user 404 are different and conflicting, which may hinder and / or reduce the user experience in the multi-user communication session.

[0066] Therefore, in some examples, a priority order is established by the first electronic device 101a and / or the second electronic device 101b, which ensures that the spatial arrangement and distribution of the virtual canvases 432a-432b are consistent between the first electronic device 101a and the second electronic device 101b. For example, as Figure 4C As shown, by synchronizing the priority order of virtual canvases 432a-432f, the first electronic device 101a and the second electronic device 101b provide the same spatial arrangement and distribution of virtual canvases 432a-432f in the three-dimensional environment 450A / 450B. Specifically, in Figure 4C In this arrangement, virtual canvases 432a-432f are arranged in two rows 433a and 433b at both electronic devices 101a and 101b, and are distributed in the same order within the two rows 433a and 433b at both electronic devices 101a and 101b, thus advantageously promoting visual consistency among local users (e.g., first user 402 and second user 404) in a multi-user communication session.

[0067] In some examples, the priority order of virtual canvases 432a-432f is determined by the first electronic device 101a and / or the second electronic device 101b based on and / or according to priority ordering data associated with remote users and collected and / or compiled by the first electronic device 101a and / or the second electronic device 101b during a multi-user communication session. In some examples, the priority ordering data includes user data and / or input data indicating user activity provided by each of the electronic devices associated with the remote users in the multi-user communication session. As an example, the priority ordering data includes data indicating camera activity and / or video activity of remote users in the multi-user communication session. For example, the electronic device associated with the remote user in the multi-user communication session sends data to the first electronic device 101a and the second electronic device 101b indicating whether the camera feed and / or video feed of a particular remote user is currently on / active. As another example, the data sent to the first electronic device 101a and the second electronic device 101b indicates whether motion or movement is detected in the video feed (e.g., whether the remote user is moving, which could indicate speaking), if the remote user's camera feed and / or video feed is currently on / active, as detected by one or more cameras of the respective electronic devices. For example, the second virtual canvas 432b and the first virtual canvas 432a represent remote users whose camera feeds are currently on, and the fourth virtual canvas 432d and the sixth virtual canvas 432f represent remote users whose camera feeds are currently off (e.g., and represented by a two-dimensional image), as... Figure 4C As shown. In some examples, the priority ordering data includes data indicating the audio activity of a remote user in a multi-user communication session. For example, an electronic device associated with a remote user in a multi-user communication session sends data to a first electronic device 101a and a second electronic device 101b indicating whether the audio function (e.g., microphone) of a particular remote user is currently on / active. Furthermore, in some examples, the data sent to the first electronic device 101a and the second electronic device 101b indicates whether one or more microphones of the respective electronic device have detected audio (e.g., whether the remote user is actively speaking or talking) if the remote user's audio function is currently on / active.

[0068] In some examples, using data indicating user activity provided by electronic devices associated with remote users in a multi-user communication session, first electronic device 101a and / or second electronic device 101b compile priority ranking data and assign a priority ranking value (e.g., a unitless number or other discrete identifier between a predefined range, such as 0 and 255)) to each remote user, determining the priority ranking order based on this value. For example, using the video and / or audio activity of remote users in a multi-user communication session as discussed above, first electronic device 101a and / or second electronic device 101b determine a ranking of remote users in the multi-user communication session from most active to least active. As an example, a remote user who has their camera feed active and is actively speaking in a multi-user communication session is assigned a higher priority ranking value compared to another remote user who has their camera feed active and speaks less or has their audio feed turned off while in the multi-user communication session. In some examples, the first electronic device 101a and / or the second electronic device 101b determine the priority order of the virtual canvases 432a-432f in the three-dimensional environment 450A / 450B after assigning priority order values ​​to each remote user according to the priority order data described above.

[0069] In some examples, the specific order or spatial distribution of virtual canvases 432a-432f within two rows 433a and 433b of the 3D environment 450A / 450B is based on the priority order discussed above. For example, in Figure 4B In the 3D environment 450A / 450B, the leftmost position within the first row 433a corresponds to the highest priority position in the horizontal spatial arrangement of the virtual canvas 432a-432f, and the rightmost position within the second row 433b corresponds to the lowest priority position in the horizontal spatial arrangement of the virtual canvas 432a-432f. Therefore, in Figure 4B In the example, the fourth user, represented by the second virtual canvas 432b, has been assigned the highest priority sorting value based on the priority sorting data discussed above. This causes the second virtual canvas 432b to be displayed at the leftmost position within the first row 433a in the three-dimensional environment 450A / 450B, from the viewpoints of the first electronic device 101a and the second electronic device 101b. Similarly, in Figure 4BIn the example, the eighth user, represented by the sixth virtual canvas 432f, has been assigned the lowest priority sorting value based on the priority sorting data discussed above. This causes the sixth virtual canvas 432f to be displayed at the rightmost position within the second row 433b in the three-dimensional environment 450A / 450B, from the viewpoint of the first electronic device 101a and the second electronic device 101b.

[0070] In some examples, Figure 4C The two rows 433a and 433b illustrated correspond to two types or groups of remote users in a multi-user communication session. For example, in Figure 4C In this example, the four virtual canvases 432a-432d displayed in the first row 433a within the horizontal arrangement of the virtual canvases correspond to the four remote users who have been assigned the first four (e.g., the four highest) priority ranking values ​​according to the priority ranking data, as discussed above. The remaining virtual canvases (e.g., virtual canvases 432e / 432f) displayed in the second row 433b correspond to the remaining remote users, optionally without a specific order. Specifically, in some examples, the four locations in the first row 433a are determined according to the priority ranking order discussed above, while two (or more) locations in the second row 433b are arbitrarily determined (e.g., priority ranking values ​​falling outside the first four values ​​are not considered). Therefore, in some examples, as mentioned above, from the viewpoint of the first electronic device 101a and the second electronic device 101b, the virtual canvas displayed in the first row 433a of the three-dimensional environment 450A / 450B is displayed at a larger size (e.g., indicating higher salience or priority) than the virtual canvas displayed in the second row 433b of the three-dimensional environment 450A / 450B. Providing virtual canvases representing more active remote users in a multi-user communication session in priority rows and / or at a size indicating higher priority ranking or salience among remote users in a multi-user communication session provides local users with a visual indication of more active participants in the multi-user communication session and / or ensures that less active participants are visually less salient in the three-dimensional environment, thereby improving user privacy as a benefit.

[0071] In some examples, the virtual canvas is updated based on and / or in response to changes in priority sorting data used to determine the priority sorting order of the virtual canvas in the 3D environment 450A / 450B. Figure 4C The order or spatial distribution in the horizontal arrangement shown. For example, from Figures 4C to 4DThe first electronic device 101a and / or the second electronic device 101b receive updated data from one or more electronic devices associated with a remote user in a multi-user communication session. This updated data causes the first electronic device 101a and / or the second electronic device 101b to update priority sorting data, and thus update the priority sorting order defining the order or spatial distribution of the virtual canvases displayed in the three-dimensional environment 450A / 450B. Specifically, in some examples, such as... Figure 4D As shown, the first electronic device 101a and the second electronic device 101b update the positioning of the first virtual canvas 432a representing the third user and the third virtual canvas 432c representing the fifth user within the first row 433a of the horizontal spatial arrangement in the three-dimensional environment 450A / 450B. For example, from Figures 4C to 4D The first electronic device 101a and the second electronic device 101b exchange the positions of the first virtual canvas 432a and the third virtual canvas 432c in the first row 433a of the three-dimensional environment 450A / 450B. In some examples, as discussed in more detail below, the updated priority ranking data indicates that the priority ranking value assigned to the fifth user represented by the third virtual canvas 432c in the three-dimensional environment 450A / 450B has exceeded or surpassed the priority ranking value assigned to the third user represented by the first virtual canvas 432a.

[0072] In some examples, detecting changes in priority ranking data includes and / or is based on data received from a third electronic device associated with a third user and / or a fifth electronic device associated with a fifth user, indicating that the user activity of the third user and / or the fifth user in a multi-user communication session has changed. Specifically, the first electronic device 101a and / or the second electronic device 101b optionally determines that the video and / or audio activity of the fifth user has exceeded that of the third user, which would cause the priority ranking value assigned to the fifth user in the multi-user communication session to exceed the priority ranking value assigned to the third user. Therefore, as discussed above and as... Figure 4D As shown, the updated priority sorting data enables the first electronic device 101a and the second electronic device 101b to update the priority sorting order of the virtual canvases 432a-432f in the three-dimensional environment 450A / 450B, especially the positioning of the first virtual canvas 432a corresponding to the third user and the third virtual canvas 432c corresponding to the fifth user in a multi-user communication session.

[0073] In some examples, one of the electronic devices associated with a local user in a multi-user communication session updates the order or spatial distribution of virtual canvases 432a-432f in its local 3D environment, under the guidance or instruction of another electronic device associated with the local user in the multi-user communication session. For example, lag or other delays in updating priority ordering data that determines the priority order of virtual canvases 432a-432f (e.g., due to connectivity issues (e.g., Wi-Fi or Bluetooth connectivity problems or lag), device operation delays, or performance degradation (e.g., due to low power or increased processing demands due to multiple applications running on the electronic device)) may cause the first electronic device 101a or the second electronic device 101b to delay updating the display of the order or spatial distribution of virtual canvases 432a-432f in the manner discussed above, which would disrupt the visual consistency between the display of virtual canvases 432a-432f between electronic devices 101a and 101b. Therefore, in some examples, the first electronic device 101a or the second electronic device 101b sends a signal, data, or other instruction (e.g., a command) to another electronic device, which causes the other electronic device to update the display of the virtual canvas 432a-432f in the three-dimensional environment 450A or 450B. For example, in Figure 4D In this context, after updating the priority ranking data based on updated user activity data provided by the third and fifth electronic devices (as discussed above), and when the second electronic device 101b is experiencing lag or other performance delays, the first electronic device 101a determines that the priority ranking order of virtual canvases 432a-432f will change according to the updated priority ranking data, and in response, sends display data (e.g., including the updated priority ranking data and other instructions) to the second electronic device 101b, which causes the second electronic device 101b to update the display of virtual canvases 432a-432f in the 3D environment 450B. As an example, the display data sent to the second electronic device 101b includes instructions for exchanging the positions of the third virtual canvas 432c and the first virtual canvas 432a in the first row 433a of the 3D environment 450B, as discussed similarly above. In some examples, the display data sent to the second electronic device 101b includes updated priority sorting data, which enables the second electronic device 101b to locally and independently update the priority sorting order of virtual canvases 432a-432f, thereby enabling the second electronic device 101b to exchange the positions of the third virtual canvas 432c and the first virtual canvas 432a in the first row 433a of the 3D environment 450B. Alternatively, in Figure 4DIn the example, after updating the priority sorting data based on updated user activity data provided by the third and fifth electronic devices (as discussed above), and while the first electronic device 101a is experiencing lag or other performance delays, the second electronic device 101b determines that the priority sorting order of the virtual canvases 432a-432f will change according to the updated priority sorting data, and in response, sends display data (e.g., including the updated priority sorting data and other instructions) to the first electronic device 101a, which causes the first electronic device 101a to update the display of the virtual canvases 432a-432f in the 3D environment 450B, as discussed similarly above.

[0074] In some examples, the transmission of the display data discussed above (e.g., including instructions or other data) (for updating the display of the order or spatial distribution of virtual canvases 432a-432f by the first electronic device 101a or the second electronic device 101b) occurs independently of or without detecting an indication that the first electronic device 101a or the second electronic device 101b is experiencing lag or other performance delays. For example, when the first electronic device 101a or the second electronic device 101b detects a change in the priority sorting data that updates the priority sorting order of the virtual canvases 432a-432f, the first electronic device 101a or the second electronic device 101b sends display data to another electronic device indicating that the priority sorting data has changed. In some such examples, after receiving display data from another electronic device, the first electronic device 101a or the second electronic device 101b compares the updated priority sorting data with the current priority sorting data at the first electronic device 101a or the second electronic device 101b, and if a difference exists, the first electronic device 101a or the second electronic device 101b updates the priority sorting order based on the updated priority sorting data (e.g., even if the first electronic device 101a or the second electronic device 101b may not necessarily be experiencing lag or performance latency). Therefore, in this way, maintaining visual consistency (e.g., virtual canvas) for local users in a multi-user communication session relative to remote users in the same session, taking into account any lag or performance latency experienced by the electronic device associated with the local user, thus serving as a benefit to maintain the user experience within the multi-user communication session.

[0075] In some examples, the first electronic device 101a and the second electronic device 101b selectively (e.g., based on the satisfaction of one or more criteria) update the order or spatial distribution of virtual canvases 432a-432f in response to detecting changes in priority ordering data associated with remote users in a multi-user communication session. Specifically, in some cases, it may be desirable to reduce the frequency of updating the order or spatial distribution of virtual canvases 432a-432f in the 3D environment 450A / 450B, for example, to help reduce eye strain or other discomfort that local users (e.g., the first user 402 and the second user 404) may experience at their respective electronic devices due to repeated updates of the display of virtual canvases 432a-432f in the 3D environment 450A / 450B. In some examples, the above references Figure 4D The changes in the described priority ranking data are insufficient to cause the first electronic device 101a and the second electronic device 101b to update the order or spatial distribution of the virtual canvases 432a-432f in the three-dimensional environment 450A / 450B. For example, although when updating the priority ranking data as discussed above, the priority ranking value assigned to the fifth user represented by the third virtual canvas 432c exceeds the priority ranking value assigned to the third user represented by the first virtual canvas 432a, the first electronic device 101a and the second electronic device 101b refrain from exchanging priority ranking data. Figure 4D The positioning of the first virtual canvas 432a and the third virtual canvas 432c in the environment. Therefore, in some examples, the order or spatial distribution of the virtual canvases 432a-432f in the 3D environment 450A / 450B is maintained as follows. Figure 4C As shown.

[0076] Return to Figure 4C Prioritizing data sufficient to cause changes in the order or spatial distribution of virtual canvases 432a-432f in the three-dimensional environment 450A / 450B (e.g., satisfying one or more criteria for updating the display of virtual canvases 432a-432f in the three-dimensional environment 450A / 450B) by the first electronic device 101a and the second electronic device 101b includes causing or indicating a change in the virtual canvas from the second row 433b to the first row 433a, and vice versa. For example, as discussed above, Figure 4C The first row 433a of the virtual canvas in the illustrated horizontal spatial arrangement corresponds to the first group of remote users who have been assigned the four highest priority ranking values ​​in the multi-user communication session, and the remaining remote users in the multi-user communication session are visually represented via their corresponding virtual canvases in the second row 433b of the virtual canvas in the horizontal spatial arrangement. Specifically, in some examples, changes in order within the same row of the horizontal spatial arrangement are considered to have less importance than changes in order from one row to another in the horizontal spatial arrangement, as discussed in more detail below.

[0077] from Figures 4C to 4E The first electronic device 101a and / or the second electronic device 101b receive updated data from one or more electronic devices associated with a remote user in a multi-user communication session. This updated data causes the first electronic device 101a and / or the second electronic device 101b to update priority sorting data that meets one or more criteria, and thus updates the priority sorting order defining the order or spatial distribution of the virtual canvases displayed in the three-dimensional environment 450A / 450B. Specifically, in some examples, such as... Figure 4E As shown, the first electronic device 101a and the second electronic device 101b update the positioning of the third virtual canvas 432c representing the fifth user and the fifth virtual canvas 432e representing the seventh user within the first row 433a and the second row 433b of the horizontal spatial arrangement in the three-dimensional environment 450A / 450B. For example, from Figures 4C to 4E The first electronic device 101a and the second electronic device 101b exchange the positions of the third virtual canvas 432c in the first row 433a and the fifth virtual canvas 432e in the second row 433b of the three-dimensional environment 450A / 450B. In some examples, as discussed in more detail below, the updated priority ranking data indicates that the priority ranking value assigned to the seventh user represented by the fifth virtual canvas 432e has exceeded or surpassed the priority ranking value assigned to the fifth user represented by the third virtual canvas 432c, and more importantly, the priority ranking value assigned to the seventh user is among the four highest priority ranking values ​​in the multi-user communication session (e.g., this would cause the fifth virtual canvas 432e to move from the second row 433b to the first row 433a in the three-dimensional environment 450A / 450B).

[0078] In some examples, as discussed similarly above, detecting changes in priority ranking data includes and / or is based on data received from a fifth electronic device associated with a fifth user and / or a seventh electronic device associated with a seventh user, indicating that the user activity of the fifth user and / or the seventh user in a multi-user communication session has changed. Specifically, the first electronic device 101a and / or the second electronic device 101b optionally determine that the video and / or audio activity of the seventh user has exceeded that of the fifth user (but not exceeding, or also exceeding, the video and / or audio activity of the third user (e.g., represented by the first virtual canvas 432a) and the sixth user (e.g., represented by the fourth virtual canvas 432d)), which would cause the priority ranking value assigned to the seventh user in the multi-user communication session to exceed the priority ranking value assigned to the fifth user. Therefore, as discussed above and as... Figure 4EAs shown, updated priority sorting data that meets one or more criteria enables the first electronic device 101a and the second electronic device 101b to update the priority sorting order of virtual canvases 432a-432f in the three-dimensional environment 450A / 450B, particularly the positioning of the third virtual canvas 432c corresponding to the fifth user and the fifth virtual canvas 432e corresponding to the seventh user in a multi-user communication session.

[0079] In some examples, detecting changes in priority ranking data includes and / or is based on indications that the number of remote users in a multi-user communication session has changed. For example, first electronic device 101a and / or second electronic device 101b update the priority ranking order based on updated priority ranking data, depending on whether a remote user has joined or left the multi-user communication session (which would change the number of active participants in the multi-user communication session). As an example, in Figure 4F In this process, the first electronic device 101a and the second electronic device 101b detect an indication that a remote user has left the multi-user communication session, which reduces the number of participants in the multi-user communication session. Specifically, in Figure 4F In this scenario, the third user of the third electronic device has left the multi-user communication session, causing the first electronic device 101a and the second electronic device 101b to stop displaying the third user's visual representation (e.g., the first virtual canvas 432a) in the three-dimensional environment 450A / 450B. In some examples, when the first virtual canvas 432a stops displaying in the three-dimensional environment 450A / 450B, the order or spatial distribution of the virtual canvases 432b-432f is updated within the horizontal spatial arrangement of the virtual canvases. For example, as... Figure 4F As shown, when a third user leaves the multi-user communication session, the first virtual canvas 432a is stopped from being displayed, causing the third virtual canvas 432c, previously displayed in the second row 433b arranged horizontally, to be transformed / moved to the first row 433a arranged horizontally (e.g., replacing the first virtual canvas 432a in the first row 433a). Specifically, in Figure 4F In the example, the departure of a third user from a multi-user communication session results in a fifth user (e.g., represented by a third virtual canvas 432c) being assigned a priority sorting value, which is and / or causes the priority sorting value already assigned to the fifth user to be among the four highest priority sorting values ​​based on the updated priority sorting data. Therefore, as discussed similarly above, the first electronic device 101a and the second electronic device 101b optionally update the display of the order or spatial distribution of the virtual canvases 432b-432f, such that the third virtual canvas 432c moves from the second row 433b to the first row 433a within the horizontal spatial arrangement of the three-dimensional environment 450A / 450B.

[0080] As mentioned above, in some examples, detecting changes in priority ranking data includes and / or indications based on a change in the number of remote users in a multi-user communication session, such as an increase in the number of participants in the multi-user communication session. For example, in Figure 4G In this context, the first electronic device 101a detects input provided by the first user 402, which corresponds to a request to add a remote user to a multi-user communication session. In some examples, such as... Figure 4G As shown, a first electronic device 101a is displaying a user interface 430 associated with a multi-user communication session in a three-dimensional environment 450A. In some examples, the user interface 430 includes multiple representations 431a-431f that can be selected to initiate a process of adding a corresponding user corresponding to the selected representation to the multi-user communication session. In some examples, the users associated with the multiple representations 431a-431f correspond to contacts associated with a contacts or telephone application running on the first electronic device 101a (and / or the second electronic device 101b). Figure 4G As shown, when user interface 430 is displayed, first electronic device 101a detects an air pinch gesture performed by hand 403 of first user 402, optionally simultaneously with the gaze 426 of first user 402 pointing to representation 431f in user interface 430. In some examples, representation 431f is associated with a ninth user (e.g., Debbie) of a ninth electronic device that is not juxtaposed with first electronic device 101a and second electronic device 101b in physical environment 400. It should be understood that additional or alternative inputs are possible, such as air tap gestures, gaze and hold inputs, verbal commands, controller inputs, etc.

[0081] In some examples, in response to detecting a selection of representation 431f in user interface 430, first electronic device 101a sends a request corresponding to an invitation to join a multi-user communication session to ninth electronic device (e.g., associated with ninth user Debbie). In some examples, after the request has been sent to the ninth electronic device and before a response to the invitation to join the multi-user communication session is received from the ninth electronic device, first electronic device 101a and second electronic device 101b update the display of a virtual canvas in the three-dimensional environment 450A / 450B to include a "waiting" canvas while the response to the invitation is pending. For example, as... Figure 4H As shown, the first electronic device 101a and the second electronic device 101b display a seventh virtual canvas 432g representing the ninth user, Debbie, within a three-dimensional environment 450A / 450B. Specifically, in some examples, such as Figure 4HAs shown, the seventh virtual canvas 432g represents a placeholder for the virtual canvas that will ultimately be displayed in the 3D environment 450A / 450B if the ninth electronic device joins the multi-user communication session and / or when the ninth electronic device joins the multi-user communication session. In some examples, the seventh virtual canvas 432g includes an indication of pending response to the invitation. For example, as... Figure 4H As shown, the seventh virtual canvas 432g includes three dots or circles (e.g., displayed below the image corresponding to the ninth user, Debbie, including an icon with the initials). Additionally, in some examples, such as... Figure 4H As shown, when the seventh virtual canvas 432g is displayed in the 3D environment 450A / 450B, the seventh virtual canvas 432g is positioned by default within the lowest priority location in the horizontal spatial arrangement of the 3D environment 450A / 450B. For example, as Figure 4H As shown, the ninth user, Debbie, has been assigned the lowest priority sorting value (e.g., optionally by default because no user activity data is available and / or the ninth electronic device has not yet sent user activity data). This gives the seventh virtual canvas 432g the lowest priority within the priority sorting order, as visually represented by the seventh virtual canvas 432g displayed at the rightmost position in the second row 433b of the 3D environment 450A / 450B. In some examples, if no response to the invitation is received from the ninth electronic device, the waiting time for the seventh virtual canvas 432g to reach a threshold is displayed in the 3D environment 450A / 450B. For example, the first electronic device 101a and the second electronic device 101b display the seventh virtual canvas 432g in the three-dimensional environment 450A / 450B for 20 seconds, 25 seconds, 30 seconds, 40 seconds, 45 seconds, 60 seconds, 70 seconds or 90 seconds, and if no response is received from the ninth electronic device during this period, the first electronic device 101a and the second electronic device 101b stop displaying the seventh virtual canvas 432g in the three-dimensional environment 450A / 450B.

[0082] from Figures 4H to 4I The first electronic device 101a and the second electronic device 101b detect responses to invitations sent to the ninth electronic device associated with the ninth user Debbie for joining a multi-user communication session. Specifically, in Figure 4I In this context, the first electronic device 101a and the second electronic device 101b detect that the ninth electronic device has accepted the invitation, which causes the ninth electronic device to be added to the multi-user communication session, thereby increasing the total number of participants in the multi-user communication session. In some examples, such as Figure 4IAs illustrated, when a ninth electronic device is added to a multi-user communication session, the first electronic device 101a and the second electronic device 101b update the display of the seventh virtual canvas 432g in the three-dimensional environment 450A / 450B to indicate that the ninth user is now active in the multi-user communication session. For example, as Figure 4I As shown, the seventh virtual canvas 432g no longer includes the previous ones. Figure 4H The example shows three dots or circles. Figure 4I In the multi-user communication session, since the user activity of the ninth user has not exceeded the user activities of other remote users in the multi-user communication session (e.g., and / or has not specifically exceeded the user activities of the remote users who have been assigned the four highest priority ranking values, as discussed similarly above), the first electronic device 101a and the second electronic device 101b maintain the display of the seventh virtual canvas 432g at the rightmost position in the second row 433b within the horizontal spatial arrangement in the three-dimensional environment 450A / 450B.

[0083] In some examples, as discussed similarly above, the horizontal spatial arrangement of the virtual canvas includes and / or is associated with a predefined number of positions within the two rows 433a and 433b. For example, in Figure 4I In the horizontal spatial arrangement, the first row 433a includes four predefined positions for displaying the virtual canvas in the three-dimensional environment 450A / 450B, and the second row 433b includes two predefined positions for displaying the virtual canvas in the three-dimensional environment 450A / 450B. However, in some examples, the number of remote users in a multi-user communication session may exceed the six predefined positions exemplified in the horizontal spatial arrangement of the virtual canvas. Therefore, in some such examples, it is not meant that each virtual canvas of a remote user in a multi-user communication session is simultaneously displayed or visible in the multi-user communication session. For example, in Figure 4I In this multi-user communication session, there are more than six remote users. However, since there are six predefined locations in the 3D environment 450A / 450B for displaying virtual canvases corresponding to the six users, one or more virtual canvases representing the respective remote users are currently not displayed or are not visible in the 3D environment 450A / 450B. Therefore, in some examples, such as when the number of remote users exceeds the number of predefined display locations for virtual canvases within the corresponding spatial arrangement in the 3D environment 450A / 450B, user input can be provided to display additional and / or alternative virtual canvases corresponding to the remote users participating in the multi-user communication session in the 3D environment 450A / 450B, as discussed below.

[0084] In some examples, such as Figure 4IAs shown, the first electronic device 101a detects input provided by the first user 402, which corresponds to a request in the second row 433b of the scrolling 3D environment 450A. For example, as Figure 4I As shown, the first electronic device 101a detects an air pinch gesture performed by the hand 403 of the first user 402, and then the hand moves in space (e.g., to the left relative to the viewpoint of the first electronic device 101a), optionally while the gaze 426 of the first user 402 is directed towards the seventh virtual canvas 432g in the second row 433b of the three-dimensional environment 450A. In some examples, the horizontally arranged second row 433b is scrollable (e.g., horizontally scrollable) in response to user input, while the first row 433a is not scrollable (e.g., horizontally scrollable). For example, as discussed above, four positions in the horizontally arranged first row 433a are reserved to represent the virtual canvases of remote users associated with the four highest priority ranking values ​​according to priority-sorted data, which are therefore greater than the priority ranking values ​​of the remote users whose virtual canvases are displayed in the horizontally arranged second row 433b. Therefore, Figure 4I The virtual canvas that is not currently displayed or visible corresponds to a remote user who has been assigned a priority sorting value lower than the four highest priority sorting values. This indicates that the virtual canvas that is not currently displayed or visible has been grouped into the second row 433b of the horizontal spatial arrangement.

[0085] In some examples, such as Figure 4J As shown, in response to detecting an input corresponding to a request in the second row 433b of the scrolling 3D environment 450A, the first electronic device 101a scrolls the second row 433b arranged in the horizontal space of the 3D environment 450A. For example, as Figure 4J As shown, the first electronic device 101a scrolls the second row 433b to the left relative to its viewpoint to display an eighth virtual canvas 432h representing the tenth user in a multi-user communication session and a ninth virtual canvas 432i representing the eleventh user in the same session, both of whom are remote users as previously discussed herein. In some examples, as previously discussed above, since the second row 433b includes two predefined positions for displaying virtual canvases, the first electronic device 101a stops displaying the sixth virtual canvas 432f and the seventh virtual canvas 432g in the three-dimensional environment 450A when scrolling the second row 433b. In some examples, the amount of scrolling of the second row 433b in the three-dimensional environment 450B (e.g., the amount of speed and / or distance) is based on (e.g., proportional to) and / or corresponds to the amount of movement of the hand 403 of the first user 402 as discussed above (e.g., the amount of speed and / or distance).

[0086] In some examples, the scrolling of the second row 433b at the first electronic device 101a discussed above constitutes an event in which the order or spatial distribution of the virtual canvas in the horizontal spatial arrangement is updated in the three-dimensional environment 450A (e.g., because the virtual canvases 432f and 432g have been updated). Figure 4J (The virtual canvases 432h and 432i are replaced). Therefore, in some examples, in order to maintain visual consistency in the order or spatial distribution of the virtual canvases corresponding to remote users in a multi-user communication session between the first electronic device 101a and the second electronic device 101b, when the second row 433b is scrolled in the three-dimensional environment 450A in the manner discussed above, the first electronic device 101a sends a signal or other data (e.g., including instructions) to the second electronic device 101b indicating that the second row 433b has been scrolled to display the virtual canvases 432h and 432i. In some examples, such as Figure 4J As shown, in response to receiving a signal or data from the first electronic device 101a, the second electronic device 101b updates the order or spatial distribution of the virtual canvas displayed in the three-dimensional environment 450B. Specifically, as Figure 4J As shown, based on the input detected at the first electronic device 101a, the second electronic device 101b replaces the display of virtual canvases 432f and 432g in the second row 433b arranged in the horizontal space with virtual canvases 432h and 432i, thereby maintaining the visual consistency of the virtual canvases in the three-dimensional environment 450B for the second user 404 at the second electronic device 101b.

[0087] Now turn our attention to an example of determining a placement location for transforming the display of a remote user's visual representation in a three-dimensional environment from a (e.g., two-dimensional) virtual canvas into (e.g., three-dimensional) avatar within a multi-user communication session that includes local and remote users as well as electronic devices.

[0088] Figures 5A to 5K Examples of how, within a multi-user communication session including co-located and non-co-located users, the display of a remote user's visual representation in a three-dimensional environment is transformed from a virtual canvas into a three-dimensional avatar, are illustrated according to some examples of this disclosure. Figure 5A In this context, a first electronic device 101a (e.g., associated with a first corresponding user 502) and a second electronic device 101b (e.g., associated with a second corresponding user 504) are juxtaposed in a physical environment 500, as discussed above, and simultaneously participate in a multi-user communication session. In some examples, the first corresponding user 502 and the second corresponding user 504 respectively correspond to... Figures 4A to 4J The first user 402 and the second user 404. In some examples, the physical environment 500 has one or more of the characteristics of the physical environment 400 discussed above.

[0089] like Figure 5A As shown, the first electronic device 101a is (e.g., via display 120a) presenting a three-dimensional environment 550A. Figure 5A As discussed similarly above, from the viewpoint of the first electronic device 101a, the three-dimensional environment 550A includes a representation (e.g., a transparent representation or a computer-generated representation) of the physical environment 500 of the first electronic device 101a. For example, as... Figure 5A As shown in the top view 510, the physical environment 500 corresponds to a room including windows 509, supports 507, and indoor plants 508. Therefore, as Figure 5A As shown, the three-dimensional environment 550A presented using the first electronic device 101a includes representations of windows 509, supports 507, and indoor plants 508 (e.g., windows 509, supports 507, and indoor plants 508 are visible within the field of view of the first electronic device 101a). Additionally, as... Figure 5A As shown, the second electronic device 101b is presenting a three-dimensional environment 550B (e.g., via display 120b). Figure 5A As discussed similarly above, from the viewpoint of the second electronic device 101b, the three-dimensional environment 550B includes a representation (e.g., a transparent representation or a computer-generated representation) of the physical environment 500 of the second electronic device 101b. For example, as... Figure 5A As shown, the three-dimensional environment 550B presented using the second electronic device 101b includes a representation of a window 509 (e.g., the window 509 is visible in the field of view of the second electronic device 101b). In some examples, the three-dimensional environment 550A / 550B has one or more of the characteristics of the three-dimensional environments 450A / 450B discussed above.

[0090] Additionally, in some examples, such as Figure 5A As shown, the first responding user 502 and the second responding user 504 are participating in a multi-user communication session with multiple remote users (e.g., non-co-located users and electronic devices), as previously described herein. Therefore, in Figure 5A In this context, first electronic device 101a and second electronic device 101b are displaying multiple visual representations corresponding to multiple remote users in a three-dimensional environment 550A / 550B. Specifically, in some examples, such as Figure 5AAs shown, first electronic device 101a and second electronic device 101b are displaying multiple virtual canvases 532 representing multiple remote users in a multi-user communication session. In some examples, as discussed similarly above, since the three-dimensional environment 550A / 550B does not include shared virtual objects in the multi-user communication session, the multiple virtual canvases 532 are arranged in a horizontal spatial arrangement within the three-dimensional environment 550A / 550B (e.g., relative to the grab bar 537), from the viewpoint of first electronic device 101a and second electronic device 101b. In some examples, as discussed earlier herein, the horizontal spatial arrangement of the virtual canvases includes two rows 533a and 533b, within which the virtual canvases are positioned in the three-dimensional environment 550A / 550B. For example, as... Figure 5A As shown, in the 3D environments 550A / 550B, virtual canvases 532a-532d are displayed in the first row 533a of the horizontal spatial arrangement, and virtual canvases 532e and 532f are displayed in the second row 533b of the horizontal spatial arrangement. In some examples, the horizontal spatial arrangement has the above reference. Figures 4A to 4J The virtual canvas is described in terms of one or more characteristics of its horizontal spatial arrangement. In some examples, as similarly described herein, virtual canvases 532a-532f correspond to different remote users in a multi-user communication session. In some examples, virtual canvases 532a-532f correspond to the virtual canvases 432a-432f previously discussed above.

[0091] Figure 5B An example is illustrated where a third electronic device 101c displays a three-dimensional environment from its viewpoint. In some examples, the third electronic device is associated with a third corresponding user 506 (e.g., Andy) located in a physical environment 560 that is different from (e.g., independent of) the physical environment 500 where the first electronic devices 101a and 101b discussed above reside. In some examples, the third electronic device 101c is in a multi-user communication session that includes the first electronic devices 101a and 101b. For example, the third corresponding user 506 is among multiple remote users represented by multiple virtual canvases 532 displayed at the first electronic devices 101a and 101b. Specifically, in some examples, the third corresponding user 506 of the third electronic device 101c is visually represented by virtual canvases 532b in the three-dimensional environments 550A / 550B at the first electronic devices 101a and 101b.

[0092] like Figure 5BAs shown, a third electronic device 101c is presenting a three-dimensional environment (e.g., via a display 120c), which, from the viewpoint of the third electronic device 101c, includes a representation (e.g., a transparent representation or a computer-generated representation) of the physical environment 560 of the third electronic device 101c. For example, as Figure 5B As shown in the top view 510, the physical environment 560 corresponds to a room including a chair 540 and a painting 541. Therefore, as... Figure 5B As shown, the three-dimensional environment presented using the third electronic device 101c includes representations of chair 540 and painting 541 (e.g., chair 540 and painting 541 are visible within the field of view of the third electronic device 101c). Additionally, as... Figure 5B As shown, the third electronic device 101c is presenting multiple visual representations of multiple users in a multi-user communication session within a three-dimensional environment. Figure 5B In this context, since the third corresponding user 506 is not co-located with any other user in the multi-user communication session in the physical environment 560, the other users, including the first corresponding user 502 and the second corresponding user 504, correspond to the third corresponding user 506 as a remote user relative to the third electronic device 101c. Therefore, as Figure 5B As shown, the third electronic device 101c optionally displays multiple virtual canvases 532 for a user who is remote from the third corresponding user 506 in a multi-user communication session, including a virtual canvas 532j corresponding to the first corresponding user 502 of the first electronic device 101a and a virtual canvas 532k corresponding to the second corresponding user 504 of the second electronic device 101b. In some examples, as discussed similarly above, from the viewpoint of the third electronic device 101c, the multiple virtual canvases 532 displayed at the third electronic device 101c are arranged in a horizontal spatial arrangement, which includes a first row 533a and a second row 533b in a three-dimensional environment. In some examples, as discussed similarly above, since the third corresponding user 506 is not located in the same physical environment as the first corresponding user 502 and the second corresponding user 504, visual consistency of the virtual canvases is not maintained among the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c, such as... Figure 5A and Figure 5B As illustrated in the examples. For instance, the order or spatial distribution of the virtual canvas and / or specific virtual canvases displayed at the third electronic device 101c differs from the order or spatial distribution of the virtual canvas and / or specific virtual canvases displayed at the first electronic device 101a and the second electronic device 101b.

[0093] like Figure 5BAs shown, the third electronic device 101c is displaying selectable options 524 associated with the spatial state of a third corresponding user 506 in a multi-user communication session. For example, the corresponding user may be associated with a first spatial state or a second spatial state, the first spatial state indicating that the corresponding user will be visually represented and will participate in the multi-user communication session spatially (e.g., via a three-dimensional avatar), and the second spatial state indicating that the corresponding user will be visually represented and will participate in the multi-user communication session non-spatially (e.g., via a two-dimensional representation). In some examples, participating in the multi-user communication session spatially (e.g., in the first spatial state) indicates that the corresponding user experiences spatial realism (e.g., as defined above) with other users also spatially participating in the multi-user communication session. For example, in Figure 5A In the first electronic device 101a, a first corresponding user 502 and a second corresponding user 504 of the second electronic device 101b participate in a multi-user communication session in space (e.g., in a first spatial state) and thus experience spatial authenticity with each other, while multiple remote users (including a third corresponding user 506) represented by virtual canvases 532a-532f participate in a non-spatial multi-user communication session and thus do not experience spatial authenticity with the first corresponding user 502 and the second corresponding user 504 (e.g., and do not experience spatial authenticity with each other).

[0094] exist Figure 5B In this context, since the third responding user 506 is currently participating in a multi-user communication session non-spatially, the third responding user 506 is visually represented as a two-dimensional representation in the three-dimensional environment 550A / 550B at the first electronic device 101a and the second electronic device 101b, such as... Figure 5A As shown. For example, as described above, the third corresponding user 506 is visually represented as a virtual canvas 532b at the first electronic device 101a and the second electronic device 101b. In some examples, a remote user currently associated with the second spatial state (e.g., participating in a non-spatial multi-user communication session) can provide input (such as via optional option 524) for transitioning from the second spatial state to the first spatial state. For example, in Figure 5B In this context, the optional option 524 can be selected to transform the third corresponding user 506 from a visually represented non-spatial representation (e.g., a two-dimensional virtual canvas 532c) to a spatial representation (e.g., a three-dimensional avatar), thereby enabling the third corresponding user 506 to participate in a multi-user communication session in space and experience spatial realism with the first corresponding user 502 and the second corresponding user 504 (e.g., via their respective electronic devices, as discussed similarly above).

[0095] exist Figure 5BIn the process, when selectable option 524 is displayed, the third electronic device 101c detects the selection of selectable option 524. For example, as Figure 5B As shown, the third electronic device 101c detects an air pinch gesture (or similar input) provided by the hand 505 of the third corresponding user 506, optionally simultaneously with the gaze 526 of the third corresponding user 506 pointing to the selectable option 524. In some examples, the third electronic device 101c displays the selectable option 524 after receiving input from the third corresponding user 506 corresponding to a request to display the selectable option 524, such as within a settings user interface or other menu associated with a multi-user communication session.

[0096] In some examples, in response to detecting a selection of selectable option 524, the third electronic device 101c associates the third corresponding user 506 with the aforementioned first spatial state. For example, the third electronic device 101c initiates a process of converting the display of the visual representation of the third corresponding user 506 (e.g., at other electronic devices in a multi-user communication session) from a non-spatial representation to a spatial representation, as described in more detail below. In some examples, the third electronic device 101c sends signals or other data (e.g., instructions) to electronic devices (including the first electronic device 101a and the second electronic device 101b) in the multi-user communication session, causing the electronic devices to update the display of the visual representation of the third corresponding user 506 from a non-spatial representation (e.g., a virtual canvas 532b) to a spatial representation (e.g., a 3D avatar).

[0097] Figure 5C An exemplary diagram 545 is illustrated, which illustrates a process for converting the display of a remote user's visual representation from a non-spatial representation to a spatial representation within a multi-user communication session. For example... Figure 5C As shown, multiple users of multiple electronic devices are participating in a multi-user communication session, including a first responding user 506, a second responding user 512, and a third responding user represented by an avatar 511. In some examples, the first responding user 506 and the second responding user 512 are juxtaposed in their respective physical environments, while the third responding user represented by avatar 511 is not juxtaposed with the first responding user 506 and the second responding user 512 in their respective physical environments. Figure 5C In this context, since the third corresponding user is visually represented by the avatar 511 (e.g., spatial representation), the third corresponding user experiences spatial authenticity along with the first corresponding user 506 and the second corresponding user 512 (e.g., via their respective electronic devices).

[0098] Additionally, such as Figure 5C As shown, a multi-user communication session includes multiple remote users currently participating in a non-spatial multi-user communication session. For example, such as... Figure 5CAs shown and discussed similarly above, multiple remote users are represented via virtual canvases 532a-532k, which correspond to non-spatial representations within a multi-user communication session. (See above reference...) Figure 5B Similarly, it is mentioned that the corresponding remote user can transition from participating in a multi-user communication session non-spatially to participating in the session spatially. This includes updating the presentation of their corresponding visual representation in the three-dimensional environment of users who are also participating in the multi-user communication session as spatial participants (e.g., the first corresponding user 506, the second corresponding user 512, and the third corresponding user). For example, the corresponding remote user transitions from being visually represented as a virtual canvas to being visually represented as a three-dimensional avatar, similar to... Figure 5C The incarnation of 511.

[0099] In some examples, in response to the detection of input or other indications (e.g., such as...) Figure 5B The selection of optional option 524 in the text), the corresponding electronic device associated with the corresponding remote user will associate the corresponding remote user with a spatial participant in a multi-user communication session (e.g., in the first spatial state described above), which will cause the non-spatial representation of the corresponding remote user to be in relation to Figure 5C The electronic devices associated with the first corresponding user 506, the second corresponding user 512, and the third corresponding user are re-displayed as spatial representations. In some examples, the transformation from displaying the visual representation of the remote user as a non-spatial representation to displaying it as a spatial representation includes determining a placement position 547 for displaying the spatial representation of the remote user at the electronic device associated with the first corresponding user 506, the second corresponding user 512, and the third corresponding user, such as... Figure 5C The illustration is shown in Figure 545. In some examples, the placement 547 for displaying the spatial representation of the remote user is determined based on the location of the non-spatial representation of the remote user. For example, in Figure 5C In diagram 545, the remote user, represented by virtual canvas 532b, has provided input at their electronic device corresponding to a request to participate in a multi-user communication session as a spatial participant. Therefore, in Figure 5C In this process, the electronic devices associated with the first corresponding user 506, the second corresponding user 512, and the third corresponding user initiate the process of transforming the display of the virtual canvas 532b corresponding to the remote user into a three-dimensional avatar displayed at the placement position 547. In some examples, Figure 5C The placement position 547 illustrated in Figure 545 is based on the position of the virtual canvas 532b within the horizontal spatial arrangement of the virtual canvas. Specifically, in some examples, the placement position 547 corresponds to a position in the three-dimensional environment that is spatially closest to the virtual canvas 532b. Figure 5CThe location within the horizontal spatial arrangement of the illustrated virtual canvas (e.g., within a threshold distance of that location (such as 0.25 meters, 0.5 meters, 0.75 meters, 1 meter, 1.5 meters, 2 meters, 3 meters, 5 meters, 10 meters, etc.)). Furthermore, as described in more detail below, placement location 547 corresponds to a location in the three-dimensional environment not occupied by another user or spatial representation within a multi-user communication session. For example, as... Figure 5C As shown in Figure 545, the placement position 547 is adjacent to and / or in front of the virtual canvas 532b in the three-dimensional environment, and is not occupied by the avatar 511, the first corresponding user 506, and the second corresponding user 512 in the three-dimensional environment.

[0100] Therefore, in some examples, in response to detecting the above-mentioned... Figure 5B In the selection of optional option 524, the third electronic device 101c sends signals or other data (e.g., instructions) to the first electronic device 101a and the second electronic device 101b, indicating that the third corresponding user 506 of the third electronic device 101c has become a spatial participant in a multi-user communication session. In some examples, in response to receiving an instruction from the third electronic device 101c, the first electronic device 101a and the second electronic device 101b determine a placement position in the three-dimensional environment 550A / 550B for displaying a spatial representation of the third corresponding user 506 in the three-dimensional environment 550A / 550B. Specifically, as referenced above... Figure 5C As described, the first electronic device 101a and the second electronic device 101b identify the location in the three-dimensional environment 550A / 550B that is spatially closest to the virtual canvas 532b previously located in... Figure 5A Positioning within the illustrated horizontal spatial arrangement. For example, in Figure 5D In the diagram, the first electronic device 101a and the second electronic device 101b identify the placement positions in the three-dimensional environment 550A / 550B, which are located at the leftmost position of the first row 533a in the horizontal spatial arrangement (e.g., Figure 5A The virtual canvas 532b is positioned adjacent to and / or in front of it, as illustrated in top view 510. Additionally, as... Figure 5D As illustrated in the top view 510, the positions corresponding to the placement locations in the three-dimensional environments 550A / 550B are not occupied by the first corresponding user 502 (e.g., and the first electronic device 101a) and the second corresponding user 504 (e.g., and the second electronic device 101b). In some examples, such as Figure 5DAs shown, after identifying the placement positions in the three-dimensional environments 550A / 550B, the first electronic device 101a and the second electronic device 101b display an avatar 511 corresponding to the third corresponding user 506 in the three-dimensional environments 550A / 550B. This indicates that the third corresponding user 506 is now participating in a multi-user communication session in space (e.g., and therefore now experiences spatial realism with the first corresponding user 502 and the second corresponding user 504). Additionally, in some examples, such as... Figure 5D As shown, when an avatar 511 corresponding to a third user 506 is displayed in a 3D environment 550A / 550B, the avatar 511 is oriented to face multiple virtual canvases 532 in the 3D environment 550A / 550B.

[0101] In some examples, such as Figure 5D As shown, when avatar 511 is displayed in the three-dimensional environment 550A / 550B, the first electronic device 101a and the second electronic device 101b cease displaying a non-spatial representation of the third corresponding user 506 in the three-dimensional environment 550A / 550B. For example, as Figure 5D As shown, when the avatar 511 corresponding to the third corresponding user 506 is displayed in the three-dimensional environment 550A / 550B, the first electronic device 101a and the second electronic device 101b no longer display the virtual canvas 532b. Additionally, as referenced above... Figures 4A to 4J Similarly, when the virtual canvas 532b stops displaying in the three-dimensional environment 550A / 550B, the first electronic device 101a and the second electronic device 101b update the order or spatial arrangement of the virtual canvas within the horizontal spatial arrangement of the three-dimensional environment 550A / 550B. For example, as... Figure 5D As shown, the first electronic device 101a and the second electronic device 101b update multiple virtual canvases 532 to include a virtual canvas 532h representing a corresponding remote user, which was previously not displayed or invisible in the three-dimensional environment 550A / 550B (e.g., due to a predefined number of display positions within the horizontal spatial arrangement, as discussed similarly above). Additionally, in some examples, such as Figure 5D As shown, from the viewpoints of the first electronic device 101a and the second electronic device 101b, the virtual canvases 532a / 532c / 532d are shifted to the left within the first row 533a of the horizontal spatial arrangement in the three-dimensional environment 550A / 550B (e.g., due to changes in priority sorting data caused by stopping the display of the virtual canvas 532b, thus updating the priority sorting order of the virtual canvases, as previously described herein).

[0102] The aforementioned method of transforming the display of a remote user's visual representation from a non-spatial representation to a spatial representation is similarly applicable to situations where the remote user's visual representation is displayed within a vertical spatial arrangement (e.g., anchored to a shared virtual object) in a multi-user communication session. For example, in Figure 5E In this context, first electronic device 101a and second electronic device 101b are in a multi-user communication session with multiple remote users, while simultaneously displaying a shared object (e.g., virtual object 520, as indicated by capsule button 522) in a three-dimensional environment 550A / 550B. Therefore, as discussed previously herein, virtual canvases 532a and 532b representing multiple remote users are arranged in a vertical spatial arrangement relative to virtual object 520 in the three-dimensional environment 550A / 550B (e.g., arranged in a column) (e.g., anchored to the side or edge of virtual object 520 from the viewpoint of first electronic device 101a and second electronic device 101b). In some examples, virtual object 520 has one or more characteristics of virtual object 420 described above.

[0103] exist Figure 5E In this context, the first electronic device 101a and the second electronic device 101b detect the following indication: the corresponding remote user has provided input at its respective electronic device corresponding to a request to participate in a multi-user communication session in space, such as as mentioned above. Figure 5B The input described is similar to the input. Figure 5E In the example, the remote user providing input for transitioning from participating in a non-spatial multi-user communication session to participating in a spatial multi-user communication session corresponds to the remote user represented by the virtual canvas 532b in the three-dimensional environment 550A / 550B. In some examples, as discussed similarly above and described in more detail below, an instruction received from the corresponding electronic device associated with the remote user represented by the virtual canvas 532b causes the first electronic device 101a and the second electronic device 101b to transition from representing the remote user as the virtual canvas 532b to representing the remote user as a virtual avatar.

[0104] Figure 5F An exemplary diagram 545 is illustrated, which illustrates a process for converting the display of a remote user's visual representation from a non-spatial representation to a spatial representation within a multi-user communication session. For example... Figure 5F As shown, multiple users of multiple electronic devices are participating in a multi-user communication session, including a first responding user 502, a second responding user 504, and a third responding user represented by an avatar 511. In some examples, the first responding user 502 and the second responding user 504 are juxtaposed in their respective physical environments, while the third responding user represented by avatar 511 is not juxtaposed with the first responding user 502 and the second responding user 504 in their respective physical environments. Figure 5FIn this context, since the third corresponding user is visually represented by the avatar 511 (e.g., spatial representation), the third corresponding user experiences spatial authenticity along with the first corresponding user 502 and the second corresponding user 504 (e.g., via their respective electronic devices).

[0105] Additionally, such as Figure 5F As shown, a multi-user communication session includes multiple remote users currently participating in a non-spatial multi-user communication session. For example, such as... Figure 5F As shown and discussed similarly above, multiple remote users are represented via virtual canvases 532a and 532b, which correspond to non-spatial representations within a multi-user communication session. In some examples, such as Figure 5F As illustrated in Figure 545, since a shared virtual object 520 is presented within a multi-user communication session, virtual canvases 532a and 532b are spatially arranged (e.g., in a column) within a vertical spatial arrangement anchored to the virtual object 520 (e.g., displayed as adjacent to the edge or side of the virtual object 520) relative to the viewpoint of the electronic device associated with the first corresponding user 502, the second corresponding user 504, and the third corresponding user. (Refer to the above...) Figure 5B Similarly, it is mentioned that the corresponding remote user can transition from participating in a multi-user communication session non-spatially to participating in the session spatially. This includes updating the presentation of their corresponding visual representation in the three-dimensional environment of users who are also participating in the multi-user communication session as spatial participants (e.g., the first corresponding user 502, the second corresponding user 504, and the third corresponding user). For example, the corresponding remote user transitions from being visually represented as a virtual canvas to being visually represented as a three-dimensional avatar, similar to... Figure 5F The incarnation of 511.

[0106] In some examples, in response to the detection of input or other indications (e.g., similar to...), Figure 5B The selection of optional option 524 in the text), the corresponding electronic device associated with the corresponding remote user will associate the corresponding remote user with a spatial participant in a multi-user communication session (e.g., in the first spatial state described above), which will cause the non-spatial representation of the corresponding remote user to be in relation to Figure 5F The electronic devices associated with the first corresponding user 502, the second corresponding user 504, and the third corresponding user are re-displayed as spatial representations. In some examples, as described similarly above, the transformation from displaying the visual representation of the remote user as a non-spatial representation to displaying it as a spatial representation includes determining a placement position 547 for displaying the spatial representation of the remote user at the electronic device associated with the first corresponding user 502, the second corresponding user 504, and the third corresponding user, such as... Figure 5FAs shown in Figure 545. In some examples, the placement 547 for displaying the spatial representation of the remote user is determined based on the location of the non-spatial representation of the remote user. For example, in Figure 5F In diagram 545, the remote user, represented by virtual canvas 532b, has provided input at their electronic device corresponding to a request to participate in a multi-user communication session as a spatial participant. Therefore, in Figure 5F In this process, the electronic devices associated with the first corresponding user 502, the second corresponding user 504, and the third corresponding user initiate the process of transforming the display of the virtual canvas 532b corresponding to the remote user into a three-dimensional avatar displayed at the placement position 547. In some examples, Figure 5F The placement position 547 illustrated in Figure 545 is based on the position of the virtual canvas 532b within the vertical spatial arrangement of the virtual canvas. Specifically, in some examples, the placement position 547 corresponds to a position in the three-dimensional environment that is spatially closest to the virtual canvas 532b. Figure 5F The illustrated virtual canvas is positioned within a vertical spatial arrangement (e.g., within a threshold distance (such as 0.25 m, 0.5 m, 0.75 m, 1 m, 1.5 m, 2 m, 3 m, 5 m, 10 m, etc.)). Furthermore, as described in more detail below, placement position 547 corresponds to a position in the three-dimensional environment not occupied by another user or spatial representation within a multi-user communication session. For example, as... Figure 5F As shown in Figure 545, the placement position 547 is adjacent to and / or in front of the virtual canvas 532b in the three-dimensional environment, and is not occupied by the avatar 511, the first corresponding user 502 and the second corresponding user 504 in the three-dimensional environment.

[0107] Therefore, in some examples, in response to the detection of the above reference Figure 5E The described remote user has provided instructions at its respective electronic device corresponding to the request to participate in a multi-user communication session in space. The first electronic device 101a and the second electronic device 101b determine a placement position in the three-dimensional environment 550A / 550B for displaying a spatial representation of the remote user within the three-dimensional environment 550A / 550B. Specifically, as referenced above... Figure 5F As described, the first electronic device 101a and the second electronic device 101b identify the location in the three-dimensional environment 550A / 550B that is spatially closest to the virtual canvas 532b previously located in... Figure 5E Positioning within the illustrated vertical spatial arrangement. For example, in Figure 5G In the diagram, the first electronic device 101a and the second electronic device 101b identify the placement position within the three-dimensional environment 550A / 550B, which is positioned relative to the bottom of the virtual canvas column adjacent to the virtual object 520 (e.g., Figure 5E The virtual canvas 532b is positioned adjacent to and / or in front of it, as illustrated in top view 510. Additionally, as... Figure 5G As illustrated in the top view 510, the positions corresponding to the placement locations in the three-dimensional environments 550A / 550B are not occupied by the first corresponding user 502 (e.g., and the first electronic device 101a) and the second corresponding user 504 (e.g., and the second electronic device 101b). In some examples, such as Figure 5G As shown, after identifying the placement positions in the three-dimensional environments 550A / 550B, the first electronic device 101a and the second electronic device 101b display avatars 511 corresponding to the remote user in the three-dimensional environments 550A / 550B. This indicates that the remote user is now participating in a multi-user communication session in space (e.g., and therefore now experiences spatial realism with the first corresponding user 502 and the second corresponding user 504). Additionally, in some examples, such as... Figure 5G As shown, when an avatar 511 corresponding to a remote user is displayed in a 3D environment 550A / 550B, the avatar 511 is oriented toward a virtual object 520 in the 3D environment 550A / 550B.

[0108] In some examples, such as Figure 5G As shown, when avatar 511 is displayed in the three-dimensional environment 550A / 550B, the first electronic device 101a and the second electronic device 101b cease displaying a non-spatial representation of the remote user in the three-dimensional environment 550A / 550B. For example, as Figure 5G As shown, when the avatar 511 corresponding to the remote user is displayed in the 3D environment 550A / 550B, the first electronic device 101a and the second electronic device 101b no longer display the virtual canvas 532b. Additionally, as referenced above... Figures 4A to 4J Similarly, when the virtual canvas 532b stops displaying in the three-dimensional environment 550A / 550B, the first electronic device 101a and the second electronic device 101b update the order or spatial arrangement of the virtual canvas within the vertical spatial arrangement of the three-dimensional environment 550A / 550B. For example, as... Figure 5G As shown, the first electronic device 101a and the second electronic device 101b update a plurality of virtual canvases 532 to include a virtual canvas 532c representing a corresponding remote user, which was previously not displayed or invisible in the three-dimensional environment 550A / 550B (e.g., due to a predefined number of display positions within the vertical spatial arrangement, as discussed similarly above).

[0109] Therefore, as outlined above, systems and methods are provided for transforming the display of a remote user's visual representation from a non-spatial representation to a spatial representation within a multi-user communication session when the remote user requests to participate in the session spatially based on a non-spatial representation of their location. This facilitates the discovery of the spatial authenticity of the remote user's experience in the local user's experience within the multi-user communication session, thereby enhancing the user experience within the multi-user communication session. Additionally, automatically determining the location for displaying the remote user's spatial representation in the 3D environment based on the previous location of the remote user's non-spatial representation in the 3D environment reduces the magnitude (e.g., magnitude of distance) of virtual elements (e.g., objects and / or representations) that are visually updated relative to the local user and / or reduces the need for user input for manually selecting placement locations. This, as another benefit, helps reduce eye strain for the local user and reduces the consumption of computational resources associated with responding to such user input.

[0110] Figures 5H to 5K Additional examples illustrate the transformation of the display of a remote user's visual representation from a non-spatial representation to a spatial representation within a multi-user communication session. Figure 5H In this configuration, a first electronic device 101a (e.g., associated with a first corresponding user 502) and a second electronic device 101b (e.g., associated with a second corresponding user 504) are juxtaposed in a physical environment 500, as illustrated in top view 510. Additionally, in some examples, such as... Figure 5H As shown, a first electronic device 101a and a second electronic device 101b are participating in a multi-user communication session with multiple remote users, including a third corresponding user of a third electronic device represented by an avatar 511 in a three-dimensional environment 550A presented at the first electronic device 101a. In some examples, as similarly described above, since the third corresponding user of the third electronic device is visually represented as an avatar 511 in the three-dimensional environment 550A (e.g., spatial representation), the first corresponding user 502 of the first electronic device 101a and the second corresponding user 504 of the second electronic device 101b experience spatial realism with the third user within the multi-user communication session. Furthermore, as... Figure 5H As shown, the multiple remote users include non-spatial participants represented by a virtual canvas in the 3D environment 550A. In some examples, as previously discussed herein, since the 3D environment 550A does not include virtual objects shared within the multi-user communication session (e.g., virtual object 520 as described above), the virtual canvas is arranged in a horizontal spatial arrangement comprising two rows 533a and 533b.

[0111] exist Figure 5HIn this process, the first electronic device 101a (and, for example, the second electronic device 101b) detects an indication that the corresponding remote user has provided input at its respective electronic device corresponding to a request to participate in a multi-user communication session in space, such as as described above. Figure 5B The input described is similar to the input. Figure 5H In the example, the remote user providing input for transitioning from participating in a non-spatial multi-user communication session to participating in a spatial multi-user communication session corresponds to the remote user represented by the virtual canvas 532c in the three-dimensional environment 550A (e.g., the fourth user of the fourth electronic device). In some examples, as similarly discussed above and described in more detail below, an instruction received from the corresponding electronic device associated with the remote user (e.g., the fourth user) represented by the virtual canvas 532c causes the first electronic device 101a (e.g., and the second electronic device 101b) to transition from representing the remote user as the virtual canvas 532c to representing the remote user as a virtual avatar (e.g., similar to...). Figure 5H The incarnation of 511).

[0112] In some examples, as previously referenced in this article Figure 5C As described, transforming the display of a remote user's (e.g., a fourth user's) visual representation from a virtual canvas 532c to a virtual avatar (e.g., a spatial representation) includes identifying the avatar's placement position within a three-dimensional environment 550A. Specifically, as previously described, a first electronic device 101a (e.g., and a second electronic device 101b) identifies a position in the three-dimensional environment 550A that is spatially closest to the positioning of the virtual canvas 532c within the first row 533a of the horizontal spatial arrangement of the virtual canvas 532c in the three-dimensional environment 550A. However, in some cases, the identified placement position of the avatar corresponding to the remote user in the three-dimensional environment 550A may be occupied by another user, virtual object, or physical object in the physical environment 500. For example, as... Figure 5H As illustrated in the top view 510, the position (e.g., in front of and / or adjacent to) the virtual canvas 532c within the first row 533a of the horizontal spatial arrangement in the 3D environment 550A is currently occupied by avatar 511. Therefore, as discussed below, the first electronic device 101a (e.g., and the second electronic device 101b) identifies alternative (e.g., different) placement positions in the 3D environment 550A, which optionally includes updating the display of one or more virtual objects or elements in the 3D environment 550A from the viewpoint of the first electronic device 101a.

[0113] In some examples, such as Figure 5IAs shown, determining an alternative placement position of the avatar corresponding to a remote user in the 3D environment 550A includes updating the display of one or more other avatars corresponding to one or more other remote users in the 3D environment 550A such that the avatar corresponding to the remote user is displayed in the 3D environment 550A at a position within the first row 533a of the virtual canvas 532c's horizontal spatial arrangement in the 3D environment 550A. For example, as discussed above and as... Figure 5K As illustrated in the top view 510, the (e.g., ideal) placement position of the virtual canvas 532c within the first row 533a of the horizontal spatial arrangement in the three-dimensional environment 550A corresponds to the position currently occupied by the avatar 511 in the three-dimensional environment 550A. In some such examples, since the avatar 511 is a virtual representation within the three-dimensional environment 550A (e.g., although representing a third corresponding user participating in a multi-user communication session in space), the position of the avatar 511 within the three-dimensional environment 550A can be updated freely and easily relative to the viewpoint of the first electronic device 101a (e.g., and the viewpoint of the second electronic device 101b). More specifically, in some examples, avatar 511 can be repositioned in the three-dimensional environment 550A from the viewpoint of the first electronic device 101a to accommodate the display of avatar 513 in the three-dimensional environment 550A, while local users (e.g., the second corresponding user 504 of the second electronic device 101b) and physical objects (e.g., the bracket 507 and / or the indoor plant 508) cannot move freely and easily in the three-dimensional environment 550A (e.g., because the local user and the physical object do not correspond to the virtual objects generated and displayed by the first electronic device 101a). Therefore, in some examples, such as Figure 5I As shown, the first electronic device 101a (and, for example, the second electronic device 101b) repositions the avatar 511 corresponding to the third corresponding user toward the first corresponding user 502, as shown in top view 510, such that the avatar 511 is now positioned in the three-dimensional environment 550A adjacent to the viewpoint of the first electronic device 101a, adjacent to the second corresponding user 504, thus the previously identified placement position of the avatar corresponding to the fourth user discussed above is not occupied in the three-dimensional environment 550A. In some examples, such as Figure 5I As shown in the top view 510, when the avatar 511 is repositioned in the three-dimensional environment 550A relative to the viewpoint of the first electronic device 101a (e.g., and the viewpoint of the second electronic device 101b), the avatar 511 is displayed in an orientation that causes the avatar 511 to continue facing multiple virtual canvases 532 in the three-dimensional environment 550A from its updated position in the three-dimensional environment 550A.

[0114] In some examples, such as Figure 5IAs shown, after repositioning the avatar 511 corresponding to the third user in the three-dimensional environment 550A, the first electronic device 101a (e.g., and the second electronic device 101b) displays the avatar 513 corresponding to the fourth user at the previously identified placement location in the three-dimensional environment 550A, indicating that the fourth user is now spatially participating in a multi-user communication session. For example, as... Figure 5I As shown in the top view 510, from the viewpoint of the first electronic device 101a, the avatar 513 is displayed in the three-dimensional environment 550A at the position previously occupied by the avatar 511 in the three-dimensional environment 550A (e.g., before the avatar 511 moved). Figure 5H (The location of the incarnation 511). Additionally, in some examples, such as... Figure 5I As indicated by the top view 510, the avatar 513 is displayed in an orientation facing multiple virtual canvases 532 in the three-dimensional environment 550A, as discussed similarly above. In some examples, as previously described herein and as... Figure 5I As illustrated, when avatar 513 is displayed in a three-dimensional environment 550A, the first electronic device 101a (and, for example, the second electronic device 101b) updates the order or spatial distribution of the virtual canvas within the horizontal spatial arrangement of the three-dimensional environment 550 based on stopping the display of the virtual canvas 532c corresponding to the fourth user in the three-dimensional environment 550A. For example, as Figure 5I As shown, after the virtual canvas 532c stops displaying in the 3D environment 550A, the virtual canvas 532e representing the corresponding remote user is moved and / or transformed to the position of the virtual canvas 532c within the first row 533a. Specifically, as shown, according to the update priority order of the virtual canvas, the virtual canvas 532e moves from the second row 533b to the first row 533a in the horizontal spatial arrangement. This update priority order is updated based on the priority order data updated after the display of the fourth user's visual representation in the 3D environment 550A changes from the virtual canvas 532c to the avatar 513, as previously described herein.

[0115] Alternatively, in some examples, such as Figure 5J As shown, determining an alternative placement location for the avatar corresponding to the remote user in the 3D environment 550A includes updating the display of the spatial arrangement (e.g., horizontal or vertical spatial arrangement) of the virtual canvas representing the remote user in the multi-user communication session, such that the avatar corresponding to the remote user can be displayed in the 3D environment 550A at a location closest to the non-spatial representation of the remote user's location in the 3D environment 550A (e.g., the location of the virtual canvas 532c within the first row 533a of the horizontal spatial arrangement in the 3D environment 550A). For example, as discussed above and as... Figure 5JAs illustrated in the top view 510, the (e.g., ideal) placement of the virtual canvas 532c within the first row 533a of the horizontal spatial arrangement in the three-dimensional environment 550A corresponds to the position currently occupied by the avatar 511 in the three-dimensional environment 550A. In some such examples, since the multiple virtual canvases 532 representing multiple (e.g., non-spatial) remote users in a multi-user communication session are virtual representations (e.g., virtual objects) within the three-dimensional environment 550A, the positions and / or orientations of the multiple virtual canvases within the three-dimensional environment 550A can be updated freely and easily relative to the viewpoint of the first electronic device 101a (e.g., and the viewpoint of the second electronic device 101b). More specifically, in some examples, multiple virtual canvases 532 (e.g., within a horizontal spatial arrangement) can be repositioned and / or reoriented (e.g., as a group) from the viewpoint of the first electronic device 101a in the three-dimensional environment 550A to accommodate the display of the avatar 513 in the three-dimensional environment 550A (e.g., because the virtual canvases correspond to virtual objects generated and displayed by the first electronic device 101a). Therefore, in some examples, such as Figure 5J As shown, a first electronic device 101a (e.g., and a second electronic device 101b) repositions and / or reorients multiple virtual canvases 532 (e.g., as a group) within a horizontal spatial arrangement relative to the viewpoint of the first electronic device 101a (e.g., while maintaining the current order and / or spatial distribution of the virtual canvases within the first row 533a and the second row 533b), as shown in top view 510, such that the multiple virtual canvases 532 are now shifted to the right and rotated clockwise (e.g., about a vertical axis passing through the center of the horizontal spatial arrangement of the virtual canvases 532) in the three-dimensional environment 550A relative to the viewpoint of the first electronic device 101a (e.g., and the viewpoint of the second electronic device 101b), thereby creating an updated, unoccupied, identified placement position in the three-dimensional environment 550A for the avatar corresponding to the fourth user discussed above. For example, as Figure 5J As illustrated in the top view 510, multiple virtual canvases 532 (e.g., as a group) in the three-dimensional environment 550A are repositioned and / or reoriented relative to the first electronic device 101a (e.g., and the second electronic device 101b) so that the avatar 511 is no longer positioned on the virtual canvas 532c. Figure 5H The location in front of and / or adjacent to the first row 533a of the horizontal spatial arrangement in the three-dimensional environment 550A shown.

[0116] In some examples, such as Figure 5JAs shown, after repositioning and / or reorienting multiple virtual canvases 532 in the three-dimensional environment 550A, the first electronic device 101a (e.g., and the second electronic device 101b) displays an avatar 513 corresponding to the fourth user at the updated placement location in the three-dimensional environment 550A, indicating that the fourth user is now spatially participating in a multi-user communication session. For example, as... Figure 5J As shown in the top view 510, the avatar 513 is displayed in the three-dimensional environment 550A at a position corresponding to the updated placement, which is spatially closest to (e.g., adjacent to and / or in front of) the first row 533a of the horizontal spatial arrangement, a position previously occupied by the virtual canvas 532c in the three-dimensional environment 550A from the viewpoint of the first electronic device 101a. Additionally, in some examples, such as Figure 5J As indicated by the top view 510, the avatar 513 is displayed in an orientation facing multiple virtual canvases 532 in the three-dimensional environment 550A, as discussed similarly above. In some examples, as previously described herein and as... Figure 5J As illustrated, when avatar 513 is displayed in a three-dimensional environment 550A, the first electronic device 101a (and, for example, the second electronic device 101b) updates the order or spatial distribution of the virtual canvas within the horizontal spatial arrangement of the three-dimensional environment 550 based on stopping the display of the virtual canvas 532c corresponding to the fourth user in the three-dimensional environment 550A. For example, as Figure 5J As shown, after the virtual canvas 532c stops displaying in the 3D environment 550A, the virtual canvas 532e representing the corresponding remote user is moved and / or transformed to the position of the virtual canvas 532c within the first row 533a. Specifically, as shown, according to the update priority order of the virtual canvas, the virtual canvas 532e moves from the second row 533b to the first row 533a in the horizontal spatial arrangement. This update priority order is updated based on the priority order data updated after the display of the fourth user's visual representation in the 3D environment 550A changes from the virtual canvas 532c to the avatar 513, as previously described herein.

[0117] As an alternative example, in Figure 5K In this process, determining an alternative placement location for the avatar corresponding to the remote user within the 3D environment 550A includes identifying the next unoccupied nearest position within the first row 533a of the horizontal spatial arrangement of the 3D environment 550A relative to the virtual canvas 532c (e.g., within a positioning threshold distance (such as 0.25 meters, 0.5 meters, 0.75 meters, 1 meter, 1.5 meters, 2 meters, 3 meters, 5 meters, 10 meters, etc.)). For example, as... Figure 5KAs illustrated in the top view 510, in the three-dimensional environment 550A, the position of the avatar 511 and the first corresponding user 502 of the first electronic device 101a relative to the virtual canvas 532c within the first row 533a is not occupied, while the remaining suitable positions in the three-dimensional environment 550A are occupied (e.g., occupied by the second corresponding user 504 of the second electronic device 101b and the stand 507). Specifically, as Figure 5K As illustrated in the top view 510, the position of the avatar 511 in the three-dimensional environment 550A and the first corresponding user 502 corresponds to and is aligned with the spatial arrangement of spatial users within a multi-user communication session. For example, in Figure 5K In the top view 510, the positioning of avatar 511, first corresponding user 502, and second corresponding user 504 corresponds to a horizontal spatial arrangement of virtual tiles representing multiple remote users in a multi-user communication session and / or forms an arc relative to that horizontal spatial arrangement. Therefore, in some examples, the updated (e.g., alternative) placement of the avatar corresponding to the fourth user discussed above is selected as an unoccupied location that is within the aforementioned threshold distance of the previous positioning in the first row 533a of the virtual canvas 532c and is unoccupied in the three-dimensional environment 550A.

[0118] In some examples, such as Figure 5K As shown, after determining the updated placement location in the three-dimensional environment 550A, the first electronic device 101a (and, for example, the second electronic device 101b) displays an avatar 513 corresponding to the fourth user at the updated placement location in the three-dimensional environment 550A, indicating that the fourth user is now participating in a multi-user communication session in space. For example, as Figure 5K As shown in the top view 510, from the viewpoint of the first electronic device 101a, the avatar 513 is displayed in the three-dimensional environment 550A between the positions of the avatar 511 and the first corresponding user 502. Additionally, in some examples, such as... Figure 5K As instructed, avatar 513 is displayed in an orientation facing multiple virtual canvases 532 within the three-dimensional environment 550A, as discussed similarly above. In some examples, as previously described herein and as... Figure 5K As illustrated, when avatar 513 is displayed in a three-dimensional environment 550A, the first electronic device 101a (and, for example, the second electronic device 101b) updates the order or spatial distribution of the virtual canvas within the horizontal spatial arrangement of the three-dimensional environment 550 based on stopping the display of the virtual canvas 532c corresponding to the fourth user in the three-dimensional environment 550A. For example, as Figure 5KAs shown, after the virtual canvas 532c stops displaying in the 3D environment 550A, the virtual canvas 532e representing the corresponding remote user is moved and / or transformed to the position of the virtual canvas 532c within the first row 533a. Specifically, according to the priority order of virtual canvas updates, the virtual canvas 532e moves from the second row 533b to the first row 533a in the horizontal spatial arrangement. This priority order is updated based on the priority order data updated after the display of the virtual canvas 532c in the 3D environment 550A is transformed into avatar 513, as described previously herein.

[0119] Therefore, as outlined above, systems and methods are provided for transforming the display of a remote user's visual representation from a non-spatial representation to a spatial representation within a multi-user communication session when the remote user requests to participate in the session spatially in a manner that adapts to the positions of virtual objects and users, as well as physical objects and users, in a three-dimensional environment. This enables the remote user to participate in the multi-user communication session spatially, while defaulting to the existing spatial arrangement of virtual objects and users, as well as physical objects and users, thereby enhancing the user experience within the multi-user communication session. Additionally, automatically determining the updated placement position for displaying the remote user's spatial representation in the three-dimensional environment based on the previous position of the remote user's non-spatial representation in the three-dimensional environment and / or automatically updating the display of the remote user's representation in the three-dimensional environment reduces the need for user input for manually selecting placement positions and / or updating the display of the remote user's representation, thus helping to reduce the consumption of computational resources associated with responding to such user input as another benefit.

[0120] It should be understood that the examples shown and described herein are merely exemplary, and additional and / or alternative elements for interacting with the exemplary content may be provided within a three-dimensional environment. It should be understood that the appearance, shape, form, and size of each of the various user interface elements and objects shown and described herein are exemplary, and alternative appearances, shapes, forms, and / or sizes may be provided. For example, virtual objects representing the user interface (e.g., virtual objects 330, 420, 430, and 520) may be provided with alternative shapes other than rectangular shapes (such as circular shapes, triangular shapes, etc.). In some examples, various selectable options described herein (e.g., capsule buttons 422 / 522, representations 431a-431f, and / or selectable option 524), user interface elements (e.g., grip bars 437 / 537), etc., may be verbally selected via user verbal commands (e.g., the verbal command "select option"). Additionally or alternatively, in some examples, the various options, user interface elements, control elements, etc., described herein can be selected and / or manipulated via user input received through one or more separate input devices in communication with the electronic device. For example, selection input can be received via a physical input device (such as a mouse, touchpad, keyboard, etc.) in communication with the electronic device.

[0121] Figure 6 This is a flowchart illustrating example processes for maintaining visual consistency of a remote user's visual representation in a three-dimensional environment within a multi-user communication session, according to some examples of this disclosure. In some examples, process 600 begins at a first electronic device communicating with one or more displays and one or more input devices, wherein the first electronic device is juxtaposed with a second electronic device in a first physical environment. In some examples, the first electronic device and the second electronic device are optionally similar to or corresponding to... Figure 2 The device is a head-mounted display with 200 units. For example... Figure 6 As shown, in some examples, at 702, when in a multi-user communication session with the second electronic device and a plurality of corresponding electronic devices, wherein the plurality of corresponding electronic devices are not juxtaposed with the first electronic device and the second electronic device in the physical environment, the first electronic device presents multiple virtual representations of the plurality of corresponding electronic devices for multiple users in a three-dimensional environment via the one or more displays, wherein, from the viewpoint of the first electronic device, the plurality of virtual representations have a first spatial arrangement in the three-dimensional environment, and wherein the first spatial arrangement is based on a first priority ordering data. For example, as Figure 4CAs shown, when the first electronic device 101a is in a multi-user communication session with multiple electronic devices, including the second electronic device 101b, the first electronic device 101a is presenting a three-dimensional environment 450A. This three-dimensional environment includes multiple virtual canvases 432 corresponding to multiple users of the multiple electronic devices, which are not juxtaposed with the first electronic device 101a and the second electronic device 101b in a physical environment 400. Additionally, in some examples, such as... Figure 4C As shown, multiple virtual canvases 432 are arranged in a horizontal spatial arrangement in a three-dimensional environment 450A, the horizontal spatial arrangement including two rows 433a and 433b of virtual canvases, wherein the virtual canvases have a specific order or spatial distribution based on priority sorting data within the two rows 433a and 433b.

[0122] In some examples, at 604, when the multiple virtual representations of the multiple users of the multiple corresponding electronic devices are presented in the 3D environment, the first electronic device detects a change in the first priority sorting data. For example, as referenced... Figure 4C As described, the first electronic device 101a detects an indication or user input that causes a change in priority sorting data controlling the priority sorting order of virtual canvases 432a-432f, such as a change in user activity (e.g., video activity and / or audio activity) of one or more remote users in a multi-user communication session.

[0123] In some examples, at 606, in response to detecting a change in the first priority sorting data, at 608, determining that the first priority sorting data meets one or more criteria based on the change, at 610, the first electronic device updates the presentation of the plurality of virtual representations via the one or more displays to have a second spatial arrangement different from the first spatial arrangement, and at 612, sends corresponding display data to the second electronic device, which causes the second electronic device to update the presentation of the plurality of virtual representations via the one or more second displays of the second electronic device to have the second spatial arrangement. For example, as referenced... Figure 4E As described, the one or more criteria include those that are satisfied when a change in priority sorting data alters the priority sorting order of virtual canvases 432a-432f, causing a virtual canvas in the second row 433b (e.g., virtual canvas 432e) to be moved to the first row 433a of the horizontal spatial arrangement of virtual canvases in the 3D environment 450A. Therefore, in some examples, such as Figure 4EAs shown, the first electronic device 101a updates the display of the order or spatial distribution of virtual canvases 432a-432f within the horizontal spatial arrangement of the virtual canvases in the three-dimensional environment 450A, including moving virtual canvas 432e from the second row 433b to the first row 433a in the three-dimensional environment 450A. In some examples, such as reference... Figure 4E As described, the first electronic device 101a sends a signal or data (e.g., including instructions) to the second electronic device 101b indicating priority sorting data associated with the order or spatial distribution of the virtual canvases 432a-432f, which causes the second electronic device 101b to update the order or spatial distribution of the virtual canvases 432a-432f in the three-dimensional environment 450B, including moving the virtual canvas 432e from the second row 433b in the three-dimensional environment 450B to the first row 433a.

[0124] In some examples, at 614, based on the change in the first priority sorting data, it is determined that the first priority sorting data fails to meet one or more criteria; at 616, the first electronic device maintains the presentation of the plurality of virtual representations with the first spatial arrangement in the three-dimensional environment; and at 618, the transmission of the corresponding display data to the second electronic device is abandoned. For example, as referenced... Figure 4E Similarly, the first electronic device 101a maintains the display of the order or spatial distribution of the virtual canvases 432a-432f in the three-dimensional environment 450A, and refrains from sending signals or data to the second electronic device 101b instructing the updating of priority sorting data associated with the order or spatial distribution of the virtual canvases 432a-432f.

[0125] It should be understood that process 600 is an example, and more, fewer, or different operations may be performed in the same or different order. Additionally, the operations in process 600 described above may optionally be performed by operating one or more functional modules in the information processing device and / or by… Figure 2 Other components are used to implement this information processing device, such as a general-purpose processor (e.g., regarding...). Figure 2 (as described) or a dedicated chip.

[0126] Figure 7 This is a flowchart illustrating example processes, according to some examples of the present disclosure, for transforming a display of a remote user's visual representation in a three-dimensional environment from a virtual canvas into a three-dimensional avatar within a multi-user communication session. In some examples, process 700 begins at a first electronic device communicating with one or more displays and one or more input devices, wherein the first electronic device is juxtaposed with a second electronic device in a first physical environment. In some examples, the first electronic device and the second electronic device are optionally similar to or corresponding to... Figure 2 The device is a head-mounted display with 200 units. For example... Figure 7As shown, in some examples, at 702, when in a multi-user communication session with the second electronic device and a plurality of corresponding electronic devices including a third electronic device, wherein the plurality of corresponding electronic devices are not juxtaposed with the first and second electronic devices in the physical environment, the first electronic device presents multiple virtual representations of the plurality of corresponding electronic devices for multiple users in a three-dimensional environment via the one or more displays, wherein, from the viewpoint of the first electronic device, the plurality of virtual representations have a first spatial arrangement in the three-dimensional environment. For example, as Figure 5A As shown, when the first electronic device 101a is in a multi-user communication session with multiple electronic devices, including the second electronic device 101b, the first electronic device 101a is presenting a three-dimensional environment 550A. This three-dimensional environment includes multiple virtual canvases 532 corresponding to multiple users of the multiple electronic devices, which are not juxtaposed with the first electronic device 101a and the second electronic device 101b in a physical environment 500. Additionally, in some examples, such as... Figure 5A As shown, multiple virtual canvases 532 are arranged in a horizontal spatial arrangement in a three-dimensional environment 550A, the horizontal spatial arrangement including two rows 533a and 533b of virtual canvases, wherein the virtual canvases have a specific order or spatial distribution within the two rows 533a and 533b.

[0127] In some examples, at 704, when the plurality of virtual representations of the plurality of users of the plurality of respective electronic devices are presented in the three-dimensional environment, the first electronic device detects an indication of an input received at the third electronic device among the plurality of respective electronic devices, the input corresponding to a request to represent the user of the third electronic device as a three-dimensional representation in the three-dimensional environment that is different from the first virtual representation of the user of the third electronic device, wherein the first virtual representation of the user of the third electronic device occupies a first position within the first spatial arrangement. For example, as Figure 5B As shown, a third electronic device 101c, within a multi-user communication session with a first electronic device 101a and a second electronic device 101b, detects a selection of an optional option 524 provided by the hand 505 of a third corresponding user 506. This optional option is used to change the spatial state of the third corresponding user 506 of the third electronic device 101c from non-spatial to spatial within the multi-user communication session. Additionally, in some examples, such as... Figure 5A As shown, when the third corresponding user 506 is associated with a non-spatial state in a multi-user communication session (e.g., before the third electronic device 101c detects a selection of the selectable option 524), from the viewpoint of the first electronic device 101a, the virtual canvas 532b representing the third corresponding user 506 is displayed at the leftmost position in the first row 533a of the horizontal spatial arrangement of the virtual canvas in the three-dimensional environment 550A.

[0128] In some examples, at 706, in response to detecting the indication of the input, the first electronic device stops presenting the first virtual representation of the plurality of virtual representations in the three-dimensional environment via the one or more displays at 708, and at 710 presents a three-dimensional representation of the user of the third electronic device in the three-dimensional environment relative to the placement position of the plurality of virtual representations via the one or more displays, wherein the placement position in the three-dimensional environment is based on the first positioning of the first virtual representation within the first spatial arrangement. For example, as referenced... Figure 5B As described and as Figure 5C As shown, the first electronic device 101a stops displaying a virtual canvas 532b representing the third corresponding user 506 within the first row 533a arranged in the horizontal space of the virtual canvas, and displays an avatar 511 corresponding to the third corresponding user 506 in the three-dimensional environment 550A. Additionally, in some examples, such as reference... Figure 5C As described, the position for displaying the avatar 511 in the three-dimensional environment 550A is based on (e.g., spatially closest) the previous positioning of the virtual canvas 432b within the first row 533a of the horizontal spatial arrangement of the virtual canvases in the three-dimensional environment 550A (e.g., before the virtual canvas 532b is no longer displayed).

[0129] It should be understood that process 700 is an example, and more, fewer, or different operations may be performed in the same or different order. Additionally, the operations in process 700 described above may optionally be performed by operating one or more functional modules in the information processing device and / or by… Figure 2 Other components are used to implement this information processing device, such as a general-purpose processor (e.g., regarding...). Figure 2 (as described) or a dedicated chip.

[0130] Therefore, based on the foregoing, some examples of this disclosure relate to a method comprising: at a first electronic device communicating with one or more displays and one or more input devices, wherein the first electronic device is juxtaposed with a second electronic device in a physical environment; while in a multi-user communication session with the second electronic device and a plurality of corresponding electronic devices, wherein the plurality of corresponding electronic devices are not juxtaposed with the first and second electronic devices in the physical environment, presenting, via the one or more displays, a plurality of virtual representations of a plurality of users of the plurality of corresponding electronic devices in a three-dimensional environment, wherein, from the viewpoint of the first electronic device, the plurality of virtual representations have a first spatial arrangement in the three-dimensional environment, and wherein the first spatial arrangement is based on a first priority ordering data; and detecting, while presenting the plurality of virtual representations of the plurality of users of the plurality of corresponding electronic devices in the three-dimensional environment, the method further comprises: detecting, detecting, and detecting, the method further comprises: ... The change in the first priority sorting data; in response to detecting the change in the first priority sorting data, determining that the first priority sorting data meets one or more criteria based on the change in the first priority sorting data: updating the presentation of the plurality of virtual representations via the one or more displays to have a second spatial arrangement different from the first spatial arrangement; and sending corresponding display data to the second electronic device, the corresponding display data causing the second electronic device to update the presentation of the plurality of virtual representations via the one or more second displays of the second electronic device to have the second spatial arrangement; and determining that the first priority sorting data fails to meet the one or more criteria based on the change in the first priority sorting data, maintaining the presentation of the plurality of virtual representations with the first spatial arrangement in the three-dimensional environment; and abandoning the sending of the corresponding display data to the second electronic device.

[0131] Additionally or alternatively, in some examples, the three-dimensional environment includes a shared application window, and presenting the plurality of virtual representations of the plurality of users in the first spatial arrangement includes: presenting the plurality of virtual representations in the three-dimensional environment relative to the shared application window. Additionally or alternatively, in some examples, presenting the plurality of virtual representations relative to the shared application window includes: presenting the plurality of virtual representations in the three-dimensional environment from the viewpoint of the first electronic device within a column of virtual representations adjacent to the shared application window. Additionally or alternatively, in some examples, the three-dimensional environment does not include a shared application window, and presenting the plurality of virtual representations of the plurality of users in the first spatial arrangement includes: presenting the plurality of virtual representations within a virtual object in the three-dimensional environment. Additionally or alternatively, in some examples, presenting the plurality of virtual representations within a virtual object includes: presenting the plurality of virtual representations in multiple rows within the virtual object in the three-dimensional environment. Additionally or alternatively, in some examples, a first subgroup of users among the plurality of users is associated with one or more first priority sorting values ​​within the first priority sorting data, and a first or more virtual representations of the first subgroup of users are presented in a first row of the plurality of rows within the virtual object, and a second subgroup of users among the plurality of users, different from the first subgroup of users, is associated with one or more second priority sorting values ​​within the first priority sorting data, different from the one or more first priority sorting values, and a second or more virtual representations of the second subgroup of users are presented in a second row below the first row of the plurality of rows within the virtual object.

[0132] Additionally or alternatively, in some examples, from the viewpoint of the first electronic device, the first one or more virtual representations are presented in the first row at a first size, and from the viewpoint of the user, the second one or more virtual representations are presented in the second row at a second size smaller than the first size. Additionally or alternatively, in some examples, the one or more criteria include criteria satisfied based on the change in the first priority ranking data: when a corresponding user among the plurality of users associated with the one or more second priority ranking values ​​becomes associated with the one or more first priority ranking values, updating the presentation of the plurality of virtual representations to have the second spatial arrangement includes: moving the corresponding virtual representation of the corresponding user from the second row in the virtual object to the first row in the virtual object. Additionally or alternatively, in some examples, when the change in the first priority ranking data is detected at the first electronic device, the second electronic device is presenting the plurality of virtual representations in the first spatial arrangement in a second three-dimensional environment via the one or more second displays based on second priority ranking data having one or more characteristics of the first priority ranking data. Additionally or alternatively, in some examples, updating the rendering of the plurality of virtual representations to have the second spatial arrangement is also based on detecting input provided by the user of the first electronic device, via the one or more input devices, pointing to one or more virtual representations among the plurality of virtual representations in the three-dimensional environment. Additionally or alternatively, in some examples, the plurality of virtual representations of the plurality of users includes a first virtual representation of a first corresponding user of a third electronic device and a second virtual representation of a second corresponding user of a fourth electronic device different from the third electronic device, and based on the first priority sorting data, the first virtual representation of the first corresponding user occupies a first location within the first spatial arrangement, and the second virtual representation of the second corresponding user occupies a second location within the first spatial arrangement different from the first location. Additionally or alternatively, in some examples, the first corresponding user is associated with a first priority ranking value of the first priority ranking data, and the second corresponding user is associated with a second priority ranking value of the first priority ranking data that is different from the first priority ranking value; based on determining that the first priority ranking value corresponds to a higher priority than the second priority ranking value, within the first spatial arrangement, the first location of the first virtual representation is prioritized relative to the second location of the second virtual representation; and based on determining that the second priority ranking value corresponds to a higher priority than the first priority ranking value, within the first spatial arrangement, the second location of the second virtual representation is prioritized relative to the first location of the first virtual representation.

[0133] Additionally or alternatively, in some examples, the first priority ranking data is based on input data corresponding to user input provided by the plurality of users and detected by the plurality of corresponding electronic devices. Additionally or alternatively, in some examples, detecting the change in the first priority ranking data includes detecting a change in the input data indicating that one or more of the plurality of corresponding electronic devices have detected user input activity. Additionally or alternatively, in some examples, the one or more criteria include a criterion satisfied based on the change in the first priority ranking data when the user input activity detected by one of the plurality of corresponding electronic devices exceeds the user input activity detected by the other corresponding electronic devices. Additionally or alternatively, in some examples, the one or more criteria include a criterion satisfied based on the change in the first priority ranking data when a first corresponding electronic device among the plurality of corresponding electronic devices has detected user input provided by a first corresponding user of the first corresponding electronic device, the user input corresponding to a request to represent the first corresponding user of the first corresponding electronic device as a three-dimensional representation in the three-dimensional environment, the request causing a first virtual representation of the first corresponding user to cease rendering within the first spatial arrangement in the three-dimensional environment. Additionally or alternatively, in some examples, the plurality of virtual representations includes one or more virtual representations of one or more corresponding users who have been invited to join the multi-user communication session but have not yet joined the multi-user communication session. Additionally or alternatively, in some examples, the first priority ordering data causes the one or more virtual representations to occupy one or more lowest priority locations within the first spatial arrangement in the three-dimensional environment.Additionally or alternatively, in some examples, the method further includes: when in the multi-user communication session with the second electronic device and the plurality of corresponding electronic devices, and when, from the viewpoint of the first electronic device, the plurality of virtual representations of the plurality of users are presented in the three-dimensional environment with the second spatial arrangement, and wherein the first spatial arrangement is based on first priority ranking data, detecting a second change in the first priority ranking data; and in response to detecting the second change in the first priority ranking data: determining, based on the second change in the first priority ranking data, that the first priority ranking data meets the one or more criteria, updating the presentation of the plurality of virtual representations via the one or more displays to have a third spatial arrangement different from the second spatial arrangement, and sending second corresponding display data to the second electronic device, the second corresponding display data causing the second electronic device to update the presentation of the plurality of virtual representations via the one or more second displays of the second electronic device to have the third spatial arrangement; and determining, based on the second change in the first priority ranking data, that the first priority ranking data fails to meet the one or more criteria, maintaining the presentation of the plurality of virtual representations with the second spatial arrangement in the three-dimensional environment, and abandoning the sending of the second corresponding display data to the second electronic device. Additionally or alternatively, in some examples, the first electronic device, the second electronic device, and the plurality of respective electronic devices each include a head-mounted display.

[0134] Some examples of this disclosure relate to a method comprising: at a first electronic device communicating with one or more displays and one or more input devices, wherein the first electronic device is juxtaposed with a second electronic device in a physical environment; while in a multi-user communication session with the second electronic device and a plurality of corresponding electronic devices including a third electronic device, wherein the plurality of corresponding electronic devices are not juxtaposed with the first and second electronic devices in the physical environment, presenting, via the one or more displays, multiple virtual representations of multiple users of the plurality of corresponding electronic devices in a three-dimensional environment, wherein, from the viewpoint of the first electronic device, the plurality of virtual representations have a first spatial arrangement in the three-dimensional environment; and when presenting the plurality of virtual representations of the plurality of users of the plurality of corresponding electronic devices in the three-dimensional environment, detecting the presence of... The instruction received at the third electronic device among the plurality of corresponding electronic devices, the input corresponding to a request to represent the user of the third electronic device as a three-dimensional representation of the user in the three-dimensional environment that is different from the first virtual representation of the user of the third electronic device, wherein the first virtual representation of the user of the third electronic device occupies a first position within the first spatial arrangement; and in response to detecting the instruction to the input: stop displaying the first virtual representation among the plurality of virtual representations in the three-dimensional environment via the one or more displays; and display the three-dimensional representation of the user of the third electronic device in the three-dimensional environment at a placement position relative to the plurality of virtual representations, wherein the placement position in the three-dimensional environment is based on the first position of the first virtual representation within the first spatial arrangement.

[0135] Additionally or alternatively, in some examples, the three-dimensional environment includes a shared application window, and presenting the plurality of virtual representations of the plurality of users in the first spatial arrangement includes: presenting the plurality of virtual representations in the three-dimensional environment relative to the shared application window. Additionally or alternatively, in some examples, presenting the plurality of virtual representations relative to the shared application window includes: presenting the plurality of virtual representations in the three-dimensional environment, from the viewpoint of the first electronic device, within a column of virtual representations adjacent to the shared application window. Additionally or alternatively, in some examples, from the viewpoint of the first electronic device, the first positioning within the first spatial arrangement corresponds to the first position within the column of virtual representations in the three-dimensional environment. Additionally or alternatively, in some examples, the three-dimensional environment does not include a shared application window, and presenting the plurality of virtual representations of the plurality of users in the first spatial arrangement includes: presenting the plurality of virtual representations within a virtual object in the three-dimensional environment. Additionally or alternatively, in some examples, presenting the plurality of virtual representations within a virtual object includes: presenting the plurality of virtual representations in multiple rows within the virtual object in the three-dimensional environment. Additionally or alternatively, in some examples, the first positioning within the first spatial arrangement corresponds to a first position within a first row of the multiple rows within the virtual object. Additionally or alternatively, in some examples, the placement position in the three-dimensional environment corresponds to a corresponding position in the three-dimensional environment, which, from the viewpoint of the first electronic device, is within a threshold distance of the first positioning within the first spatial arrangement.

[0136] Additionally or alternatively, in some examples, based on determining that the corresponding position in the three-dimensional environment within the threshold distance of the first positioning is occupied by a corresponding user of a corresponding electronic device in the multi-user communication session that is different from the third electronic device, the placement position in the three-dimensional environment corresponds to a second corresponding position within the threshold distance of the first positioning that is different from the corresponding position. Additionally or alternatively, in some examples, presenting the three-dimensional representation of the user of the third electronic device at the placement position in the three-dimensional environment includes: based on determining that the corresponding position in the three-dimensional environment within the threshold distance of the first positioning is occupied by a second three-dimensional representation of a corresponding user of a corresponding electronic device among a plurality of corresponding electronic devices that is different from the third electronic device, updating the presentation of the second three-dimensional representation of the corresponding user so that, from the viewpoint of the first electronic device, the corresponding position in the three-dimensional environment is no longer occupied. Additionally or alternatively, in some examples, presenting the three-dimensional representation of the user of the third electronic device at the placement location in the three-dimensional environment includes: determining that a corresponding position in the three-dimensional environment within the threshold distance of the first positioning is occupied by a second three-dimensional representation of the user of a corresponding electronic device different from the third electronic device; moving the plurality of virtual representations in the three-dimensional environment to generate a second corresponding position in the three-dimensional environment, the second corresponding position being within the threshold distance of the first positioning from the viewpoint of the first electronic device within the first spatial arrangement; and presenting the three-dimensional representation of the user of the third electronic device at the second corresponding position in the three-dimensional environment. Additionally or alternatively, in some examples, when an indication corresponding to an input received at the third electronic device among the plurality of corresponding electronic devices as a three-dimensional representation of the user of the third electronic device in the three-dimensional environment is detected at the first electronic device, the second electronic device presents the plurality of virtual representations in the first spatial arrangement in the second three-dimensional environment via one or more second displays of the second electronic device.

[0137] Additionally or alternatively, in some examples, when the three-dimensional representation of the user of the third electronic device is presented at the placement location in the three-dimensional environment at the first electronic device in response to the detection of the instruction, the second electronic device presents the three-dimensional representation of the user of the third electronic device at the placement location relative to the plurality of virtual representations in the second three-dimensional environment via the one or more second displays. Additionally or alternatively, in some examples, the plurality of virtual representations includes a second virtual representation of the user of a fourth electronic device different from that of the third electronic device. In some examples, the method further includes: in response to the detection of the instruction on the input, after stopping the presentation of the first virtual representation among the plurality of virtual representations in the three-dimensional environment, moving the second virtual representation of the user of the fourth electronic device to the first location within the first spatial arrangement in the three-dimensional environment. Additionally or alternatively, in some examples, the method further includes: when in the multi-user communication session with the second electronic device and the plurality of corresponding electronic devices including the third electronic device, and when the plurality of virtual representations of the plurality of users and the three-dimensional representation of the user of the third electronic device are presented in the three-dimensional environment, detecting a second indication of input received at a fourth electronic device among the plurality of corresponding electronic devices that is different from the third electronic device, the input corresponding to a request in the three-dimensional environment to represent the user of the fourth electronic device as a three-dimensional representation of the user of the fourth electronic device that is different from the second virtual representation of the user of the fourth electronic device, wherein the second virtual representation of the user of the fourth electronic device occupies a second position within the first spatial arrangement; and in response to detecting the second indication of the input, stopping the presentation of the second virtual representation among the plurality of virtual representations in the three-dimensional environment via the one or more displays, and presenting the three-dimensional representation of the user of the fourth electronic device at a second placement position relative to the plurality of virtual representations in the three-dimensional environment via the one or more displays, the second placement position relative to the plurality of virtual representations in the three-dimensional environment, wherein the second placement position in the three-dimensional environment is based on the second position of the second virtual representation within the first spatial arrangement. Additionally or alternatively, in some examples, the first electronic device, the second electronic device, and a plurality of respective electronic devices including the third electronic device each include a head-mounted display.

[0138] Some examples of this disclosure relate to a first electronic device comprising: one or more processors; a memory; and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described above.

[0139] Some examples of this disclosure relate to a non-transitory computer-readable storage medium that stores one or more programs, the one or more programs including instructions that, when executed by one or more processors of a first electronic device, cause the first electronic device to perform any of the methods described above.

[0140] Some examples of this disclosure relate to a first electronic device that includes one or more processors, a memory, and components for performing any of the methods described above.

[0141] Some examples of this disclosure relate to an information processing apparatus for use in a first electronic device, the information processing apparatus including components for performing any of the methods described above.

[0142] For purposes of explanation, the foregoing description has been presented with reference to specific examples. However, the illustrative discussion above is not intended to be exhaustive or to limit this disclosure to the precise form disclosed. Many modifications and variations are possible based on the teachings above. The examples were chosen and described to best elucidate the principles of this disclosure and its practical application, thereby enabling others skilled in the art to make optimal use of this disclosure with various modifications suitable for the particular intended purpose, as well as the various described examples.

Claims

1. A method, the method comprising: At a first electronic device that communicates with one or more displays and one or more input devices, wherein the first electronic device is placed alongside the second electronic device in a physical environment: When in a multi-user communication session with the second electronic device and a plurality of corresponding electronic devices, wherein the plurality of corresponding electronic devices are not juxtaposed with the first electronic device and the second electronic device in the physical environment, multiple virtual representations of the plurality of corresponding electronic devices for multiple users are presented in a three-dimensional environment via the one or more displays, wherein, from the viewpoint of the first electronic device, the plurality of virtual representations have a first spatial arrangement in the three-dimensional environment, and wherein the first spatial arrangement is based on a first priority sorting data; When the multiple virtual representations of the multiple users of the multiple corresponding electronic devices are presented in the three-dimensional environment, changes in the first priority ranking data are detected; and In response to the detection of the change in the first priority sorting data: Based on the changes in the first priority sorting data, it is determined that the first priority sorting data meets one or more criteria: Update the presentation of the plurality of virtual representations via the one or more displays to have a second spatial arrangement that is different from the first spatial arrangement; as well as Send corresponding display data to the second electronic device, the corresponding display data causing the second electronic device to update the presentation of the plurality of virtual representations via one or more second displays of the second electronic device to have the second spatial arrangement; as well as Based on the changes in the first priority ranking data, it is determined that the first priority ranking data fails to meet one or more of the following criteria: Maintaining the presentation of the plurality of virtual representations having the first spatial arrangement within the three-dimensional environment; and The transmission of the corresponding display data to the second electronic device is abandoned.

2. The method according to claim 1, wherein: The three-dimensional environment includes a shared application window; and Presenting the multiple virtual representations of the multiple users in the first spatial arrangement includes: presenting the multiple virtual representations relative to the shared application window in the three-dimensional environment.

3. The method of claim 2, wherein presenting the plurality of virtual representations relative to the shared application window comprises: In the three-dimensional environment, the plurality of virtual representations are presented from the viewpoint of the first electronic device in a column of virtual representations adjacent to the shared application window.

4. The method according to claim 1, wherein: The three-dimensional environment does not include shared application windows; and Presenting the multiple virtual representations of the multiple users in the first spatial arrangement includes presenting the multiple virtual representations within virtual objects in the three-dimensional environment.

5. The method of claim 4, wherein presenting the plurality of virtual representations within the virtual object comprises: The multiple virtual representations are presented in multiple rows within the virtual object in the three-dimensional environment.

6. The method according to claim 5, wherein: A first subgroup of users among the plurality of users is associated with one or more first priority ranking values ​​within the first priority ranking data, and a first or more virtual representations of the first subgroup of users are presented in the first row of the plurality of rows within the virtual object; and A second subgroup user, which is different from the first subgroup user, is associated with one or more second priority sort values ​​in the first priority sort data that are different from the one or more first priority sort values, and a second or more virtual representation of the second subgroup user is presented in a second row below the first row in the multiple rows of the virtual object.

7. The method according to claim 6, wherein: From the viewpoint of the first electronic device, the first one or more virtual representations are presented in the first row at a first size; and From the viewpoint of the first electronic device, the second or more virtual representations are presented in the second row at a second size smaller than the first size.

8. The method according to claim 6, wherein: The one or more criteria include those satisfied based on the changes in the first priority ranking data when a corresponding user among the plurality of users associated with the one or more second priority ranking values ​​becomes associated with the one or more first priority ranking values; and Updating the presentation of the plurality of virtual representations to have the second spatial arrangement includes: moving the corresponding virtual representation of the corresponding user from the second row in the virtual object to the first row in the virtual object.

9. The method of claim 1, wherein, When the change in the first priority sorting data is detected at the first electronic device, the second electronic device is presenting the plurality of virtual representations in the first spatial arrangement in a second three-dimensional environment via the one or more second displays based on second priority sorting data having one or more characteristics of the first priority sorting data.

10. The method according to claim 1, wherein: The plurality of virtual representations of the plurality of users include a first virtual representation of a first corresponding user of a third electronic device and a second virtual representation of a second corresponding user of a fourth electronic device, which is different from the third electronic device; and Based on the first priority sorting data, the first virtual representation of the first corresponding user occupies a first position within the first spatial arrangement, and the second virtual representation of the second corresponding user occupies a second position within the first spatial arrangement that is different from the first position.

11. The method of claim 10, wherein: The first corresponding user is associated with a first priority ranking value of the first priority ranking data, and the second corresponding user is associated with a second priority ranking value of the first priority ranking data that is different from the first priority ranking value. Based on the determination that the first priority ranking value corresponds to a higher priority than the second priority ranking value, within the first spatial arrangement, the first location of the first virtual representation is prioritized relative to the second location of the second virtual representation; and Based on the determination that the second priority sorting value corresponds to a higher priority than the first priority sorting value, within the first spatial arrangement, the second location of the second virtual representation is prioritized relative to the first location of the first virtual representation.

12. The method of claim 1, wherein the first priority sorting data is based on input data corresponding to user inputs provided by the plurality of users and detected by the plurality of corresponding electronic devices.

13. The method of claim 12, wherein detecting the change in the first prioritization data comprises: Detect changes in the input data, the changes indicating that one or more of the plurality of corresponding electronic devices have detected user input activity.

14. The method of claim 13, wherein the one or more criteria include criteria satisfied based on the change in the first priority sorting data when the user input activity detected by one of the plurality of corresponding electronic devices exceeds the user input activity detected by the other corresponding electronic devices among the plurality of corresponding electronic devices.

15. The method of claim 1, wherein the one or more criteria include criteria satisfied based on the change in the first priority sorting data: when a first corresponding electronic device among the plurality of corresponding electronic devices has detected user input provided by a first corresponding user of the first corresponding electronic device, the user input corresponds to a request to represent the first corresponding user of the first corresponding electronic device as a three-dimensional representation in the three-dimensional environment, the request causing a first virtual representation of the first corresponding user to stop being rendered within the first spatial arrangement in the three-dimensional environment.

16. The method of claim 1, wherein the plurality of virtual representations includes one or more virtual representations of one or more corresponding users who have been invited to join the multi-user communication session but have not yet joined the multi-user communication session.

17. The method of claim 16, wherein the first priority sorting data causes the one or more virtual representations to occupy one or more lowest priority locations within the first spatial arrangement in the three-dimensional environment.

18. The method according to claim 1, further comprising: When in a multi-user communication session with the second electronic device and the plurality of corresponding electronic devices, and when, from the viewpoint of the first electronic device, the plurality of virtual representations of the plurality of users are presented in the three-dimensional environment in the second spatial arrangement, and wherein the first spatial arrangement is based on first priority ranking data, a second change in the first priority ranking data is detected; and In response to detecting the second change in the first priority sorting data: Based on the second change in the first priority sorting data, it is determined that the first priority sorting data meets one or more criteria: The rendering of the plurality of virtual representations via the one or more displays is updated to have a third spatial arrangement that is different from the second spatial arrangement; as well as Send second corresponding display data to the second electronic device, the second corresponding display data causing the second electronic device to update the presentation of the plurality of virtual representations via one or more second displays of the second electronic device to have the third spatial arrangement; as well as Based on the second change in the first priority sorting data, it is determined that the first priority sorting data fails to meet one or more of the criteria: Maintaining the presentation of the plurality of virtual representations having the second spatial arrangement within the three-dimensional environment; and The transmission of the second corresponding display data to the second electronic device is abandoned.

19. A first electronic device, the first electronic device comprising: One or more processors; Memory; and One or more programs, the programs being stored in the memory and configured to be executed by the one or more processors, the programs including instructions for performing the method according to any one of claims 1 to 18.

20. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by one or more processors of a first electronic device, cause the first electronic device to perform the method according to any one of claims 1 to 18.

Images