SURFACE-BASED INTERACTIONS AND PROCESSES IN MULTI-USER COMMUNICATION SESSIONS

Abstract

Some examples of the disclosure relate to systems and methods for displaying virtual content relative to a surface of a three-dimensional environment in a multi-user communication session. Some examples of the disclosure relate to systems and methods for moving virtual content in a three-dimensional environment relative to a surface of the three-dimensional environment in a multi-user communication session.

Description

CROSS-REFERENCE TO RELATED REGISTRATIONS

[0001] This application claims the benefits of U.S. Preliminary Application No. 63 / 698,847, filed on September 25, 2024, U.S. Preliminary Application No. 63 / 817,642, filed on June 4, 2025, and U.S. Preliminary Application No. 19 / 299,087, filed on August 13, 2025, the contents of which are incorporated herein by reference in their entirety for all purposes. AREA OF REVELATION

[0002] This document generally refers to systems and methods for establishing spatial truth (e.g., based on synchronized reference points) and for facilitating vertical movement of virtual content based on surfaces (e.g., physical surfaces) contained in a three-dimensional environment in multi-user communications. BACKGROUND OF THE REVELATION

[0003] Some computer graphics environments provide two-dimensional and / or three-dimensional environments in which at least some objects displayed to a user are virtual and generated by a computer. In some examples, the three-dimensional environments are presented by multiple devices communicating in a multi-user communication session. In some examples, an avatar (e.g., a representation) of each user not located in the same place who participates in the multi-user communication session (e.g., via the computing devices) is displayed in the three-dimensional environment of the multi-user communication session. In some examples, the content in the three-dimensional environment can be shared for viewing and interaction by multiple users participating in the multi-user communication session. SUMMARY OF THE REVELATION

[0004] Some examples of the disclosure relate to systems and methods for displaying virtual content relative to a surface of a three-dimensional environment in a multi-user communication session. In some examples, a method is performed on an electronic device that communicates with one or more displays and one or more input devices.In some examples, during a communication session with a second electronic device, and while a visual representation of a user of the second electronic device in a three-dimensional environment is displayed on one or more displays, the first electronic device receives a first specification of a request to share content of a first type in the communication session. The first electronic device then displays a first object corresponding to the content of the first type in the three-dimensional environment on one or more displays.In some examples, while displaying the first object and the user of the second electronic device is visualizing it in the three-dimensional environment, the first electronic device receives a second instruction requesting that the first object be linked to a second surface, distinct from the first, within the three-dimensional environment. In some examples, in response to receiving this second instruction, the first electronic device links the first object to the second surface in the three-dimensional environment.In some examples, the first electronic device updates the user's visual representation of the second electronic device via one or more displays, so that it is displayed from the viewpoint of the first electronic device at a second height that differs from the first height, relative to the second surface in the three-dimensional environment.

[0005] Some examples of the disclosure relate to systems and methods for moving virtual content in a three-dimensional environment relative to a surface of the three-dimensional environment in a multi-user communication session. In some examples, a method is performed on an electronic device that communicates with one or more displays and one or more input devices. In some examples, during a communication session with a second electronic device, the first electronic device displays, via the one or more displays, a visual representation of a user of the second electronic device and a first object corresponding to shared content in a three-dimensional environment, wherein the visual representation is displayed from a viewpoint of the first electronic device at a first height relative to a first surface in the three-dimensional environment.In some examples, while displaying the user's visual representation of the first object in the three-dimensional environment to the user of the second electronic device, the first electronic device detects an initial input via one or more input devices. This initial input is a request to move the first object vertically in the three-dimensional environment relative to the viewing angle of the first electronic device. In some examples, in response to the detection of the initial input, the first electronic device moves the first object and the user's visual representation of the second electronic device vertically in the three-dimensional environment relative to the viewing angle of the first electronic device, according to the initial input.In some examples, the first electronic device displays, via one or more displays, the user's visual representation of the second electronic device at a second height that differs from the first height, relative to the first surface in the three-dimensional environment.

[0006] The complete descriptions of these examples are included in the drawings and the detailed description, and it is understood that this summary does not in any way limit the scope of the revelation. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] For a better understanding of the various examples described herein, please refer to the detailed description below, along with the following drawings. The same reference numerals throughout the drawings often refer to corresponding parts. Fig. Figure 1 illustrates an electronic device that represents an augmented reality environment, according to some examples from the revelation. Fig. Figure 2 illustrates a block diagram of an exemplary architecture for a system according to some examples from the Revelation. Fig. Figure 3 illustrates an example of a spatial group in a multi-user communication session, which includes a first electronic device and a second electronic device according to some examples of the disclosure. Fig. 4A to 4I illustrate examples of presenting content in a three-dimensional environment relative to surfaces in the three-dimensional environment in a multi-user communication session according to some examples of the revelation. Fig. 5A to 5K illustrate examples of vertical movement of shared content in a three-dimensional environment relative to surfaces in the three-dimensional environment in a multi-user communication session according to some examples of the revelation. Fig. Figure 6 illustrates a flowchart that demonstrates an exemplary process for presenting content in a three-dimensional environment relative to surfaces in the three-dimensional environment in a multi-user communication session according to some examples of the Revelation. Fig. Figure 7 illustrates a flowchart that demonstrates an exemplary process for virtually moving shared content in a three-dimensional environment relative to surfaces in the three-dimensional environment in a multi-user communication session according to some examples from the Revelation. DETAILED DESCRIPTION

[0008] Some examples of the disclosure relate to systems and methods for displaying virtual content relative to a surface of a three-dimensional environment in a multi-user communication session. In some examples, a method is performed on an electronic device that communicates with one or more displays and one or more input devices.In some examples, during a communication session with a second electronic device, and while a visual representation of a user of the second electronic device in a three-dimensional environment is displayed on one or more displays, the first electronic device receives a first specification of a request to share content of a first type in the communication session. The first electronic device then displays a first object corresponding to the content of the first type in the three-dimensional environment on one or more displays.In some examples, while displaying the first object and the user of the second electronic device is visualizing it in the three-dimensional environment, the first electronic device receives a second instruction requesting that the first object be linked to a second surface, distinct from the first, within the three-dimensional environment. In some examples, in response to receiving this second instruction, the first electronic device links the first object to the second surface in the three-dimensional environment.In some examples, the first electronic device updates the user's visual representation of the second electronic device via one or more displays, so that it is displayed from the viewpoint of the first electronic device at a second height that differs from the first height, relative to the second surface in the three-dimensional environment.

[0009] Some examples of the disclosure relate to systems and methods for moving virtual content in a three-dimensional environment relative to a surface of the three-dimensional environment in a multi-user communication session. In some examples, a method is performed on an electronic device that communicates with one or more displays and one or more input devices. In some examples, during a communication session with a second electronic device, the first electronic device displays, via the one or more displays, a visual representation of a user of the second electronic device and a first object corresponding to shared content in a three-dimensional environment, wherein the visual representation is displayed from a viewpoint of the first electronic device at a first height relative to a first surface in the three-dimensional environment.In some examples, while displaying the user's visual representation of the first object in the three-dimensional environment to the user of the second electronic device, the first electronic device detects an initial input via one or more input devices. This initial input is a request to move the first object vertically in the three-dimensional environment relative to the viewing angle of the first electronic device. In some examples, in response to the detection of the initial input, the first electronic device moves the first object and the user's visual representation of the second electronic device vertically in the three-dimensional environment relative to the viewing angle of the first electronic device, according to the initial input.In some examples, the first electronic device displays, via one or more displays, the user's visual representation of the second electronic device at a second height that differs from the first height, relative to the first surface in the three-dimensional environment.

[0010] As used herein, a spatial group corresponds to a group or number of participants (e.g., users) in a multi-user communication session. In some examples, a spatial group in the multi-user communication session has a spatial arrangement that dictates the locations of users and content within the spatial group. In some examples, users within the same spatial group in the multi-user communication session experience spatial truth according to the spatial arrangement of the spatial group. In some examples, spatial truth requires a consistent spatial arrangement between users (or their representations) and virtual objects.

[0011] As used herein, the movement of a shared virtual object in a multi-user communication session triggers spatial refinement in a shared three-dimensional environment of the multi-user communication session. In some examples, the spatial refinement corresponds to the movement and / or repositioning of avatars and / or shared objects (e.g., triggered by the movement of a shared object), thereby maintaining spatial truth within a given spatial group of users, as discussed in detail below.

[0012] In some examples, initiating a multi-user communication session may involve interaction with one or more user interface elements. In some examples, a user's gaze may be tracked by an electronic device as input for orienting a selectable option / affordance within a corresponding user interface element displayed in the three-dimensional environment. The gaze may be used, for example, to identify one or more option(s) / affordance(s) to be selected using another selection input. In some examples, a particular option / affordance may be selected using a hand-tracking input detected by an input device connected to the electronic device.In some examples, objects displayed in the three-dimensional environment can be moved and / or reoriented in the three-dimensional environment in accordance with the motion input detected by the device.

[0013] Fig. Figure 1 illustrates an electronic device 101 that presents an augmented reality (XR) environment (e.g., a computer-generated environment that optionally includes representations of physical and / or virtual objects) according to some examples of the disclosure. In some examples, as in Fig. As shown in Figure 1, the electronic device 101 is a head-mounted display or other head-mounted device configured to be worn on the head of a user of the electronic device 101. Examples of the electronic device 101 are given below with reference to the architecture of the block diagram from Figure 1. Fig. 2 described. As in Fig. As shown in Figure 1, the electronic device 101 and the table 106 are located in a physical environment. The physical environment can include physical features such as a physical surface (e.g., floor, walls) or a physical object (e.g., table, lamp, etc.). In some embodiments, the electronic device 101 can be configured to detect and / or capture images of the physical environment, including the table 106 (illustrated in the field of view of the electronic device 101).

[0014] In some examples, such as in Fig. As shown in Figure 1, the electronic device 101 includes one or more internal image sensors 114a oriented towards the user's face (e.g., eye-tracking cameras, which are described below in relation to Fig. 2 (described). In some examples, internal image sensors 114a are used for eye tracking (e.g., to detect the user's gaze). Internal image sensors 114a are optionally arranged in the left and right areas of the display 120 to enable eye tracking of the user's left and right eyes. In some examples, the electronic device 101 also includes external image sensors 114b and 114c, which are directed outwards from the user, to detect and / or capture the physical environment of the electronic device 101 and / or movements of the user's hands or other body parts.

[0015] In some examples, the display 120 has a user-visible field of view (e.g., which may or may not correspond to the field of view of the external image sensors 114b and 114c). Since display 120 is optionally part of a head-mounted device, its field of view is optionally the same as, or similar to, the user's eye field of view. In other examples, the field of view of display 120 may be smaller than the user's eye field of view. In some examples, the electronic device 101 may be an optically transparent device, in which display 120 is a transparent or translucent display through which parts of the physical environment can be directly viewed. In some examples, display 120 may be contained within a transparent lens and overlap all or part of the transparent lens.In other examples, the electronic device can be a video passthrough device, where the display 120 is an opaque display configured to show images of the physical environment captured by the external image sensors 114b and 114c. Although only a single display 120 is shown, it is understood that display 120 can also include a stereo pair of displays.

[0016] In some examples, the electronic device 101 can be configured, in response to a trigger, to display a virtual object 104 in the XR environment, which is defined by a Fig. The illustrated cube is shown, which does not exist in the physical environment but is displayed positioned on the tabletop of the real table 106 (or a representation thereof) in the XR environment. Optionally, the virtual object 104 can be displayed on the surface of the table 106 in the XR environment via the display 120 of the electronic device 101 in response to the detection of the flat surface of the table 106 in the physical environment 100.

[0017] It is understood that virtual object 104 is a representative virtual object, and one or more different virtual objects (e.g., of different dimensions, such as two-dimensional or other three-dimensional virtual objects) can be enclosed and rendered within a three-dimensional XR environment. For example, the virtual object can represent an application or a user interface displayed in the XR environment. In some examples, the virtual object can represent content corresponding to the application and / or displayed via the user interface in the XR environment. In some examples, virtual object 104 is optionally configured to be interactive and responsive to user input (e.g., air gestures such as pinch air gestures, tap air gestures, and / or touch air gestures), allowing a user to virtually touch, tap, move, rotate, or otherwise interact with virtual object 104.

[0018] In some examples, displaying an object in a three-dimensional environment may involve interaction with one or more user interface objects within that environment. For instance, initiating the display of the object in the three-dimensional environment may involve interaction with one or more virtual options / affordances displayed within that environment. In some examples, a user's gaze may be tracked as input to identify one or more virtual options / affordances available for selection when the display of an object in the three-dimensional environment is initiated. The gaze may, for example, be used to identify one or more virtual option(s) / affordance(s) to be selected using further selection input.In some examples, a virtual option / affordance can be selected using hand tracking input, detected via an input device connected to the electronic device. In some examples, objects displayed in the three-dimensional environment can be moved and / or reoriented in the three-dimensional environment in accordance with the motion input detected by the device.

[0019] The following description outlines an electronic device that communicates with a display generation component and one or more input devices. It is understood that the electronic device optionally communicates with one or more other physical user interface devices, such as a touch-sensitive surface, a physical keyboard, a mouse, a joystick, a hand tracking device, an eye tracking device, a stylus, etc. Furthermore, as described above, it is understood that the described electronic device, the display, and the touch-sensitive surface are optionally distributed among two or more devices.Therefore, as used in this disclosure, information displayed on or by the electronic device is optionally used to describe information output by the electronic device for display on a separate display device (touch-sensitive or not). Similarly, as used in this disclosure, input received by the electronic device (e.g., touch input received on a touch-sensitive surface of the electronic device or touch input received on the surface of an input pen) is optionally used to describe input received on a separate input device from which the electronic device receives input information.

[0020] The device typically supports a variety of applications, such as one or more of the following: a drawing application, a presentation application, a word processing application, a website creation application, a disc authoring software development application, a spreadsheet application, a gaming application, a telephone application, a video conferencing application, an email application, an instant messaging application, a training support application, a photo management application, a digital camera application, a digital video camera application, an internet browsing application, a digital music playback application, a television channel browsing application, and / or a digital video playback application.

[0021] Fig. Figure 2 illustrates a block diagram of an exemplary architecture for a System 201 according to some examples of the disclosure. In some examples, the System 201 includes several devices. For example, the System 201 includes a first electronic device 260 and a second electronic device 270, wherein the first electronic device 260 and the second electronic device 270 are in communication with each other. In some examples, the first electronic device 260 and the second electronic device 270 are portable devices, such as a mobile phone, a smartphone, a tablet computer, a laptop computer, an auxiliary device in communication with another device, a head-mounted display, etc. In some examples, the first electronic device 260 and the second electronic device 270 correspond to the above with respect to Fig. 1 described electronic device 101.

[0022] As in Fig. As illustrated in Figure 2, the first device 260 optionally includes various sensors (e.g., one or more hand tracking sensors 202A, one or more position sensors 204A, one or more image sensors 206A, one or more touch-sensitive surfaces 209A, one or more motion and / or orientation sensors 210A, one or more eye tracking sensors 212A, one or more microphones 213A or other audio sensors, one or more body tracking sensors (e.g., upper body and / or head tracking sensors), one or more display generating components 214A, one or more loudspeakers 216A, one or more processors 218A, one or more memories 220A, and / or the communication circuit 222A). In some examples, the second electronic device 270 optionally includes various sensors (e.g.,one or more hand tracking sensors 202B, one or more position sensors 204B, one or more image sensors 206B, one or more touch-sensitive surfaces 209B, one or more motion and / or orientation sensors 210B, one or more eye tracking sensors 212B, one or more microphones 213B or other audio sensors, one or more body tracking sensors (e.g., upper body and / or head tracking sensors), one or more display generation components 214B, one or more loudspeakers 216, one or more processors 218B, one or more memories 220B, and / or the communication circuit 222B. In some examples, the one or more display generation components 214A, 214B correspond to the display 120 in . Fig. 1. One or more communication buses 208A and 208B are optionally used for communication between the aforementioned components of the electronic devices 260 and 270, respectively. The first electronic device 260 and the second electronic device 270 optionally communicate via a wired or wireless connection (e.g., via the communication circuits 222A, 222B) between the two devices.

[0023] The 222A and 222B communication switching logic optionally includes switching logic for communicating with electronic devices and networks such as the Internet, intranets, wired and / or wireless networks, cellular networks, and wireless local area networks (LANs). The 222A and 222B communication switching logic also optionally includes switching logic for communicating using near-field communication (NFC) and / or short-range communication such as Bluetooth®.

[0024] The one or more processors 218A, 218B include one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, the memory 220A, 220B is a non-transitory, computer-readable storage medium (e.g., flash memory, random-access memory, or other volatile or non-volatile memory or storage) that stores computer-readable instructions configured to be executed by one or more processors 218A, 218B to perform the techniques, processes, and / or procedures described below. In some examples, the memory 220A, 220B may include more than one non-transitory, computer-readable storage medium. A non-transitory, computer-readable storage medium can be any medium (e.g.,(with the exception of a signal), which can contain or store tangible computer-executable instructions for use by, or in connection with, the instruction execution system, instruction execution device, or instruction execution apparatus. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transient computer-readable storage medium. The non-transient computer-readable storage medium may include, but is not limited to, magnetic, optical, and / or semiconductor storage. Examples of such storage include magnetic disks, optical disks based on Compact Disc (CD), Digital Versatile Disc (DVD), or Blu-ray technologies, and persistent semiconductor storage such as flash memory, solid-state drives, and the like.

[0025] In some examples, one or more display generating components 214A, 214B include a single display (e.g., a liquid crystal display (LCD), organic light-emitting diode (OLED), or other types of displays). In some examples, one or more display generating components 214A, 214B include multiple displays. In some examples, one or more display generating components 214A, 214B may include a touch-enabled display (e.g., a touchscreen), a projector, a holographic projector, a retinal projector, a transparent or translucent display, etc. In some examples, the electronic devices 260 and 270 include touch-sensitive surface(s) 209A and 209B for receiving user input such as typing, swiping, or other gestures.In some examples, the display generating component(s) 214A, 214B and the touch-sensitive surface(s) 209A, 209B form touch-sensitive displays (e.g., a touchscreen integrated into or located outside the electronic devices 260 and 270, respectively, which communicates with the electronic devices 260 and 270).

[0026] The electronic devices 260 and 270 optionally include image sensor(s) 206A and 206B, respectively. The one or more image sensors 206A / 206B optionally include one or more visible light image sensors, such as CCD (charge-coupled device) sensors and / or CMOS (complementary metal-oxide semiconductor) sensors, which can be operated to obtain images of physical objects from the real environment. The image sensor(s) 206A / 206B also optionally include one or more infrared (IR) sensors, such as a passive or an active IR sensor, for detecting infrared light from the real environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real environment. The 206A / 206B image sensor(s) optionally also include one or more cameras configured to detect movement of physical objects in the real environment.The image sensor(s) 206A / 206B optionally include one or more depth sensors configured to detect the distance of physical objects from the electronic device 260 / 270. In some examples, information from one or more depth sensors can enable the device to identify objects in the real-world environment and distinguish them from other objects in the real-world environment. In some examples, one or more depth sensors can enable the device to determine the texture and / or topography of objects in the real-world environment.

[0027] In some examples, the electronic devices 260 and / or 270 use CCD sensors, event cameras, and depth sensors in combination to detect the physical environment around the electronic devices 260 and / or 270. In some examples, the image sensor(s) 206A / 206B include a first image sensor and a second image sensor. The first and second image sensors work together and are optionally configured to capture different information about physical objects in the real-world environment. In some examples, the first image sensor is a visible light image sensor, and the second image sensor is a depth sensor. In some examples, the electronic device 260 / 270 uses image sensor(s) 206A / 206B to detect the position and orientation of the device 260 / 270 and / or the display generating component(s) 214A / 214B in the real-world environment.For example, the electronic device 260 / 270 uses image sensor(s) 206A / 206B to track the position and orientation of the display generating component(s) 214A / 214B relative to one or more fixed objects in the real environment.

[0028] In some examples, the electronic device 260 / 270 includes microphone(s) 213A / 213B or other audio sensors. The device 260 / 270 uses the microphone(s) 213A / 213B to detect sound from the user and / or the user's real-world environment. In some examples, the microphone(s) 213A / 213B include an array of microphones (a plurality of microphones) that optionally work together to identify ambient noise or to localize the sound source within the real-world environment.

[0029] In some examples, the device 260 / 270 includes one or more location sensors 204A / 204B to detect the location of the device 260 / 270 and / or the display generating component(s) 214A / 214B. The location sensor(s) 204A / 204B may include a GPS (Global Positioning System) receiver that receives data from one or more satellites and enables the electronic device 260 / 270 to determine the absolute position of the device in the physical world.

[0030] In some examples, the electronic device 260 / 270 includes one or more orientation sensors 210A / 210B to detect the orientation and / or movement of the device 260 / 270 and / or the display generating component(s) 214A / 214B. For example, the electronic device 260 / 270 uses one or more orientation sensors 210A / 210B to track changes in the position and / or orientation of the electronic device 260 / 270 and / or the one or more display generating component(s) 214A / 214B, as with respect to physical objects in the real environment. The orientation sensor(s) 210A / 210B optionally include one or more gyroscopes and / or one or more accelerometers.

[0031] The electronic device 260 / 270 includes, in some examples, hand tracking sensor(s) 202A / 202B and / or eye tracking sensor(s) 212A / 212B (and / or other body tracking sensor(s), such as leg, torso, and / or head tracking sensor(s)). The one or more hand tracking sensors 202A / 202B are configured to track the position / location of one or more sections of the user's hands and / or movements of one or more sections of the user's hands in relation to the cross-reality environment relative to the one or more display-generating components 214A / 214A and / or relative to another defined coordinate system.The one or more eye-tracking sensors 212A / 212B are configured to track the position and movement of a user's gaze (eyes, face, or generally the head) with respect to the real world or the cross-reality environment and / or relative to the one or more display-generating components 214A / 214A. In some examples, the one or more hand-tracking sensors 202A / 202B and / or the one or more eye-tracking sensors 212A / 212B are implemented together with the one or more display-generating components 214A / 214A. In other examples, the one or more hand-tracking sensors 202A / 202B and / or the one or more eye-tracking sensors 212A / 212B are implemented separately from the one or more display-generating components 214A / 214A.

[0032] In some examples, one or more hand tracking sensors 202A / 202B (and / or other body tracking sensors such as leg, torso, and / or head tracking sensors) can use one or more image sensors 206A / 206B (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that detect three-dimensional information from the real world, including one or more body parts (e.g., hands, legs, or torso of a human user). In some examples, the hands can be resolved with sufficient resolution to distinguish fingers and their respective positions.In some examples, one or more image sensors 206A / 206B are positioned relative to the user to define a field of view for the one or more image sensors 206A / 206B and an interaction space in which finger / hand position, orientation, and / or movement captured by the image sensors are used as input (e.g., to distinguish between the user's hand at rest and the hands of other people in the real-world environment). Tracking fingers / hands for input (e.g., gestures, touch, taps, etc.) can be advantageous because the user does not need to touch, hold, or wear any kind of beacon, sensor, or other device.

[0033] In some examples, the eye-tracking sensor(s) 212A / 212B include at least one eye-tracking camera (e.g., infrared cameras (IR cameras)) and / or light sources (e.g., IR light sources such as LEDs) that emit light toward a user's eyes. The eye-tracking cameras can be directed toward a user's eyes to receive reflected IR light from the light sources directly or indirectly from the eyes. In some examples, both eyes are tracked separately by their respective eye-tracking cameras and light sources, and a gaze can be determined from tracking both eyes. In other examples, one eye (e.g., a dominant eye) is tracked by one or more of their respective eye-tracking cameras / light sources.

[0034] The electronic device 260 / 270 and the system 201 are not based on the components and configuration of Fig. 2 is limited, but can include fewer, different, or additional components in several configurations. In some examples, System 201 may be a single device. A person or persons using System 201 are optionally referred to herein as the device user(s). Attention now turns to exemplary simultaneous displays of a three-dimensional environment on a first electronic device (e.g., corresponding to Electronic Device 260) and a second electronic device (e.g., corresponding to Electronic Device 270). As discussed below, the first electronic device may be in communication with the second electronic device in a multi-user communication session. In some examples, an avatar (e.g.,A representation of a user of the first electronic device in the three-dimensional environment can be displayed on the second electronic device, and an avatar of a user of the second electronic device can be displayed in the three-dimensional environment on the first electronic device. In some examples, the user of the first electronic device and the user of the second electronic device can be assigned to a spatial group in the multi-user communication session.In some examples, interactions with content in the three-dimensional environment, while the first electronic device and the second electronic device are in the multi-user communication session, can result in the user of the first electronic device and the user of the second electronic device being connected to different spatial groups in the multi-user communication session.

[0035] Fig. Figure 3 illustrates an example of a spatial group 340 in a multi-user communication session, which includes a first electronic device 360 ​​and a second electronic device 370 according to some examples of the disclosure. In some examples, the first electronic device 360 ​​may represent a three-dimensional environment 350A, and the second electronic device 370 may represent a three-dimensional environment 350B. The first electronic device 360 ​​and the second electronic device 370 may be similar to electronic device 101 or 260 / 270 and / or be a head-worn system / device and / or a projection-based system / device (including a hologram-based system / device) configured to generate and present a three-dimensional environment, such as...Heads-up displays (HUDs), head-mounted displays (HMDs), windows with integrated display function, displays in the form of lenses designed to be placed on a person's eyes (e.g., similar to contact lenses), etc. In the example of . Fig. 3. A first user optionally wears the first electronic device 360 ​​and a second user optionally wears the second electronic device 370, so that the three-dimensional environment 350A / 350B can be defined by X, Y and Z axes as viewed from a perspective of the electronic devices (e.g. a viewing angle associated with the electronic device 360 / 370, which may be, for example, a head-mounted display).

[0036] As in Fig. As shown in Figure 3, the first electronic device 360 ​​can be located in a first physical environment that includes a table 306 and a window 309. Thus, the three-dimensional environment 350A represented by the first electronic device 360 ​​can optionally include parts of the physical environment around the first electronic device 360, such as a representation of the table 306' and a representation of the window 309'. Similarly, the second electronic device 370 can be located in a second physical environment that is different from the first physical environment (e.g., separate from the first physical environment), which includes a floor lamp 307 and a coffee table 308. Therefore, the three-dimensional environment 350B represented by the second electronic device 370 can include parts of the physical environment around the second electronic device 370, e.g.,a representation of the floor lamp 307' and a representation of the coffee table 308'. In addition, the three-dimensional environments 350A and 350B can include representations of the floor, ceiling and walls of the room in which the first electronic device 360 ​​and the second electronic device 370 are located, respectively.

[0037] As mentioned above, in some examples the first electronic device 360 ​​is optional in a multi-user communication session with the second electronic device 370. For example, the first electronic device 360 ​​and the second electronic device 370 are configured (e.g., via the communication circuits 222A / 222B) to present a common three-dimensional environment 350A / 350B that includes one or more common virtual objects (e.g., content such as images, video, audio, and the like, representations of application user interfaces, etc.).As used herein, the term “shared three-dimensional environment” refers to a three-dimensional environment that is independently rendered, displayed, and / or visible on two or more electronic devices and through which content, applications, data, and the like can be shared and / or presented to the users of the two or more electronic devices. In some examples, while the first electronic device 360 ​​is in the multi-user communication session with the second electronic device 370, an avatar corresponding to the user of one electronic device is optionally displayed in the three-dimensional environment displayed by the other electronic device. As in . Fig. As shown in Figure 3, for example, an avatar 315, corresponding to the user of the second electronic device 370, is displayed on the first electronic device 360 ​​in the three-dimensional environment 350A. Similarly, an avatar 317, corresponding to the user of the first electronic device 360, is displayed on the second electronic device 370 in the three-dimensional environment 350B.

[0038] In some examples, the display of avatars 315 / 317 as part of a common three-dimensional environment is optionally accompanied by an audio effect corresponding to the voice of the users of electronic devices 370 / 360. For example, avatar 315, displayed in the three-dimensional environment 350A by means of the first electronic device 360, is optionally accompanied by an audio effect corresponding to the voice of the user of the second electronic device 370. In some such examples, when the user of the second electronic device 370 speaks, the user's voice can be captured by the second electronic device 370 (e.g., via the microphone(s) 213B) and transmitted to the first electronic device 360 ​​(e.g., via the microphone(s) 213B).via the communication circuit 222B / 222A) so that the captured voice of the user of the second electronic device 370 can be presented to the user of the first electronic device 360 ​​in the three-dimensional environment 350A as audio (e.g., using the loudspeaker(s) 216A). In some examples, the audio effect corresponding to the voice of the user of the second electronic device 370 can be spatialized in such a way that, to the user of the first electronic device 360, it appears as if it is coming from the location of the avatar 315 in the shared three-dimensional environment 350A (e.g., although it is output from the loudspeakers of the first electronic device 360).Similarly, the avatar 317, displayed in the three-dimensional environment 350B by means of the second electronic device 370, is optionally accompanied by an audio effect corresponding to the voice of the user of the first electronic device 360. In some such examples, when the user of the first electronic device 360 ​​speaks, the user's voice can be captured by the first electronic device 360 ​​(e.g., via the microphone(s) 213A) and transmitted to the second electronic device 370 (e.g., via the communication circuit 222A / 222B), so that the captured voice of the user of the first electronic device 360 ​​can be presented as audio (e.g., using the speaker(s) 216B) to the user of the second electronic device 370 in the three-dimensional environment 350B.In some examples, the audio effect corresponding to the voice of the user of the first electronic device 360 ​​can be spatialized in such a way that, to the user of the second electronic device 370, it appears as if it is coming from the location of the avatar 317 in the shared three-dimensional environment 350B (e.g., although it is output from the loudspeakers of the first electronic device 360).

[0039] In some examples, the avatars 315 / 317 are displayed during the multi-user communication session in the three-dimensional environments 350A / 350B with corresponding orientations that correspond to and / or are based on the orientations of the electronic devices 360 / 370 (and / or the users of the electronic devices 360 / 370) in the physical environments surrounding the electronic devices 360 / 370. As in Fig. As shown in Figure 3, avatar 315 in the three-dimensional environment 350A is optionally oriented towards the viewpoint of the user of the first electronic device 360, and in the three-dimensional environment 350B, avatar 317 is optionally oriented towards the viewpoint of the user of the second electronic device 370. If a particular user moves the electronic device (and / or themselves) in the physical environment, the user's viewpoint changes according to the movement, which can also change the orientation of the user's avatar in the three-dimensional environment. For example, if, with reference to Fig. 3 If the user of the first electronic device 360 ​​in the three-dimensional environment 350A were to look to the left, so that the first electronic device 360 ​​is rotated to the left (e.g., counterclockwise) by a corresponding amount, the user of the second electronic device 370 would see that the avatar 317, corresponding to the user of the first electronic device 360, rotates to the right (e.g., clockwise) with respect to the viewing angle of the user of the second electronic device 370 in accordance with the movement of the first electronic device 360.

[0040] Additionally, in some examples, during the multi-user communication session, the viewing angle of the three-dimensional environments 350A / 350B and / or the location of the viewing angle of the three-dimensional environments 350A / 350B optionally changes in accordance with the movement of the electronic devices 360 / 370 (e.g., by the users of the electronic devices 360 / 370). For example, if during the communication session the first electronic device 360 ​​is moved closer to the representation of the table 306' and / or the avatar 315 (e.g., because the user of the first electronic device 360 ​​moves forward in the physical environment surrounding the first electronic device 360), the viewing angle of the three-dimensional environment 350A would change accordingly, so that the representation of the table 306', the representation of the window 309', and the avatar 315 appear larger in the field of view.In some examples, each user can interact independently with the three-dimensional environment 350A / 350B, so that changes to the viewpoints of the three-dimensional environment 350A and / or interactions with virtual objects in the three-dimensional environment 350A by the first electronic device 360 ​​optionally have no effect on what is displayed in the three-dimensional environment 350B on the second electronic device 370, and vice versa.

[0041] In some examples, the avatars 315 / 317 are representations (e.g., a full-body rendering) of the users of the electronic devices 370 / 360. In some examples, the avatar 315 / 317 is a representation of a part (e.g., a rendering of a head, face, head and torso, etc.) of the users of the electronic devices 370 / 360. In some examples, the avatars 315 / 317 are user-personalized, user-selected, and / or user-created representations that are displayed in the three-dimensional environments 350A / 350B and are representative of the users of the electronic devices 370 / 360. It is understood that the in Fig. Although the avatars 315 / 317 shown correspond to full-body representations of the users of the electronic devices 370 / 360, other avatars, such as those described above, can also be provided.

[0042] As mentioned above, the three-dimensional environments 350A / 350B can be a shared three-dimensional environment represented by the electronic devices 360 / 370, while the first electronic device 360 ​​and the second electronic device 370 are in the multi-user communication session. In some examples, content viewed by one user on one electronic device can be shared with another user on a different electronic device in the multi-user communication session. In some of these examples, the content can be experienced (e.g., viewed and / or interacted with) by both users (e.g., via their respective electronic devices) in the shared three-dimensional environment. As in Fig. As shown in Figure 3, the three-dimensional environments 350A / 350B, for example, include a shared virtual object 310 (e.g., optionally a three-dimensional virtual sculpture) that can be viewed and interacted with by both users. As shown in Fig. As shown in Figure 3, the common virtual object 310 can be displayed with a gripper option (e.g., a steering rod) 335, which can be selected to initiate a movement of the common virtual object 310 within the three-dimensional environments 350A / 350B.

[0043] In some examples, the three-dimensional environments 350A / 350B include non-shared content intended for only one user in the multi-user communication session. Fig. For example, the first electronic device 360 ​​displays a private application window 330 in the three-dimensional environment 350A, which is optionally an object not shared between the first electronic device 360 ​​and the second electronic device 370 in the multi-user communication session. In some examples, the private application window 330 may be associated with a corresponding application running on the first electronic device 360 ​​(e.g., a media player application, a web browsing application, a messaging application, etc.). Since the private application window 330 is not shared with the second electronic device 370, the second electronic device 370 optionally displays a representation of the private application window 330 in the three-dimensional environment 350B. As in Fig. As shown in Figure 3, in some examples the representation of the private application window 330" may be a faded, obscured, discolored and / or translucent representation of the private application window 330, which prevents the user of the second electronic device 370 from seeing the contents of the private application window 330.

[0044] As mentioned above, in some examples, the user of the first electronic device 360 ​​and the user of the second electronic device 370 are in a spatial group 340 within the multi-user communication session. In some examples, the spatial group 340 can be a base group (e.g., a first or default group) within the multi-user communication session. For example, when the user of the first electronic device 360 ​​and the user of the second electronic device 370 initially join the multi-user communication session, the user of the first electronic device 360 ​​and the user of the second electronic device 370 are automatically (and initially, as discussed in more detail below) assigned to spatial group 340 within the multi-user communication session (e.g., grouped).In some examples, the user of the first electronic device 360 ​​and the user of the second electronic device 370 have a first spatial arrangement (e.g., a first spatial template) within the common three-dimensional environment, while the users are located in the spatial group 340, as in . Fig. Figure 3 illustrates this. For example, the user of the first electronic device 360 ​​and the user of the second electronic device 370, including the objects displayed in the common three-dimensional environment, have spatial truth within the spatial group 340. In some examples, spatial truth requires a consistent spatial arrangement between users (or their representations) and virtual objects. For example, the distance between the viewpoint of the user of the first electronic device 360 ​​and the avatar 315 corresponding to the user of the second electronic device 370 may be the same as the distance between the viewpoint of the user of the second electronic device 370 and the avatar 317 corresponding to the user of the first electronic device 360.As described here, the avatar 317, corresponding to the user of the first electronic device 360, moves in the three-dimensional environment 350B according to the movement of the user's viewing angle relative to the viewing angle of the user of the second electronic device 370, when the viewing angle of the user of the first electronic device 360 ​​moves. When the user of the first electronic device 360 ​​interacts with the shared virtual object 310 (e.g., moves the virtual object 310 in the three-dimensional environment 350A), the second electronic device 370 changes the display of the shared virtual object 310 in the three-dimensional environment 350B in accordance with the interaction (e.g., it moves the virtual object 310 in the three-dimensional environment 350B).

[0045] It is understood that in some examples, more than two electronic devices may be communicatively connected in a multi-user communication session. For instance, in a situation where three electronic devices are communicatively connected in a multi-user communication session, a first electronic device would display two avatars, not just one, corresponding to the users of the other two electronic devices. It should therefore be understood that the various processes and exemplary interactions described here with respect to the first electronic device 360 ​​and the second electronic device 370 in the multi-user communication session are optional for situations where more than two electronic devices are communicatively connected in a multi-user communication session.

[0046] In some examples, it may be advantageous to provide mechanisms for presenting content in a three-dimensional environment relative to surfaces in that three-dimensional environment during a multi-user communication session. For example, it may be desirable to share virtual content in a three-dimensional environment that is optionally visible and / or interactive to users in the multi-user communication session, in a way that accommodates the different heights of the users in the multi-user communication session. As discussed above in a similar manner, the three-dimensional environment optionally includes avatars corresponding to the users of the electronic devices in the multi-user communication session. In some examples, as discussed below, the representation of virtual objects (e.g.,Avatars and shared virtual content) in the three-dimensional environment in a multi-user communication session are selected to be relative to a physical surface in a physical environment and / or relative to a virtual surface of the three-dimensional environment in order to allow the virtual objects to be clearly and easily visible and / or interactive relative to the different heights of the viewpoints associated with the users in the multi-user communication session.

[0047] Fig. Figures 4A to 4I illustrate examples of presenting content in a three-dimensional environment relative to surfaces in the three-dimensional environment during a multi-user communication session, as described in some examples of the disclosure. In some examples, while a first electronic device 101a is in a multi-user communication session with a second electronic device 101b, a three-dimensional environment 450A is presented using the first electronic device 101a (e.g., via a display 120a), and a three-dimensional environment 450B is presented using the second electronic device 101b (e.g., via a display 120b). In some examples, the electronic devices 101a / 101b optionally correspond to the electronic devices 360 / 370 and / or the electronic devices 260 / 270 discussed above. Fig. 2, or are similar to them. In some examples, such as in Fig. As shown in Figure 4A, the first electronic device 101a is used by a first user 402 in side view 410 (e.g. worn on the head), and the second electronic device 101b is used by a second user 404 in side view 412 (e.g. worn on the head).

[0048] In some examples, the three-dimensional environments 450A / 450B include captured sections of the physical environments in which the electronic devices 101a / 101b are located. As in Fig. As shown in Figure 4A, the physical environment of the first electronic device 101a includes, for example, a desk 406 and a window 409. Similarly, in some examples, the three-dimensional environment 450A includes the desk 406 (e.g., a representation of the desk) and the window 409 (e.g., a representation of the window 409). Likewise, the physical environment of the second electronic device 101b, as shown in Figure 4A, includes... Fig. 4A shows a houseplant 408. Accordingly, the three-dimensional environment 450B includes in some examples, as in Fig. Figure 4A shows the houseplant 408 (e.g., a representation of the houseplant). In some examples, the representations may include sections of the physical environments viewed through a transparent or translucent display of the first electronic device 101a and the second electronic device 101b. In some examples, the three-dimensional environments 450A / 450B have one or more properties of those described above with reference to Fig. 3 described three-dimensional environments 350A / 350B.

[0049] As above with reference to Fig. As described in section 3, users can be represented by avatars corresponding to the users of the electronic devices while electronic devices are communicatively connected in a multi-user communication session. As described in Fig. As shown in Figure 4A, the three-dimensional environment 450A presented on the first electronic device 101a includes, for example, an avatar 414 corresponding to the second user 404 (e.g., a three-dimensional visual representation of the second user 404), and the three-dimensional environment 450B presented on the second electronic device 101b includes an avatar 416 corresponding to the first user 402 (e.g., a three-dimensional visual representation of the first user 402). Additionally, in some examples, as above similarly with respect to Fig. As described in section 3, while electronic devices are communicatively connected in a multi-user communication session, virtual content (e.g., user interfaces, three-dimensional shapes or models, interactive virtual video games, etc.) is configured to be displayed and / or shared among the users of the electronic devices. For example, in Fig. 4A the first electronic device 101a offers a virtual object 407, which optionally is or includes a user interface with content (e.g., a two-dimensional image of an airplane). In some examples, as in Fig. As shown in Figure 4A, the virtual object 407 includes or is shown with a gripper or guide 411, which is selectable to initiate a movement of the virtual object 407 in the three-dimensional environment 450A relative to the viewing angle of the first electronic device 101a. In some examples, as in Fig. As specified in 4A, virtual object 407 corresponds to a private virtual object in the multi-user communication session. As shown in Fig. As shown in Figure 4A, the virtual object 407 includes, for example, the pill-shaped box 417, which indicates that the virtual object 407 is currently a private virtual object (e.g., a virtual object that is visible and / or interactive only to the first user 402 on the first electronic device 101a, as described above similarly with respect to Fig. 3 discussed). In some examples, the pill-shaped box 417 can be selected to initiate a process for changing a privacy state of the virtual object 407, including sharing the virtual object 407 with other users in the multi-user communication session.

[0050] As above, similarly with regard to Fig. As described in Section 3, while the first electronic device 101a is in the multi-user communication session with the second electronic device 101b, the first user 402 and the second user 404 can be in a first spatial group in the multi-user communication session. In some examples, the first spatial group has one or more properties of the above with respect to Fig. 3. Spatial group 340 discussed. As described above in a similar manner, while the first user 402 and the second user 404 are in the first spatial group in the multi-user communication session, the users have an initial spatial arrangement in the shared three-dimensional environment (e.g., represented by the locations and / or distance between users 402 and 404 in side views 410 / 412 in Fig. 4A), which is determined by the physical locations of the first electronic device 101a and the second electronic device 101b in their respective physical environments. In particular, the first electronic device 101a and the second electronic device 101b experience spatial truth within the first spatial group as specified by the physical locations and / or orientations of the first user 402 and the second user 404, respectively.

[0051] In some examples, as discussed herein, the avatars 414 / 416 are displayed with respective heights based on heights associated with the users of the electronic devices 101a / 101b. In some examples, the users' heights are determined by the electronic devices 101a / 101b as relative to a floor or subsurface of the three-dimensional environments 450A / 450B (e.g., the floors of the physical environments in which the electronic devices 101a / 101b are located). In some examples, a user's height corresponds to a vertical distance between the floor or subsurface and the electronic device associated with that user (e.g., worn on their body).For example, in side view 410, the height of the first user 402 is determined as the vertical distance between the floor of the first user 402's physical environment and the first electronic device 101a (which is worn, for example, on the first user 402's head), even though the first user 402 is sitting on a chair in front of the desk 406. Similarly, in side view 412, the height of the second user 404 is determined as the vertical distance between the floor of the second user 404's physical environment and the second electronic device 101b (which is worn, for example, on the second user 404's head). As mentioned above, in some examples, the height of a given avatar is based on a specific height of the corresponding user relative to the floor or surface of the three-dimensional environment (e.g., it corresponds to and / or is equal to it). As in side view 410 in . Fig. As illustrated in Figure 4A, for example, an avatar height of 414 corresponds to the height of the second user 404 (e.g., a height of 449 in side view 412). Fig. 4D), and in side view 412, an avatar height of 416 corresponds to the height of the first user 402 (e.g., a height of 447 in side view 410 in Fig. 4D) relative to the ground or subsurface of their respective physical environments.

[0052] In some cases, it may be desirable and / or advantageous to redisplay the avatars corresponding to users in a multi-user communication session relative to alternative surfaces in the three-dimensional environment (e.g., surfaces that differ from the floor or ground of the users' respective physical environments). In particular, in some examples, it may be desirable to reposition avatars relative to alternative surfaces in the three-dimensional environment to allow the avatars to be at or near eye level with users at their respective electronic devices, such as when users view and / or interact with shared content (e.g., horizontally oriented content, as discussed below).

[0053] In Fig. 4A, while the first electronic device 101a is in the multi-user communication session with the second electronic device 110b, the first electronic device 101a detects an input corresponding to a request to change a privacy state of the virtual object 407 in the three-dimensional environment 450A. As in Fig. As shown in Figure 4A, the first electronic device 101a, for example, recognizes a selection input directed at the pill-shaped box 417, such as an air-tap gesture (e.g., contact between index finger and thumb) performed by a hand 403 of the first user 402, while optionally a gaze 426 of the first user 402 is directed at the pill-shaped box 417 in the three-dimensional environment 450A. It is understood that additional or alternative inputs are possible, such as air-tap gestures, gaze and dwell inputs, voice commands, etc.

[0054] In some examples, such as in Fig. As shown in Figure 4B, the first electronic device 101a, in response to the detection of the selection of the pill-shaped box 417, displays the menu item 424 in the three-dimensional environment 450A. As shown in Figure 4B, the first electronic device 101a displays the menu item 424 in the three-dimensional environment 450A. Fig. As shown in Figure 4B, the first electronic device 101a, for example, displays the menu item 424, which includes one or more options for sharing the contents of the virtual object 407 in the multi-user communication session (allowing, for example, other users in the multi-user communication session to view and / or interact with the contents of the virtual object 407). Fig. 4B detects the first electronic device 101a when menu item 424 is displayed and option 423 is selected in menu item 424 in the three-dimensional environment 450A. For example, as in Fig. As shown in Figure 4B, the first electronic device 101a detects an air pinch gesture performed by hand 403, while optionally the gaze 426 of the first user 402 is directed towards option 423 in the three-dimensional environment 450A.

[0055] In some examples, such as in Fig. As shown in Figure 4C, the first electronic device 101a, in response to the detection of the selection of option 423 in menu item 424, shares the virtual object 407 in the multi-user communication session. As shown by the pill-shaped box 417 in Fig. When 4C is displayed, for example, virtual object 407 becomes a shared virtual object in the multi-user communication session, so that the content of virtual object 407 is now also visible and / or interactive for the second user 404 on the second electronic device 101b. In some examples, as in Fig. As shown in Figure 4C, the second electronic device 101b updates the display of the three-dimensional environment 450B to include the virtual object 407 when the virtual object 407 is shared with the second electronic device 101b. Additionally, in some examples, as shown in side views 410 and 412 in Figure 4C, the second electronic device 101b updates the display of the three-dimensional environment 450B to include the virtual object 407 when the virtual object 407 is shared with the second electronic device 101b. Fig. As shown in Figure 4C, the first electronic device 101a maintains the display of the avatar 414 (e.g., at the same height in the three-dimensional environment 450A relative to the ground) and the second electronic device 101b maintains the display of the avatar 416 (e.g., at the same height in the three-dimensional environment 450B relative to the ground) when the virtual object 407 is shared in the multi-user communication session.

[0056] In Fig. 4C The first electronic device 101a recognizes an input that corresponds to a request to associate the virtual object 407 with a particular surface in the three-dimensional environment 450A (e.g., a surface that differs from the surface of the floor or subsurface of the physical environment of the first electronic device 101a). In some examples, the input that corresponds to the request to associate the virtual object 407 with the particular surface includes and / or corresponds to a motion input directed at the virtual object 407. As in Fig. As shown in Figure 4C, for example, the first electronic device 101a recognizes an air pinch and pull gesture provided by the hand 403, while optionally the gaze 426 is directed at the gripper 411, which is linked to the virtual object 407. In some examples, as in Fig. As specified in 4C, the air pinch and pull gesture involves a movement of the hand 403 in the direction of the viewing angle of the first electronic device 101a and in the direction of an upper surface of the desk 406 (e.g., the representation of the desk) in the three-dimensional environment 450A. Accordingly, in some examples, the input corresponding to the requirement to link the virtual object 407 with a respective surface in the three-dimensional environment 450A corresponds to a requirement to display (e.g., and / or anchor) the virtual object 407 on the surface of the desk 406 in the three-dimensional environment 450A.

[0057] In some examples, the second electronic device 101b updates the display of the avatar 416 corresponding to the first user 402 in the three-dimensional environment 450B, while the first electronic device 101a detects the air pinch-and-pull gesture directed at the virtual object 407, provided by the hand 403 of the first user 402. As in Fig. As shown in 4D, for example, the second electronic device 101b replaces the avatar 416 with a two-dimensional representation of the first user 402, such as a spatial coin shape 418, in the three-dimensional environment 450B. In some examples, as in Fig. As shown in 4D, the spatial coin shape 418 includes an indication of the first user 402's name (e.g., John Doe) and a two-dimensional visual representation (e.g., picture, photo, symbol, cartoon, sketch, etc.) of the first user 402. In some examples, the spatial coin shape 418 provides the second user 404 with a visual indication that the first user 402 is interacting with the virtual object 407 (e.g., moving the virtual object 407 as described above). In some examples, when the input is recognized by the first electronic device 101a, the first electronic device 101a transmits an indication of the input provided by the first user 402 to the second electronic device 101b, thereby enabling and / or causing the second electronic device 101b to update the display of the first user 402's visual representation in the three-dimensional environment 450B (e.g.,from the three-dimensional avatar into the two-dimensional spatial coin shape).

[0058] In some examples, the first electronic device 101a and the second electronic device 101b update the heights at which the avatars 414 and 416 are displayed in their respective three-dimensional environments when the first electronic device 101a links the virtual object 407 to the surface of the desk 406 in the three-dimensional environment 450A. Specifically, the first electronic device 101a updates the height of the avatar 414 in the three-dimensional environment 450A so that it no longer corresponds to the height 449 of the second user 404 relative to the floor, as shown in the side view 412. Fig. 4D is specified, and the second electronic device 101b updates the height of the avatar 416 in the three-dimensional environment 450B so that it no longer corresponds to the height 447 of the first user 402 relative to the ground, which is specified in the side view 410, as described in more detail below.

[0059] In some examples, linking virtual object 407 to the surface of desk 406 in the three-dimensional environment 450A causes the heights of avatars 414 / 416 to be updated in their respective three-dimensional environments, since virtual object 407 corresponds to and / or is a first-type virtual object when linked to the surface of desk 406 (e.g., when displayed on the surface of desk 406). In some examples, a first-type object corresponds to a virtual object that is or includes horizontally oriented content. As in Fig. As shown in Figure 4E, the virtual object 407 is displayed, for example, on the surface of the desk 406, such that the content (e.g., the image of the airplane) of the virtual object 407 has a horizontal orientation relative to the viewing angle of the first electronic device 101a, which corresponds to the horizontal (e.g., flat) orientation of the surface of the desk 406, as illustrated in side view 410.

[0060] In some examples, the first electronic device 101a updates the way in which the height of the first user 402 is measured in the physical environment of the first electronic device 101a when the virtual object 407 is moved onto and displayed on the surface of the desk 406, such that the virtual object 407 has a horizontal orientation in the three-dimensional environment 450A, as discussed above. In particular, the first electronic device 101a determines in Fig. 4E The height of the first user 402 is defined as relative to the surface of the desk 406 and not to the surface of the floor of the physical environment of the first electronic device 101a, as indicated by a height 427 in the side view 410. For example, the height 427 of the first user 402 is measured vertically between the first electronic device 101a and the surface of the desk 406 (e.g., as detected by one or more sensors (e.g., image sensor(s)) of the first electronic device 101a). As an example, the height 427 in the side view 410 is shown in Fig. 4E different (e.g. smaller than) the height 447 in the side view 410 in Fig. 4D. In some examples, as discussed below, the first electronic device 101a transmits data to the second electronic device 101b corresponding to the updated height 427 of the first user 402, which is relative to the surface of the desk 406 in the physical environment of the first electronic device 101a.

[0061] Since, in some examples, the physical environment of the second electronic device 101b does not include any surface other than the floor or subfloor of the physical environment and / or does not include a surface similar to the desk 406 in the physical environment of the first electronic device 101a, the second electronic device 101b generates a virtual surface 428 on which the virtual object 407 is displayed (e.g., anchored) in the three-dimensional environment 450B, after the virtual object 407 has been displayed (e.g., anchored) on the surface of the desk 406 in the three-dimensional environment 450A of the first electronic device 101a. In some examples, the virtual surface 428 in the three-dimensional environment 450B resembles the surface of the desk 406 in the three-dimensional environment 450A.For example, the virtual surface 428 has the same or a similar orientation to the surface of the desk 406 (e.g., a horizontal orientation relative to the viewing angles of the electronic devices 101a / 101b). Additionally or alternatively, in some examples, the virtual surface 428 has the same or a similar size to the surface of the desk 406 (e.g., the same or a similar surface area on which content can be displayed). In some examples, the height of the virtual surface 428 relative to the viewing angle of the second electronic device 101b in the three-dimensional environment 450B is equal to or similar to the height of the surface of the desk 406 relative to the viewing angle of the first electronic device 101a in the three-dimensional environment 450A. As shown in the side view 412 in Figure 450A. Fig. As illustrated in Figure 4E, for example, a height of 429, in which the virtual surface 428 is displayed in the three-dimensional environment 450B relative to the viewing angle of the second electronic device 101b, corresponds to a height of 427 in the side view 410 (is, for example, the same).

[0062] In some examples, the second electronic device 101b additionally or alternatively generates and displays the virtual surface 428 in the three-dimensional environment 450B based on one or more physical properties of the physical environment in which the second electronic device 101b is located. For example, the second electronic device 101b determines a size, shape, orientation, and / or height of the virtual surface 428 relative to the viewing angle of the second electronic device 101b in the three-dimensional environment 450B based on a size of the physical environment of the second electronic device 101b and / or locations, sizes, and / or orientations of physical objects in the physical environment (e.g., the houseplant 408) as viewed from the viewing angle of the second electronic device 101b. As an example, in Fig. 4E The virtual surface 428 is spatially positioned in the three-dimensional environment 450B relative to the viewing angle of the second electronic device 101b such that it does not coincide with a location of the houseplant 408 in the three-dimensional environment 450B (e.g., it overlaps and / or intersects it). In some examples, the virtual surface 428 is positioned in the center of the viewing angle of the second electronic device 101b. In some examples, the virtual surface 428 is positioned at a predefined height (e.g., determined without user input) relative to the viewing angle of the second electronic device 101b in the three-dimensional environment 450B (e.g., and optionally independent of the height 427 of the first user 402 relative to the surface of the desk 406 in the side view 410).

[0063] As mentioned above, in some examples the second electronic device 101b redisplays the avatar 416 in the three-dimensional environment 450B (e.g., redisplays the visual representation of the first user 402 as a three-dimensional avatar) when the virtual object 407 is linked to the surface of the desk 406 in the three-dimensional environment 450A (e.g., when the movement of the virtual object 407 to the surface of the desk 406 is complete). Additionally, in some examples, as in Fig. 4E shows the height of the avatar 414, which corresponds to the second user 404, so that it is relative to the surface of the desk 406 and not relative to the surface of the floor or subfloor. As shown in the side view 410 in Fig. As illustrated in Figure 4E, for example, avatar 414 has an updated height that corresponds to height 429 in side view 410 (which corresponds, for example, to the height of the second user 404 relative to the virtual surface 428). Similarly, in some examples, such as in side view 412 in Fig. As shown in Figure 4E, the avatar 416, corresponding to the first user 402, has an updated height that corresponds to the height 427 of the first user 402 relative to the surface of the desk 406 in the side view 410. In this way, as shown in Fig. Figure 4E illustrates that the avatar 414, corresponding to the second user 404, is displayed at eye level with the first user 402 on the first electronic device 101a, and the avatar 416, corresponding to the first user 402, is displayed at eye level with the second user 404 on the second electronic device 101b, which, as an advantage, allows for simpler and more lifelike interaction with the content of the virtual object 407.

[0064] In some examples, the first electronic device 101a and the second electronic device 101b display a visual indication that the heights of avatars 414 / 416 have been updated when the heights of avatars 414 / 416 are updated in their respective three-dimensional environments 450A / 450B. For example, the first electronic device 101a and the second electronic device 101b display a notification, message, icon, or other virtual element that visually indicates that the heights of avatars 414 / 416 have been updated and optionally specifies that the heights are now relative to a surface (e.g., the surface of the desk 406 and / or the virtual surface 428) that is different from the surface of the floor or subsurface.

[0065] In some examples, in addition to updating the heights of the avatars 414 / 416 in their respective three-dimensional environments 450A / 450B, the positions of the avatars 414 / 416 are optionally updated based on the input that links the virtual object 407 to the surface of the desk 406 in the three-dimensional environment 450A. For example, as described above, linking the virtual object 407 to the surface of the desk 406 is effected by moving the virtual object 407 within the three-dimensional environment 450A. In some examples, as defined herein, moving the virtual object 407, which is a shared object in the multi-user communication session, triggers spatial refinement. In particular, in some examples, as in Fig. 4C to Fig. Figure 4E shows the avatar 414 with the virtual object 407 in the three-dimensional environment 450A relative to the viewing angle of the first electronic device 101a. For example, the first electronic device 101a moves the avatar 414 and the virtual object 407 simultaneously in the direction of the viewing angle of the first electronic device 101a in the three-dimensional environment 450A according to the movement of the hand 403 in Fig. 4C. To maintain spatial truth within the spatial group of the first user 402 and the second user 404, as defined herein, the second electronic device 101b similarly moves, in some examples, the avatar 416 corresponding to the first user 402 in the three-dimensional environment 450B, based on the motion input detected by the first electronic device 101a. In particular, because the motion input detected by the first electronic device 101a causes a reduction in the distance between the viewing angle of the first electronic device 101a and the shared virtual objects (e.g., the avatar 414 and the virtual object 407) in the three-dimensional environment 450A, the second electronic device 101b reduces the distance (e.g., by the same or a similar amount or degree) between the viewing angle of the second electronic device 101b and the avatar 416 (e.g.,by moving the avatar 416 in the direction of the viewing angle of the second electronic device 101b) in the three-dimensional environment 450B, as in . Fig. 4E shown.

[0066] Fig. Figure 4F illustrates an example of the automatic linking of the virtual object 407 to the surface of the desk 406 in the three-dimensional environment 450A when sharing the virtual object 407 in the multi-user communication session. For example, in Fig. 4F the first electronic device 101a an input corresponding to a request to share the contents of the virtual object 407 with the second user 404 in the three-dimensional environment 450B, while the virtual object 407 is a private object in the multi-user communication session (e.g., private to the first user 402 in the three-dimensional environment 450A, as previously described herein), as indicated by the pill-shaped box 417. In some examples, as in Fig. As shown in Figure 4F, the input detected by the first electronic device 101a includes an air pinch gesture provided by hand 403, directed at the pill-shaped box 417 in the three-dimensional environment 450A (e.g., while gaze 426 is directed at the pill-shaped box 417), as similarly described above. In some examples, when the input provided by hand 403 is detected by the first electronic device 101a, the second electronic device 101b displays the virtual object 432 in the three-dimensional environment 450B. In some examples, as in Fig. As shown in Figure 4F, the virtual object 432 is content (e.g., a user interface) that is linked to, or includes, an email application running on the second electronic device 101b. In some examples, as in Fig. As shown in Figure 4F, the virtual object 432 includes a gripper or guide 433 and / or is displayed with one that can be selected to initiate movement of the virtual object 432 in the three-dimensional environment 450B relative to the viewing angle of the second electronic device 101b. Additionally, in the example of Fig. 4F, as indicated by the pill-shaped box 431, the contents of the virtual object 432 for the second user 404 on the second electronic device 101b are private (e.g., the virtual object 432 corresponds to a private object). In the example of Fig. 4F includes the physical environment in which the second electronic device 101b is located, a table 430, when the input discussed above is detected by the first electronic device 101a. Accordingly, in some examples, the three-dimensional environment 450B includes a representation of the table 430, as shown in Fig. 4F shown. In some examples, as already described above, while the virtual object 407 is private to the first user 402 in the multi-user communication session, the avatars 414 / 416 are displayed in their respective three-dimensional environments 450A / 450B at heights based on the heights of their respective users relative to the ground or subsurface of the three-dimensional environments 450A / 450B, as shown in the side views 410 / 412 in Fig. 4F illustrates.

[0067] In some examples, such as in Fig. As shown in Figure 4G, the first electronic device 101a, in response to the recognition of input provided by hand 403, shares the contents of the virtual object 407 with the second user 404 on the second electronic device 101b. For example, as discussed similarly herein, the second electronic device 101b displays the virtual object 407 in the three-dimensional environment 450B. Additionally, in some examples, the first electronic device 101a updates the display of the pill-shaped box 417 in the three-dimensional environment 450A to indicate that the virtual object 407 corresponds to a shared object in the multi-user communication session.

[0068] Additionally, in some examples, the first electronic device 101a automatically links the virtual object 407 to the surface of the desk 406 in the three-dimensional environment 450A when the virtual object 407 is shared in the multi-user communication session. For example, the virtual object 407 is displayed and / or anchored on the surface of the desk 406 in the three-dimensional environment 450A without requiring any input from the first user 402 to link the virtual object 407 to the surface of the desk 406 (e.g., the one described above in relation to Fig. (4C described motion input). In some examples, the virtual object 407 is automatically linked to the surface of the desk 406 because the virtual object 407 is an object of the first type described above (e.g., a horizontally oriented virtual object). In some examples, the virtual object 407 is automatically linked to the surface of the desk 406 in the three-dimensional environment 450A because the virtual object 407 is located above and / or near (e.g., within a threshold distance of 0.1, 0.2, 0.5, 0.75, 1, 1.5, 2, 3, etc. meters from) the surface of the desk 406 when the virtual object 407 is shared in the multi-user communication session.

[0069] In some examples, such as in Fig. As shown in Figure 4G, the second electronic device 101b displays (e.g., anchors) the virtual object 407 on the surface of the table 430 in the three-dimensional environment 450B (e.g., instead of on a virtual surface, such as the virtual surface 428 discussed above) when the virtual object 407 is shared with the second electronic device 101b. In some examples, the second electronic device 101b displays the virtual object 407 on the surface of the table 430 in the three-dimensional environment 450B because one or more properties of the table 430 are similar to one or more properties of the desk 406. For example, desk 406 and table 430 have similar shapes and / or sizes and a similar surface area, occupy similar parts of the field of view of electronic devices 101a / 101b and / or have similar heights relative to the viewing angles of electronic devices 101a / 101b.In some examples, such as in page views 410 / 412 in . Fig. As illustrated in Figure 4G, the height of the table 430 relative to the floor or surface of the physical environment of the second electronic device 101b is greater than the height of the desk 406 relative to the floor or surface of the physical environment of the first electronic device 101a.

[0070] In some examples, as discussed above in a similar manner, the electronic devices 101a / 101b update the display of the avatars 414 / 416 when the virtual object 407 is shared in the multi-user communication session and linked to the surfaces in the three-dimensional environments 450A / 450B that differ from the surfaces of the floors in the three-dimensional environments 450A / 450B (e.g., the surfaces of the desk 406 and the table 430). As in Fig. As shown in Figure 4G, for example, the first electronic device 101a updates the display of the height of the avatar 414 in the three-dimensional environment 450A, and the second electronic device 101b updates the display of the height of the avatar 416 in the three-dimensional environment 450B. In particular, in some examples, the first electronic device 101a updates the height of the avatar 414 so that it corresponds to a height of the second user 404 relative to the surface of the table 430, and the second electronic device 101b updates the height of the avatar 416 so that it corresponds to a height of the first user 402 relative to the surface of the desk 406.As illustrated, for example, in side view 410, the first electronic device 101a determines the height 427 of the first user 402, which is relative to the surface of the desk 406 in the three-dimensional environment 450A, and transmits data corresponding to the determined height 427 to the second electronic device 101b. Similarly, as illustrated in side view 412, the second electronic device 101b determines the height 439 of the second user 404, which is relative to the surface of the table 430 in the three-dimensional environment 450B, and transmits data corresponding to the determined height 439 to the first electronic device 101a. Then the first electronic device 101a, as similarly described herein, in some examples positions the avatar 414 at a height 437 relative to the surface of the desk 406 in the three-dimensional environment 450A, as shown in the side view 410 in . Fig. Figure 4G illustrates this, where the height 437 corresponds to (e.g., is equal to) the height 439 of the second user 404 relative to the surface of the table 430 in the side view 412. Additionally, in some examples, the second electronic device 101b positions the avatar 416 at a height 429 relative to the surface of the table 430 in the three-dimensional environment 450B, as shown in the side view 412. Fig. Figure 4G illustrates this, where the height 429 corresponds to the height 427 of the first user 402 relative to the surface of the desk 406 in the side view 410 (e.g., is the same). Although the users and the avatars are not perfectly aligned at eye level when the virtual object 407 is shared in the multi-user communication session, the avatars remain sufficiently visible to the users in their respective three-dimensional environments, while simulating near-real heights of the users in their respective physical environments. This has the advantage of allowing for easier and more lifelike interaction with the content of the virtual object 407.

[0071] In Fig. 4H detects the second electronic device 101b as an input corresponding to a request to share the virtual object 432 (e.g., instead of the virtual object and / or instead of it) in the multi-user communication session. For example, as in Fig. As shown in Figure 4H, the second electronic device 101b detects an air pinch gesture performed by the hand 405 of the second user 404, while optionally the gaze 426 of the second user 404 is directed at the pill-shaped box 431 of the virtual object 432. In some examples, as in Fig. As shown in Figure 4H, the second electronic device 101b recognizes the input provided by the hand 405, while the virtual object 407 is shared in the multi-user communication session.

[0072] In some examples, such as in Fig. As shown in Figure 4I, the second electronic device 101b, in response to the recognition of the input provided by the hand 405, shares the virtual object 432 with the first user 402 on the first electronic device 101a. As shown in Fig. As shown in Figure 4I, for example, the first electronic device 101a displays the virtual object 432 in the three-dimensional environment 450A, with the pill-shaped box 431 being updated to indicate that the virtual object 432 is a shared object in the multi-user communication session. In some examples, as discussed similarly above, the content (e.g., the email user interface) of the virtual object 432 becomes visible and / or interactive to the first user 402 in the three-dimensional environment 450A when the virtual object 432 is shared with the first user 402 on the first electronic device 101a, as shown in Figure 4I. Fig. Figure 4I shows that in some examples, the first electronic device 101a and the second electronic device 101b limit the number of applications (e.g., content) that can be shared in the multi-user communication session. For example, in Fig. 4I Each time, a single application (e.g., optionally a single virtual object) is shared in the multi-user communication session. Accordingly, in some examples, as in Fig. As shown in Figure 4, when virtual object 432 is shared in the multi-user communication session, virtual object 407 in the multi-user communication session is made private again (e.g., it reverts to this state). As in Fig. As shown in 4I, the virtual object 407 is no longer displayed in the three-dimensional environment 450B (e.g., because the virtual object 407 is no longer shared with the second electronic device 101b), and the pill-shaped box 417 of the virtual object 407 in the three-dimensional environment 450A is updated to reflect that the virtual object 407 is again private to the first user 402 on the first electronic device 101a.

[0073] In some examples, the display of avatars 414 / 416 in their respective three-dimensional environments is updated when the second electronic device 101b shares the content of the virtual object 432 with the first electronic device 101a. Specifically, in some examples, the first electronic device 101a updates the height of avatar 414 in the three-dimensional environment 450A, and the second electronic device 101b updates the height of avatar 416 in the three-dimensional environment 450B. In some examples, as in side views 410 / 412 in Fig. As specified in 4I, the heights of avatars 414 / 416 are updated so that they are no longer relative to the surfaces of desk 406 and table 430, but relative to the floor of the physical environments of electronic devices 101a / 101b. As in Fig. As shown in Figure 4I, for example, the first electronic device 101a updates the height 437 of the avatar 414 so that it corresponds to (e.g., is equal to) the height of the second user 404 relative to the floor of the physical environment of the second electronic device 101b, as shown in side view 410. Similarly, in some examples, as shown in side view 412, the second electronic device 101b updates Fig. Figure 4I shows the height 429 of avatar 416, such that it corresponds to the height of the first user 402 relative to the floor of the physical environment of the first electronic device 101a. In some examples, the heights of avatars 414 / 416 are updated when virtual object 432 is shared in the multi-user communication session, since virtual object 432 corresponds to an object of a second type, which is different from the object of the first type discussed above. For example, virtual object 432 in Fig. 4I content or includes content that has a vertical orientation in the three-dimensional environments 450A / 450B (e.g., an object of the second type corresponds to a vertically oriented virtual object). As described above, in some examples, the electronic devices 101a / 101b update the display of the heights of the avatars 414 / 416 for horizontally oriented objects to allow the avatars 414 / 416 to be displayed at or near the eye level of the users of the electronic devices 101a / 101b, thus enhancing mutual and / or cooperative interaction with the horizontally oriented objects. However, this may not be the case for vertically oriented objects (e.g., maintaining eye level between users and avatars does not necessarily enhance and / or facilitate interaction with content in a vertically oriented object).In order to help conserve computing resources and device performance, the avatars 414 / 416 are accordingly displayed again at heights relative to the ground or subsurface of the physical environments in which the electronic devices 101a / 101b are located (e.g., according to the standard display mode of the avatars).

[0074] It is understood that the method described above for updating the height at which avatars 414 / 416 are displayed in the three-dimensional environments 450A / 450B applies similarly to multi-user communication sessions involving more than two users. For example, if the above multi-user communication session includes a third user of a third electronic device (not shown), the display of an avatar corresponding to the third user will be updated in the same or a similar manner as the display of avatars 414 / 416 (e.g., so that they are relative to a physical surface or a virtual surface that differs from the floor or subsurface of the three-dimensional environments 450A / 450B) when horizontally oriented virtual objects (e.g., the above virtual object 407) are shared and / or interacted with.Attention now turns to examples of facilitating the spatial refinement of virtual objects (e.g., avatars and / or virtual content) in a multi-user communication session.

[0075] Fig. 5A to 5K illustrate examples of vertical movement of shared content in a three-dimensional environment relative to surfaces in the three-dimensional environment during a multi-user communication session, according to some examples from the revelation. Fig. 5A are the first electronic device 101a (e.g., associated with the first user 502), the second electronic device 101b (e.g., associated with the second user 504), and the third electronic device 101c (e.g., associated with a third user) in a multi-user communication session, as discussed similarly above. In some examples, the first user 502 and the second user 504 correspond to the first user 402 and the second user 404, respectively, of Fig. 4A to 4I.

[0076] As in Fig. As shown in Figure 5A, the first electronic device 101a (e.g., via display 120a) presents a three-dimensional environment 550A. In Fig. 5A includes the three-dimensional environment 550A, as discussed above in a similar manner, representations (e.g., passthrough representations or computer-generated representations) of the physical environment of the first electronic device 101a. As, for example, in Fig. As shown in Figure 5A, the physical environment corresponds to a room that includes the desk 506 and the window 509. Accordingly, as shown in Fig. Figure 5A shows the three-dimensional environment 550A presented using the first electronic device 101a, including representations of the desk 506 and the window 509 (e.g., the desk 506 and the window 509 are visible in a field of view of the first electronic device 101a). Likewise, as shown in Fig. As shown in Figure 5A, the second electronic device 101b (e.g., via display 120b) represents a three-dimensional environment 550B. Fig. 5A includes the three-dimensional environment 550B, as discussed above in a similar manner, representations (e.g., passthrough representations or computer-generated representations) of the physical environment of the first electronic device 101a. As in Fig. As shown in Figure 5A, for example, the physical environment corresponds to a room that includes a table 530 and a houseplant 508. Accordingly, as shown in Fig. Figure 5A shows the three-dimensional environment 550B presented using the second electronic device 101b, which includes representations of the table 530 and the houseplant 508 (e.g., the table 530 and the houseplant 508 are visible in a field of view of the second electronic device 101b). In some examples, the three-dimensional environments 550A / 550B exhibit one or more features of the three-dimensional environments 450A / 450B discussed above.

[0077] Additionally, close in Fig. 5A The three-dimensional environments presented by the electronic devices in the multi-user communication session include visual representations (e.g., avatars) that correspond to the users of the electronic devices. As discussed similarly above, for example, in Fig. 5A the three-dimensional environment 550A presented on the first electronic device 101a includes an avatar 514 corresponding to the second user 504 and the avatar 519 corresponding to the third user of the third electronic device (not shown). Similarly, as in Fig. Figure 5A shows that, in some examples, the three-dimensional environment 550B presented on the second electronic device 101b includes an avatar 516 corresponding to the first user 502 and the avatar 519 corresponding to the third user. In some examples, the avatars 514 / 516 correspond to the avatars 414 / 416 described above. Additionally, in some examples, as shown in Figure 5A, the three-dimensional environments 550A / 550B include an avatar 516 corresponding to the first user 502 and the avatar 519 corresponding to the third user. Fig. As shown in Figure 5A, the virtual object 507 is included. In some examples, the virtual object 507 is associated with a particular application running on the electronic devices 101a / 101b, such as an image processing or image display application, a video game or board game application, etc. In some examples, the virtual object 507 includes or is shown with a gripper or handle 511 that can be selected to initiate movement of the virtual object 507 in the three-dimensional environment 550A / 550B. Additionally, in some examples, the virtual object 507 corresponds to a shared object in the multi-user communication session. As indicated by the pill-shaped box 517 associated with the virtual object 507 in Figure 5A, the virtual object 507 is also included in Figure 5A. Fig. When 5A is displayed, for example, the content of virtual object 507 (e.g., the image of the aircraft) is visible and / or interactive (e.g., via their respective electronic devices) to the first user 502, the second user 504, and the third user in the multi-user communication session. In some examples, virtual object 507 corresponds to virtual object 407 discussed above.

[0078] In some examples, as discussed above in a similar manner, the first electronic device 101a, the second electronic device 101b, and the third electronic device are communicatively connected in the multi-user communication session. However, the first user 502, the second user 504, and the third user are associated in the multi-user communication session with the same spatial group, according to which spatial truth is defined for the first user 502, the second user 504, and the third user. In some examples, maintaining spatial truth within the spatial group involves triggering spatial refinement according to a determination that a motion input is recognized as directed at a shared object in the multi-user communication session.In some examples, the electronic devices in the multi-user communication session facilitate vertical spatial refinement, as discussed below, in response to the detection of a motion input corresponding to a vertical movement of a shared object in the multi-user communication session.

[0079] In Fig. 5A The first electronic device 101a detects an input provided by a hand 503 of the first user 502, corresponding to a request to move the virtual object 507 in the three-dimensional environment 550A. As in Fig. As shown in Figure 5A, the first electronic device 101a, for example, recognizes an air pinch and pull gesture performed by hand 503, while optionally the gaze 526 of the first user 502 is directed at the gripper 511, which is associated with the virtual object 507 in the three-dimensional environment 550A. In some examples, the movement of the virtual object 507 corresponds to a vertical (e.g., upward) movement of the virtual object 507 relative to the viewing angle of the first electronic device 101a, as shown in Figure 5A. Fig. 5A is specified.

[0080] In some examples, as mentioned above, the first electronic device 101a, in response to the recognition of input from the hand 503 of the first user 502, which corresponds to the request to move the virtual object 507 in the three-dimensional environment 550A, initiates a spatial refinement in the multi-user communication session. Specifically, in some examples, such as in Fig. 5A to Fig. Figure 5B-1 shows the (e.g., divided) virtual objects within the three-dimensional environment 550A vertically relative to the viewing angle of the first electronic device 101a, in particular the virtual object 507 and the visual representations of the users of the second electronic device 101b and the third electronic device 101c, according to the upward movement of the hand 503. In some examples, as in Fig. As shown in Figure 5B-1, the first electronic device 101a replaces the display of the avatars 514 / 519 in the three-dimensional environment 550A with spatial coin shapes 534 / 536 when and / or during the spatial refinement is triggered in the multi-user communication session (e.g., while the first user 502 continues to display the in Fig. 5B-1 provides an illustrated air pinch and pull gesture). As in Fig. As shown in Figure 5B-1, for example, the first electronic device 101a replaces the display of the avatar 514 corresponding to the second user 504 with a two-dimensional representation of the second user 504 (e.g., the three-dimensional coin shape 534) and replaces the display of the avatar 536 corresponding to the third user with a two-dimensional representation of the third user (e.g., the three-dimensional coin shape 536), as illustrated in the side view 510. In some examples, as in Fig. As shown in Figure 5B-1, the spatial coin shape 534 includes a statement of the name of the second user 504 (e.g., Mary Smith) and a two-dimensional visual representation (e.g., picture, photograph, symbol, cartoon, sketch, etc.) of the second user 504. Similarly, in some examples, the spatial coin shape 536 includes a statement of the name of the third user (e.g., Ava Jones) and a two-dimensional visual representation of the third user, as shown in Figure 5B-1. Fig. 5B-1 shown. In some examples, the spatial coin shapes 534 / 536 provide the first user 502 with a visual indication that the interaction with the virtual object 507 has triggered a spatial refinement in the multi-user communication session (and thus leads, for example, to the repositioning of the virtual object 507 and the visual representations of the users of the second electronic device 101b and the third electronic device, as described below).

[0081] Additionally, the first electronic device 101a shows in some examples how in Fig. 5B-1 shows that when and / or during a spatial refinement in the multi-user communication session, a virtual element 545 (e.g., a virtual ring, circle, plate, disk, or other virtual display) is displayed in the three-dimensional environment 550A. In some examples, as in Fig. As illustrated in Figure 5B-1, the virtual element 545 is displayed among the virtual objects moved according to the principles of spatial refinement, in particular the virtual object 507 and the spatial coin shapes 534 / 536 discussed above, in the three-dimensional environment 550A from the viewpoint of the first electronic device 101a. In some examples, a location where the virtual element 545 is displayed in the three-dimensional environment 550A is based on one or more boundaries of the three-dimensional environment 550A that correspond to one or more boundaries for the vertical movement of the virtual object 507 (e.g., while spatial refinement is active). For example, Figure 5B-1 shows that the virtual element 545 is displayed in the three-dimensional environment 550A. Fig. 5B-1 displays the first electronic device 101a and the virtual element 545 on a floor or substrate of the physical environment visible from the viewpoint of the first electronic device 101a in the three-dimensional environment 550A, and visually indicates to the first user 502 that the floor or substrate of the physical environment corresponds to a boundary for the vertical movement of the virtual object 507 in the three-dimensional environment 550A. In some examples, the virtual element 545 remains displayed in the three-dimensional environment 550A while spatial refinement is active in the multi-user communication session (e.g., for the duration that the first user 502 interacts with the virtual object 507 in the three-dimensional environment 550A). Additionally, in some examples, the virtual element 545 does not move (e.g.,(It is not moved laterally and / or raised or lowered vertically) in the three-dimensional environment 550A while spatial refinement is active in the multi-user communication session and while the virtual object 507 is moved vertically in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a.

[0082] It is understood that in some examples, the first electronic device 101a displays the virtual element 545 in the three-dimensional environment 550A for additional and / or alternative types of movement that trigger spatial refinement, such as a horizontal movement of the virtual object 507 in the three-dimensional environment 550A in the multi-user communication session. It is also understood that in some examples, the first electronic device 101a displays the virtual element 545 at additional and / or alternative locations in the three-dimensional environment 550A that correspond to the limits for the vertical movement of the virtual object 507 in the three-dimensional environment 550A, such as on a ceiling of the physical environment visible from the viewing angle of the first electronic device 101a in the three-dimensional environment 550A.

[0083] In some examples, such as in Fig. As shown in Figure 5B-2, the second electronic device 101b, as discussed above in a similar manner, updates the display of the avatar 516 corresponding to the first user 502 in the three-dimensional environment 550B, while the first electronic device 101a detects the air pinch-and-pull gesture directed at the virtual object 507, provided by the hand 503 of the first user 502. As shown in Figure 5B-2, the second electronic device 101b updates the display of the avatar 516 corresponding to the first user 502 in the three-dimensional environment 550B, while the first electronic device 101a detects the air pinch-and-pull gesture directed at the virtual object 507, provided by the hand 503 of the first user 502. Fig. As shown in Figure 5B-2, for example, the second electronic device 101b replaces the avatar 516, as shown in a side view 512, with a two-dimensional representation of the first user 502, such as a spatial coin shape 518, in the three-dimensional environment 550B. In some examples, as in Fig. As shown in Figure 5B-2, the spatial coin shape 518 includes an indication of the first user's name 502 (e.g., John Doe) and a two-dimensional visual representation (e.g., picture, photo, symbol, cartoon, sketch, etc.) of the first user 502. In some examples, the spatial coin shape 518 provides the second user 504 with a visual indication that the first user 502 is interacting with the virtual object 507 (e.g., moving the virtual object 507 as described above). In some examples, when the input is recognized by the first electronic device 101a, the first electronic device 101a transmits an indication of the input provided by the first user 502 to the second electronic device 101b, thereby enabling and / or causing the second electronic device 101b to update the display of the first user 502's visual representation in the three-dimensional environment 550B (e.g.,from the three-dimensional avatar to the two-dimensional spatial coin shape). In some examples, the spatial coin shape 518 exhibits one or more properties of the spatial coin shape 418 discussed above. Additionally, in some examples, as in . Fig. As shown in Figure 5B-2, the second electronic device 101b displays the visual representation of the third user in the three-dimensional environment 550B as the three-dimensional avatar 519 (e.g., thus foregoing the replacement of the avatar 519 with a spatial coin shape corresponding to the third user), since the motion input directed at the virtual object 507 is provided by the first user 502 on the first electronic device 101a. Furthermore, in some examples, as in Fig. 5B-2 shows a display of the virtual element 545 discussed above in the three-dimensional environment 550B, since the motion input directed towards the virtual object 507 is provided by the first user 502 of the first electronic device 101a.

[0084] In some examples, such as in Fig. As shown in Figure 5C, the first electronic device 101a, in response to the detection of input provided by hand 503, moves the virtual object 507 vertically in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a according to the movement of hand 503. Fig. As shown in Figure 5C, for example, the first electronic device 101a moves the virtual object 507 in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a according to the upward movement of the hand 503. Additionally, in some examples, the second electronic device 101b displays the avatar 516 again in the three-dimensional environment 550B in response to and / or after the first electronic device 101a has detected a completion (e.g., termination) of the input provided by the hand 503 (e.g., a release of the air pinch gesture provided by the hand 503) (e.g., displays the visual representation of the first user 502 again as a three-dimensional avatar).

[0085] Additionally, the above movement of the virtual object 507, which is a shared object in the multi-user communication session, as defined herein, optionally triggers spatial refinement. In particular, in some examples, as in Fig. 5A to Fig. Figure 5C shows the avatars 514 / 519 with the virtual object 507 in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a. For example, the first electronic device 101a moves the avatars 514 / 519 and the virtual object 507 simultaneously vertically upwards in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a according to the movement of the hand 503 in Fig. 5A. In order to maintain spatial truth within the spatial group of the first user 502, the second user 504, and the third user, as defined herein, the second electronic device 101b (e.g., and the third electronic device) in some examples moves the avatar 516 corresponding to the first user 502 in the three-dimensional environment 550B, based on the motion input detected on the first electronic device 101a (e.g., without moving the avatar 519 and the virtual object 507 in the three-dimensional environment 550B). In particular, since the motion input detected on the first electronic device 101a causes the shared virtual objects (e.g., the avatars 514 / 516 and the virtual object 507) in the three-dimensional environment 550A to be raised relative to the viewing angle of the first electronic device 101a (e.g., vertically), the second electronic device 101b (e.g.,and the third electronic device) the avatar 516 (e.g., by the same or a similar amount or degree, such as a distance) in the three-dimensional environment 550B relative to the viewing angle of the second electronic device 101b (e.g., such that a spatial arrangement and distribution of the avatars 514 / 519 and the virtual object 507 relative to the viewing angle of the first electronic device 101a in the three-dimensional environment 550A corresponds to a spatial arrangement and distribution of the avatar 519, the viewing angle of the second electronic device 101b, and the virtual object 507 relative to the avatar 516 in the three-dimensional environment 550B (e.g., is equal)), as in . Fig. 5C shown.

[0086] In some examples, the first electronic device 101a spatially refines the avatars 514 / 519 and the virtual object 507 in the three-dimensional environment 550A according to the movement of the hand 503 by moving the avatars 514 / 519 and the virtual object 507 relative to a respective surface in the three-dimensional environment 550A. For example, in Fig. 5C the first electronic device 101a a height of each of the avatars 514 / 519 and the virtual object 507 in the three-dimensional environment 550A, based on the movement of the hand 503, relative to the surface of the desk 506 in the three-dimensional environment 550A. In particular, in some examples, the first electronic device 101a correlates a magnitude (e.g., a distance) of the movement of the hand 503 in space relative to the viewing angle of the first electronic device 101a with a magnitude (e.g., a distance) of a vertical movement of the avatars 514 / 519 and the virtual object 507 relative to the surface of the desk 506. In some examples, the vertical movement of the avatars 514 / 519 and the virtual object 507 is chosen such that it is relative to the surface of the desk 506 in the three-dimensional environment 550A, since the virtual object 507 is positioned on the surface of the desk 506 (e.g.,displayed and / or anchored therein) when the input provided above by the hand 503 is recognized by the first electronic device 101a. Additionally or alternatively, in some examples, the vertical movement of the avatars 514 / 519 and the virtual object 507 is selected such that it is relative to the surface of the desk 506 in the three-dimensional environment 550A, since the virtual object 507 is a horizontally oriented virtual object (e.g., an object of the first type, as described above) or corresponds to one when the input provided above by the hand 503 is recognized by the first electronic device 101a.

[0087] In Fig. 5C detects the second electronic device 101b an input provided by the hand 505 of the second user 504, which corresponds to a request to move the virtual object 507 in the three-dimensional environment 550B relative to the viewing angle of the second electronic device 101b. As in Fig. As shown in Figure 5C, the second electronic device 101b, for example, recognizes an air pinch and pull gesture performed by hand 505, while optionally the gaze 526 of the second user 504 is directed at the gripper 511, which is linked to the virtual object 507 in the three-dimensional environment 550B. In some examples, as in Fig. 5C indicates that the input directed to the virtual object 507 corresponds to a request to move the virtual object 507 in the three-dimensional environment 550B relative to the viewing angle of the second electronic device 101b vertically downwards.

[0088] In some examples, such as in Fig. As shown in Figure 5D, the second electronic device 101b, in response to the detection of the movement of the second user's hand 505, moves the virtual object 507 vertically downwards in the three-dimensional environment 550B relative to the viewing angle of the second electronic device 101b, according to the movement of the hand 505. Additionally, in some examples, as discussed above, the second electronic device 101b performs spatial refinement according to the movement of the hand 505, since the motion input provided by the hand 505 is directed at a split virtual object. As shown in the side view 512 in Figure 550B, the second electronic device 101b moves the virtual object 507 vertically downwards relative to the viewing angle of the second electronic device 101b, according to the movement of the hand 505. Additionally, in some examples, as discussed above, the second electronic device 101b performs spatial refinement according to the movement of the hand 505, since the motion input provided by the hand 505 is directed at a split virtual object. Fig. As shown in 5D, for example, the second electronic device 101b moves (e.g., simultaneously) the avatars 516 / 519 and the virtual object 507 vertically downwards in the three-dimensional environment 550B according to the downward movement of the hand 505 (e.g., according to a magnitude of the movement of the hand 505). Additionally, in some examples, as in side view 510, it moves (e.g., vertically upwards). Fig. 5D shown, the first electronic device 101a (e.g., and the third electronic device) moves the avatar 514, corresponding to the second user 504, in the three-dimensional environment 550A based on the motion input detected on the second electronic device 101b (e.g., without moving the avatar 519 and the virtual object 507 in the three-dimensional environment 550A) to maintain spatial truth within the spatial group of the first user 502, the second user 504, and the third user, as discussed above in a similar manner.

[0089] In some examples, the spatial refinement of the virtual objects in the multi-user communication session is limited (e.g., restricted) by one or more boundaries associated with the three-dimensional environments 550A / 550B. In some examples, the one or more boundaries associated with the three-dimensional environments 550A / 550B correspond to physical boundaries of the physical environments within the three-dimensional environments 550A / 550B (e.g., they are defined by them). As in Fig. As shown in 5D, for example, the walls, ceiling, and / or floor of the physical environments of the first electronic device 101a and the second electronic device 101b define boundaries of the three-dimensional environments 550A / 550B. In some examples, the one or more boundaries associated with the three-dimensional environments 550A / 550B correspond to physical objects and / or surfaces of the physical objects in the physical environment within the three-dimensional environments 550A / 550B (e.g., they are defined by them). Fig. For example, the surfaces of the desk 506 and the table 530 define the boundaries of the three-dimensional environments 550A / 550B. In some examples, the one or more boundaries of the three-dimensional environment 550A on the first electronic device 101a may differ from the one or more boundaries of the three-dimensional environment 550B on the second electronic device 101b. For example, a boundary defined by a physical boundary of a physical environment in the three-dimensional environment 550A on the first electronic device 101a, such as a physical wall or ceiling, may be smaller (e.g., closer to the viewing angle of the first electronic device 101a) than a similar boundary defined by a physical boundary of the physical environment in the three-dimensional environment 550B on the second electronic device 101b.As another example, a boundary defined by a physical surface of a physical object in the physical environment in the three-dimensional environment 550A on the first electronic device 101a, such as the surface of the desk 506, can be at a height and / or location relative to the viewing angle of the first electronic device 101a like a similar boundary defined by a physical surface of a physical object (e.g. a surface of the table 530) in the physical environment in the three-dimensional environment 550B on the second electronic device 101b.

[0090] In some examples, the respective electronic device updates the display of the three-dimensional visual representation into a two-dimensional visual representation according to a stipulation that the vertical movement of the virtual object 507 (which, for example, triggers the spatial refinement discussed above) causes a respective user's three-dimensional visual representation to intersect and / or cross a boundary of one or more boundaries of the three-dimensional environment presented on a respective electronic device. For example, the second electronic device 101b in Fig. 5D, that when the virtual object 507, the avatar 516, and the avatar 519 are moved vertically downwards in the three-dimensional environment 550B relative to the viewing angle of the second electronic device 101b according to the movement of the hand 505, the avatar 516 corresponding to the first user 502 crosses (e.g., exceeds) a boundary of the three-dimensional environment 550B defined by the physical surface of the table 530, as illustrated by the location of the avatar 516 in the side view 512. Accordingly, the second electronic device 101b replaces, as in Fig. 5D shown, optionally the display of the avatar 516 by the spatial coin shape 518, which corresponds to a two-dimensional visual representation of the first user 502 in the three-dimensional environment 550B. In some examples, as in Fig. As shown in 5D, the movement of the virtual object 507, the avatar 514, and the avatar 519 according to the movement of the hand 503 does not cause the avatar 519 to cross a boundary of the three-dimensional environment 550A, and therefore the first electronic device 101a maintains the display of the avatar 519 in the three-dimensional environment 550A. In some examples, in Fig. In 5D, the first electronic device 101a further determines that when the avatar 514 is moved vertically upwards in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a according to the motion input detected on the second electronic device 101b, the avatar 514 corresponding to the second user 504 crosses (e.g., exceeds) a boundary of the three-dimensional environment 550A defined by the ceiling of the physical environment in which the first electronic device 101a is located, as illustrated by the location of the avatar 514 in the side view 510. Accordingly, the first electronic device 101a, as shown in Fig. 5D shown, optionally the display of the avatar 514 by the spatial coin shape 534, which corresponds to a two-dimensional visual representation of the second user 504 in the three-dimensional environment 550A. In some examples, the spatial coin shape 534 exhibits one or more properties of the spatial coin shape 518 discussed above.

[0091] In some examples, such as in Fig. In 5D illustration, the spatial coin shapes 518 / 524 are displayed at their respective locations in the three-dimensional environments 550A / 550B, which differ from the locations to which the avatars 516 / 514 are moved in their respective three-dimensional environments. For example, in Fig. 5D the spatial coin shape 534 is displayed in the field of view of the first electronic device 101a (e.g., although the avatar 514, which corresponds to the three-dimensional visual representation of the second user 504, is moved into the ceiling in the side view 510), and the spatial coin shape 518 remains displayed in the field of view of the second electronic device 101b (e.g., although the avatar 516, which corresponds to the three-dimensional visual representation of the first user 502, is moved into the surface of the table 530 in the side view 512).

[0092] In Fig. 5D, the first electronic device 101a detects an input made by the hand 503 of the first user 502, which corresponds to a request to move the virtual object 507 in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a. For example, as in Fig. As shown in 5D, the first electronic device 101a recognizes that the hand 503 is performing an air pinch and pull gesture, while optionally the gaze 526 of the first user 502 is directed towards the gripper 511 in the three-dimensional environment 550A. In some examples, as in Fig. 5D specified, the motion input provided by hand 503 corresponds to a requirement to move the virtual object 507 vertically upwards in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a.

[0093] In some examples, such as in Fig. As shown in Figure 5E, the first electronic device 101a, in response to the detection of the movement of the first user's hand 503, moves the virtual object 507 vertically upwards in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a, in accordance with the movement of the hand 503. Additionally, as discussed above, in some examples the first electronic device 101a performs spatial refinement in accordance with the movement of the hand 503, since the motion input provided by the hand 503 is directed at a split virtual object. For example, as shown in side view 510 in Figure 5E, the first electronic device 101a moves the virtual object vertically upwards in response to the detection of the movement of the hand 503. Fig. As shown in Figure 5E, the first electronic device 101a moves (e.g., simultaneously) the avatars 514 / 519 and the virtual object 507 vertically upwards in the three-dimensional environment 550A according to the upward movement of the hand 503 (e.g., according to a magnitude of the movement of the hand 503). Additionally, in some examples, as in side view 512 in Figure 5E, the device 507 moves the avatars 514 / 519 and the virtual object 507 vertically upwards. Fig. 5E shown, the second electronic device 101b (e.g., and the third electronic device) moves the avatar 516 corresponding to the first user 502 (e.g., vertically downwards) in the three-dimensional environment 550B based on the motion input detected on the first electronic device 101a (e.g., without moving the avatar 519 and the virtual object 507 in the three-dimensional environment 550B) to maintain spatial truth within the spatial group of the first user 502, the second user 504, and the third user, as discussed above in a similar manner.

[0094] As in Fig. As shown in Figure 5E, in some examples the vertical movement of the virtual object 507, the avatar 514, and the avatar 519 in the three-dimensional environment 550A corresponds to a movement of the avatar 514 further upwards in the three-dimensional environment 550A and into the upper boundary of the three-dimensional environment 550A (e.g., defined by the ceiling), as illustrated in the side view 510. Additionally, in the example of Fig. 5E the vertical movement of the virtual object 507, the avatar 514, and the avatar 519, a movement of the avatar 519, corresponding to the third user, into the upper boundary of the three-dimensional environment 550A, as illustrated in the side view 510. Accordingly, the first electronic device 101a, as discussed above in a similar manner, replaces the display of the avatar 519 in the three-dimensional environment 550A with the spatial coin shape 536, which corresponds to a two-dimensional visual representation of the third user. In some examples, the spatial coin shape 536 has one or more features of the spatial coin shapes 518 / 534 discussed above.

[0095] In some examples, a given electronic device updates, according to a provision that a given three-dimensional visual representation of a given user is redisplayed as a two-dimensional visual representation (e.g., a spatial coin shape) because the three-dimensional visual representation at least partially intersects a boundary of the three-dimensional environment in which the three-dimensional visual representation is displayed, the display of all three-dimensional visual representations of users of the electronic devices in such a way that they are two-dimensional visual representations in the three-dimensional environment (e.g., regardless of which three-dimensional visual representations actually intersect a boundary of a given three-dimensional environment). For example, in Fig. 5E, as illustrated in side view 512, the avatar 519 does not move into a boundary of the three-dimensional environment 550B on the second electronic device 101b (e.g., in response to user input detected on the first electronic device 101a or on the second electronic device 101b). However, since, as in Fig. As shown in Figure 5E, when at least one of the avatars representing a user of an electronic device in multi-user communication has been moved into a boundary of the three-dimensional environment on the electronic device, the second electronic device 101b replaces the display of avatar 519, corresponding to the third user, with the spatial coin shape 536 already discussed above. For example, on the second electronic device 101b, avatar 516, corresponding to the first user 502, has been moved into the boundary of the three-dimensional environment 550B, defined by the surface of the table 530, as shown in side view 512 in Figure 5E. Fig. Figure 5E illustrates how the second electronic device 101b replaces the display of both avatars 516 / 519 with their respective spatial coin forms 518 / 536. This maintains display consistency of the visual representations across all electronic devices in the multi-user communication session, regardless of whether users of the electronic devices are represented spatially (e.g., via a three-dimensional visual representation) or non-spatially (e.g., via a two-dimensional visual representation) in the three-dimensional environment. This helps to improve the overall user experience in the multi-user communication session by synchronizing the specific display of the visual representations and the interactions between them.

[0096] Fig. Sections 5F through 5G illustrate alternative examples for facilitating vertical movement of a shared virtual object, thereby triggering spatial refinement of the shared virtual objects in the multi-user communication session. For example, in Fig. 5F, while the first user 502, the second user 504, and the third user (e.g., not shown) are in the multi-user communication session via their respective electronic devices, the first electronic device 101a and the second electronic device 101b display the virtual object 507, which corresponds to a shared object in the multi-user communication session, as discussed similarly above. Additionally, in some examples, as in Fig. Figure 5F shows the spatial coin shape 534, corresponding to the second user 504, and the spatial coin shape 536, corresponding to the third user, in the three-dimensional environment 550A. In some examples, as in Fig. As shown in Figure 5F, the second electronic device 101b displays the avatar 516, corresponding to the first user 502, and the avatar 519, corresponding to the third user, in the three-dimensional environment 550B.

[0097] In some examples, the display status of the users' visual representations in the multi-user communication session does not need to be maintained (e.g., synchronized) on all electronic devices associated with the users in the multi-user communication session. In particular, some examples show how in Fig. As shown in Figure 5F, the first electronic device 101a displays the visual representations of the second user 504 and the third user as non-spatial two-dimensional representations (e.g., the spatial coin shapes 534 / 536) in the three-dimensional environment 550A, while the second electronic device 101b displays the visual representations of the first user 502 and the third user as spatial three-dimensional representations (e.g., the avatars 516 / 519) in the three-dimensional environment 550B. In some examples, the asynchronous display states of the users' visual representations in the multi-user communication session are specific to the locations of the visual representations relative to one or more boundaries of the users' three-dimensional environments in the multi-user communication session. As shown in side view 510 in Fig. As shown in Figure 5F, for example, the visual representations of the second user 504 and the third user are positioned at least partially outside and / or beyond a boundary of the three-dimensional environment 550A (e.g., as in a ceiling of the physical environment and / or outside a vertical threshold distance 547 (e.g., a threshold elevation and / or height) relative to the head of the first user 502, such as more than 0.25, 0.5, 0.75, 1, 1.5, 2, or 3 meters above the head of the first user 502), which causes the first electronic device 101a to display the visual representations as the spatial coin shapes 534 / 536 in the three-dimensional environment 550A. On the other hand, as shown in the side view 512 in Fig. 5F illustrates that the visual representations of the first user 502 and the third user are positioned within the boundaries of the three-dimensional environment 550B (e.g., as above the floor or subsurface of the physical environment visible in the three-dimensional environment 550B), enabling the second electronic device 101b to display the visual representations as the avatars 516 / 519 in the three-dimensional environment 550B.

[0098] Although in some examples the display of users' visual representations in the multi-user communication session is not limited and / or restricted by one or more boundaries of a respective user's three-dimensional environment, the display of the content shared in the multi-user communication session is limited and / or restricted by one or more boundaries of a respective user's three-dimensional environment. Fig. 5F detects the first electronic device 101a an input provided by the hand 503 of the first user 502, which corresponds to a request to move the virtual object 507 in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a. As in Fig. As shown in Figure 5F, for example, the first electronic device 101a recognizes that the hand 503 is performing an air pinch and pull gesture, while optionally the gaze 526 of the first user 502 is directed towards the gripper 511, which is linked to the virtual object 507 in the three-dimensional environment 550A. In some examples, as in Fig. As specified in Figure 5F, the movement of hand 503 corresponds to a request to move the virtual object 507 vertically upwards in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a. In some examples, the input provided by hand 503 is recognized by the first electronic device 101a while the virtual object 507 is located at a position in the three-dimensional environment 550A that lies within one or more boundaries of the three-dimensional environment 550A. As shown in the side view 510 in Figure 5F, the movement of hand 503 corresponds to a request to move the virtual object 507 vertically upwards in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a. In some examples, the input provided by hand 503 is recognized by the first electronic device 101a while the virtual object 507 is located at a position in the three-dimensional environment 550A that lies within one or more boundaries of the three-dimensional environment 550A. Fig. As illustrated in Figure 5F, for example, the first electronic device 101a recognizes the input discussed above, while the virtual object 507 is displayed at a location that has a elevation (e.g., a height) above the floor or subsurface of the physical environment in which the first user 502 is located (e.g., corresponding to a minimum limit) and below a maximum limit defined by the threshold distance 547. In some examples, the threshold distance 547 is measured relative to the position of the first electronic device 101a on the head of the first user 502. In other examples, the threshold distance 547 is measured relative to the head of the first user 502, regardless of the posture of the first user 502 in the physical environment of the first user 502 (e.g., regardless of whether the first user 502 is sitting (e.g., as in the side view 510 in Figure 5F). Fig. 5F illustrates) or stands (e.g., as similarly illustrated by the second user 504 in page view 512 in Fig. 5F illustrates)). In some examples, the maximum limit corresponds to the ceiling or other physical object or boundary of the physical environment in which the first user 502 is located.

[0099] In some examples, such as in Fig. As illustrated in 5G, the input directed to the virtual object 507 corresponds to a request to move the virtual object 507 vertically beyond the maximum or upper limit of the three-dimensional environment 550A. As shown in the side view 510 in Fig. As illustrated by 5G, for example, the first electronic device 101a moves the virtual object 507 upwards and outside the threshold distance 547 relative to the head of the first user 502 in the three-dimensional environment 550A, in accordance with the movement of the hand 503. Additionally, in some examples, as described above, the vertical movement of the virtual object 507 in the three-dimensional environment 550A in response to the input provided by the hand 503 triggers spatial refinement in the multi-user communication session (e.g., because the virtual object 507 is shared in the multi-user communication session). Accordingly, in some examples, as in Fig. 5G demonstrates that when the virtual object 507 is moved upwards in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a, the spatial coin shapes 534 / 536 move further vertically upwards in the three-dimensional environment 550A along with the virtual object 507, such that the spatial coin shapes 534 / 536 are no longer visible (e.g., not currently visible) from the viewing angle of the first electronic device 101a in the three-dimensional environment 550A. In some examples, a "rubber band" effect is applied to the movement of the shared virtual object in the three-dimensional environment in response to the detection of an input that causes a shared virtual object to move vertically out of and / or beyond a boundary of a three-dimensional environment in a multi-user communication session. In particular, in the example of Fig. 5G applies a physics-based motion model (e.g., a set of simulated physical laws, such as a spring-based model (e.g., Hooke's Law)) to the vertical motion of the virtual object 507 beyond the upper boundary in the three-dimensional environment 550A, as discussed in more detail below. Fig. 5G detects the first electronic device 101a that the hand 503 of the first user 502 continues to perform the air pinch gesture, but optionally stops detecting the movement of hand 503.

[0100] While in some examples the physics-based motion model is applied to the vertical movement of the virtual object 507 beyond the upper boundary in the three-dimensional environment 550A, further and / or additional movement of the virtual object 507, which causes the virtual object 507 to move further beyond the upper boundary in the three-dimensional environment 550A, encounters a higher simulated resistance. For example, if in Fig. When the first electronic device 101a detects a further movement of the hand 503 while maintaining the air pinch gesture, the distance by which the virtual object 507 is moved vertically beyond the upper limit (e.g., beyond the threshold distance 547) is proportional to the magnitude (e.g., of a distance) of the movement of the hand 503. However, due to the simulated resistance in response to the detection of continued movement of the hand 503, the distance moved is progressively smaller. In some examples, the simulated resistance increases proportionally, exponentially, and / or in another combination based on the distance by which the virtual object 507 is moved beyond the limit in the three-dimensional environment 550A.

[0101] In some examples, such as in Fig. As shown in Figure 5H, the first electronic device 101a, in response to detecting the termination (e.g., of an end) of the air pinch gesture performed by hand 503, binds the virtual object 507 as if with a rubber band, so that the virtual object 507 is no longer displayed at a location beyond the upper limit (e.g., outside the threshold distance 547), but is moved to a location that, from the viewing angle of the first electronic device 101a, lies within one or more limits (e.g., within the threshold distance 547 relative to the head of the first user 502) in the three-dimensional environment 550A. For example, although the movement of the virtual object 507 according to the movement of hand 503 causes the virtual object 507 to move beyond the maximum limit and (e.g.,(temporarily) displayed at a location outside the maximum limit in the three-dimensional environment 550A, in response to the detection of the end of the input provided by hand 503, the rubber band effect is applied to the virtual object 507 to maintain the display of the virtual object 507 at a location that, from the viewing angle of the first electronic device 101a, lies below the maximum limit in the three-dimensional environment 550A. In some examples, the detection of the termination of input by hand 503 of the first user 502 includes the detection of a release of the air pinch gesture, such as when the index finger and thumb of hand 503 are no longer in contact.In some examples, detecting the termination of the input provided by the hand 503 of the first user 502 includes detecting an updated posture of the hand 503 and / or arm of the first user 502, such as a movement of the hand 503 into a resting position or a resting state (e.g., at one side of the first user 502). In some examples, the first electronic device 101a animates the movement of the virtual object 507 from the location outside and / or beyond the upper boundary (e.g., in ). Fig. 5G) to the location below and / or within the upper limit (e.g. in Fig. 5H) according to the set of simulated physical laws discussed above. For example, the first electronic device 101a, in response to the detection of the termination of the input provided by hand 503, determines Fig. 5G until Fig. 5H a displacement amount (e.g. a distance) between the location of the virtual object 507 in Fig. 5G and the location of virtual object 507 in Fig. 5H and animates a translation of the virtual object 507 with a quantity (e.g., an acceleration) based on the displacement. In some examples, the relationship between the displacement and the simulated acceleration of the virtual object 507, which moves back to the location within one or more boundaries in the three-dimensional environment 550A, is linear, exponential, logarithmic, and / or exhibits another nonlinear relationship.

[0102] In some examples, the rubber band effect described above, which is applied to the movement of the virtual object 507, is similarly applied to the visual representations (e.g., the spatial coin shapes 534 / 536) of the second user 504 and the third user in the three-dimensional environment 550A when the visual representations are moved relative to the viewing angle of the first electronic device 101a. For example, if in Fig. 5H when the termination of the input provided by hand 503 is detected, the first electronic device 101a moves the spatial coin shapes 534 / 536 in the three-dimensional environment 550A (e.g., downwards in height / rise towards the upper boundary of the three-dimensional environment 550A) with the downward movement of the virtual object 507 relative to the viewing angle of the first electronic device 101a (e.g., according to the set of simulated physical laws discussed above). Since in the example of Fig. Since visual representations of users in the multi-user communication session can be displayed and / or positioned beyond one or more boundaries of the three-dimensional environment 550A, the application of the rubber band effect does not require that the spatial coin shapes 534 / 536, like the virtual object 507, also be moved to a location within the upper boundary in the three-dimensional environment 550A, causing the spatial coin shapes 534 / 536 to remain outside the current field of view of the first electronic device 101a in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a.

[0103] In some examples, as described above, the movement of the virtual object 507 in the three-dimensional environment 550A by the first electronic device 101a triggers a spatial refinement in the multi-user communication session (e.g., because the virtual object 507 is shared in the multi-user communication session). Accordingly, when the input provided by the hand 503 of the first user 502 is terminated, the second electronic device 101b moves the visual representation of the first user 502 (e.g., the avatar 516) in the three-dimensional environment 550B by a corresponding size and / or direction, although this in Fig. 5H is not illustrated. For example, as discussed similarly herein, the first electronic device 101a represents the virtual object 507 in the three-dimensional environment 550A of Fig. 5F after Fig. After moving 5H vertically upwards, the second electronic device 101b moves the avatar 516 in the three-dimensional environment 550B relative to the viewing angle of the second electronic device 101b and optionally by a corresponding displacement amount (e.g. a corresponding vertical distance) vertically downwards.

[0104] In some examples, the first user 502 receives visual feedback regarding the response to the input to move the virtual object 507 within the three-dimensional environment 550A by being able to move it (e.g., temporarily) beyond a boundary. Additionally, animating the movement of the virtual object 507 from the location beyond the boundary to the location within the boundary in the three-dimensional environment 550A, according to the simulated physical laws outlined above, helps to avoid eye strain and general difficulties for the user associated with moving the virtual object 507 so that it is within the boundary in the three-dimensional environment 550A, and maintains the visibility of the content of the virtual object 507 from the user's perspective, which is advantageous.It is understood that the rubber band effect described above is applied in a similar way to the movement of a shared virtual object beyond additional or alternative boundaries in the three-dimensional environment, such as the minimum boundary discussed above, which corresponds to the ground or subsurface of the physical environment in which a particular user is located.

[0105] Fig. Sections 5I to 5K illustrate alternative examples for facilitating vertical movement of a shared virtual object, thereby triggering spatial refinement of the shared virtual objects in the multi-user communication session. For example, in Fig. 5I, while the first user 502, the second user 504, and the third user (e.g., not shown) are in the multi-user communication session via their respective electronic devices, the first electronic device 101a and the second electronic device 101b access the virtual object 540, which corresponds to a shared object in the multi-user communication session. In some examples, as in Fig. As shown in Figure 5I, the virtual object 540 is displayed with a gripper or handle 541 (e.g., with one or more properties of the gripper 511 described above) and a pill-shaped box 542, indicating that the virtual object 540 is shared in the multi-user communication session, as discussed above in a similar manner. Additionally, in some examples, the virtual object 540 is associated with a respective application running on the electronic device 101a / 101b, such as a text editing application. In some examples, as discussed above, the three-dimensional environment 550A presented on the first electronic device 101a includes the avatar 514 corresponding to the second user 504 and the avatar 519 corresponding to the third user.Similarly, in some examples, the three-dimensional environment 550B presented on the second electronic device 101b includes the avatar 516, corresponding to the first user 502, and the avatar 519.

[0106] In Fig. 5I The first electronic device 101a detects an input provided by the hand 503 of the first user 502, which corresponds to a request to move the virtual object 540 in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a. As in Fig. As shown in Figure 5I, for example, the first electronic device 101a recognizes that the hand 503 is performing an air pinch and pull gesture, while optionally the gaze 526 of the first user 502 is directed towards the gripper 541, which is linked to the virtual object 540 in the three-dimensional environment 550A. In some examples, as in Fig. As specified in 5I, the movement of the hand 503 corresponds to a requirement to move the virtual object 540 in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a vertically downwards.

[0107] In some examples, such as in Fig. As shown in Figure 5J, the first electronic device 101a, in response to the detection of input from the hand 503 of the first user 502, moves the virtual object 540 in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a, vertically downwards according to the movement of the hand 503. Additionally, as discussed above, in some examples the first electronic device 101a performs spatial refinement according to the movement of the hand 503, since the motion input provided by the hand 503 is directed at a split virtual object. As shown in the side view 510 in Fig. As shown in Figure 5J, for example, the first electronic device 101a moves (e.g., simultaneously) the avatars 514 / 519 and the virtual object 540 in the three-dimensional environment 550A vertically downwards according to the downward movement of the hand 503 (e.g., according to a magnitude of the movement of the hand 503). Additionally, in some examples, as in side view 512, it moves (e.g., vertically upwards). Fig. 5J shown, the second electronic device 101b (e.g., and the third electronic device) moves the avatar 516 corresponding to the first user 502 in the three-dimensional environment 550B based on the motion input detected on the first electronic device 101a (e.g., without moving the avatar 519 and the virtual object 540 in the three-dimensional environment 550B) to maintain spatial truth within the spatial group of the first user 502, the second user 504, and the third user, as discussed above in a similar manner.

[0108] In some examples, the first electronic device 101a refines the avatars 514 / 519 and the virtual object 540 spatially in the three-dimensional environment 550A according to the movement of the hand 503 by moving the avatars 514 / 519 and the virtual object 507 relative to a respective surface in the three-dimensional environment 550A. For example, in Fig. 5C the first electronic device 101a a height of each of the avatars 514 / 519 and the virtual object 540 in the three-dimensional environment 550A, based on the movement of the hand 503, relative to a floor or ground of the physical environment in which the first electronic device 101a is located and which is visible in the three-dimensional environment 550A. In particular, in some examples, the first electronic device 101a correlates a magnitude (e.g., a distance) of the movement of the hand 503 in space relative to the viewing angle of the first electronic device 101a with a magnitude (e.g., a distance) of a vertical movement of the avatars 514 / 519 and the virtual object 540 relative to the floor or ground in the three-dimensional environment 550A.In some examples, the vertical movement of the avatars 514 / 519 and the virtual object 540 is chosen to be relative to the ground or substrate in the three-dimensional environment 550A, since the virtual object 540 is not positioned (e.g., displayed and / or anchored) on a surface of a physical object in the three-dimensional environment 550A, such as the desk 506, when the input provided by the hand 503 above is detected by the first electronic device 101a. Additionally or alternatively, in some examples, the vertical movement of the avatars 514 / 519 and the virtual object 540 is chosen to be relative to the ground or substrate in the three-dimensional environment 550A, since the virtual object 540 is a vertically oriented virtual object (e.g.,an object of the second type, as described above) is or corresponds to such an object if the input provided by hand 503 above is recognized by the first electronic device 101a.

[0109] Alternatively, in some examples, in response to the detection of input directed at a shared virtual object in the multi-user communication session that corresponds to a vertical movement of the shared virtual object, spatial refinement of the virtual objects in the multi-user communication session is selectively triggered based on a type of shared virtual object. In some examples, the type of shared virtual object is based on an orientation and / or dimensionality (e.g., volume) of the shared virtual object. For example, in Fig. 5I Virtual object 540 is a vertically oriented virtual object, as illustrated in side views 510 and 512, such that the contents of virtual object 540 (e.g., the text editing user interface illustrated in virtual object 540) are displayed on and / or contained within a front-facing surface of virtual object 540. As another example, virtual object 540 is a two-dimensional virtual object (e.g., a two-dimensional virtual window, as discussed similarly above). On the other hand, virtual object 507, again with reference to Fig. 5A above, to create a different type of shared virtual object in the multi-user communication session. For example, in Fig. 5A Virtual object 507 is a horizontally oriented virtual object, as illustrated in side views 510 and 512, such that the content of virtual object 507 (e.g., the image of the aircraft illustrated in virtual object 507) is displayed on and / or contained within a top surface of virtual object 507. As another example, virtual object 540 is a three-dimensional virtual object (e.g., a volumetric virtual object).

[0110] In some examples, in response to the detection of input directed at a shared virtual object in the multi-user communication session and corresponding to a vertical movement of the shared virtual object, according to a determination that the shared virtual object is a first type of virtual object (e.g., a horizontally oriented and / or a volumetric virtual object, such as virtual object 507 in Fig. 5A), a spatial refinement is triggered in the multi-user communication session. In some examples, according to a determination that the shared virtual object is a second type of virtual object (e.g., a vertically oriented and / or a two-dimensional virtual object, such as virtual object 540 in Fig. 5I) does not trigger spatial refinement in the multi-user communication session. In some examples, the determination of the type of shared virtual object is based on data provided by the application to which the shared virtual object is associated. For example, in Fig. 5I the first electronic device 101a, that the virtual object 540 is a vertically oriented and / or a two-dimensional virtual object, based on data provided by the text editing application running on the first electronic device 101a and associated with the virtual object 540 (e.g., the data provided by the text editing application identify the virtual object 540 as vertically oriented and / or as not being a volumetric virtual object).

[0111] As discussed above, the first electronic device 101a determines in the example of Fig. 5I, in response to the recognition of the input provided by the hand 503 of the first user 502, which corresponds to the request to move the virtual object 540 vertically in the three-dimensional environment 550A, determines the type of the virtual object 540 and, based on this determination, selectively triggers a spatial refinement in the multi-user communication session. As mentioned above, the virtual object 540 corresponds to Fig. 5I a vertically oriented and / or two-dimensional virtual object. Accordingly, in some examples, the first electronic device 101a, in response to the recognition of the input provided by the first user's hand 503, which corresponds to the request to move the virtual object 540 vertically in the three-dimensional environment 550A, refrains from triggering spatial refinement in the multi-user communication session. In particular, the first electronic device 101a, as in Fig. 5I to Fig. 5K shows the movement of the virtual object 540 vertically (e.g. downwards) in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a. Furthermore, in some examples, as in Fig. 5K shown, on the movement of avatars 514 / 519 in the three-dimensional environment 550A according to the in Fig. 5I illustrated the movement of hand 503. Accordingly, the second electronic device 101b omits this in some examples, as in Fig. 5K shows the updating of the display of the avatar 516, which corresponds to the first user 502 in the three-dimensional environment 550B, since no spatial refinement is triggered in the multi-user communication session in response to the recognition of the input provided by the hand 503 of the first user 502. As shown in Fig. As shown in Figure 5K, the second electronic device 101b, for example, refrains from moving the avatar 516 vertically (e.g., upwards) in the three-dimensional environment 550B relative to the viewing angle of the second electronic device 101b. In the case where the vertical movement of a shared virtual object, and thus spatial refinement in a multi-user communication session, is based on the type of shared virtual object, an input directed at the shared virtual object to move it vertically will cause the shared virtual object to move vertically and therefore trigger spatial refinement if the shared virtual object is a horizontally oriented virtual object, as discussed above. For example, in Fig. 5A the first electronic device 101a the vertical movement of the virtual object 507 in the three-dimensional environment 550A and triggers a spatial refinement in the multi-user communication session in response to the recognition of the input provided by the hand 503 of the first user 502, since the virtual object 507 is a horizontally oriented virtual object and / or a volumetric (e.g. three-dimensional) virtual object, as mentioned above.

[0112] Accordingly, as outlined above, providing systems and methods to facilitate vertical movement of virtual content in a shared three-dimensional environment during a multi-user communication session advantageously enables users to participate in the multi-user communication session and experience and interact with synchronized movement of the virtual content, thereby improving user-device interaction.Additionally, the vertical movement of virtual content in the shared three-dimensional environment relative to a specific surface, selected based on an orientation associated with the virtual content, helps to improve the accuracy of the movement of the virtual content based on the motion input directed at the virtual content. This helps to reduce user input that would be required to correct inconsistencies in the movement in the shared three-dimensional environment, thereby saving computing resources that would otherwise be consumed to respond to such user input, which is a further advantage.

[0113] It is understood that the examples shown and described here are merely illustrative and that additional and / or alternative elements within the three-dimensional environment may be provided for interaction with the illustrated content. It is also understood that the appearance, shape, form, and size of the various user interface elements and objects shown and described here are exemplary and that alternative appearances, shapes, and / or sizes may be provided. For example, the virtual objects representing application windows (e.g., virtual objects 407, 432, 507, and 540) may be provided in a shape other than rectangular, such as circular or triangular, etc. In some examples, the various selectable options (e.g.,Option 423 and / or pill-shaped boxes 417, 431, 517, and / or 542), user interface elements (e.g., menu item 424), etc., described herein, can be selected verbally via verbal user commands (e.g., the verbal command "Select Option"). Additionally or alternatively, in some examples, the various options, user interface elements, controls, etc., described herein can be selected and / or manipulated by user input received via one or more separate input devices communicating with the electronic device(s). Selection can be made, for example, via physical input devices such as a mouse, trackpad, keyboard, etc., communicating with the electronic device(s).

[0114] Fig. Figure 6 illustrates a flowchart that demonstrates an exemplary process for presenting content in a three-dimensional environment relative to surfaces in the three-dimensional environment in a multi-user communication session according to some examples of the disclosure. In some examples, a process 600 begins at a first electronic device that communicates with one or more displays and one or more input devices. In some examples, the first electronic device is optionally a head-mounted display connected to the electronic device 260. Fig. 2 resembles or corresponds to. As in Fig. As shown in Figure 6, in some examples at 602, during a communication session with a second electronic device, and while a visual representation of a user of the second electronic device in a three-dimensional environment is displayed on one or more displays, wherein the visual representation is displayed from a viewpoint of the first electronic device at a first height relative to a first surface in the three-dimensional environment, a first indication of a request to share content of a first type in the communication session. As shown in Figure 6, the first electronic device receives, in some examples at 602, during a communication session with a second electronic device and while a visual representation of a user of the second electronic device in a three-dimensional environment is displayed on one or more displays, wherein the visual representation is displayed from a viewpoint of the first electronic device at a first height relative to a first surface in the three-dimensional environment, a first indication of a request to share content of a first type in the communication session. Fig. As shown in Figure 4A, for example, the first electronic device 101a recognizes a selection of the pill-shaped box 417, which is associated with the virtual object 407 in the three-dimensional environment 450A, thus fulfilling a requirement to initiate the sharing of the contents of the virtual object 407 with the second electronic device 101b. Additionally, the first electronic device shows

[0115] Device 101a in Fig. 4A, when the selection of the pill-shaped box 417 is detected, the avatar 414, corresponding to the second user 404, is displayed at a respective height relative to a ground or subsurface of the three-dimensional environment 450A.

[0116] In some examples, the first electronic device at 604, in response to the recognition of the first specification, displays a first object on one or more displays that corresponds to the content of the first type in the three-dimensional environment. As in Fig. As shown in Figure 4C, for example, the first electronic device 101a updates the display of the pill-shaped box 417 of the virtual object 407 to indicate that the virtual object 407 has been shared in the multi-user communication session. In some examples, while displaying the first object and the user of the second electronic device in the three-dimensional environment, the first electronic device at 606 receives a second instruction to link the first object to a second surface, different from the first surface, in the three-dimensional environment. As shown in Fig. As shown in Figure 4C, for example, the first electronic device 101a recognizes a motion input provided by the hand 403 of the first user 402, which corresponds to a request to move the virtual object 407 onto the surface of the desk 406 in the three-dimensional environment 450A.

[0117] In some examples, the first electronic device at 608, in response to receiving the second input at 610, links the first object to the second surface in the three-dimensional environment. As in Fig. As shown in Figure 4E, for example, the first electronic device 101a displays the virtual object 407 on the surface of the desk 406 and with a horizontal orientation in the three-dimensional environment 450A. In some examples, the first electronic device at 612 updates the user's visual representation of the second electronic device via one or more displays, so that it is displayed from the viewpoint of the first electronic device at a second height, different from the first height, relative to the second surface in the three-dimensional environment. As shown in the side view 410 in Fig. As shown in Figure 4E, for example, the first electronic device 101a updates the display of the avatar 414, which corresponds to the second user 404, so that it is displayed at height 427, which is relative to the surface of the desk 406 in the three-dimensional environment 450A.

[0118] It is understood that Process 600 is an example and that more, fewer, or different operations can be performed in the same or a different sequence. Additionally, the operations described above in Process 600 are optionally implemented by one or more functional modules in an information processing device, such as general-purpose processors (e.g., with reference to...). Fig. 2 described) or application-specific chips, and / or by other components of Fig. 2 is / will be executed.

[0119] Fig. Figure 7 illustrates a flowchart that demonstrates an exemplary process for virtually moving shared content in a three-dimensional environment relative to surfaces in the three-dimensional environment in a multi-user communication session according to some examples of the disclosure. In some examples, a process 700 begins at a first electronic device that communicates with one or more displays and one or more input devices. In some examples, the first electronic device is optionally a head-mounted display connected to the electronic device 260. Fig. 2 resembles or corresponds to. As in Fig. As shown in Figure 7, in some examples at 702, during a communication session with a second electronic device, the first electronic device displays, via one or more displays, a visual representation of a user of the second electronic device and a first object corresponding to shared content in a three-dimensional environment, wherein the visual representation is displayed from a viewpoint of the first electronic device at a first height relative to a first surface in the three-dimensional environment. For example, as shown in Figure 7, the first electronic device displays a visual representation of a user of the second electronic device and a first object corresponding to shared content in a three-dimensional environment, from a viewpoint of the first electronic device at a first height relative to a first surface in the three-dimensional environment. Fig. Figure 5A shows the first electronic device 101a, while the first electronic device 101a is in a multi-user communication session with the second electronic device 101b, displaying the virtual object 507 in the three-dimensional environment 550A, which corresponds to a shared virtual object in the multi-user communication session. Additionally, Figure 5A shows the first electronic device 101a, while the first electronic device 101a is in a multi-user communication session with the second electronic device 101b, displaying the virtual object 507 in the three-dimensional environment 550A, which corresponds to a shared virtual object in the multi-user communication session. Fig. 5A the first electronic device 101a, while the virtual object 507 is displayed in the three-dimensional environment 550A, the avatar 514, corresponding to the second user 504, at a respective height relative to a surface of the desk 506 in the three-dimensional environment 550A.

[0120] In some examples, the first electronic device detects at 704, while the user's visual representation of the second electronic device and the first object in the three-dimensional environment is displayed, via which one or more input devices receive a first input corresponding to a request to move the first object vertically in the three-dimensional environment relative to the viewing angle of the first electronic device. As in Fig. As shown in Figure 5A, for example, the first electronic device 101a recognizes an input provided by the hand 503, which corresponds to a request to move the virtual object 507 in the three-dimensional environment 550A vertically upwards relative to the viewing angle of the first electronic device 101a.

[0121] In some examples, at 706, in response to the detection of the first input, the first electronic device moves the first object and the user's visual representation of the second electronic device vertically in the three-dimensional environment relative to the viewing angle of the first electronic device according to the first input. As in Fig. As shown in Figure 5C, for example, the first electronic device 101a moves the virtual object 507 and the avatar 514 in the three-dimensional environment 550A relative to the viewing angle of the first electronic device 101a according to the movement of the hand 503 vertically upwards. In some examples, at Figure 708, the first electronic device displays the user's visual representation of the second electronic device at a second height, which differs from the first height, relative to the first surface in the three-dimensional environment, via one or more displays. For example, if in Fig. 5C When the avatar 514 is moved vertically upwards in the three-dimensional environment 550A, the avatar 514 is displayed in the three-dimensional environment 550A at a height relative to the surface of the desk 506 that is higher than the height of the avatar 514 relative to the surface of the desk 506 when the input provided by the hand 503 is in Fig. 5A is detected.

[0122] It is understood that Process 700 is an example and that more, fewer, or different operations can be performed in the same or a different sequence. Additionally, the operations described above in Process 700 are optionally implemented by one or more functional modules in an information processing device, such as general-purpose processors (e.g., with reference to...). Fig. 2 described) or application-specific chips, and / or by other components of Fig. 2 is / will be executed.

[0123] Therefore, according to the above, some examples of the disclosure relate to a method comprising, on a first electronic device communicating with one or more display(s) and one or more input device(s): during a communication session with a second electronic device and while displaying a visual representation to a user of the second electronic device in a three-dimensional environment via the one or more display(s), wherein the visual representation is displayed from a viewpoint of the first electronic device at a first height relative to a first surface in the three-dimensional environment, receiving a first indication of a request to share content of a first type in the communication session;in response to receiving the first instruction, displaying a first object corresponding to the content of the first type in the three-dimensional environment via one or more displays; while the first object and the user's visual representation are displayed to the second electronic device in the three-dimensional environment, receiving a second instruction requesting the first object to link to a second surface in the three-dimensional environment that is different from the first surface;and in response to receiving the second input, linking the first object to the second surface in the three-dimensional environment and updating the display of the user's visual representation of the second electronic device via the one or more displays, so that it is displayed from the viewpoint of the first electronic device at a second height that differs from the first height, relative to the second surface in the three-dimensional environment.

[0124] Additionally or alternatively, in some examples, the first surface in the three-dimensional environment corresponds to a physical ground or subsurface of a physical surface visible in the three-dimensional environment from the viewpoint of the first electronic device. Additionally or alternatively, in some examples, the second surface in the three-dimensional environment corresponds to a physical surface of a physical object visible in the three-dimensional environment from the viewpoint of the first electronic device. Additionally or alternatively, in some examples, the first surface in the three-dimensional environment corresponds to a physical ground or subsurface of a physical surface visible in the three-dimensional environment from the viewpoint of the first electronic device.Additionally or alternatively, in some examples, the method further comprises, during the recognition of the first input, displaying, via one or more displays, a visual indication of a boundary associated with the movement of the first object in the three-dimensional environment, wherein the visual indication of the boundary is displayed on the first surface in the three-dimensional environment. Additionally or alternatively, in some examples, the second surface in the three-dimensional environment corresponds to a virtual surface of a virtual object displayed in the three-dimensional environment.Additionally or alternatively, in some examples, the virtual surface of the virtual object is positioned at a specific height from the first surface in the three-dimensional environment relative to the viewing angle of the first electronic device, and this specific height is determined based on data provided by the second electronic device. Additionally or alternatively, in some examples, the virtual surface of the virtual object is positioned at a specific height from the first surface in the three-dimensional environment relative to the viewing angle of the first electronic device, and this specific height is determined based on one or more physical properties of a physical environment of the first electronic device.Additionally or alternatively, in some examples, the height of the second surface in the three-dimensional environment is greater than the height of the first surface in the three-dimensional environment relative to the viewing angle of the first electronic device. Additionally or alternatively, in some examples, receiving the second specification of the request to associate the first object with the second surface in the three-dimensional environment includes detecting an input via one or more input devices that corresponds to a request to move the first object across the second surface in the three-dimensional environment. Additionally or alternatively, in some examples, associateing the first object with the second surface in the three-dimensional environment includes anchoring the first object to the second surface in the three-dimensional environment.

[0125] Additionally or alternatively, in some examples, receiving the second specification of the requirement to link the first object with the second surface in the three-dimensional environment includes visually identifying the second surface in the three-dimensional environment via the one or more input devices, without recognizing an input to link the first object with the second surface in the three-dimensional environment.Additionally or alternatively, in some examples, the method further comprises: after receiving the second instruction, receiving a third instruction requesting the first object to associate with a third surface, different from the first and second surfaces, in the three-dimensional environment; and in response to receiving the third instruction, association of the first object with the third surface in the three-dimensional environment and updating the display of the user's visual representation of the second electronic device via the one or more displays, such that it is displayed from the viewpoint of the first electronic device at a third height, different from the first and second heights, relative to the third surface in the three-dimensional environment.Additionally or alternatively, in some examples, the content of the first type corresponds to content linked to a horizontally aligned virtual object.Additionally or alternatively, in some examples, the method further comprises: receiving a third specification of a request to share content of a second type, different from the first type, in the communication session; and in response to receiving the third specification, ceasing the display of the first object in the three-dimensional environment, displaying a second object corresponding to the content of the second type in the three-dimensional environment on the one or more displays, and updating the display of the user's visual representation of the second electronic device on the one or more displays so that it is displayed from the viewpoint of the first electronic device at the first height relative to the first surface in the three-dimensional environment.Additionally or alternatively, in some examples, the content of the second type corresponds to content linked to a vertically aligned virtual object.

[0126] Additionally or alternatively, in some examples the procedure also includes: Before receiving the first request to share content of the first type in the communication session, receiving a third request to share content of a second type, different from the first type, in the communication session; and in response to receiving the third request, displaying a second object corresponding to the content of the second type in the three-dimensional environment via one or more displays; while the second object and the user's visual representation of the second electronic device are displayed in the three-dimensional environment, receiving a fourth request to link the second object to the second surface in the three-dimensional environment; and in response to receiving the fourth request, Linking the second object to the second surface in the three-dimensional environment and maintaining the display of the user's visual representation of the second electronic device at the first height relative to the first surface in the three-dimensional environment from the viewpoint of the first electronic device via the one or more displays.Additionally or alternatively, in some examples, the method further comprises: while the first object is associated with the second surface in the three-dimensional environment, receiving a third indication of a request to stop sharing the content of the first type in the three-dimensional environment; and in response to receiving the third indication, stopping the display of the first object in the three-dimensional environment and updating the display of the user's visual representation of the second electronic device via the one or more displays, so that it is again displayed from the viewpoint of the first electronic device at the first height relative to the first surface in the three-dimensional environment.Additionally or alternatively, in some examples the first electronic device remains in the communication session with a third electronic device, the three-dimensional environment includes a second visual representation of a user of the third electronic device, and. The second visual representation is displayed from the viewpoint of the first electronic device at a third height relative to the first surface in the three-dimensional environment. In some examples, the method further comprises, in response to receiving the second input, updating the display of the second visual representation of the user of the third electronic device via one or more displays, so that it is displayed from the viewpoint of the first electronic device at a fourth height, which differs from the third height, relative to the second surface in the three-dimensional environment.Additionally or alternatively, in some examples, the method further comprises, in response to receiving the second indication, displaying a visual indication via the one or more displays indicating that the user's visual representation of the second electronic device is being updated to be relative to the second surface in the three-dimensional environment.

[0127] Some examples of the disclosure relate to a method comprising, on a first electronic device communicating with one or more display(s) and one or more input device(s): during a communication session with a second electronic device, displaying a visual representation of a user of the second electronic device and of a first object corresponding to shared content in a three-dimensional environment, via the one or more display(s), wherein the visual representation is displayed from a viewpoint of the first electronic device at a first height relative to a first surface in the three-dimensional environment;While the user's visual representation of the second electronic device and the first object in the three-dimensional environment is displayed, the device detects, via one or more input devices, a first input corresponding to a request to move the first object vertically in the three-dimensional environment relative to the viewing angle of the first electronic device; and in response to the detection of the first input, the device moves the first object and the user's visual representation of the second electronic device vertically in the three-dimensional environment relative to the viewing angle of the first electronic device according to the first input, including displaying the user's visual representation of the second electronic device at a second height different from the first height, relative to the first surface in the three-dimensional environment via the one or more displays.

[0128] Additionally or alternatively, in some examples, the first object corresponds to a vertically oriented virtual object. Additionally or alternatively, in some examples, the first surface in the three-dimensional environment corresponds to a physical floor or substructure of a physical surface visible in the three-dimensional environment from the viewpoint of the first electronic device. Additionally or alternatively, in some examples, the first object corresponds to a horizontally oriented virtual object. Additionally or alternatively, in some examples, the first surface in the three-dimensional environment corresponds to a physical surface of a physical object visible in the three-dimensional environment from the viewpoint of the first electronic device.Additionally or alternatively, in some examples, the visual representation of the user of the second electronic device corresponds to a three-dimensional avatar of the user of the second electronic device. Additionally or alternatively, in some examples, the method further comprises: while displaying the visual representation of the user of the second electronic device and the first object in the three-dimensional environment, receiving a signal initiating an input recognized by the second electronic device, corresponding to a request to move the first object vertically in the three-dimensional environment; and, while receiving the signal, updating the display of the visual representation of the user of the second electronic device on the one or more displays so that it corresponds to a two-dimensional representation in the three-dimensional environment.Additionally or alternatively, in some examples, the method further comprises: receiving a signal indicating the completion of the input recognized by the second electronic device, which corresponds to the request to move the first object vertically in the three-dimensional environment; and, in response to receiving the signal, updating the display of the user's visual representation of the second electronic device on the one or more displays so that it corresponds to the three-dimensional avatar of the user of the second electronic device without moving the first object in the three-dimensional environment; wherein the user's visual representation of the second electronic device is displayed at a third height, different from the first and second heights, relative to the first surface in the three-dimensional environment.Additionally or alternatively, in some examples, the third height is based on a vertical distance of a movement of the first object in a second three-dimensional environment, presented on the second electronic device according to the input recognized by the second electronic device.

[0129] Additionally or alternatively, in some examples, according to a stipulation that the vertical movement of the first object in the three-dimensional environment corresponds to a movement away from the first surface in the three-dimensional environment, the second height is greater than the first height; and according to a stipulation that the vertical movement of the first object in the three-dimensional environment corresponds to a movement toward the first surface in the three-dimensional environment, the second height is less than the first height. Additionally or alternatively, in some examples, the first input includes an air gesture performed by the hand of a user of the first electronic device. Additionally or alternatively, in some examples, the method further includes: while displaying the visual representation of the user of the second electronic device and the first object in the three-dimensional environment, recognizing,via one or more input devices, a second input corresponding to a request to move the first object vertically in the three-dimensional environment relative to the viewing angle of the first electronic device; and in response to the detection of the second input, according to a determination that the vertical movement of the first object in the three-dimensional environment corresponds to a movement of the first object to a location in the three-dimensional environment beyond a boundary associated with the three-dimensional environment, moving the first object to a first location in the three-dimensional environment that lies within the boundary relative to the viewing angle of the first electronic device, including updating the display of the user's visual representation of the second electronic device via the one or more displays,so that it corresponds to a two-dimensional representation in the three-dimensional environment.

[0130] Additionally or alternatively, in some examples, the boundary associated with the three-dimensional environment is defined as the threshold distance from the first surface in the three-dimensional environment. Additionally or alternatively, in some examples, the boundary associated with the three-dimensional environment is defined as the threshold distance from the head of a user of the first electronic device in the three-dimensional environment. Additionally or alternatively, in some examples, the method further comprises: while displaying the visual representation of the user of the second electronic device and the first object in the three-dimensional environment, detecting, via one or more input devices, a second input corresponding to a request to move the first object in the three-dimensional environment vertically relative to the viewing angle of the first electronic device.and in response to the detection of the second input, according to a determination that the vertical movement of the first object in the three-dimensional environment corresponds to a movement of the first object to a location in the three-dimensional environment beyond a boundary associated with the three-dimensional environment, moving the first object to a first location in the three-dimensional environment that lies beyond the boundary relative to the viewing angle of the first electronic device, according to the second input;and after detecting the completion of the second input, moving the first object to a second location, different from the first location, in the three-dimensional environment, which is within the boundary relative to the viewing angle of the first electronic device. Additionally or alternatively, in some examples, the second input corresponds to a requirement to move the first object vertically in the three-dimensional environment by a first distance beyond the boundary, followed by a second distance beyond the boundary, and moving the first object to the first location in the three-dimensional environment according to the second input includes: moving the first object by the first distance beyond the boundary in the three-dimensional environment relative to the viewing angle of the first electronic device;and after moving the first object by the first distance beyond the boundary, moving the first object by a third distance, which is smaller than the second distance, beyond the boundary in the three-dimensional environment relative to the viewing angle of the first electronic device.

[0131] Additionally or alternatively, in some examples, the method further comprises: while displaying the user's visual representation of the second electronic device and the first object in the three-dimensional environment, receiving a signal of input, which is recognized by the second electronic device and corresponds to a request to move the first object vertically in the three-dimensional environment; and in response to receiving the signal, vertically moving the user's visual representation of the second electronic device in the three-dimensional environment relative to the viewing angle of the first electronic device.Additionally or alternatively, in some examples, the method further includes, in response to receiving the information and according to a determination that the vertical movement of the visual representation of the second electronic device in the three-dimensional environment corresponds to a movement of the first object to a location in the three-dimensional environment beyond a boundary associated with the three-dimensional environment, updating the display of the user's visual representation of the second electronic device via the one or more displays so that it corresponds to a two-dimensional representation in the three-dimensional environment.

[0132] Some examples of the disclosure relate to a first electronic device comprising: one or more processors; a working memory; and one or more programs stored in the working memory and configured to be executed by the one or more processors, the one or more programs containing instructions for carrying out one of the above-mentioned procedures.

[0133] Some examples of the disclosure relate to a non-transitory, computer-readable storage medium that stores one or more programs, wherein the one or more programs comprise instructions which, when executed by one or more processors of a first electronic device, cause the first electronic device to perform one of the aforementioned procedures.

[0134] Some examples of the disclosure relate to a first electronic device comprising one or more processors, memory and means for carrying out one of the foregoing methods.

[0135] Some examples of the disclosure relate to an information processing device for use in a first electronic device, the information processing device comprising means for carrying out one of the methods disclosed above.

[0136] The foregoing description was provided for the purpose of explanation with reference to specific examples. However, the preceding illustrative discussions are neither intended to be exhaustive nor to limit revelation to the exact forms revealed. In light of the foregoing teachings, many modifications and variations are possible. The examples were chosen and described to best explain the principles of revelation and their practical application, thus enabling other professionals to make the best possible use of revelation and the various examples described, with different modifications, as appropriate for their intended particular use. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 63 / 698,847

[0001] US 63 / 817,642

[0001] US 19 / 299,087

[0001]

Claims

[1] Procedure, encompassing: on a first electronic device that communicates with one or more displays and one or more input devices: during a communication session with a second electronic device and during a display, via one or more displays, of a visual representation of a user of the second electronic device in a three-dimensional environment, wherein the visual representation is displayed from a view of the first electronic device at a first height relative to a first surface in the three-dimensional environment, receiving a first display of a request to share content of a first kind in the communication session; in response to receiving the first indication, indications, about the one or more indications, of a first object that corresponds to the content of the first kind in the three-dimensional environment; during the display of the first object and the visual representation of the user of the second electronic device in the three-dimensional environment, receiving a second request to link the first object with a second surface that differs from the first surface in the three-dimensional environment; and in response to receiving the second piece of information: Linking the first object with the second surface in the three-dimensional environment; and Updating the display, via one or more displays, of the user's visual representation of the second electronic device, to be displayed from the perspective of the first electronic device at a second height that differs from the first height, relative to the second surface in the three-dimensional environment. [2] Method according to claim 1, wherein the first surface in the three-dimensional environment corresponds to a physical ground or subsurface of a physical surface that is visible in the three-dimensional environment from the viewpoint of the first electronic device. [3] Method according to one of claims 1 to 2, wherein the second surface in the three-dimensional environment corresponds to a physical surface of a physical object that is visible in the three-dimensional environment from the viewpoint of the first electronic device. [4] Method according to one of claims 1 to 2, wherein the second surface in the three-dimensional environment corresponds to a virtual surface of a virtual object that is displayed in the three-dimensional environment. [5] Method according to claim 4, wherein: the virtual surface of the virtual object in the three-dimensional environment is positioned at a respective height from the first surface relative to the view of the first electronic device; and The respective height is determined based on data provided by the second electronic device. [6] Method according to claim 4, wherein: the virtual surface of the virtual object in the three-dimensional environment is positioned at a respective height from the first surface relative to the view of the first electronic device; and the respective height is determined based on one or more physical properties of a physical environment of the first electronic device. [7] Method according to any one of claims 1 to 6, wherein a height of the second surface in the three-dimensional environment relative to the view of the first electronic device is greater than a height of the first surface in the three-dimensional environment. [8] Method according to any one of claims 1 to 7, wherein receiving the second specification of the request to link the first object with the second surface in the three-dimensional environment includes acquiring, via one or more input devices, an input corresponding to a request to move the first object over the second surface in the three-dimensional environment. [9] Method according to claim 8, wherein linking the first object to the second surface in the three-dimensional environment includes anchoring the first object to the second surface in the three-dimensional environment. [10] Method according to any one of claims 1 to 7, wherein receiving the second specification of the request to link the first object with the second surface in the three-dimensional environment includes the visual identification, via one or more input devices, of the second surface in the three-dimensional environment, without capturing the input to link the first object with the second surface in the three-dimensional environment. [11] Method according to any one of claims 1 to 10, further comprising: After receiving the second piece of information, receiving a third piece of information from a request, to link the first object with a third surface, which differs from the first and second surfaces, in the three-dimensional environment; and in response to receiving the third piece of information: Linking the first object with the third surface in the three-dimensional environment; and Updating the display, via one or more displays, of the user's visual representation of the second electronic device, to be displayed from the perspective of the first electronic device at a third height, which differs from the first height and the second height, relative to the third surface in the three-dimensional environment. [12] Method according to any one of claims 1 to 11, further comprising: Receiving a third piece of information regarding a request, content of a second kind that differs from the first kind, to share in the communication session; and in response to receiving the third piece of information: Finishing displaying the first object in the three-dimensional environment; Displays, about which one or more displays of a second object corresponding to the content of the second kind, in the three-dimensional environment; and Updating the display, via one or more displays, of the user's visual representation of the second electronic device, to be displayed from the perspective of the first electronic device at the first height relative to the first surface in the three-dimensional environment. [13] Method according to any one of claims 1 to 12, further comprising: Before receiving the first piece of information in the request, to share content of the first kind in the communication session; upon receiving a third piece of information in the request, to share content of a second kind, which differs from the first kind, in the communication session; and in response to the recognition of the third piece of information, advertisements about which one or the multiple displays of a second object corresponding to the content of the second kind in the three-dimensional environment; during the display of the second object and the visual representation of the user of the second electronic device in the three-dimensional environment, receiving a fourth indication of a request to link the second object with the second surface in the three-dimensional environment; and in response to receiving the fourth indication: Linking the second object to the second surface in the three-dimensional environment; and Maintaining the display, via one or more displays, of the user's visual representation of the second electronic device from the perspective of the first electronic device at the first height relative to the first surface in the three-dimensional environment. [14] First electronic device comprising: one or more processors; a storage facility; and one or more programs that are stored in memory and configured to be executed by the one or more processors, wherein one or more programs include instructions for carrying out a method according to any one of claims 1 to 13. [15] Non-volatile, computer-readable storage medium that stores one or more programs comprising instructions which, when executed by one or more processors of an electronic device, cause the electronic device to perform a method according to any one of claims 1 to 13.

Images