System and method for determining location of parallel electronic devices

Abstract

The present disclosure generally relates to systems and methods for determining locations of collocated electronic devices. A first device (1) displays elements selectable to establish a multi-user communication session between a first user of the first device and a second user of a second device based on a pose of the second device in a physical environment in which the first device and the second device are juxtaposed; 2) starting a different type of session based on the type of device via which the second user accepts the request to join the session; 3) when in a multi-user communication session of a third user including the first user, the second user and not including the third device, sending to the third device a shared coordinate system of a physical environment between the first device and the second device; and / or 4) sending the shared origin of the different devices to the third device in response to detecting that the first device and the third device are juxtaposed in the physical environment.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 698,520, filed September 24, 2024; U.S. Provisional Application No. 63 / 876,750, filed September 5, 2025; and U.S. Patent Application No. 19 / 323,961, filed September 9, 2025, the contents of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] This disclosure relates in its entirety to systems and methods for determining the location of electronic devices juxtaposed in a physical environment. Background Technology

[0004] Some computer graphics environments provide two-dimensional and / or three-dimensional environments in which at least some of the objects displayed for user viewing are virtual and computer-generated. In some examples, the three-dimensional environment is presented by multiple electronic devices communicating with each other. In some examples, a portal for visual communication with a specific user is displayed within the three-dimensional environment presented at the respective electronic device. Summary of the Invention

[0005] Some examples of this disclosure relate to systems and methods for determining the location of juxtaposed electronic devices in a physical environment. For example, before establishing a multi-user communication session between a first electronic device and a second electronic device, the first and / or second electronic devices can determine the location of another electronic device according to this disclosure. For example, the first and second electronic devices can be juxtaposed in a physical environment, and a shared reference origin in the physical environment can be determined based on map data determined by the first electronic device and / or by the second electronic device. The first electronic device can determine the offset between the origin of its coordinate system and the shared reference origin. The first electronic device can transmit data indicating the attitude (e.g., positioning and orientation) of the first electronic device relative to the origin of its coordinate system and data indicating the offset to the second electronic device. In response to receiving such data, the second electronic device can determine the location of the first electronic device relative to the shared reference origin.

[0006] In some examples, the first electronic device initiates a multi-user communication session of a different type with the second user of the second electronic device based on the type of electronic device from which the second user of the second electronic device accepts a request to join the multi-user communication session with the first user of the first electronic device.

[0007] In some examples, a first electronic device detects and responds to a third user of a third electronic device being physically juxtaposed with a first user of the first electronic device in a multi-user communication session, the multi-user communication session including a first user of the first electronic device and a second user of the second electronic device but excluding a third user, wherein the first user and the second user are juxtaposed in a physical environment, and wherein a shared spatial coordinate system of the physical environment between the first electronic device and the second electronic device has been established.

[0008] In some examples, the first electronic device performs a different position tracking process together with the different electronic devices in response to detecting juxtaposition with them.

[0009] A full description of these examples is provided in the accompanying drawings and detailed embodiments, and it should be understood that the content of this invention does not limit the scope of this disclosure in any way. Attached Figure Description

[0010] To better understand the various examples described herein, reference should be made to the following detailed embodiments and accompanying drawings. Throughout the drawings, similar reference numerals generally refer to corresponding parts.

[0011] Figure 1 Electronic devices for presenting a three-dimensional environment according to some examples of this disclosure are generally illustrated.

[0012] Figures 2A to 2C Block diagrams of example architectures for electronic devices or systems are generally illustrated according to some examples of this disclosure.

[0013] Figure 3 Examples of space groups in a multi-user communication session including a first electronic device and a second electronic device are generally illustrated according to some examples of this disclosure.

[0014] Figures 4A to 4G Examples of performing a location tracking process between a first electronic device and a second electronic device prior to establishing a multi-user communication session between the two devices are generally illustrated according to some examples of this disclosure.

[0015] Figure 4H Examples of electronic devices that present shared virtual content when in a multi-user communication session are generally illustrated according to some examples of this disclosure.

[0016] Figure 5 A flowchart illustrating a method for displaying a user interface element according to some examples of this disclosure is generally shown, the user interface element being selectable to establish a multi-user communication session between a first electronic device and a second electronic device.

[0017] Figure 6Flowcharts illustrating methods for performing operations based on specific information are generally shown according to some examples of this disclosure.

[0018] Figures 7A to 7E-1 Generally, examples according to some examples of this disclosure are illustrated where a first electronic device initiates a multi-user communication session of a different type with a second user of a second electronic device based on the type of electronic device from which a second user of a second electronic device accepts a request to join a multi-user communication session with the first user of the first electronic device.

[0019] Figure 8 A flowchart illustrating, according to some examples of this disclosure, is generally provided as a method for initiating a multi-user communication session of a different type with a second user of a second electronic device based on the type of electronic device from which the second user of the second electronic device accepts a request to join a multi-user communication session with a first user of a first electronic device.

[0020] Figures 9A to 9F Generally, examples according to some examples of this disclosure are illustrated, such as a first electronic device detecting and responding to a third user of a third electronic device being juxtaposed with a first user of the first electronic device in a physical environment when in a multi-user communication session, the multi-user communication session including a first user of the first electronic device and a second user of a second electronic device but excluding a third user of the third electronic device, wherein the first user and the second user are juxtaposed in a physical environment, and wherein a shared spatial coordinate system of the physical environment between the first electronic device and the second electronic device has been established.

[0021] Figure 10 A flowchart illustrating, generally according to some examples of the present disclosure, is provided for detecting and responding to a method for juxtaposing a third user of a third electronic device with a first user of a first electronic device in a physical environment while in a multi-user communication session, the multi-user communication session including a first user of the first electronic device and a second user of a second electronic device but excluding the third user, wherein the first user and the second user are juxtaposed in a physical environment, and wherein a shared spatial coordinate system of the physical environment between the first electronic device and the second electronic device has been established.

[0022] Figures 11A to 11O Examples of first electronic devices according to some examples of this disclosure performing different position tracking processes together with different electronic devices in response to detecting juxtaposition with different electronic devices are generally illustrated.

[0023] Figure 12 A flowchart illustrating, generally speaking, a method for performing different position tracking processes with different electronic devices in response to the detection of juxtaposition with different electronic devices, according to some examples of the present disclosure. Detailed Implementation

[0024] Some examples of this disclosure relate to systems and methods for determining the location of juxtaposed electronic devices in a physical environment. For example, before establishing a multi-user communication session between a first electronic device and a second electronic device, the first and / or second electronic devices can determine the location of another electronic device according to this disclosure. For example, the first and second electronic devices can be juxtaposed in a physical environment, and a shared reference origin in the physical environment can be determined based on map data determined by the first and / or the second electronic device. The first electronic device can determine the offset between the origin of its coordinate system and the shared reference origin. The first electronic device can transmit data indicating the attitude (e.g., positioning and / or orientation) of the first electronic device relative to the origin of its coordinate system and data indicating the offset to the second electronic device. In response to receiving such data, the second electronic device can determine the location of the first electronic device relative to the shared reference origin.

[0025] In some examples, the first electronic device initiates a multi-user communication session of a different type with the second user of the second electronic device based on the type of electronic device from which the second user of the second electronic device accepts a request to join the multi-user communication session with the first user of the first electronic device.

[0026] In some examples, a first electronic device detects and responds to a third user of a third electronic device being physically juxtaposed with a first user of the first electronic device in a multi-user communication session, the multi-user communication session including a first user of the first electronic device and a second user of the second electronic device but excluding a third user, wherein the first user and the second user are juxtaposed in a physical environment, and wherein a shared spatial coordinate system of the physical environment between the first electronic device and the second electronic device has been established.

[0027] In some examples, the first electronic device performs a different position tracking process together with the different electronic devices in response to detecting juxtaposition with them.

[0028] In some examples, initiating a multi-user communication session may include interaction with one or more user interface elements. In some examples, the electronic device tracks a user's gaze as input for aiming at selectable options / enabled representations within a corresponding user interface element displayed in a 3D environment. For example, a gaze can be used to identify one or more options / enabled representations selected as a target using another selection input. In some examples, a corresponding option / enabled representation can be selected using hand-tracking input detected via an input device communicating with the electronic device. In some examples, objects displayed in a 3D environment may be moved and / or reoriented in the 3D environment based on movement input detected via an input device.

[0029] In some examples, a spatial group or state representation in a multi-user communication session indicates the spatial arrangement or template of the location of users and content located within or otherwise associated with the spatial group. In some examples, users in the same spatial group within a multi-user communication session experience spatial realism based on the spatial arrangement of the spatial group. In some examples, spatial realism requires a consistent spatial arrangement between the user (or its representation) and virtual objects. In some examples, when a user of a first electronic device is in a first spatial group in a multi-user communication session and a user of a second electronic device is in a second spatial group in a multi-user communication session, the user experiences spatial realism localized to their respective spatial group. In some examples, when a user of a first electronic device and a user of a second electronic device are grouped into separate spatial groups or states within a multi-user communication session, if the first and second electronic devices return to the same operating state, the users of the first and second electronic devices are regrouped into the same spatial group within the multi-user communication session.

[0030] As used herein, a hybrid space group corresponds to a group or number of participants (e.g., users) in a multi-user communication session, in which at least a subset of the participants are not juxtaposed in a physical environment. For example, as described via one or more examples in this disclosure, a hybrid space group includes at least two participants juxtaposed in a first physical environment and at least one participant not juxtaposed with the at least two participants in the first physical environment (e.g., the at least one participant is located in a second physical environment different from the first physical environment). In some examples, the hybrid space group in the multi-user communication session has a spatial arrangement that determines the location of users and content within the space group. In some examples, users in the same hybrid space group within the multi-user communication session experience spatial realism according to the spatial arrangement of the space group, as discussed similarly above.

[0031] Figure 1 An electronic device 101 is illustrated according to some examples of this disclosure, which presents a three-dimensional environment (e.g., an extended reality (XR) environment or a computer-generated reality (CGR) environment, optionally including representations of physical and / or virtual objects). In some examples, such as Figure 1 As shown, electronic device 101 is a head-mounted display or other head-mountable device configured to be worn on the head of a user of electronic device 101. See below for reference. Figure 2A An example of an architecture block diagram to describe electronic device 101. For example... Figure 1As shown, electronic device 101 and table 106 are located in a physical environment. The physical environment may include physical features such as physical surfaces (e.g., floor, wall) or physical objects (e.g., table, lamp, etc.). In some examples, electronic device 101 may be configured to detect and / or capture images of the physical environment including table 106 (exemplified in the field of view of electronic device 101).

[0032] In some examples, such as Figure 1 As shown, the electronic device 101 includes one or more internal image sensors 114a oriented toward the user's face (e.g., referred to below). Figures 2A to 2B (The described eye-tracking camera). In some examples, an internal image sensor 114a is used for eye tracking (e.g., detecting the user's gaze). The internal image sensor 114a is optionally arranged on the left and right portions of the display 120 to enable eye tracking of the user's left and right eyes. In some examples, the electronic device 101 also includes external image sensors 114b and 114c facing outwards from the user to detect and / or capture the physical environment of the electronic device 101 and / or movement of the user's hands or other body parts.

[0033] In some examples, display 120 has a field of view visible to the user. In some examples, the user-visible field of view is the same as the field of view of external image sensors 114b and 114c. For example, when display 120 is optionally part of a head-mounted device, the field of view of display 120 may be the same as or similar to the field of view of the user's eyes. In some examples, the user-visible field of view is different from the field of view of external image sensors 114b and 114c (e.g., narrower than the field of view of external image sensors 114b and 114c). In other examples, the field of view of display 120 may be smaller than the field of view of the user's eyes. The user's viewpoint determines what is visible in the field of view, and the viewpoint typically specifies the position and orientation relative to the three-dimensional environment. As the user's viewpoint shifts, the field of view of the three-dimensional environment also shifts accordingly. In some examples, electronic device 101 may be an optically transparent device, through which display 120 is a transparent or translucent display through which parts of the physical environment can be directly viewed. In some examples, display 120 may be included within a transparent lens and may overlap entirely or only partially with the transparent lens. In other examples, the electronics may be a video pass-through device, wherein display 120 is an opaque display configured to display images of the physical environment captured by external image sensors 114b and 114c. Although Figure 1 A single display is shown, but it should be understood that display 120 may optionally include more than one display. For example, display 120 may optionally include displays that are merged (e.g., merged by the user's brain) to create Figure 1The view of the content shown is displayed on a pair of stereo displays (e.g., a left display panel and a right display panel for the user's left and right eyes, respectively). In some examples, see the following references. Figures 2A to 2B As detailed, the display 120 includes or corresponds to a transparent or translucent surface (e.g., a lens) that does not have display capabilities (e.g., and therefore cannot generate and display virtual objects 104), and alternatively presents a direct view of the physical environment in the user's field of view (e.g., the user's eye's field of view).

[0034] In some examples, electronic device 101 is configured to display (e.g., in response to a trigger) virtual object 104 in a three-dimensional environment. Virtual object 104 is... Figure 1 The illustrated cube represents a cube that does not exist in the physical environment but is displayed in a three-dimensional environment positioned on top of table 106 (e.g., a real-world table or a representation thereof). Optionally, in response to detecting a flat surface of table 106 in the physical environment 100, virtual object 104 is displayed on the surface of table 106 in the three-dimensional environment displayed via display 120 of electronic device 101.

[0035] It should be understood that virtual object 104 is a representative virtual object and may include and render one or more different virtual objects (e.g., virtual objects with various dimensions, such as two-dimensional or other three-dimensional virtual objects) in a three-dimensional environment. For example, a virtual object may represent an application or user interface displayed in a three-dimensional environment. In some examples, a virtual object may represent content corresponding to an application and / or displayed via a user interface in a three-dimensional environment. In some examples, virtual object 104 may optionally be configured to be interactive and responsive to user input (e.g., air gestures, such as air pinch gestures, air tap gestures, and / or air touch gestures), allowing the user to virtually touch, tap, move, rotate, or otherwise interact with virtual object 104.

[0036] As described in this article, by the user (e.g., using) Figure 1One or more air pinch gestures performed by the hand (103) are detected by one or more input devices of the electronic device 101 and interpreted as one or more user inputs pointing to content displayed by the electronic device 101. Additionally or alternatively, in some examples, one or more user inputs interpreted by the electronic device 101 as pointing to content displayed by the electronic device 101 (e.g., virtual object 104) are detected via one or more hardware input devices (e.g., controllers, touchpads, proximity sensors, buttons, sliders, knobs, etc.) rather than via one or more input devices configured to detect air gestures such as one or more air pinch gestures performed by the user. This description is intended to be exemplary and not limiting; the user may optionally use different air gestures and / or other forms of input to provide user input.

[0037] In some examples, electronic device 101 may be configured to communicate with a second electronic device (such as an accompanying device). For example, as Figure 1 As illustrated, electronic device 101 may optionally communicate with electronic device 160. In some examples, electronic device 160 corresponds to a mobile electronic device, such as a smartphone, tablet, smartwatch, laptop, or other electronic device. In some examples, electronic device 160 corresponds to a non-mobile electronic device, which is typically stationary and not easily moved within a physical environment (e.g., a desktop computer, server, etc.). See below for reference. Figure 2B The following are additional examples of architectural diagrams used to describe electronic device 160. In some examples, electronic device 101 and electronic device 160 are associated with the same user. For example, in Figure 1 In this configuration, electronic device 101 may be positioned (e.g., mounted on) a user's head, and electronic device 160 may be positioned near electronic device 101, such as in the user's hand 103 (e.g., the hand 103 holding electronic device 160), in the user's pocket or bag, or on a surface near the user. Electronic devices 101 and 160 may optionally be associated with the same user account (e.g., the user's account logged into both electronic devices 101 and 160). See below for further details. Figures 2A to 2C Additional details regarding the communication between electronic device 101 and electronic device 160 are provided.

[0038] In some examples, the display of an object in a 3D environment is triggered or enabled by interaction with one or more user interface objects in the 3D environment. For example, initiating the display of an object in a 3D environment may include interaction with one or more virtual option / power representations displayed in the 3D environment. In some examples, when initiating the display of an object in a 3D environment, the electronic device may track the user's gaze as input for identifying one or more virtual option / power representations as a target for selection. For example, a gaze may be used to identify one or more virtual option / power representations as a target for selection using another selection input. In some examples, a virtual option / power representation may be selected using hand-tracking input detected via an input device communicating with the electronic device. In some examples, an object displayed in a 3D environment may be moved and / or reoriented in the 3D environment based on movement input detected via an input device.

[0039] In the following description, an electronic device that communicates with one or more displays and one or more input devices is described. It should be understood that the electronic device may optionally communicate with one or more other physical user interface devices, such as touch-sensitive surfaces, physical keyboards, mice, joysticks, hand-tracking devices, eye-tracking devices, styluses, etc. Furthermore, as described above, it should be understood that the described electronic device, display, and touch-sensitive surface may optionally be distributed among two or more devices. Therefore, as used in this disclosure, information on or displayed by the electronic device may optionally be used to describe information output by the electronic device for display on a separate display device (touch-sensitive or non-touch-sensitive). Similarly, as used in this disclosure, input received on the electronic device (e.g., touch input received on a touch-sensitive surface of the electronic device, or touch input received on the surface of a stylus) may optionally be used to describe input received on a separate input device from which the electronic device receives input information.

[0040] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, TV channel browsing applications, and / or digital video player applications.

[0041] Figures 2A to 2CBlock diagrams illustrating example architectures for electronic devices according to some examples of this disclosure are shown. In some examples, electronic device 201 and / or device 260 include one or more electronic devices. For example, electronic device 201 may be a portable device, an auxiliary device for communicating with another device, a head-mounted display, a head-mounted speaker, etc. In some examples, electronic device 201 corresponds to the above reference. Figure 1 The described electronic device 101. In some examples, electronic device 260 corresponds to the above reference. Figure 1 The described electronic device 160.

[0042] like Figure 2A As illustrated, electronic device 201 may optionally include one or more sensors, such as one or more hand tracking sensors 202, one or more position sensors 204A, and one or more image sensors 206A (optionally corresponding to...). Figure 1 The electronic device 201 may include an internal image sensor 114a and / or external image sensors 114b and 114c, one or more touch-sensitive surfaces 209A, one or more motion and / or orientation sensors 210A, one or more eye-tracking sensors 212, one or more microphones 213A or other audio sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors), etc. The electronic device 201 may optionally include one or more output devices, such as one or more display generation components 214A (optionally corresponding to...). Figure 1 The electronic device 201 may include a display 120, one or more speakers 216A, one or more tactile output devices (not shown), etc. Optionally, the electronic device 201 may include one or more processors 218A, one or more memories 220A, and / or communication circuitry 222A. One or more communication buses 208A may optionally be used for communication between the components of the electronic device 201 described above.

[0043] Additionally, electronic device 260 may optionally include components that are the same as or similar to those of electronic device 201. For example, such as Figure 2B As shown, the electronic device 260 optionally includes one or more position sensors 204B, one or more image sensors 206B, one or more touch-sensitive surfaces 209B, one or more orientation sensors 210B, one or more microphones 213B, one or more display generating components 214B, one or more speakers 216B, one or more processors 218B, one or more memories 220B, and / or communication circuitry 222B. One or more communication buses 208B are optionally used for communication between the components of the electronic device 260 described above.

[0044] Electronic devices 201 and 260 are optionally configured to communicate via a wired or wireless connection between the two electronic devices (e.g., via communication circuits 222A, 222B). For example, as Figure 2A As indicated, electronic device 260 can be used as an accompanying device of electronic device 201. For example, in some examples, electronic device 260 processes sensor inputs from electronic devices 201 and 260 and / or uses the display generation component 214A of electronic device 201 to generate content for display.

[0045] Communication circuits 222A and 222B optionally include circuitry for communicating with electronic devices and networks such as the Internet, intranets, wired and / or wireless networks, cellular networks, and wireless local area networks (LANs). Communication circuits 222A and 222B optionally include circuitry for using near-field communication (NFC) and / or other communication methods. The communication circuits 222A and 222B are used for short-range communication. In some examples, the communication circuits 222A and 222B include or support Wi-Fi (e.g., the 802.11 protocol), Ethernet, Ultra-Wideband (“UWB”), high-frequency systems (e.g., 900MHz, 2.4GHz, and 5.6GHz communication systems) or any other communication protocol or any combination thereof.

[0046] One or more processors 218A, 218B include one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, one or more processors 218A, 218B include one or more microprocessors, one or more central processing units, one or more application-specific integrated circuits, one or more field-programmable gate arrays, one or more programmable logic devices, or combinations of such devices. In some examples, memories 220A and / or 220B are non-transitory computer-readable storage media (e.g., flash memory, random access memory, or other volatile or non-volatile memory or storage device) storing computer-readable instructions configured to be executed by one or more processors 218A, 218B to perform the techniques, processes, and / or methods described below. In some examples, memories 220A and / or 220B may include more than one non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium can be any medium (e.g., excluding signals) that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, and device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on compact disc (CD), digital versatile optical disc (DVD), or Blu-ray technology, and persistent solid-state storage devices (such as flash memory, solid-state drives, etc.).

[0047] In some examples, one or more display generating components 214A, 214B include a single display (e.g., a liquid crystal display (LCD), an organic light-emitting diode (OLED), or other type of display). In some examples, one or more display generating components 214A, 214B include multiple displays. In some examples, one or more display generating components 214A, 214B may include a touch-enabled display (e.g., a touchscreen), a projector, a holographic projector, a retinal projector, a transparent or translucent display, etc. In some examples, the electronic device does not include one or more display generating components 214A or 214B. For example, instead of one or more display generating components 214A or 214B, some electronic devices include transparent or translucent lenses or other surfaces not configured to display or present virtual content. However, it should be understood that in this case, electronic device 201 and / or electronic device 260 may optionally be equipped with Figure 2A and Figure 2BOne or more of the other components illustrated and described herein, such as one or more hand-tracking sensors 202, one or more eye-tracking sensors 212, one or more image sensors 206A, and / or one or more motion and / or orientation sensors 210A. Alternatively, in some examples, one or more display generation components 214A or 214B are provided separately from electronic devices 201 and / or 260. For example, one or more display generation components 214A, 214B communicate with electronic device 201 (and / or electronic device 260) but are not integrated with electronic device 201 and / or electronic device 260 (e.g., within the housing of electronic devices 201, 260). In some examples, electronic devices 201 and 260 respectively include one or more touch-sensitive surfaces 209A and 209B for receiving user input, such as tap input and swipe input or other gestures (e.g., hand-based or finger-based gestures). In some examples, one or more display generating components 214A, 214B and one or more touch-sensitive surfaces 209A, 209B form a touch-sensitive display (e.g., a touchscreen integrated with each of the electronic devices 201 and 260 or a touchscreen external to each of the electronic devices 201 and 260 that communicates with each of the electronic devices 201 and 260).

[0048] Electronic devices 201 and 260 optionally include one or more image sensors 206A and 206B, respectively. The image sensors 206A and 206B optionally include one or more visible light image sensors (such as charge-coupled device (CCD) sensors) and / or complementary metal-oxide-semiconductor (CMOS) sensors operable to obtain images of physical objects from a real-world environment. The image sensors 206A and 206B also optionally include one or more infrared (IR) sensors, such as passive or active IR sensors, for detecting infrared light from the real-world environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real-world environment. The image sensors 206A and 206B also optionally include one or more cameras configured to capture movement of physical objects in the real-world environment. The image sensors 206A and 206B also optionally include one or more depth sensors configured to detect the distance between the physical object and the electronic devices 201 and 260. In some examples, information from one or more depth sensors allows a device to identify objects in a real-world environment and distinguish them from other objects in the real-world environment. In some examples, one or more depth sensors allow a device to determine the texture and / or shape of objects in a real-world environment. In some examples, one or more image sensors 206A or 206B are included in an electronic device different from electronic devices 201 and / or 260. For example, one or more image sensors 206A, 206B communicate with electronic devices 201, 260 but are not integrated with electronic devices 201, 260 (e.g., within the housing of electronic devices 201, 260). Specifically, in some examples, one or more cameras of one or more image sensors 206A, 206B are integrated into and / or coupled to one or more devices separate from electronic devices 201 and / or 260 (e.g., but communicating with electronic devices 201 and / or 260), such as one or more input and / or output devices (e.g., one or more speakers and / or one or more microphones, such as headphones or headsets) that include one or more image sensors 206A, 206B. In some examples, electronic device 201 or electronic device 260 corresponds to a headphone speaker (e.g., headphones or earbuds). In this case, electronic device 201 or electronic device 260 is equipped with Figure 2A and Figure 2B A subset of the other components illustrated and described herein. In some such examples, electronic device 201 or electronic device 260 is equipped with one or more image sensors 206A, 206B, one or more motion and / or orientation sensors 210A, 210B, and / or speakers 216A, 216B.

[0049] In some examples, electronic devices 201 and 260 combine a CCD sensor, an event camera, and a depth sensor to detect the physical environment surrounding them. In some examples, one or more image sensors 206A and 206B include a first image sensor and a second image sensor. The first and second image sensors work cooperatively and are optionally configured to capture different information about physical objects in the real-world environment. In some examples, the first image sensor is a visible light image sensor, and the second image sensor is a depth sensor. In some examples, electronic devices 201 and 260 use one or more image sensors 206A and 206B to detect the location and orientation of electronic devices 201 and 260 and / or one or more display generation components 214A and 214B in the real-world environment. For example, electronic devices 201 and 260 use one or more image sensors 206A and 206B to track the location and orientation of one or more display generation components 214A and 214B relative to one or more stationary objects in the real-world environment.

[0050] In some examples, electronic devices 201 and 260 include one or more microphones 213A and 213B or other audio sensors, respectively. Electronic devices 201 and 260 may optionally use one or more microphones 213A and 213B to detect sound from a user and / or the user's real-world environment. In some examples, the one or more microphones 213A and 213B include an array of microphones (multiple microphones) that may optionally operate in cooperation to identify ambient noise or locate sound sources in the space of a real-world environment.

[0051] Electronic devices 201 and 260 each include one or more position sensors 204A and 204B for detecting the positions of electronic device 201 and / or one or more display generating components 214A and electronic device 260 and / or one or more display generating components 214B, respectively. For example, the one or more position sensors 204A, 204B may include a Global Positioning System (GPS) receiver that receives data from one or more satellites and allows electronic devices 201, 260 to determine the absolute position of the electronic device in the physical world.

[0052] Electronic devices 201 and 260 each include one or more orientation sensors 210A and 210B for detecting the orientation and / or movement of electronic device 201 and / or one or more display generation components 214A and the orientation and / or movement of electronic device 260 and / or one or more display generation components 214B, respectively. For example, electronic devices 201 and 260 use one or more orientation sensors 210A and 210B to track changes in the positioning and / or orientation of electronic devices 201 and 260 and / or one or more display generation components 214A and 214B, such as changes relative to physical objects in a real-world environment. The one or more orientation sensors 210A and 210B may optionally include one or more gyroscopes and / or one or more accelerometers.

[0053] In some examples, device 201 includes one or more hand-tracking sensors 202 and / or one or more eye-tracking sensors 212. It should be understood that, although referred to as hand-tracking or eye-tracking sensors, electronic device 201 additionally or optionally includes one or more other body-tracking sensors, such as one or more leg-tracking sensors, one or more torso-tracking sensors, and / or one or more head-tracking sensors. One or more hand-tracking sensors 202 are configured to track the localization and / or position of one or more portions of a user's hand, and / or the movement of one or more portions of the user's hand relative to a three-dimensional environment, relative to one or more display generation components 214A, and / or relative to another defined coordinate system. One or more eye-tracking sensors 212 are configured to track the localization and movement of a user's gaze (e.g., the user's attention, more generally, the eyes, face, or head) relative to the real world or three-dimensional environment and / or relative to one or more display generation components 214A. In some examples, one or more hand-tracking sensors 202 and / or one or more eye-tracking sensors 212 are implemented together with one or more display generation components 214A. In some examples, one or more hand-tracking sensors 202 and / or one or more eye-tracking sensors 212 are implemented separately from one or more display generation components 214A. In some examples, the electronic device 201 alternatively does not include one or more hand-tracking sensors 202 and / or one or more eye-tracking sensors 212. In some such examples, one or more display generation components 214A may be used by the electronic device 260 to provide a three-dimensional environment, and the electronic device 260 may utilize input and other data collected via one or more other sensors of the electronic device 201 (e.g., one or more position sensors 204A, one or more image sensors 206A, one or more touch-sensitive surfaces 209A, one or more motion and / or orientation sensors 210A, and / or one or more microphones 213A or other audio sensors) as input and data processed by the processor 218B of the electronic device 260. Additionally or alternatively, the electronic device 260 may optionally not include... Figure 2B Other components shown include one or more position sensors 204B, one or more image sensors 206B, one or more touch-sensitive surfaces 209B, etc. In some such examples, one or more display generation components 214A may be used by electronic device 260 to provide a three-dimensional environment, and electronic device 260 may use input and other data collected via one or more motion and / or orientation sensors 210A (and / or one or more microphones 213A) of electronic device 201 as input.

[0054] In some examples, one or more hand-tracking sensors 202 (and / or other body-tracking sensors, such as leg-tracking sensors, torso-tracking sensors, and / or head-tracking sensors) may use one or more image sensors 206 (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that capture 3D information from the real world, including one or more body parts (e.g., a human user's hand, leg, torso). In some examples, sufficient resolution is available to distinguish the hand to differentiate the fingers and their corresponding positions. In some examples, one or more image sensors 206A are positioned relative to the user to define a field of view and interaction space for one or more image sensors 206A, in which the finger / hand positions, orientations, and / or movements captured by the image sensors are used as input (e.g., to differentiate from the user's resting hand or other hands of other people in the real-world environment). Tracking the fingers / hands used for input (e.g., gestures, touches, taps, etc.) may be advantageous because it does not require the user to touch, hold, or wear any type of beacon, sensor, or other marker.

[0055] In some examples, one or more eye-tracking sensors 212 include at least one eye-tracking camera (e.g., an IR camera) and / or an illumination source (e.g., an IR light source, such as an LED) that emits light toward the user's eyes. The eye-tracking camera may be pointed at the user's eyes to receive reflected IR light from the light source directly or indirectly from the eyes. In some examples, both eyes are tracked separately by the respective eye-tracking camera and illumination source, and focus / gaze can be determined by tracking both eyes. In some examples, one eye (e.g., the dominant eye) is tracked by one or more respective eye-tracking cameras / illumination sources.

[0056] Electronic devices 201 and 260 are not limited to Figures 2A to 2B The components and configurations may include, but are not limited to, a few more components, additional components, or supplementary components in various configurations. In some examples, electronic device 201 and / or electronic device 260 may be implemented in various ways among multiple electronic devices (e.g., as a system). In some such examples, each (or more) of the electronic devices may include one or more of the same components discussed above, such as various sensors, one or more display generation components, one or more speakers, one or more processors, one or more memories, and / or communication circuitry. One or more persons using electronic device 201 and / or electronic device 260 may optionally be referred to herein as one or more users of the devices.

[0057] Figure 2C A block diagram illustrating an example architecture of a system 200 according to some examples of this disclosure is shown. In some examples, the system 200 includes multiple electronic devices. For example, in Figure 2CIn this system 200, a first electronic device 201A and a second electronic device 201B are included, wherein the first electronic device 201A and the second electronic device 201B communicate with each other. In some examples, the first electronic device 201A and the second electronic device 201B correspond to the above reference. Figure 2A The described electronic device 201 and / or has one or more characteristics of the electronic device. In some examples, the first electronic device 201A and the second electronic device 201B correspond to the above reference. Figure 1 The described electronic device 101. For example, such as... Figure 2C As shown, the first electronic device 201A optionally includes various sensors (e.g., one or more hand tracking sensors 202A, one or more position sensors 204A, one or more image sensors 206A, one or more touch-sensitive surfaces 209A, one or more motion and / or orientation sensors 210A, one or more eye tracking sensors 212A, one or more microphones 213A or other audio sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors), one or more display generation components 214A, one or more speakers 216A, one or more processors 218A, one or more memories 220A, and / or communication circuitry 222A, as referenced above). Figures 2A to 2B As described above. Similarly, in some examples, the second electronic device 201B may optionally include various sensors (e.g., one or more hand tracking sensors 202B, one or more position sensors 204B, one or more image sensors 206B, one or more touch-sensitive surfaces 209B, one or more motion and / or orientation sensors 210B, one or more eye tracking sensors 212B, one or more microphones 213B or other audio sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors), one or more display generation components 214B, one or more speakers 216B, one or more processors 218B, one or more memories 220B, and / or communication circuitry 222B, as referenced above). Figures 2A to 2B As described. In some examples, one or more hand tracking sensors 202B have one or more characteristics of one or more hand tracking sensors 202A described above. In some examples, one or more eye tracking sensors 212B have one or more characteristics of one or more eye tracking sensors 212A. One or more communication buses 208A and 208B are optionally used for communication between the aforementioned components of the first electronic device 201A and the second electronic device 201B, respectively. The first electronic device 201A and the second electronic device 201B may optionally communicate via a wired or wireless connection between the two devices (e.g., via communication circuits 222A, 222B).

[0058] System 200 is not limited to Figure 2C The system 200 may include, but is not limited to, components and configurations, but may include fewer components, other components, or additional components in various configurations. In some examples, the system 200 may be implemented in a single device. One or more persons using the system 200 may optionally be referred to herein as one or more users of the system and / or one or more devices.

[0059] Attention now turns to the exemplary simultaneous display of a three-dimensional environment on a first electronic device (e.g., corresponding to electronic device 260) and a second electronic device (e.g., corresponding to electronic device 270). As discussed below, the first electronic device can communicate with the second electronic device in a multi-user communication session. In some examples, an avatar (e.g., a representation) of a user of the first electronic device can be displayed in the three-dimensional environment at the second electronic device, and an avatar of a user of the second electronic device can be displayed in the three-dimensional environment at the first electronic device. In some examples, the users of the first and second electronic devices can be associated with a spatial group in the multi-user communication session.

[0060] Figure 3 Examples of space groups 340 in a multi-user communication session including a first electronic device 360 ​​and a second electronic device 370, according to some examples of this disclosure, are illustrated.

[0061] In some examples, a first electronic device 360 ​​may present a three-dimensional environment 350A, and a second electronic device 370 may present a three-dimensional environment 350B. The first electronic device 360 ​​and the second electronic device 370 may be similar to electronic devices 101 or 201A / 201B, and / or may be head-mounted systems / devices and / or projection-based systems / devices (including hologram-based systems / devices) configured to generate and present three-dimensional environments, such as head-up displays (HUDs), head-mounted displays (HMDs), windows with integrated display capabilities, and displays formed as lenses (e.g., similar to contact lenses) designed to be placed on a person's eyes. Figure 3 In the example, a first user may optionally wear a first electronic device 360 ​​and a second user may optionally wear a second electronic device 370, such that the three-dimensional environment 350A / 350B may be defined by the X-axis, Y-axis, and Z-axis viewed from the perspective of the electronic device (e.g., the viewpoint associated with the electronic device 360 / 370, which may be, for example, a head-mounted display).

[0062] like Figure 3As shown, the first electronic device 360 ​​may be situated in a first physical environment including a table 306 and a window 309. Therefore, the three-dimensional environment 350A presented using the first electronic device 360 ​​may optionally include captured portions of the physical environment surrounding the first electronic device 360, such as a representation of the table 306' and a representation of the window 309'. Similarly, the second electronic device 370 may be situated in a second physical environment different from the first physical environment (e.g., separate from the first physical environment), which includes a floor lamp 307 and a coffee table 308. Therefore, the three-dimensional environment 350B presented using the second electronic device 370 may optionally include captured portions of the physical environment surrounding the second electronic device 370, such as a representation of the floor lamp 307' and a representation of the coffee table 308'. Additionally, the three-dimensional environments 350A and 350B may include representations of the floor, ceiling, and walls of the rooms where the first electronic device 360 ​​and the second electronic device 370 are respectively located.

[0063] As described above, in some examples, the first electronic device 360 ​​may optionally be in a multi-user communication session with the second electronic device 370. For example, the first electronic device 360 ​​and the second electronic device 370 (e.g., via communication circuits 222A / 222B) are configured to present a shared three-dimensional environment 350A / 350B including one or more shared virtual objects (e.g., content such as images, videos, audio, etc., representations of the user interface of an application, etc.). As used herein, the term "shared three-dimensional environment" refers to a three-dimensional environment independently presented, displayed, and / or visible at two or more electronic devices, through which content, applications, data, etc., can be shared and / or presented to users of the two or more electronic devices. In some examples, when the first electronic device 360 ​​is in a multi-user communication session with the second electronic device 370, an avatar corresponding to a user of one electronic device is optionally displayed in the three-dimensional environment displayed via the other electronic device. For example, as Figure 3 As shown, at the first electronic device 360, an avatar 315 corresponding to the user of the second electronic device 370 is displayed in the three-dimensional environment 350A. Similarly, at the second electronic device 370, an avatar 317 corresponding to the user of the first electronic device 360 ​​is displayed in the three-dimensional environment 350B.

[0064] In some examples, the presentation of avatars 315 / 317 as part of a shared 3D environment may optionally be accompanied by audio effects corresponding to the speech of a user of electronic devices 370 / 360. For example, avatar 315 displayed in 3D environment 350A using first electronic device 360 ​​may optionally be accompanied by audio effects corresponding to the speech of a user of second electronic device 370. In some such examples, when a user of second electronic device 370 speaks, the user's speech may be detected by second electronic device 370 (e.g., via microphone 213B) and transmitted to first electronic device 360 ​​(e.g., via communication circuits 222B / 222A), such that the detected speech of the user of second electronic device 370 may be presented as audio (e.g., using speaker 216A) to the user of first electronic device 360 ​​in 3D environment 350A. In some examples, the audio effects corresponding to the speech of the user of second electronic device 370 may be spatialized so that they appear to the user of first electronic device 360 ​​to originate from the location of avatar 315 in shared 3D environment 350A (e.g., although output from a speaker of first electronic device 360). Similarly, the avatar 317 displayed in the three-dimensional environment 350B using the second electronic device 370 may optionally be accompanied by audio effects corresponding to the voice of the user of the first electronic device 360. In some such examples, when the user of the first electronic device 360 ​​speaks, the user's voice may be detected by the first electronic device 360 ​​(e.g., via microphone 213A) and transmitted to the second electronic device 370 (e.g., via communication circuits 222A / 222B), such that the detected voice of the user of the first electronic device 360 ​​may be presented as audio (e.g., using speaker 216B) to the user of the second electronic device 370 in the three-dimensional environment 350B. In some examples, the audio effects corresponding to the voice of the user of the first electronic device 360 ​​may be spatialized so that they appear to the user of the second electronic device 370 to originate from the location of the avatar 317 in the shared three-dimensional environment 350B (e.g., although output from the speaker of the first electronic device 360).

[0065] In some examples, when in a multi-user communication session, avatars 315 / 317 are displayed in the three-dimensional environment 350A / 350B in an orientation corresponding to and / or based on the orientation of electronic devices 360 / 370 (and / or users of electronic devices 360 / 370) in the physical environment surrounding electronic devices 360 / 370. For example, as Figure 3As shown, in three-dimensional environment 350A, avatar 315 can select the user's viewpoint from the ground towards the first electronic device 360, and in three-dimensional environment 350B, avatar 317 can select the user's viewpoint from the ground towards the second electronic device 370. When a particular user moves the electronic device (and / or itself) in the physical environment, the user's viewpoint changes according to the movement, which can therefore also change the orientation of the user's avatar in the three-dimensional environment. For example, see reference... Figure 3 If the user of the first electronic device 360 ​​looks to the left in the three-dimensional environment 350A, causing the first electronic device 360 ​​to rotate to the left (e.g., counterclockwise) (e.g., by a corresponding amount), then the user of the second electronic device 370 will see the avatar 317 corresponding to the user of the first electronic device 360 ​​rotate to the right (e.g., clockwise) relative to the viewpoint of the user of the second electronic device 370 according to the movement of the first electronic device 360.

[0066] Additionally, in some examples, when in a multi-user communication session, the viewpoint of the 3D environment 350A / 350B and / or the position of the viewpoint of the 3D environment 350A / 350B may optionally change according to the movement of the electronic device 360 / 370 (e.g., by the user of the electronic device 360 / 370). For example, when in a communication session, if the first electronic device 360 ​​moves closer toward the table representation 306' and / or the avatar 315 (e.g., because the user of the first electronic device 360 ​​moves forward in the physical environment surrounding the first electronic device 360), the viewpoint of the 3D environment 350A will change accordingly, making the table representation 306', the window representation 309', and the avatar 315 appear larger in the field of view. In some examples, each user can interact independently with the 3D environment 350A / 350B, such that changes in the viewpoint of the 3D environment 350A and / or interactions between the first electronic device 360 ​​and virtual objects in the 3D environment 350A may optionally not affect the content displayed in the 3D environment 350B at the second electronic device 370, and vice versa.

[0067] In some examples, avatar 315 / 317 is a representation of the user of electronic device 370 / 360 (e.g., full-body rendering). In some examples, avatar 315 / 317 is a representation of a portion of the user of electronic device 370 / 360 (e.g., rendering of head, face, head and torso, etc.). In some examples, avatar 315 / 317 is a representation of user personalization, user selection, and / or user creation displayed in 3D environment 350A / 350B, representing the user of electronic device 370 / 360. It should be understood that, although Figure 3 The avatars 315 / 317 illustrated correspond to the full-body representation of the user of the electronic devices 370 / 360, respectively, but alternative avatars, such as those described above, may be provided.

[0068] As described above, when the first electronic device 360 ​​and the second electronic device 370 are in a multi-user communication session, the three-dimensional environment 350A / 350B can be a shared three-dimensional environment presented using electronic devices 360 / 370. In some examples, in a multi-user communication session, content viewed by a user at one electronic device can be shared with another user at another electronic device. In some such examples, content can be experienced (e.g., viewed and / or interacted with) by two users (e.g., via their respective electronic devices) in a shared three-dimensional environment. For example, as... Figure 3 As shown, the 3D environment 350A / 350B includes a shared virtual object 310 (e.g., optionally a 3D virtual sculpture) that can be viewed and interacted with by two users. Figure 3 As shown, the shared virtual object 310 can be displayed together with a grabber indication (e.g., a control bar) 335 that can be selected to initiate movement of the shared virtual object 310 within the three-dimensional environment 350A / 350B.

[0069] In some examples, the 3D environment 350A / 350B includes non-shared content that is private to one user in a multi-user communication session. For example, in Figure 3 In this context, the first electronic device 360 ​​is displaying a private application window 330 in a three-dimensional environment 350A, which may optionally be an object in a multi-user communication session that is not shared between the first electronic device 360 ​​and the second electronic device 370. In some examples, the private application window 330 may be associated with a corresponding application operating on the first electronic device 360 ​​(e.g., a media player application, a web browsing application, a messaging application, etc.). Because the private application window 330 is not shared with the second electronic device 370, the second electronic device 370 may optionally display a representation 330 of the private application window in a three-dimensional environment 350B. Figure 3 As shown, in some examples, the representation 330 of the private application window can be a faded, obscured, discolored, and / or semi-transparent representation of the private application window 330, which prevents the user of the second electronic device 370 from viewing the content of the private application window 330.

[0070] As mentioned earlier, in some examples, users of the first electronic device 360 ​​and users of the second electronic device 370 are in space group 340 within a multi-user communication session. In some examples, space group 340 may be a baseline (e.g., first or default) space group within the multi-user communication session. For example, when users of the first electronic device 360 ​​and users of the second electronic device 370 initially join the multi-user communication session, users of the first electronic device 360 ​​and users of the second electronic device 370 are automatically (and initially, as discussed in more detail below) associated with (e.g., grouped into) space group 340 within the multi-user communication session. In some examples, when users in such... Figure 3 In the spatial group 340 shown, the user of the first electronic device 360 ​​and the user of the second electronic device 370 have a first spatial arrangement (e.g., a first spatial template) within a shared three-dimensional environment. For example, the user of the first electronic device 360 ​​and the user of the second electronic device 370 (including objects displayed in the shared three-dimensional environment) have a spatial reality within the spatial group 340. In some examples, spatial reality requires a consistent spatial arrangement between the user (or its representation) and the virtual objects. For example, the distance between the viewpoint of the user of the first electronic device 360 ​​and the avatar 315 corresponding to the user of the second electronic device 360 ​​may be the same as the distance between the viewpoint of the user of the second electronic device 370 and the avatar 317 corresponding to the user of the first electronic device 370. As described herein, if the position of the viewpoint of the user of the first electronic device 360 ​​moves, the avatar 317 corresponding to the user of the first electronic device 360 ​​moves within the three-dimensional environment 350B according to the movement of the user's viewpoint relative to the viewpoint of the user of the second electronic device 370. Additionally, if a user of the first electronic device 360 ​​interacts with the shared virtual object 310 (e.g., moves the virtual object 310 in the three-dimensional environment 350A), the second electronic device 370 changes the display of the shared virtual object 310 in the three-dimensional environment 350B based on the interaction (e.g., moves the virtual object 310 in the three-dimensional environment 350B).

[0071] It should be understood that in some examples, more than two electronic devices can be communicatively linked in a multi-user communication session. For example, in the case where three electronic devices are communicatively linked in a multi-user communication session, the first electronic device will display two avatars corresponding to the users of the other two electronic devices, instead of just one avatar. Therefore, it should be understood that the various processes and exemplary interactions described herein with reference to the first electronic device 360 ​​and the second electronic device 370 in a multi-user communication session are optionally applicable to cases where more than two electronic devices are communicatively linked in a multi-user communication session.

[0072] In some examples, it may be advantageous to provide mechanisms for facilitating multi-user communication sessions that include co-located users (e.g., co-located electronic devices associated with users). For example, it might be desirable to enable users co-located in a first physical environment to establish a multi-user communication session, allowing virtual content to be shared and presented in a three-dimensional environment, optionally content that co-located users in the multi-user communication session can view and / or interact with. In some examples, see references below. Figure 4H The presentation of virtual objects (e.g., avatars and shared virtual content) within a three-dimensional environment in a multi-user communication session including (e.g., relative to a first electronic device) co-located users is based on a shared coordinate space / system established according to the positioning and / or orientation of the co-located users in the physical environment of the first electronic device. Specifically, in a multi-user communication session consisting only of remote users (e.g., non-co-located users), the shared origin of the three-dimensional environment (e.g., as the basis for presenting content) can be determined / placed anywhere in the physical environment relative to the first user. In contrast, a multi-user communication session including only co-located users may involve agreements and / or collaboration between electronic devices regarding the placement of the shared origin of the three-dimensional environment.

[0073] In some examples, it may be advantageous to provide a mechanism for determining the location of co-located users (e.g., co-located electronic devices associated with users) before establishing a multi-user communication session. For example, it may be desirable to enable a user to select one or more co-located users to establish a multi-user communication session. Furthermore, it may be desirable to perform one or more operations relative to the co-located electronic device before establishing a multi-user communication session with it, such as operations involving determining the location of the co-located users and / or displaying a user interface and / or user interface elements that can be selected to establish a multi-user communication session.

[0074] In some examples, in response to determining that a first electronic device 101a and a second electronic device 101b are juxtaposed, the first electronic device 101a performs a location tracking process to locate and / or track the location of the second electronic device 101b. In some examples, in response to determining that a second electronic device 101b is juxtaposed with the first electronic device 101a, the second electronic device 101b performs a location tracking process to locate and / or track the location of the first electronic device 101a. In some examples, the location tracking process utilizes map data from the first electronic device 101a and map data from the second electronic device 101b. For example, the first electronic device may store Simultaneous Localization and Mapping (SLAM) map data determined by the first electronic device 101a (e.g., independently), and the second electronic device 101b may store SLAM map data determined by the second electronic device 101b (e.g., independently). In some examples, the location tracking process is performed before a multi-user communication session between the first electronic device 101a and the second electronic device 101b has been established.

[0075] Figures 4A to 4G Examples of location tracking processes performed between a first electronic device 101a and a second electronic device 101b prior to the establishment of a multi-user communication session between the two devices are generally illustrated according to some examples of this disclosure.

[0076] Figure 4A A first electronic device 101a is shown, which is in the same physical environment 400 as the second electronic device 101b and the third electronic device 101c. Figure 4A As illustrated in top view 410 (e.g., top-to-bottom view), a first user 402 wears a first electronic device 101a, a second user 404 wears a second electronic device 101b, and a third user 406 wears a third electronic device 101c. Note that... Figure 4A and Figures 4E to 4H This includes a corresponding top view 410, which is applicable to the corresponding diagram in which it is located. Figure 4AIn this context, the display 120a of the first electronic device 101a shows a first three-dimensional environment 450A including a second user 404 of the second electronic device 101b and a third user 406 of the third electronic device 101c. In some examples, assuming the first electronic device 101a and the second electronic device 101b are juxtaposed, the first electronic device 101a performs a position tracking process with the second electronic device 101b, as described below. In some examples, assuming the first electronic device 101a and the third electronic device 101c are juxtaposed, the first electronic device 101a performs a position tracking process with the third electronic device 101c, as described below. It should be noted that the position tracking process can be performed without specific user input requesting the execution of the position tracking process. For example, as described above, a position tracking process between the first electronic device 101a and the second electronic device 101b can be performed in response to determining the juxtaposition between the first electronic device 101a and the second electronic device 101b.

[0077] exist Figure 4A In this configuration, the first electronic device 101a and the second electronic device 101b are juxtaposed in physical environment 400. In some examples, the first electronic device 101a and the second electronic device 101b are determined to be juxtaposed because they are located in the same physical room. In some examples, if the first electronic device 101a and the second electronic device 101b are not in the same physical room, then it will be determined that the first electronic device 101a and the second electronic device 101b are not juxtaposed.

[0078] In some examples, determining the juxtaposition of the first electronic device 101a and the second electronic device 101b in physical environment 400 is based on the distance between the first electronic device 101a and the second electronic device 101b. For example, in Figure 4A In this configuration, the first electronic device 101a and the second electronic device 101b are placed side-by-side in the physical environment 400 because the first electronic device 101a is within a threshold distance (e.g., 0.1 m, 0.5 m, 1 m, 2 m, 3 m, 5 m, 10 m, 15 m, 20 m, etc.) of the second electronic device 101b. If the first electronic device 101a and the second electronic device 101b are not within the threshold distance, then the first electronic device 101a and the second electronic device 101b may not be placed side-by-side.

[0079] In some examples, determining the juxtaposition of the first electronic device 101a and the second electronic device 101b in physical environment 400 is based on communication between the first electronic device 101a and the second electronic device 101b. For example, in Figure 4AIn this configuration, the first electronic device 101a and the second electronic device 101b can communicate with each other (e.g., wirelessly, such as via Bluetooth, Wi-Fi, or a server (e.g., a wireless communication terminal)). For example, one or more signals transmitted by the first electronic device 101a may be detected by a receiver at the second electronic device 101b, and / or one or more signals transmitted by the second electronic device 101b may be detected by a receiver at the first electronic device 101a. If no signal is transmitted from the first electronic device 101a or detected by the second electronic device 101b, it can be determined that the first electronic device 101a and the second electronic device 101b are not side-by-side. Similarly, if no signal is transmitted from the second electronic device 101b or detected by the first electronic device 101a, it can be determined that the first electronic device 101a and the second electronic device 101b are not side-by-side.

[0080] In some examples, the juxtaposition of the first electronic device 101a and the second electronic device 101b in physical environment 400 is determined based on the strength of wireless signals transmitted and detected between the first electronic device 101a and the second electronic device 101b. For example, in Figure 4A In this configuration, the first electronic device 101a and the second electronic device 101b are juxtaposed in physical environment 400 because the strength of the Bluetooth signal (or other wireless signal) transmitted and detected between the first electronic device 101a and the second electronic device 101b is greater than a threshold strength. For example, if the second electronic device 101b detects a Bluetooth signal from the first electronic device 101a with a signal strength greater than the threshold, it can be determined that the first electronic device 101a and the second electronic device 101b are juxtaposed. Similarly, if the first electronic device 101a detects a Bluetooth signal from the second electronic device 101b with a signal strength greater than the threshold, it can be determined that the first electronic device 101a and the second electronic device 101b are not juxtaposed and / or the wireless transmission or reception features are turned off at the first electronic device 101a and / or the second electronic device 101b. Similarly, if the first electronic device 101a detects a Bluetooth signal with a signal strength less than a threshold from the second electronic device 101b, it can be determined that the first electronic device 101a and the second electronic device 101b are not co-located and / or the wireless transmission or wireless reception features are turned off at the second electronic device 101b and / or the first electronic device 101a.

[0081] In some examples, the determination of the co-location of the first electronic device 101a and the second electronic device 101b is based on the fact that the first electronic device 101a and the second electronic device 101b are connected to the same network (e.g., a wireless network) in the physical environment 400. For example, if it is determined that the first electronic device 101a and the second electronic device 101b are connected to the same wireless network, it can be determined that the first electronic device 101a and the second electronic device 101b are co-located. Continuing with this example, if it is determined that the first electronic device 101a and the second electronic device 101b are not connected to the same wireless network, it can be determined that the first electronic device 101a and the second electronic device 101b are not co-located.

[0082] In some examples, determining that the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400 is based on visual detection of the first electronic device 101a and the second electronic device 101b in the physical environment 400. For example, as Figure 4A As shown, the second electronic device 101b is positioned within the field of view of the first electronic device 101a, enabling the first electronic device 101a to visually detect (e.g., identify or scan, such as via object detection / recognition or other image processing techniques) the second electronic device 101b (e.g., in one or more images captured by the first electronic device 101a, such as via external image sensors 114b-i and 114c-i). Furthermore, if the first electronic device 101a is within the field of view of the second electronic device 101b, it can be determined that the first electronic device 101a and the second electronic device 101b are juxtaposed. Conversely, if the second electronic device 101b is not within the field of view of the first electronic device 101a, it can be determined that the first electronic device 101a and the second electronic device 101b are not juxtaposed. Similarly, if the first electronic device 101a is not within the field of view of the second electronic device 101b, it can be determined that the first electronic device 101a and the second electronic device 101b are not juxtaposed. It should be noted that if the first electronic device 101a detects the second electronic device 101b via its image sensor, but the second electronic device 101b does not detect the first electronic device 101a via its image sensor, then it can be determined that the first electronic device 101a and the second electronic device 101b are juxtaposed. Similarly, it should be noted that if the second electronic device 101b detects the first electronic device 101a via its image sensor, but the first electronic device 101a does not detect the second electronic device 101b via its image sensor, then it can be determined that the first electronic device 101a and the second electronic device 101b are juxtaposed.

[0083] In some examples, determining that a first electronic device 101a and a second electronic device 101b are juxtaposed in physical environment 400 is based on the contact list of the other electronic device's user on the electronic device. For example, the first electronic device 101a and the second electronic device 101b may be juxtaposed in physical environment 400 because a first user 402 of the first electronic device 101a is in the contact list of the second electronic device 101b (e.g., a contact list of an application accessible via the second electronic device 101b, such as a phone application, a contact list application, an email application, a communication application, or another type of application). Additionally or alternatively, for example, the first electronic device 101a and the second electronic device 101b may be juxtaposed in physical environment 400 because a second user 404 of the second electronic device 101b is in the contact list of the first electronic device 101a (e.g., a contact list of an application accessible via the first electronic device 101a, such as a phone application, a contact list application, an email application, a communication application, or another type of application). In some examples, if the user of the first electronic device 101a is not in the contact list of the second electronic device 101b and / or if the user of the second electronic device 101b is not in the contact list of the first electronic device 101a, it can be determined that the first electronic device 101a and the second electronic device 101b are not co-located in the physical environment 400, even though they are in the physical environment 400. It should be noted that in some examples, if the second user 404 of the second electronic device 101b is in the contact list of the first electronic device 101a, while the first user 402 is not in the contact list of the second electronic device 101b, it can be determined that the first electronic device 101a and the second electronic device 101b are co-located. It should be noted that in some examples, if the second user 404 of the second electronic device 101b is in the contact list of the first electronic device 101a, while the first user 402 is not in the contact list of the second electronic device 101b, it can be determined that the first electronic device 101a and the second electronic device 101b are not co-located.

[0084] It should be noted that, Figure 4A In this context, it can be determined that the first electronic device 101a and the third electronic device 101c are co-located (or non-co-located) in the physical environment 400, as described above with reference to determining that the first electronic device 101a and the second electronic device 101b are co-located (or non-co-located) in the physical environment 400. It should be noted that in... Figure 4AIn this context, the distance between the first electronic device 101a and the third electronic device 101c is greater than the distance between the first electronic device 101a and the second electronic device 101b. In some examples, this distance is greater than a threshold distance, making it possible to determine that the first electronic device 101a and the third electronic device 101c are not co-located in physical environment 400. In some examples, it is still possible to determine that the first electronic device 101a and the third electronic device 101c are not co-located in physical environment 400 if the distance between the first electronic device 101a and the third electronic device 101c is the same as the distance between the first electronic device 101a and the second electronic device 101b. For example, as described above, co-location can be further based on the user of the electronic device being in the contact list of the other electronic device, and if the user of the third electronic device 101c is not in the contact list of the first electronic device 101a, it can be determined that the first electronic device 101a and the third electronic device 101c are not co-located in physical environment 400.

[0085] Figure 4B Examples of the attitude (e.g., positioning and / or orientation) of a first electronic device 101a and the attitude (e.g., positioning and / or orientation) of a second electronic device 101b based on their respective reference origins (e.g., respective reference positioning and orientation). Figure 4BIn this embodiment, a first electronic device 101a has a first reference origin 412a (e.g., a first reference position) by which it determines its own posture (e.g., positioning and / or orientation). For example, the first reference origin 412a of the first electronic device 101a can be used by the first electronic device 101a to determine how it is positioned and / or oriented in the physical environment 400, and can be used to determine how to present virtual content (e.g., virtual content private to the first electronic device 101a (e.g., not shared)) at the first electronic device 101a. For example, using the posture (e.g., positioning and / or orientation) of the first electronic device 101a relative to the first reference origin 412a, the first electronic device 101a can display content at a specific location and / or can present spatialized audio at different locations relative to the current location of the first electronic device 101a. Therefore, the first electronic device 101a can present virtual content relative to the first reference origin 412a, thereby enabling the first electronic device 101a to maintain spatial realism when interacting with virtual content in the physical environment 400. In some examples, the first electronic device 101a determines its first reference origin 412a before initiating a position tracking process between the first electronic device 101a and the second electronic device 101b. In some examples, the first electronic device 101a determines its first reference origin 412a before determining that the first electronic device 101a and the second electronic device 101b are juxtaposed. For example, the first electronic device 101a can determine its first reference origin 412a in the physical environment 400 when it is first used and / or powered on by the first user 402.

[0086] exist Figure 4B In this example, vector 414a (e.g., an arrow) extends from the first reference origin 412a to the current position of the first electronic device 101a and indicates a first attitude (e.g., positioning and / or orientation) of the first electronic device 101a. For example, vector 414a may be based on a Cartesian coordinate system and / or another coordinate system. Furthermore, the orientation (e.g., angular orientation) of the first electronic device 101a at its current position may also be tracked and included in the attitude information of the first electronic device 101a relative to the first reference origin 412a. Thus, in some examples, vector 414a represents the positioning of the first electronic device 101a relative to the first reference origin 412a and the orientation of the first electronic device 101a relative to the first reference origin 412a. The attitude (e.g., positioning and / or orientation) of the first electronic device 101a may follow Expression 1:

[0087] First attitude = attitude of the first electronic device relative to the first reference origin.

[0088] In addition, Figure 4B In this embodiment, the second electronic device 101b has a second reference origin 412b (e.g., a second reference position) through which it determines its own posture (e.g., positioning and / or orientation). For example, the second reference origin 412b of the second electronic device 101b can be used to determine how the second electronic device 101b is positioned and / or oriented in the physical environment 400, and can be used by the second electronic device 101b to determine how virtual content (e.g., virtual content private to the second electronic device 101b (e.g., not shared)) is presented at the second electronic device 101b. For example, using the posture (e.g., positioning and / or orientation) of the second electronic device 101b relative to the second reference origin 412b, the second electronic device 101b can display content at a specific location and / or can present spatialized audio at different locations relative to the current location of the second electronic device 101b. Therefore, the second electronic device 101b can present virtual content relative to the second reference origin 412b, thereby enabling the second electronic device 101b to maintain spatial realism when interacting with virtual content in the physical environment 400. In some examples, the second electronic device 101b determines its second reference origin 412b before initiating a position tracking process between the first electronic device 101a and the second electronic device 101b. In some examples, the second electronic device 101b determines its second reference origin 412b before determining that the first electronic device 101a and the second electronic device 101b are juxtaposed. For example, the second electronic device 101b can determine its second reference origin 412b in the physical environment 400 when it is first used and / or powered on by the user 404.

[0089] exist Figure 4BIn this example, vector 414b (e.g., an arrow) extends from the second reference origin 412b to the current position of the second electronic device 101b. For example, vector 414b may be based on a Cartesian coordinate system and / or another coordinate system. Furthermore, the orientation (e.g., angular orientation) of the second electronic device 101b at its current position can also be tracked and included in the attitude information of the second electronic device 101b relative to the second reference origin 412b. Therefore, in some examples, vector 414b represents the positioning of the second electronic device 101b relative to the second reference origin 412b and the orientation of the first electronic device 101a relative to the second reference origin 412b. The attitude (e.g., positioning and / or orientation) of the second electronic device 101b can follow Expression 2:

[0090] Second attitude = attitude of the second electronic device relative to the second reference origin.

[0091] It should be noted that, Figure 4B In the vector 414a, vector 414b is private to the first electronic device 101a, and vector 414b is private to the second electronic device 101b; therefore, in Figure 4B In this context, vector 414a may not be shared with the second electronic device 101b, and vector 414b may not be shared with the second electronic device 101b. For example, in... Figure 4B In this process, the first electronic device 101a may not share data indicating its attitude (e.g., positioning and / or orientation) with the second electronic device 101b, and the second electronic device 101b may not share data indicating its attitude (e.g., positioning and / or orientation) with the first electronic device 101a.

[0092] In some examples, the first reference origin 412a is shared with the second electronic device 101b (e.g., data indicating the first reference origin 412a is shared with the second electronic device 101b). In some examples, the second reference origin 412b is shared with the first electronic device 101a (e.g., data indicating the second reference origin 412b is shared with the first electronic device 101a). In some examples, the first reference origin 412a is shared with the second electronic device 101b without sharing the first pose of the first electronic device 101a (e.g., expression 1, vector 414b). In some examples, the second reference origin 412b is shared with the first electronic device 101a without sharing the second pose of the second electronic device 101b (e.g., expression 2, vector 414b). Therefore, in some examples, the first electronic device 101a announces the first reference origin 412a (e.g., wirelessly transmits data indicating the first reference origin), and / or the second electronic device 101b announces the second reference origin 412b (e.g., wirelessly transmits data indicating the second reference origin).

[0093] Figure 4C This is an example flowchart illustrating communication between a first electronic device 101a and a second electronic device 101b that can occur in response to determining that the first electronic device 101a and the second electronic device 101b are juxtaposed. Note that... Figure 4C The illustrated operations can be performed before establishing a multi-user communication session between the first electronic device 101a and the second electronic device 101b (e.g., as detailed below). Figure 4C The illustrated operations include transmitting data between the first electronic device 101a and the second electronic device 101b. Therefore, before a multi-user communication session is established between the first electronic device 101a and the second electronic device 101b, data can be transmitted from the first electronic device 101a to the second electronic device 101b and / or from the second electronic device 101b to the first electronic device 101a. For example, the first electronic device 101a may announce first information via Bluetooth or another wireless protocol, such as those described below. Figure 4C The information associated with box 428, and the second electronic device 101b can detect the first information without establishing a multi-user communication session between the first electronic device 101a and the second electronic device 101b. Similarly, the second electronic device 101b can announce the second information via Bluetooth or another wireless protocol, such as with... Figure 4C The information associated with box 430, and the first electronic device 101a can detect the second information without establishing a multi-user communication session between the first electronic device 101a and the second electronic device 101b.

[0094] In addition, in execution Figure 4C During one or more of the operations, the first electronic device 101a and the second electronic device 101b may not have yet accepted a request to join or initiate a multi-user communication session with each other. For example, the first electronic device 101a and the second electronic device 101b may not present or display content being shared from another electronic device because, during the execution of one or more of the operations, the first electronic device 101a and the second electronic device 101b may not present or display content being shared from another electronic device. Figure 4C During operation, a multi-user communication session has not yet been initiated or established between the first electronic device 101a and the second electronic device 101b. Furthermore, in some examples, Figure 4C The illustrated operations can be performed by the first electronic device 101a and / or the second electronic device 101b without requiring a request from the first user 402 of the first electronic device 101a or the second user 404 of the second electronic device 101b. Figure 4C The specific input for the illustrated operation. Note that in some examples, Figure 4C The operations shown are executed in the order shown. Note that, alternatively, in some examples, Figure 4C The operations shown are executed in a different order than the order shown.

[0095] At box 414, first electronic device 101a sends a message (e.g., a wireless signal, such as via Bluetooth communication) to second electronic device 101b. In some examples, the message corresponds to a request from second electronic device 101b to send map data to first electronic device 101a. In some examples, the sending of the message initiates a location tracking process between first electronic device 101a and second electronic device 101b. In some examples, before sending the message, it is determined that first electronic device 101a and second electronic device 101b are juxtaposed, and first electronic device 101a sends the message based on this determination. In some examples, if it is determined that first electronic device 101a and second electronic device 101b are not juxtaposed in physical environment 400, first electronic device 101a may not send the message illustrated at box 414 to second electronic device 101b.

[0096] exist Figure 4CIn this context, the second electronic device 101b detects a message from the first electronic device 101a and responds by sending map data determined by the second electronic device 101b to the first electronic device 101a (box 416). For example, in physical environment 400, the second electronic device 101b may detect image data via one or more external sensors and may use the image data to determine a map of physical environment 400 relative to the viewpoint of the second electronic device 101b (e.g., the positions of walls, floors, objects in a room, etc.). The map data sent by the second electronic device 101b to the first electronic device 101a may be based on such data. In some examples, the map data sent by the second electronic device 101b is SLAM data determined by the second electronic device 101b.

[0097] In addition, such as Figure 4C As shown, a first electronic device 101a sends its map data to a second electronic device 101b (box 418). For example, in physical environment 400, the first electronic device 101a can detect image data via one or more external sensors of the first electronic device 101a, and can use the image data to determine a map drawing of the physical environment 400 relative to the viewpoint of the first electronic device 101a. The map data sent by the first electronic device 101a to the second electronic device 101b can be based on such data. In some examples, the map data sent by the first electronic device 101a is SLAM data determined by the first electronic device 101a (e.g., independently). Note that reference... Figure 4C The data transmission described may be as an indication of map data, as bits of encrypted data, and / or may be decrypted by the receiving electronic device.

[0098] In some examples, the first electronic device 101a processes its map data and map data sent to it by the second electronic device 101b. In some examples, the second electronic device 101b processes its map data and map data sent to it by the first electronic device 101a. In some examples, the first electronic device 101a receives map data from the second electronic device 101b without sending its own map data to the second electronic device 101b. In some examples, the second electronic device 101b receives map data from the first electronic device 101a without sending its own map data to the first electronic device 101a.

[0099] like Figure 4CAs shown, the first electronic device 101a and the second electronic device 101b can continue to transmit map data until the alignment of the map data is determined at either or both of the first electronic device 101a and the second electronic device 101b (e.g., box 420). The map data of the first electronic device 101a and the second electronic device 101b (e.g., processed by the first electronic device 101a and / or the second electronic device 101b) is processed until the merging of the map data is completed. For example, the first electronic device 101a and the second electronic device 101b can continuously transmit their respective map data to each other until the map data are aligned. Additionally, in some examples, the operations involved in box 420 can take into account changes in the pose (e.g., positioning and / or orientation) of the first electronic device 101a and / or the second electronic device 101b. For example, if one of the users (e.g., user 402 or user 404) moves, resulting in a change in the viewpoint of one of the electronic devices, the operations involved in box 420 may include transmitting map data at the transmitting electronic device based on the updated viewpoint determined by the transmitting electronic device (e.g., the first electronic device 101a or the second electronic device 101b).

[0100] In some examples, in response to the merging of map data (e.g., based on determining map data alignment), the first electronic device 101a or the second electronic device 101b selects (box 422) a reference origin to use as a shared reference origin (e.g., a shared reference location) for the position tracking process between the first electronic device 101a and the second electronic device 101b, such as Figure 4D A shared reference origin 424 (e.g., a shared reference location) is used. The shared reference origin 424 can serve as the origin for a first electronic device 101a that tracks the location of the second electronic device 101b, the origin for the second electronic device 101b that tracks the location of the first electronic device 101a, and / or the origin for other operations performed at either electronic device that may include the location of the other electronic device as input. In some examples, the first electronic device 101a is selected. In some examples, the second electronic device 101b is selected. In some examples, which electronic device is selected is arbitrary. In some examples, which electronic device is selected is not arbitrary (e.g., the electronic device performing the operation associated with box 414 is selected).

[0101] In some examples, a shared reference origin 424 is selected from multiple potential reference locations. In some examples, multiple potential reference locations for the location tracking process are obtained in response to processing (e.g., comparison) of map data of the first electronic device 101a and the second electronic device 101b. In some examples, only one potential reference location for the location tracking process between the first electronic device 101a and the second electronic device 101b is obtained in response to comparing map data of the first electronic device 101a and the second electronic device 101b. In some examples, either the first electronic device 101a or the second electronic device 101b selects a potential reference location as the shared reference origin for the location tracking process between the first electronic device 101a and the second electronic device 101b, such as... Figure 4D The shared reference origin is 424. In some examples, the first electronic device 101a is selected. In some examples, the second electronic device 101b is selected. In some examples, which electronic device makes the selection is not arbitrary (e.g., the electronic device performing the operation associated with box 414 makes the selection).

[0102] In some examples, in response to the selection of a shared reference origin 424, the selecting electronic device sends a message notifying the other electronic device of the selection. For example, if the first electronic device 101a makes a selection, it may send a message indicating the shared reference origin 424 to the second electronic device 101b, and if the second electronic device 101b makes a selection, it may send a message indicating the shared reference origin 424 to the first electronic device 101a. In some examples, the shared reference origin 424 is located between the positions of the first electronic device 101a and the second electronic device 101b (e.g., from the perspective of the first electronic device 101a and / or the second electronic device 101b). It should be noted that in some examples, during position tracking between the first electronic device 101a and the second electronic device 101b, communication between them is based on Bluetooth and / or another wireless signal-based communication protocol.

[0103] It should be noted that the first electronic device 101a can determine the positioning and orientation of the shared reference origin 424 relative to the first reference origin 412a of the first electronic device 101a, such as in Figure 4C In box 422. In some examples, in response to determining a shared reference origin 424 between the first electronic device 101a and the second electronic device 101b, the first electronic device 101a determines a first offset (e.g., displacement) between the shared reference origin 424 and a first reference origin 412a of the first electronic device 101a, such as by... Figure 4DThe first offset vector 422a (e.g., the first offset arrow) is illustrated. Note that... Figure 4D The first offset vector 422a indicates the positioning offset and orientation (e.g., direction) offset between the positioning and orientation of the shared reference origin 424 and the positioning and orientation of the first reference origin 412a. For example, the origin indicated by the first reference origin 412a of the first electronic device 101a may not be aligned in positioning and / or orientation with the origin indicated by the shared reference origin 424. Figure 4D The first offset vector 414a between the shared reference origin 424 and the first reference origin 412a of the first electronic device 101a can follow expression 3:

[0104] First offset vector = First reference origin – Shared reference origin

[0105] In some examples, the first electronic device 101a transmits data indicating a first offset vector 414a and data indicating a first attitude (e.g., positioning and / or orientation) to the second electronic device 101b, such as... Figure 4C Box 428 is shown. In some examples, the data also includes the shared reference origin 424 and the identifier of the first electronic device 101a. Note that the position data (e.g., the first offset vector 414a and / or the first attitude) can be encrypted; therefore, encryption keys can be exchanged between the first electronic device 101a and the second electronic device 101b. Having data indicating the first offset vector 414a and data indicating the first attitude (e.g., positioning and / or orientation) of the second electronic device 101b, the second electronic device 101a can determine the attitude (e.g., positioning and / or orientation) of the first electronic device 101a relative to the shared reference origin 424 using the following expression 4:

[0106] The position of the first electronic device relative to the shared reference origin = the position of the first electronic device relative to the first reference origin – the first offset vector

[0107] Expression 4 can also be expressed as:

[0108] Expression 4 = Expression 1 – Expression 3

[0109] The second electronic device 101b can calculate expression 4 in response to receiving data indicating expression 1 and data indicating expression 3 from the first electronic device 101a, to determine the position of the first electronic device 101a relative to the shared reference origin 424 (e.g., relative to the second electronic device 101b's understanding of the location of the shared reference origin 424, such as in...). Figure 4C The determination is made at box 422 in the diagram. Therefore, the first electronic device 101a can send the first offset and its attitude relative to the first reference origin 412a (e.g., by...). Figure 4D The vector 414a in the first electronic device 101a indicates the first attitude, but does not send its attitude (e.g., positioning and / or orientation) relative to different locations (e.g., different origins), and the second electronic device 101b can calculate expression 4 to determine the position of the first electronic device 101a relative to the shared reference origin 424, which the second electronic device 101b already knows in its local coordinate system.

[0110] It should be noted that the second electronic device 101b can determine the positioning and orientation of the shared reference origin 424 relative to the second reference origin 412b of the second electronic device 101b, such as in Figure 4C In box 422. In some examples, in response to determining a shared reference origin 424 between the first electronic device 101a and the second electronic device 101b, the second electronic device 101b determines a second offset (e.g., displacement) between the shared reference origin 424 and the second reference origin 412b of the second electronic device 101b, such as by... Figure 4D The second offset vector 422b (e.g., the second offset arrow) is illustrated. Note that... Figure 4D The second offset vector 422b indicates the positioning offset and orientation (e.g., direction) offset between the positioning and orientation of the shared reference origin 424 and the positioning and orientation of the second reference origin 412b. For example, the origin indicated by the second reference origin 412b of the second electronic device 101b may not be aligned with the origin indicated by the shared reference origin 424 in positioning and / or orientation. Figure 4D The second offset vector 422b between the shared reference origin 424 and the second reference origin 412b of the second electronic device 101b can follow expression 5:

[0111] Second offset vector = Second reference origin – Shared reference origin

[0112] In some examples, the second electronic device 101b transmits data indicating the second offset vector 414b and data indicating the second attitude (e.g., positioning and / or orientation) to the first electronic device 101a, such as... Figure 4C Box 430 is shown. In some examples, the data also includes the shared reference origin 424 and the identifier of the second electronic device 101b. Note that the position data (e.g., the second offset vector 414b and / or the second attitude) can be encrypted; therefore, encryption keys can be exchanged between the first electronic device 101a and the second electronic device 101b. With data indicating the second offset vector 414b and data indicating the second attitude (e.g., positioning and / or orientation) of the second electronic device 101b, the first electronic device 101a can determine the attitude (e.g., positioning and / or orientation) of the second electronic device 101b relative to the shared reference origin 424 using the following expression 6:

[0113] The position of the second electronic device relative to the shared reference origin = the position of the second electronic device relative to the second reference origin – the second offset vector

[0114] Expression 6 can also be represented as:

[0115] Expression 6 = Expression 2 – Expression 4

[0116] The first electronic device 101a can calculate expression 6 in response to receiving data indicating expression 2 and data indicating expression 4 from the second electronic device 101b, to determine the position of the second electronic device 101b relative to the shared reference origin 424 (e.g., relative to the first electronic device 101a's understanding of the location of the shared reference origin 424, such as in...). Figure 4C The determination is made at box 422 in the diagram. Therefore, the second electronic device 101b can transmit the second offset and its attitude relative to the second reference origin 412b (e.g., by...). Figure 4D The vector 414b in the first electronic device 101a indicates the second attitude, without sending its attitude (e.g., positioning and / or orientation) relative to a different location (e.g., a different origin), and the second electronic device 101b can calculate expression 6 to determine the position of the first electronic device 101a relative to the shared reference origin 424, which the first electronic device 101a already knows in its local coordinate system.

[0117] Therefore, the first electronic device 101a can send data indicating a first attitude (e.g., according to Expression 1) and a first offset vector (e.g., according to Expression 3) to the second electronic device 101b, and the second electronic device 101b can use the data according to Expression 4 to determine the position of the first electronic device 101a. Similarly, the second electronic device 101b can send data indicating a second attitude (e.g., according to Expression 2) and a second offset vector (e.g., according to Expression 4) to the first electronic device 101a, and the second electronic device 101b can use the data according to Expression 6 to determine the position of the first electronic device 101a.

[0118] Before establishing a multi-user communication session between the first electronic device 101a and the second electronic device 101b (e.g., before either electronic device accepts a request to establish a multi-user communication session with the other electronic device), expression 6 is determined, and the first electronic device 101a may use this information in different ways. For example, based on the fact that the first electronic device 101a has determined where the second electronic device 101b will be located, the first electronic device 101a may display user interface elements that can be selected to establish a multi-user communication session between the first electronic device 101a and the second electronic device 101b, such as... Figure 4EUser interface element 440a is displayed in the first electronic device 101a. For example, if the second electronic device 101b is in the field of view of the user of the first electronic device 101a, the first electronic device 101a displays the user interface element 440a at a position based on the position of the second electronic device 101b in the field of view of the user of the first electronic device 101a. Continuing this example, if the second electronic device 101b is not in the field of view of the user of the first electronic device 101a, the second electronic device 101b may not display the user interface element 440a. Furthermore, the first electronic device 101a may display the user interface element 440a facing the user of the first electronic device 101a, such that if the first electronic device 101a moves around the position of the second electronic device 101b, the user interface element 440a moves accordingly (e.g., rotates around the position of the second electronic device 101b). In some examples, user interface element 440a includes one or more areas that can be selected to perform different operations, such as a first area that can be selected to establish a multi-user communication session (e.g., to share content to be presented with the second electronic device 101b), and a second area that is different from the first area that can be selected to stop displaying user interface element 440a without establishing a multi-user communication session.

[0119] Figure 4F An example is illustrated where a first electronic device 101a simultaneously displays a user interface element 440a (which can be selected to establish a multi-user communication session between the first electronic device 101a and the second electronic device 101b) and a user interface element 440b (which can be selected to establish a multi-user communication session between the first electronic device 101a and the third electronic device 101c). For example, the first electronic device 101a and the third electronic device 101c may have been determined to be juxtaposed in a physical environment 400, and operations performed between the first electronic device 101a and the second electronic device 101b may also have been performed between the first electronic device 101a and the third electronic device 101c. For example, refer to... Figure 4CThe operations described by the first electronic device 101a and the second electronic device 101b can be performed independently between the first electronic device 101a and the third electronic device 101c. Therefore, the first electronic device 101a can maintain a shared reference origin 424 between the first electronic device 101a and the second electronic device 101b, and can maintain a shared reference position 432 (e.g., a shared reference origin) between the first electronic device 101a and the third electronic device 101c. Thus, in some examples, the first electronic device 101a can display multiple user interface elements that can be selected to establish a multi-user communication session with different electronic devices based on corresponding shared reference positions determined between the respective electronic devices. Furthermore, the first electronic device 101a can understand the position of the second electronic device 101b relative to the shared reference origin 424 (e.g., according to Expression 6), and can understand the position of the third electronic device 101c relative to... Figure 4F The position of the shared reference position 432 of the user interface element 440b in the middle.

[0120] Returning to the location tracking process between the first electronic device 101a and the second electronic device 101b, before establishing a multi-user communication session between the two devices (e.g., before either device accepts a request to establish a multi-user communication session with the other), expression 5 is determined. The second electronic device 101b may use this information in different ways. For example, but based on the fact that the second electronic device 101b has determined where the first electronic device 101a will be located, as described above with reference to the user's field of view of the second electronic device 101b, the second electronic device 101b may display user interface elements that can be selected to establish a multi-user communication session between the first electronic device 101a and the second electronic device 101b, such as... Figure 4E User interface element 440a.

[0121] It should be noted that in some examples, only the first electronic device 101a displays user interface element 440a, or the second electronic device 101b displays user interface elements that can be selected to initiate a multi-user communication session between the first electronic device 101a and the second electronic device 101b (e.g., similar to user interface element 440a). Additionally, it should be noted that user interface element 440a is not shared from the first electronic device 101a to the second electronic device 101b. For example, user interface element 440a may not be visible to the second electronic device 101b (e.g., not displayed by the second electronic device 101b); however, the second electronic device 101b may display user interface elements that can be selected to establish a multi-user communication session between the first electronic device 101a and the second electronic device 101b, as described with reference to the display of user interface element 440a. It should be noted that when user interface element 440a is displayed, the multi-user communication session between the first electronic device 101a and the second electronic device 101b has not yet been established. For example, visual and / or audio content is not shared between the first electronic device 101a and the second electronic device 101b when displaying user interface elements and / or when performing position tracking between the first electronic device 101a and the second electronic device 101b.

[0122] In some examples, from the perspective of a user of the first electronic device 101a, the first electronic device 101a displays user interface elements between the location of the first electronic device 101a and the location of the second electronic device 101b, such as Figure 4E The position of the user interface element 440a. In some examples, the first electronic device 101a displays the user interface element at the midpoint between the position of the first electronic device 101a and the position of the second electronic device 101b. In some examples, from the viewpoint of the first electronic device 101a, the first electronic device 101a displays the user interface element closer to its position than the position of the second electronic device 101b. In some examples, from the viewpoint of the first electronic device 101a, the first electronic device 101a displays the user interface element closer to the position of the second electronic device 101b than the position of the first electronic device 101a. In some examples, from the viewpoint of the first electronic device 101a, the position of the second electronic device 101b does not overlap with the position of the user interface element 440a.

[0123] In some examples, if the position of the second electronic device 101b changes while the first electronic device 101a is displaying user interface elements that can be selected to initiate a multi-user communication session with the second electronic device 101b, the first electronic device 101a can change the display position of the user interface elements, such as... Figure 4EThe display position of the user interface element 440a in the middle is different due to the change in the position of the second electronic device 101b. Figure 4G The display position of the user interface element 440a is shown in the diagram. Therefore, the first electronic device 101a can update the display position of the user interface element 440a based on the detection of an update to the position of the second electronic device 101b.

[0124] In some examples, when user interface element 440a is displayed, the first electronic device 101a detects a selection of user interface element 440a. For example, the first electronic device 101a may detect a gesture (e.g., a pinch-to-open gesture) pointing at user interface element 440a when the user's gaze is directed at it. In response, a multi-user communication session between the first electronic device 101a and the second electronic device 101b may be initiated. In some examples, a multi-user communication session between the first electronic device 101a and the second electronic device 101b may be initiated in response to input from the first user 402 corresponding to a selection of user interface element 440a and in response to user input from the second user 404 of the second electronic device 101b accepting a request to initiate a multi-user communication session with the first electronic device 101a. In some examples, the second electronic device 101b does not present the second user 404 with any notifications regarding the location tracking process of the second electronic device 101b with the first electronic device 101a. In some examples, the first electronic device 101a does not present the first user 402 with any notifications regarding the location tracking process of the first electronic device 101a with the second electronic device 101b. In some examples, the display of user interface element 440a serves to notify the first user 402 that the first electronic device 101a is involved in a location tracking process with the second electronic device 101b. In some examples, when a multi-user communication session is active, either the first electronic device 101a or the second electronic device 101b may request to share virtual content (e.g., virtual visual and / or audio content) with another electronic device in the multi-user communication session. In response to detecting the request, a process for presenting the virtual content is initiated. For example, if the first electronic device 101a requests to share the display of the user interface with the second electronic device 101b, the second electronic device 101b may optionally display the user interface at the corresponding location in the physical environment where the first electronic device 101a displays the user interface, such as... Figure 4H The user interface 434 is shown in the example. For example, in Figure 4HIn this configuration, the user interface 434 is displayed in a first three-dimensional environment 450A presented via a display 120a of the first electronic device 101a, and in a second three-dimensional environment 450B presented via a display 120b of the second electronic device 101b, which are located at the same corresponding position in the physical environment 400. Therefore, in some examples, shared visual virtual content between the first electronic device 101a and the second electronic device 101b may be placed at the same corresponding position in the physical environment 400. Furthermore, a first user interface element 431a (e.g., for repositioning the user interface 434) and a second user interface element 431b indicating that the user interface 434 is shared are displayed in the respective three-dimensional environments 450A / 450B.

[0125] In some examples, when a multi-user communication session is established between a first electronic device 101a and a second electronic device 101b, i.e., the origin of the multi-user communication session, virtual content is presented in a way that maintains the spatial realism between the first electronic device 101a and the second electronic device 101b. In other examples, when a multi-user communication session is established between the first electronic device 101a and the second electronic device 101b, the first electronic device 101a is configured to present virtual content from the second electronic device 101b to a first user 402 of the first electronic device 101a. For example, when a multi-user communication session is established between the first electronic device 101a and the second electronic device 101b, virtual content can be presented at the first electronic device 101a as if the second electronic device 101b is streaming virtual content to the first electronic device 101a while simultaneously presenting virtual content to the second user 404. For example, when a multi-user communication session is established between the first electronic device 101a and the second electronic device 101b, the second electronic device 101b can display virtual content. If the second electronic device 101b shares virtual content with the first electronic device 101a, the first electronic device 101a will optionally display the virtual content in the physical environment 400 at the same corresponding location as the virtual content displayed via the second electronic device 101b. Continuing this example, while the first electronic device 101a and the second electronic device 101b are displaying virtual content, operations resulting from interactions with virtual content from the first user 402 of the first electronic device 101a or the second user 404 of the second electronic device 101b can be displayed at both the first electronic device 101a and the second electronic device 101b, as referenced above. Figure 3 Similarly, describe it.

[0126] In some examples, in response to detecting a selection of user interface element 440a, the above (e.g., reference) Figures 4A to 4GThe location tracking process described between the first electronic device 101a and the second electronic device 101b ceases. For example, in response to detecting a selection of user interface element 440a, the first electronic device 101a and / or the second electronic device 101b may use a different process than described above to begin tracking the other electronic device. For example, a reference location (e.g., the reference origin) different from the shared reference origin 424 may be selected (and / or otherwise agreed upon by the first electronic device 101a and / or the second electronic device 101b) for use in the location tracking of the other electronic device and / or for presenting shared virtual content at the first electronic device 101a and the second electronic device 101b in a multi-user communication session. For example, content used by the first electronic device 101a and the second electronic device 101b to display... Figure 4H The reference origin of the user interface 434 is different from the shared reference origin 424.

[0127] In some examples, when in physical environment 400, the first electronic device 101a associates different locations (e.g., one or more locations) in physical environment 400 with different anchor points (e.g., one or more anchor points) in the coordinate system of physical environment 400. For example, the first electronic device 101a may assign (e.g., create and assign) a first anchor point of the coordinate system of physical environment 400 to a first location in physical environment 400, and assign a second anchor point of the coordinate system of physical environment 400 to a second location in physical environment 400. In some examples, when in physical environment 400... Figure 4H In a multi-user communication session including a first user 402 of a first electronic device 101a and a second user 404 of a second electronic device 101b, the first electronic device 101a uses one or more indicators to mark the anchor points of the coordinate system of the physical environment 400 (e.g., to associate them), as described below.

[0128] In some examples, when in Figure 4H In a multi-user communication session, the first electronic device 101a uses indicators of users of electronic devices that are together with the first electronic device 101a in the multi-user communication session (e.g., juxtaposed with the first electronic device 101a in the physical environment 400) to mark the first anchor point of the coordinate system of the physical environment 400 (e.g., to associate them), such as Figure 4H The character shape 460a is shown in the image. For example, when in... Figure 4H In a multi-user communication session, the first electronic device 101a uses the indicator of the second user 404 to mark the first anchor point of the coordinate system of the physical environment 400. In some examples, if Figure 4HThe multi-user communication session also includes a third user 406 of a third electronic device 101c. The first electronic device 101a then uses the indicators of the second user 404 and the third user 406 to mark a first anchor point in the coordinate system of the physical environment 400. In some examples, the first anchor point belongs to a coordinate system of the physical environment 400 that is private to the first electronic device 101a (e.g., not shared with the second electronic device 101b or the third electronic device 101c). For example, the first anchor point may optionally be one of multiple anchor points included in the SLAM map data of the physical environment 400 determined by the first electronic device 101a (such as the SLAM map data determined by the first electronic device 101a described above) and / or generated using that SLAM map data.

[0129] Similarly, in some examples, when in Figure 4H In a multi-user communication session, the second electronic device 101b uses the indicators of the users of the electronic devices that are together with the second electronic device 101b in the multi-user communication session (e.g., juxtaposed with the second electronic device 101b in the physical environment 400) to mark the second anchor point of the coordinate system of the physical environment 400, such as... Figure 4H The glyph 460b is shown in the text. For example, when in... Figure 4H In a multi-user communication session, the second electronic device 101b uses the indicator of the first user 402 to mark the second anchor point of the coordinate system of the physical environment 400. In some examples, if Figure 4H The multi-user communication session also includes a third user 406 of a third electronic device 101c. The second electronic device 101b then uses the indicators of the first user 402 and the third user 406 to mark a second anchor point in the coordinate system of the physical environment 400. In some examples, the second anchor point belongs to a coordinate system of the physical environment 400 that is private to the second electronic device 101b (e.g., not shared with the first electronic device 101a or the third electronic device 101c). For example, the second anchor point may optionally be one of multiple anchor points included in SLAM map data of the physical environment 400 determined by the second electronic device 101b (such as the SLAM map data determined by the second electronic device 101b described above) and / or generated using that SLAM map data. It should be noted that in some examples, the anchor point (e.g., the first anchor point and the second anchor point) is not visible or displayed by the first electronic device 101a and the second electronic device 101b, but is instead part of the respective coordinate systems of the first electronic device 101a and the second electronic device 101b. For example, the first electronic device 101a may optionally not display the first anchor point, and the second electronic device 101b may optionally not display the second anchor point.

[0130] In some examples, when a first electronic device is in a multi-user communication session with another electronic device and is juxtaposed with another electronic device in a physical environment, the first electronic device 101a uses the indicator of another user of the other electronic device to mark the first anchor point of the coordinate system of the physical environment (e.g., to associate them). For example, if the first electronic device 101a is not in a multi-user communication session with another electronic device juxtaposed with the first electronic device in a physical environment, the first electronic device does not use the indicator of the user of the other electronic device to mark the first anchor point. Similarly, if the first electronic device is in a first multi-user communication session and the other electronic device is not in a first multi-user communication session, and if the first electronic device and the other electronic device are juxtaposed in a physical environment, the first electronic device 101a does not use the indicator of the user of the other electronic device to mark the first anchor point of the coordinate system.

[0131] In some examples, when Figure 4H When a multi-user communication session is interrupted, the first electronic device 101a maintains awareness of the aforementioned first marker anchor point. For example, even when... Figure 4H After the multi-user communication session is terminated, the first electronic device 101a may also optionally maintain access to the indicator marked with the second user 404 (and the indicator marked with the third user 406), provided that... Figure 4H The multi-user communication session includes knowledge of a first anchor point of a first electronic device 101a (which has a second user 404 and a third user 406). In some examples, the first electronic device 101a retains knowledge of the first anchor point (e.g., knowledge of an indicator associated with the first anchor point) for a predetermined amount of time. In some examples, the predetermined amount of time is 3 hours, 1 day, 10 days, 45 days, or another amount of time. In some examples, the predetermined amount of time that the first electronic device 101a retains knowledge of the first anchor point marked with an indicator of the second user 404 is relative to the time since the first electronic device 101a became one of the second users 404 in the multi-user communication session with the second electronic device 101b when the first electronic device 101a was juxtaposed with the second electronic device 101b in the physical environment 400.

[0132] In some examples, the first electronic device 101a and the second electronic device 101b respectively perform a reference operation in response to detecting a second user 404 whose first anchor point is marked with the second electronic device 101b and a first user 402 whose second anchor point is marked with the first electronic device 101a. Figure 4C The illustrated and / or described processes. For example, when the first electronic device 101a is in a multi-user communication session with the second user 404, it may optionally mark the first anchor point with an indicator of the second user 404, such as... Figure 4H As shown. Continuing with this example, in Figure 4HAfter the multi-user communication session stops, if it is determined that the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400 and the first anchor point is still marked with the indicator of the second user 404 of the second electronic device 101b, then the first electronic device 101a may optionally perform a reference together with the second electronic device 101b. Figure 4C The illustrated and / or described process. Continuing with this example, in Figure 4H After the multi-user communication session stops, if it is determined that the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400 and the first anchor point is no longer marked with the indicator of the second user 404 of the second electronic device 101b, then the first electronic device 101a may optionally abandon the reference operation with the second electronic device 101b. Figure 4C The illustrated and / or described processes. Similarly, for example, when the second electronic device 101b is in a multi-user communication session with the first user 402, it may optionally mark the second anchor point with an indicator of the first user 402, such as... Figure 4H As shown. Continuing with this example, in Figure 4H After the multi-user communication session is terminated, if it is determined that the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400 and the second anchor point is still marked with the indicator of the first user 402 of the first electronic device 101a, then the second electronic device 101b may optionally perform a reference together with the first electronic device 101a. Figure 4C The illustrated and / or described process. Continuing with this example, in Figure 4H After the multi-user communication session stops, if it is determined that the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400 and the second anchor point is no longer marked with the indicator of the first user 402 of the first electronic device 101a, then the second electronic device 101b may optionally abandon the reference operation performed with the first electronic device 101a. Figure 4C The illustrated and / or described processes.

[0133] In some examples, the first electronic device 101a updates the indicator of the first marker anchor on a per-user basis. For example, as described above, if Figure 4HThe multi-user communication session also includes a third user 406 of a third electronic device 101c. The first electronic device 101a uses the indicator of the second user 404 and the indicator of the third user 406 to mark a first anchor point in the coordinate system of the physical environment 400. Continuing the example, after the multi-user communication session is stopped, if it is determined that the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400 without the third electronic device 101c, the first electronic device 101a may optionally determine whether the marked anchor point in the coordinate system of the physical environment 400 indicates the second user 404 of the second electronic device 101b (e.g., whether the first marked anchor point indicates the second user 404 of the second electronic device 101b), and optionally may not determine whether the marked anchor point in the coordinate system of the physical environment 400 indicates the third user 406 of the third electronic device 101c. Continuing this example, since the first marker anchor point indicates the second user 404 of the second electronic device 101b, the first electronic device 101a and the second electronic device 101b may optionally perform reference operations in the illustrated order or in an order different from the illustrated order. Figure 4C The operation described. Additionally, in this example, the first electronic device 101a may optionally reset a predetermined amount of time for holding the indicator of the second user 404 of the second electronic device 101b, without resetting the predetermined amount of time for holding the indicator of the third user 406 of the third electronic device 101c (e.g., because it is determined that the first user 402 of the first electronic device 101a and the second user 404 of the second electronic device 101b are again co-located). In some examples, the first electronic device 101a resets the predetermined amount of time for holding the indicator of the second user 404 of the second electronic device 101b, provided that there is a new multi-user communication session including the first user 402 of the first electronic device 101a and the second user 404 of the second electronic device 101b co-located in the physical environment 400.

[0134] As described above, in some examples, in response to determining that the first electronic device 101a and the second electronic device 101b are juxtaposed in a physical environment, the first electronic device 101a and the second electronic device 101b perform reference... Figure 4C The described operation. Additionally, as described above, in some examples, the first electronic device 101a and the second electronic device 101b are... Figure 4A In the physical environment 400, they are juxtaposed because the first electronic device 101a and the second electronic device 101b satisfy the reference... Figure 4A The determination criteria described. For example, as described above, the determination of the first electronic device 101a and the second electronic device 101b may optionally be based on the following: Figure 4AThe following are considered in the juxtaposition within physical environment 400: the distance between the first electronic device 101a and the second electronic device 101b; communication between the first electronic device 101a and the second electronic device 101b; the strength of wireless signals transmitted and detected between the first electronic device 101a and the second electronic device 101b; the first electronic device 101a and the second electronic device 101b being connected to the same network (e.g., a wireless network) in physical environment 400; visual detection of the first electronic device 101a and the second electronic device 101b in physical environment 400; the user of the other electronic device being in the contact list of the electronic device; and / or because the first electronic device 101a and the second electronic device 101b are in the same physical room.

[0135] Additionally, in some examples, the first electronic device 101a and the second electronic device 101b perform reference... Figure 4C The illustrated and / or described process is because the first electronic device 101a and the second electronic device 101b are parties to a multi-user communication session previously established when the first electronic device 101a and the second electronic device 101b were juxtaposed in physical environment 400. For example, if the first electronic device 101a and the second electronic device 101b are parties to a multi-user communication session previously established when the first electronic device 101a and the second electronic device 101b were juxtaposed in physical environment 400, then the first electronic device 101a and the second electronic device 101b may optionally initiate a reference... Figure 4C The illustrated and / or described process. Continuing this example, if the first electronic device 101a and the second electronic device 101b are not parties to a previously established multi-user communication session when the first electronic device 101a and the second electronic device 101b were juxtaposed in physical environment 400, then the first electronic device 101a and the second electronic device 101b abandon initiating a reference... Figure 4C The illustrated and / or described processes. For example, if a first electronic device 101a and a second electronic device 101b are currently juxtaposed in physical environment 400, and if the first electronic device 101a and the second electronic device 101b are not parties to a multi-user communication session previously established when the first electronic device 101a and the second electronic device 101b were juxtaposed in physical environment 400, then the first electronic device 101a may perform the implementation reference. Figure 4C The first process of executing the illustrated and / or described process. In some examples, the first process includes presenting a method for implementing the reference. Figure 4C Instructions for the execution of the illustrated and / or described processes. In some examples, these instructions include providing (e.g., detecting) a corresponding input at the second electronic device 101b to establish a reference between the first electronic device 101a and the second electronic device 101b. Figure 4CInstructions for the execution of the illustrated and / or described processes. In some examples, these instructions include instructions to provide a code (e.g., a password or PIN) to the second electronic device 101b. In some examples, if the first electronic device 101a detects an instruction to execute a corresponding input from the second electronic device 101b, the first electronic device 101a enables (e.g., initiates) the execution of a reference process together with the second electronic device 101b. Figure 4C The illustrated and / or described processes.

[0136] In some examples, the first electronic device 101a and the second electronic device 101b perform reference Figure 4C The illustrated and / or described process is as follows: the first electronic device 101a and the second electronic device 101b are the last party in a multi-user communication session previously established when the first electronic device 101a and the second electronic device 101b were juxtaposed in the physical environment 400 for a threshold time length (e.g., 3 hours, 1 day, 10 days, 45 days, or another time amount). For example, if the first electronic device 101a and the second electronic device 101b are the last party in a multi-user communication session previously established when the first electronic device 101a and the second electronic device 101b were juxtaposed in the physical environment 400 for a threshold time length, then the first electronic device 101a and the second electronic device 101b may optionally initiate a reference... Figure 4C The illustrated and / or described process. Continuing this example, if the first electronic device 101a and the second electronic device 101b are the last parties to a multi-user communication session previously established when the first electronic device 101a and the second electronic device 101b were concatenated in physical environment 400 for a period exceeding a threshold time, then the first electronic device 101a and the second electronic device 101b abandon initiating the reference. Figure 4C The illustrated and / or described processes.

[0137] It should be understood that the examples shown and described herein are merely exemplary and additional and / or alternative elements may be provided within a three-dimensional environment to facilitate sharing content in a multi-user communication session including co-located users. It should be understood that the appearance, shape, form, and size of each of the various user interface elements and objects shown and described herein are exemplary and alternative appearances, shapes, forms, and / or sizes may be provided. For example, virtual objects representing user interfaces (user interface element 440a and user interface element 440b) may be provided in alternative shapes (such as circular shapes, triangular shapes, etc.) other than rectangular shapes. In some examples, the various selectable functional representations described herein (e.g., user interface element 440a and user interface element 440b) may be verbally selected via user verbal commands (e.g., verbal commands such as “select an option” or “select a virtual object”). Additionally or alternatively, in some examples, the various options, user interface elements, control elements, etc., described herein may be selected and / or manipulated via user input received through one or more separate input devices in communication with an electronic device. For example, selection input can be received via a physical input device (such as a mouse, touchpad, keyboard, etc.) that communicates with an electronic device.

[0138] Figure 5 A flowchart illustrating a method 500 for displaying user interface elements according to some examples of this disclosure is shown, the user interface elements being selectable to establish a multi-user communication session between a first electronic device and a second electronic device. One or more examples of method 500 are referenced above. Figures 4A to 4H One or more of the methods described below are illustrated and / or described. It should be understood that method 500 is an example and more, fewer, or different operations may be performed in the same or different order. Additionally, the operations in method 500 described below may optionally be performed by running an information processing device such as a general-purpose processor (e.g., as per [reference to...]). Figures 2A to 2C (as described) or one or more functional modules in a dedicated chip and / or through Figures 2A to 2C It is implemented using other components.

[0139] Therefore, based on the above description, some examples of this disclosure relate to a method (e.g., Figure 5Method 500 (illustrated). Method 500 can be performed at a first electronic device communicating with one or more first displays and one or more first input devices, wherein the first electronic device and a second electronic device are juxtaposed in a physical environment, wherein the first electronic device has a first pose (e.g., positioning and / or orientation) relative to a first reference origin in the physical environment, and wherein the second electronic device has a second pose (e.g., positioning and / or orientation) relative to a second reference origin in the physical environment. Additionally, method 500 can be performed before a multi-user communication session is established between the first electronic device and the second electronic device. Method 500 includes: determining (502) a shared reference origin in a physical environment based on first map data determined by a first electronic device and second map data determined by a second electronic device; after determining the shared reference origin, receiving (504) first information from the second electronic device, the first information including a second pose (e.g., positioning and / or orientation) of the second electronic device relative to the second reference origin of the second electronic device and an offset of the second reference origin of the second electronic device relative to the shared reference origin; and displaying (506) a user interface element at a location based on the first information via one or more first displays, the user interface element being selectable to establish a multi-user communication session between the first electronic device and the second electronic device.

[0140] Additionally or alternatively, in some examples, the first electronic device is juxtaposed with the second electronic device based on determining that the second electronic device is within the field of view of the first electronic device in the physical environment.

[0141] Additionally or alternatively, in some examples, the first electronic device is juxtaposed with the second electronic device based on determining that the second electronic device is within a signal-based distance range of the first electronic device in the physical environment.

[0142] Additionally or alternatively, in some examples, the first electronic device is juxtaposed with the second electronic device based on determining that the second electronic device is within a threshold distance of the first electronic device.

[0143] Additionally or alternatively, in some examples, the first electronic device is juxtaposed with the second electronic device based on the user of the second electronic device being in the contact list of the first electronic device.

[0144] Additionally or alternatively, in some examples, the determination of the shared reference origin is performed based on at least one of the following (e.g., one or more or all of the following): a first determination of the second electronic device within a signal-based distance range of the first electronic device in a physical environment; a second determination of the second electronic device within a threshold distance of the first electronic device; and a third determination of the user of the second electronic device in the contact list of the first electronic device.

[0145] Additionally or alternatively, in some examples, the shared reference origin differs from the first reference origin of the first electronic device and the second reference origin of the second electronic device.

[0146] Additionally or alternatively, in some examples, the first reference origin of the first electronic device is different from the second reference origin of the second electronic device.

[0147] Additionally or alternatively, in some examples, the location of the displayed user interface element corresponds to a corresponding location in the physical environment, which, from the viewpoint of the first electronic device, is located between a first location of the first electronic device in the physical environment and a second location of the second electronic device in the physical environment.

[0148] Additionally or alternatively, in some examples, the distance between the corresponding position and the first position is a first distance, the distance between the corresponding position and the second position is a second distance, and the first distance is less than the second distance.

[0149] Additionally or alternatively, in some examples, the distance between the corresponding position and the first position is a first distance, the distance between the corresponding position and the second position is a second distance, and the second distance is less than the first distance.

[0150] Additionally or alternatively, in some examples, the second pose (e.g., positioning and / or orientation) of the second electronic device relative to a second reference origin of the second electronic device included in the first information is a first corresponding pose (e.g., positioning and / or orientation) of the second electronic device at a first time, and method 500 includes: after receiving the first information from the second electronic device and after displaying a user interface element at a location based on the first information, receiving updated first information from the second electronic device, the updated first information including the pose (e.g., positioning and / or orientation) of the second electronic device at a second time after the first time, wherein the pose of the second electronic device at the second time is a second corresponding pose of the second electronic device relative to a second reference origin of the second electronic device that differs from the first corresponding pose and an offset of the second reference origin of the second electronic device relative to a shared reference origin; and displaying the user interface element at a corresponding location based on the updated first information via one or more first displays.

[0151] Additionally or alternatively, in some examples, the shared reference origin is determined based on an indication that a second electronic device has selected the shared reference origin as the shared reference origin.

[0152] Additionally or alternatively, in some examples, determining a shared reference origin includes identifying multiple potential shared reference origins and selecting a shared reference origin from among these multiple potential shared reference origins.

[0153] Additionally or alternatively, in some examples, method 500 is part of a first location detection process for determining the location of a second electronic device and not part of a second location detection process for determining the location of a second electronic device, and method 500 includes detecting input corresponding to a selection of a user interface element via one or more input devices, and in response to detecting input corresponding to a selection of a user interface element, performing a second location detection process for determining the location of the second electronic device without performing the first location detection process for determining the location of the second electronic device. For example, the first location detection process may optionally include establishing a shared spatial coordinate system of the physical environment for determining the location of the second electronic device (e.g., to display an optional option to request initiating a multi-user communication session with the second electronic device) before establishing a multi-user communication session between the first and second electronic devices. Continuing with this example, the second location detection process may optionally include establishing a shared spatial coordinate system of the physical environment 400 for the multi-user communication session between the first and second electronic devices, such as for displaying virtual content for the shared multi-user communication session between the first and second electronic devices.

[0154] Additionally or alternatively, in some examples, the second electronic device communicates with one or more second displays, and method 500 includes: detecting input corresponding to a selection of the user interface element via one or more first input devices when a user interface element is displayed at a location based on first information; establishing a multi-user communication session between the first electronic device and the second electronic device in response to detecting the input corresponding to the selection of the user interface element; and displaying shared virtual content at a corresponding location in the physical environment via one or more first displays while the multi-user communication session between the first electronic device and the second electronic device is active, wherein the shared virtual content is also displayed at a corresponding location in the physical environment via one or more second displays.

[0155] Additionally or alternatively, in some examples, the first electronic device is also juxtaposed with the third electronic device in a physical environment, wherein the third electronic device has a third pose (e.g., positioning and / or orientation) relative to a third reference origin in the physical environment, and method 500 includes: when a multi-user communication session has not yet been established between the first electronic device and the third electronic device, determining a corresponding shared reference origin in the physical environment based on first map data detected by the first electronic device and third map data received from the third electronic device; after determining the corresponding shared reference origin, receiving corresponding information from the third electronic device, the corresponding information including the third pose (e.g., positioning and / or orientation) of the third electronic device relative to the third reference origin of the third electronic device and a corresponding offset of the third reference origin of the third electronic device relative to the corresponding shared reference origin; and displaying a corresponding user interface element at a corresponding location based on the corresponding information via one or more first displays, the corresponding user interface element being selectable to establish a multi-user communication session between the first electronic device and the third electronic device.

[0156] Additionally or alternatively, in some examples, method 500 includes: when a user interface element is displayed, detecting a selection of the user interface element via one or more first input devices, such as a selection of a user interface element. Figure 4G The selection of user interface element 440a in the process; and in response to detecting the selection of the user interface element, establishing a multi-user communication session between the first electronic device and the second electronic device, such as Figure 4H A multi-user communication session between a first electronic device 101a and a second electronic device 101b, wherein the multi-user communication session between the first electronic device and the second electronic device is a first multi-user communication session between the first electronic device and the second electronic device. Additionally or alternatively, in some examples, method 500 includes: when in the first multi-user communication session with the second electronic device, associating a first anchor point of the coordinate system of the physical environment of the first electronic device with an indicator of the second user of the second electronic device, such as... Figure 4H The glyph 460a is shown in the text; and after associating a first anchor point of the coordinate system of the physical environment of the first electronic device with an indicator of the second user of the second electronic device, the first multi-user communication session is stopped. Additionally or alternatively, in some examples, method 500 includes performing a first process, such as referencing..., after stopping the first multi-user communication session, and before establishing a second multi-user communication session between the first electronic device and the second electronic device, based on determining that the first electronic device and the second electronic device are juxtaposed in the physical environment after stopping the first multi-user communication session, ... Figure 4CThe illustrated and / or described process between the first electronic device 101a and the second electronic device 101b. Additionally or alternatively, in some examples, method 500 includes, after stopping the first multi-user communication session, abandoning the execution of a first process, such as abandoning the execution of a reference process between the first electronic device 101a and the second electronic device 101b, based on the determination that the first electronic device and the second electronic device are not physically juxtaposed after stopping the first multi-user communication session. Figure 4C The illustrated and / or described processes. Additionally or alternatively, in some examples, determining the juxtaposition of the first electronic device and the second electronic device in the physical environment after the first multi-user communication session is terminated includes one or more of the following operations: determining the contact list of the second user's application on the first electronic device after the first multi-user communication session is terminated; determining the second electronic device within a signal-based distance range of the first electronic device in the physical environment after the first multi-user communication session is terminated; determining that the amount of time elapsed since the termination of the first multi-user communication session is less than a threshold time amount; and determining that a first anchor point in the coordinate system of the physical environment of the first electronic device remains associated with the indicator of the second user of the second electronic device after the termination of the first multi-user communication session. Additionally or alternatively, in some examples, determining that the first anchor point in the coordinate system of the physical environment of the first electronic device still bears the indicator of the second user of the second electronic device after the termination of the first multi-user communication session includes determining that the amount of time elapsed since the termination of the first multi-user communication session is less than a threshold time amount. Additionally or alternatively, in some examples, determining that the first anchor point of the coordinate system of the physical environment of the first electronic device is still not marked with an indicator of the second user of the second electronic device after the first multi-user communication session is terminated includes determining that the amount of time elapsed since the termination of the first multi-user communication session is not less than a threshold amount of time. Additionally or alternatively, in some examples, after the termination of the first multi-user communication session, the first electronic device and the second electronic device are juxtaposed in the physical environment, the first electronic device having a first corresponding pose relative to a first corresponding reference origin in the physical environment, and the second electronic device having a second corresponding pose relative to a second corresponding reference origin in the physical environment. Additionally or alternatively, in some examples, the first process includes determining a corresponding shared reference origin in the physical environment, such as a reference map, based on first corresponding map data determined by the first electronic device and second corresponding map data determined by the second electronic device, before establishing a second multi-user communication session between the first and second electronic devices. Figure 4CThe first process is described in box 422. Additionally or alternatively, in some examples, the first process includes, after determining a corresponding shared reference origin, receiving first corresponding information from the second electronic device, the first corresponding information including a second corresponding attitude of the second electronic device relative to a second corresponding reference origin of the second electronic device and a corresponding offset of the second corresponding reference origin of the second electronic device relative to the corresponding shared reference origin, such as a reference... Figure 4C The first process is described in box 430. Additionally or alternatively, in some examples, the first process includes displaying a user interface element at a corresponding location based on first response information via one or more first displays, the user interface element being selectable to establish a second multi-user communication session between the first electronic device and the second electronic device, such as... Figure 4G User interface element 440a. Additionally or alternatively, in some examples, method 500 may also include one or more operations described with reference to method 600.

[0157] Some examples of this disclosure relate to an electronic device comprising: one or more processors; a memory; and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described above.

[0158] Some examples of this disclosure relate to a non-transitory computer-readable storage medium that stores one or more programs, the one or more programs including instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform any of the methods described above.

[0159] Some examples of this disclosure relate to an electronic device that includes one or more processors, a memory, and components for performing any of the methods described above.

[0160] Some examples of this disclosure relate to an information processing apparatus used in an electronic device, the information processing apparatus including components for performing any of the methods described above.

[0161] Figure 6 A flowchart illustrating a method 600 for performing an operation based on specific information, according to some examples of this disclosure, is shown. One or more examples of method 600 are referenced above. Figures 4A to 4H One or more of the methods described herein are illustrated and / or described. It should be understood that method 600 is an example and more, fewer, or different operations may be performed in the same or different order. Additionally, the operations in method 600 described below may optionally be performed by running an information processing device such as a general-purpose processor (e.g., as per [reference to...]). Figures 2A to 2C(as described) or one or more functional modules in a dedicated chip and / or through Figures 2A to 2C It is implemented using other components.

[0162] Therefore, based on the above description, some examples of this disclosure relate to a method (e.g., Figure 6 Method 600 (illustrated). Method 600 can be performed at a first electronic device communicating with one or more first displays and one or more first input devices, wherein the first electronic device and a second electronic device are juxtaposed in a physical environment, wherein the first electronic device has a first pose (e.g., positioning and / or orientation) relative to a first reference origin in the physical environment, and wherein the second electronic device has a second pose (e.g., positioning and / or orientation) relative to a second reference origin in the physical environment. Additionally, method 600 can be performed before a multi-user communication session is established between the first and second electronic devices. Method 600 includes: determining (602) a shared reference origin in the physical environment based on first map data determined by the first electronic device and second map data determined by the second electronic device; after determining the shared reference origin, receiving (604) first information from the second electronic device, the first information including a second pose (e.g., positioning and / or orientation) of the second electronic device relative to the second reference origin and an offset of the second reference origin relative to the shared reference origin; and performing (606) operations based on the first information. Additionally or alternatively, in some examples, method 600 may also include one or more operations described with reference to method 500. For example, at block 606 of method 600, the following can be performed: Figure 5 The operation of box 506 and / or different operations can be performed.

[0163] Some examples of this disclosure relate to an electronic device including: one or more processors; a memory; and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described above. Some examples of this disclosure relate to a non-transitory computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by one or more processors of the electronic device, cause the electronic device to perform any of the methods described above. Some examples of this disclosure relate to an electronic device including one or more processors, a memory, and components for performing any of the methods described above. Some examples of this disclosure relate to an information processing apparatus used in an electronic device, the information processing apparatus including components for performing any of the methods described above.

[0164] It should be noted that Figure 3 The multi-user communication session between the user of the first electronic device 360 ​​and the user of the second electronic device 370 is a first-type multi-user communication session. (See reference...) Figure 3 As illustrated and / or described, in some examples, in a first type of multi-user communication session, the three-dimensional environment presented at the corresponding electronic device in the first type of multi-user communication session includes visual representations (e.g., two-dimensional and / or three-dimensional representations, such as avatars) of other users in the first type of multi-user communication session. For example, in Figure 3 In this context, the first electronic device 360 ​​presents the user's avatar 315 of the second electronic device 370. (See reference...) Figure 3 As illustrated and / or described, in some examples, in a first type of multi-user communication session, the respective electronic device communicates via its audio input device (e.g., a microphone, such as...). Figure 2A Audio detection is achieved using one or more microphones 213A. For example, the first electronic device 360 ​​performs audio detection via its audio input device. Alternatively, the first electronic device 360 ​​may send the detected audio to the second electronic device 370. Additionally, as referenced... Figure 3 As illustrated and / or described, in some examples, in a first type of multi-user communication session, a respective electronic device renders audio associated with other users in the first type of multi-user communication session via its audio output device (e.g., internal and / or external speakers, such as speaker 216A). For example, a first electronic device 360 ​​renders audio captured by a second electronic device 370. Additionally, in some examples, the shared content of the multi-user communication session is rendered by the respective electronic devices in the first type of multi-user communication session at different anchor points (e.g., the origin) relative to the coordinate system of the physical environment in which the respective electronic device is located.

[0165] It should be noted that Figure 4H The multi-user communication session between the first user 402 of the first electronic device 101a and the second user 404 of the second electronic device 101b is a second-type multi-user communication session, distinct from the first type. In some examples, the users of the electronic devices in the second-type multi-user communication session are juxtaposed in a physical environment and presented relative to each other via their physical bodies rather than via avatars, such as references. Figure 4HAs illustrated and / or described. Additionally, in some examples, electronic devices associated with users in a second-type multi-user communication session do not present the audio of users in the same second-type multi-user communication session (e.g., their presentation is disabled). Additionally, in some examples, electronic devices of users in a second-type multi-user communication session have established a shared spatial coordinate system of their physical environment to align the presentation of virtual content in the multi-user communication session (e.g., to have spatial realism in the physical environment (e.g., a consistent spatial arrangement between users (or their representations) and shared virtual objects)). For example, two electronic devices in a second-type multi-user communication session present the shared content of the multi-user communication session at the same corresponding location in their physical environment. In some examples, users in a second-type multi-user communication session are juxtaposed with each other in the physical environment. In some examples, the second-type multi-user communication session does not include users of electronic devices not juxtaposed in the physical environment. In some examples, a first electronic device may initiate a hybrid multi-user communication session that is both a first-type and a second-type multi-user communication session. For example, when in a first type of multi-user communication session with a group of users, the first electronic device may be in a second type of multi-user communication session with a subgroup of those users.

[0166] A first user of a first electronic device may wish to initiate a multi-user communication session. For example, the first user of the first electronic device may wish to initiate a multi-user communication session to share content (e.g., an application's user interface, such as user interface 434) with a second user of a second electronic device. In some examples, the first electronic device may detect user input requesting to initiate a multi-user communication session with one or more users and respond to that user input by sending a request to the electronic devices associated with those users. In some examples, after sending the request, the first electronic device detects an indication that one or more users have accepted the request from one or more electronic devices of one or more users. In some examples, in response to detecting the indication, based on the type of the corresponding electronic device from which the corresponding user accepted the request to join the multi-user communication session, the first electronic device initiates a first-type or second-type multi-user communication session with the corresponding user of the corresponding electronic device among the one or more users of the electronic device. For example, if the corresponding user of the corresponding electronic device is juxtaposed with the first user of the first electronic device in a physical environment, if the corresponding user of the corresponding electronic device accepts the request to join the multi-user communication session with the first user of the first electronic device via the corresponding electronic device, and if the corresponding electronic device is a first-type electronic device, then the first electronic device initiates a first-type multi-user communication session with the corresponding user of the corresponding electronic device. Continuing the example, if the corresponding user of the corresponding electronic device is juxtaposed with the first user of the first electronic device in a physical environment, if the corresponding user of the corresponding electronic device accepts a request to join a multi-user communication session with the first user of the first electronic device via the corresponding electronic device, and if the corresponding electronic device is a second type of electronic device different from the first type of electronic device, then the first electronic device initiates a second type of multi-user communication session with the corresponding user of the corresponding electronic device, instead of initiating a first type of multi-user communication session with the corresponding user of the corresponding electronic device.

[0167] Figures 7A to 7E-1 Generally, examples according to some examples of this disclosure are illustrated where a first electronic device initiates a multi-user communication session of a different type with a second user of a second electronic device based on the type of electronic device from which a second user of a second electronic device accepts a request to join a multi-user communication session with the first user of the first electronic device.

[0168] Figure 7A An example of a first electronic device 101a, according to some examples of this disclosure, detects an input to initiate a multi-user communication session with a second user 404 of a second electronic device 101b and a third user 406 of a third electronic device 101c, wherein the second user 404 is in physical environment 400 together with the first user 402, and wherein the third user 406 is not in physical environment 400 together with the first user 402.

[0169] exist Figure 7A In the first electronic device 101a, the first electronic device 101a (e.g., a wearable electronic device) worn by the first user 402 presents a first three-dimensional environment 450A from the viewpoint of the first electronic device 101a (such as the viewpoint of the first electronic device 101a illustrated in top view 716a). Figure 7A , Figures 7C to 7E-1 as well as Figures 9A to 9C The top view 716a generally shows the relative positioning of objects in the first three-dimensional environment 450A in the horizontal and depth dimensions and shows the viewpoint of the first electronic device 101a, as indicated by the arrows extending from the first electronic device 101a in the corresponding figures. Similarly, in Figure 7A and Figures 7C to 7E-1 In the middle, the display 120a has respectively corresponding to Figure 7A and Figures 7C to 7E-1 The field of view of the content shown on the display 120a (e.g., the field of view captured by and / or visible to the first user 402 via the display 120a by the external image sensors 114b and 114c of the first electronic device 101a (e.g., external image sensors 114b-i and 114c-i of the first electronic device 101a)). For example, in Figure 7A In the view 716a, the viewing boundary of the first user 402 via the first electronic device 101a is given by the viewing boundary 711. Because the display 120a is optionally a head-mounted device, the field of view of the display 120a may be the same as or similar to the field of view of the eyes of the first user 402. For example, the view of the first three-dimensional environment 450A depicts the content visible to the first user 402 when the viewpoint of the first electronic device 101a is positioned as shown in the top view 716a of the first three-dimensional environment 450A and the first electronic device 101a is optionally oriented in the direction indicated by the directional arrow emanating from the first electronic device 101a in the top view 716a (via the display 120a).

[0170] exist Figure 7A In the second electronic device 101b, the second electronic device 101b (e.g., a wearable electronic device) worn by the second user 404 presents a second three-dimensional environment 450B from the viewpoint of the second electronic device 101b (such as the viewpoint of the second electronic device 101b illustrated in top view 716b). Figures 7A to 7E-1 The top view 716b generally illustrates the relative positioning of objects in the second three-dimensional environment 450B in the horizontal and depth dimensions and shows the viewpoint of the second electronic device 101b, as indicated by the arrows extending from the second electronic device 101b in the corresponding figures. Similarly, in Figures 7A to 7E-1 In the middle, the display 120b has respectively corresponding to Figures 7A to 7E-1The field of view of the content shown on the display 120b (e.g., the field of view captured by and / or visible to the second user 404 via the display 120b by the external image sensors 114b and 114c of the second electronic device 101b (e.g., external image sensors 114b-ii and 114c-ii of the second electronic device 101b)). For example, in Figure 7A In the top view 716b, the viewing boundary of the second user 404 via the second electronic device 101b is given by the viewing boundary 713. Because the display 120b is optionally a head-mounted device, the field of view of the display 120b may optionally be the same as or similar to the field of view of the second user 404's eyes. For example, the view of the second three-dimensional environment 450B depicts what is visible to the second user 404 when the viewpoint of the second electronic device 101b is positioned as shown in the top view 716b of the second three-dimensional environment 450B and the second electronic device 101b is optionally oriented in the direction indicated by the directional arrow emanating from the second electronic device 101b in the top view 716b (via the display 120b).

[0171] exist Figure 7B In the third electronic device 101c, the third user 406 wears the third electronic device 101c (e.g., a wearable electronic device) and presents a third three-dimensional environment 450C from the viewpoint of the third electronic device 101c (such as the viewpoint of the third electronic device 101c illustrated in top view 718). Figures 7A to 7E-1 Top view 718 and Figures 9D to 9F The top view 716c generally illustrates the relative positioning of objects in the third three-dimensional environment 450C in the horizontal and depth dimensions and shows the viewpoint of the third electronic device 101c, as indicated by the arrows extending from the third electronic device 101c in the corresponding figures. Similarly, in Figure 7B and Figures 9D to 9F In this context, the display 120c has a field of view corresponding to the content shown on the display 120c in the corresponding figures (e.g., a field of view captured by and / or visible to the third user 406 via the display 120c by external image sensors 114b and 114c of the third electronic device 101c (e.g., external image sensors 114b-iii and 114c-iii of the third electronic device 101c)). For example, in Figure 7B and Figures 9D to 9F In the accompanying drawings, the viewing boundary of the third user 406 via the third electronic device 101c is given by the viewing boundary 715 in the top views 718 and 716c. Because the display 120c is optionally a head-mounted device, the field of view of the display 120c may be the same as or similar to the field of view of the third user 406. For example, in Figure 7BIn the view of the third three-dimensional environment 450C, the content visible to the third user 406 is shown when the viewpoint of the third electronic device 101c is positioned as shown in the top view 718 of the third three-dimensional environment 450C and the third electronic device 101c is optionally oriented in the direction indicated by the directional arrow emanating from the third electronic device 101c in the top view 718 (via display 120c).

[0172] exist Figure 7A In this context, the second user 404 of the second electronic device 101b is also a user of the second corresponding electronic device 101b-1. For example, in Figure 7A In this scenario, a second user 404 logs into the second electronic device 101b and the second corresponding electronic device 101b-1. For example, the operating systems of the second electronic device 101b and the second corresponding electronic device 101b-1 are associated with the same user account (e.g., the same username and / or password) because the second user 404 logs into the second electronic device 101b and the second corresponding electronic device 101b-1 using the same username and password. Figure 7A In this context, the second corresponding electronic device 101b-1 is a mobile phone (e.g., a handheld electronic device or a wearable electronic device); however, in some examples, the second corresponding electronic device 101b-1 is alternatively a tablet computer, a computer, a laptop computer, a watch, or a device connected to a mobile phone. Figure 7A The second electronic device 101b is of a different type than another type of electronic device. In some examples, the second corresponding electronic device 101b-1 includes one or more features described in reference electronic device 160 / 260. In some examples, the second electronic device 101b is a head-mounted display system.

[0173] exist Figure 7A In this scenario, a second user 404 wears and holds (e.g., in the hand 404a of the second user 404) a second corresponding electronic device 101b-1. In some examples, the second user 404, while wearing the second electronic device 101b, interacts with... Figure 7A The second corresponding electronic device 101b-1 interacts with the second electronic device 101b. In some examples, the second user 404 does not interact with the second electronic device 101b while wearing it. Figure 7A The second corresponding electronic device 101b-1 interacts with it.

[0174] exist Figure 7A In this context, the first user 402 of the first electronic device 101a and the second user 404 of the second electronic device 101b are determined to be juxtaposed in the physical environment 400, as described in the reference above. Figure 4AThe first electronic device 101a and the second electronic device 101b are juxtaposed in physical environment 400 as described. In some examples, if the second user 404 is only a user of the corresponding electronic device 101b-1 (e.g., if the second electronic device 101b is not in physical environment 400 and / or if the second electronic device 101b is not worn or used by the second user 404), then the first user 402 and the second user 404 will not be determined to be juxtaposed in physical environment 400. In some examples, if the second user 404 is only a user of the second electronic device 101b, then the first user 402 and the second user 404 will be determined to be juxtaposed in physical environment 400. In some examples, if the second user 404 is a user of both the second electronic device 101b and the second corresponding electronic device 101b-1, then the first user 402 and the second user 404 will be determined to be juxtaposed in physical environment 400. Figure 7A In the third electronic device 101c, the first user 402 and the third user 406 are in different physical environments. Figure 7A In this context, the first user 402 and the second user 404 are both in physical environment 400, and the third user 406 is in physical environment 400a. Figure 7A In this context, a first user 402 of a first electronic device 101a wishes to initiate a first type of multi-user communication session with a second user 404 of a second electronic device 101b and a third user 406 of a third electronic device 101c.

[0175] exist Figure 7A In this configuration, the first electronic device 101a displays option 702, which can be selected to send requests to the second user 404 and the third user 406 to join a multi-user communication session including the first user 402. For example, option 702 can be selected to cause the first electronic device 101a to send a first request to one or more electronic devices associated with the second user 404, instructing the first user 402 to request the second user 404 to join the multi-user communication session with the first user 402, and a second request to one or more electronic devices associated with the third user 406, instructing the first user 402 to request the third user 406 to join the multi-user communication session with the first user 402. Figure 7A In this process, the first electronic device 101a detects user input pointing to option 702. For example, in Figure 7A In the process, user input includes the first user 402's attention 704a (e.g., gaze) when the first user 402's hand 402a performs an air pinch gesture. In response to detection... Figure 7A In response to user input, the first electronic device 101a sends a request to the electronic devices associated with the second user 404 and the third user 406, such as... Figure 7B As shown.

[0176] In some examples, the first electronic device 101a sends different types of requests to the electronic devices associated with the second user 404 and the third user 406 based on whether the corresponding user is co-located with the first user 402 in the physical environment 400. For example, if when a detection Figure 7A During user input, the second user 404 and the first user 402 are juxtaposed in the physical environment 400. Figure 7A In the case illustrated herein, the first electronic device 101a responds to Figure 7A The input in [the context] sends a first-type request to one or more electronic devices associated with the second user 404. Alternatively, continuing the example, if when [the input] is detected... Figure 7A If, during user input, the second user 404 is not co-located with the first user 402 in the physical environment 400, then the first electronic device 101a responds to... Figure 7A The input in the request will be used to send a second type of request, different from the first type of request, to one or more electronic devices associated with the second user 404. Similarly, if when the input is detected... Figure 7A When a third user 406 and a first user 402 are co-located in physical environment 400 during user input, the first electronic device 101a responds to... Figure 7A The input in the code sends a first-type request to one or more electronic devices associated with the third user 406. Alternatively, continuing the example, if when a first-type request is detected... Figure 7A When a user inputs data, the third user 406 is not co-located with the first user 402 in the physical environment 400. This is... Figure 7A In the case illustrated herein, the first electronic device 101a responds to Figure 7A The input in the request is used to send a second type of request, different from the first type of request, to one or more electronic devices associated with the third user 406.

[0177] In some examples, sending a first-type request enables the receiver to join a first-type or second-type multi-user communication session with the first user 402 based on the electronic device used by the receiver to join. In some examples, sending a second-type request enables the receiver only to join a first-type multi-user communication session with the first user 402, without enabling the receiver to join a second-type multi-user communication session, regardless of the electronic device used by the receiver to join.

[0178] For example, as described above, in Figure 7A In this context, the first user 402 and the second user 404 are juxtaposed in the physical environment 400. Therefore, when detected... Figure 7ADuring user input, first user 402 and second user 404 are juxtaposed in physical environment 400. Therefore, first electronic device 101a sends a first type request to initiate a multi-user communication session with second user 404, wherein the first type request enables second user 404 to join a first type or second type multi-user communication session with first user 402 based on the electronic device used by second user 404 to accept the request from first user 402. Continuing this example, as described above, in Figure 7A In this context, the first user 402 and the third user 406 are not co-located in physical environment 400. Therefore, when detected... Figure 7A During user input, the first user 402 and the third user 406 are not co-located in the physical environment 400. Therefore, the first electronic device 101a sends a second type of request to initiate a multi-user communication session with the third user 406. This second type of request only enables the third user 406 to join the first type of multi-user communication session with the first user 402, but does not enable the third user 406 to join the second type of multi-user communication session with the first user 402, regardless of the electronic device used by the third user 406 to join.

[0179] Figure 7B Examples illustrate how an electronic device associated with a second user 404 detects and / or responds to a first type of request from a first electronic device 101a to initiate a multi-user communication session with the second user 404 (e.g., in response to...). Figure 7A The request sent from the first electronic device 101a in response to user input, and the electronic device associated with the third user 406 detects and / or responds to a second type of request for initiating a multi-user communication session with the third user 406 (e.g., in response to...). Figure 7A (A request sent from the first electronic device 101a in response to user input). Specifically, as described above, in response to detecting... Figure 7A The user input corresponds to a request to initiate a multi-user communication session including first user 402, second user 404, and third user 406. The first electronic device 101a sends a request to the electronic device associated with second user 404 and third user 406 to join the multi-user communication session with first user 402.

[0180] like Figure 7B As shown, in response to detecting a request from a second user 404 to join a multi-user communication session including a first user 402, the second electronic device 101b presents a user interface element 712a (e.g., a visual notification) and an audio notification 712b. Figure 7BIn this context, user interface element 712a can be selected to accept a request from first user 402 (“Bella”) to join a multi-user communication session involving first user 402. In some examples, if the request to join the multi-user communication session with first user 402 includes content to be shared by first user 402 in the multi-user communication session, then user interface element 712a will further visually indicate that first user 402 is requesting a specific application to share with second user 404 in the multi-user communication session. Additionally, in Figure 7B In response to detecting a request from the second user 404 to join a multi-user communication session including the first user 402, the second electronic device 101b presents an audio notification 712b (e.g., a ringtone or ringing) instructing the first user 402 to request the second user 404 to join the multi-user communication session with the first user 402.

[0181] In addition, such as Figure 7B As shown, in response to detecting a request from the second user 404 to join a multi-user communication session including the first user 402, the second corresponding electronic device 101b-1 presents a notification (e.g., a visual notification) 714a instructing the first user 402 to request the second user 404 to join the multi-user communication session with the first user 402. It should be noted that in Figure 7B In this context, the second corresponding electronic device 101b-1 includes a display and an audio output device, and the second corresponding electronic device 101b-1 can present notifications via the display and / or audio output device; however, in Figure 7BIn this process, the second corresponding electronic device 101b-1 presents a notification (e.g., a visual notification) 714a via a display, instructing the first user 402 to request the second user 404 to join the multi-user communication session with the first user 402, without presenting an audio notification instructing the first user 402 to request the second user 404 to join the multi-user communication session with the first user 402. Optionally, the first user 402 is guided to provide user input for accepting the request via the second electronic device 101b rather than via the second corresponding electronic device 101b-1. Specifically, if the second corresponding electronic device 101b-1 detects user input from the second user 404 accepting a request for the second user 404 to join a multi-user communication session with the first user 402, a first type of multi-user communication session will be initiated between the first electronic device 101a and the second corresponding electronic device 101b-1. Furthermore, if the second electronic device 101b detects user input from the second user 404 accepting a request for the second user 404 to join a multi-user communication session with the first user 402, a second type of multi-user communication session will be initiated between the first electronic device 101a and the second electronic device 101b, even if the first electronic device 101a and the second electronic device 101b are juxtaposed in the physical environment 400. The user experience in the second type of multi-user communication session between juxtaposed users may optionally be more desirable than the first type of user experience. For example, the second type of multi-user communication session between juxtaposed users in the physical environment is advantageous in reducing problems that may be caused by juxtaposed electronic devices in the first type of multi-user communication session (e.g., audio feedback, inconsistent spatial alignment of virtual content in the multi-user communication session, etc.). Therefore, in some examples, different electronic devices associated with the co-location receiver present different types or groups of notifications instructing the first user 402 to request the co-location receiver to join a multi-user communication session with the first user 402, and different types of multi-user communication sessions can be initiated based on which electronic device the co-location receiver uses to accept the request. These features are referenced below. Figure 7C and Figure 7D Further examples and / or descriptions.

[0182] Similarly, as Figure 7B As shown, in response to detecting a request from a third user 406 to join a multi-user communication session including the first user 402, the third electronic device 101c presents a user interface element 712c (e.g., a visual notification) that can be selected to accept the request and presents an audio notification 712d. Figure 7BIn this context, user interface element 712c includes one or more features described with reference to user interface element 712a, but this user interface element can be selected to accept a request from first user 402 (“Bella”) for third user 406 (“Charlie”) to join a multi-user communication session including first user 402. Audio notification 712d (e.g., a ringtone or ringing) instructs first user 402 to request third user 406 (“Charlie”) to join the multi-user communication session with first user 402. Audio notification 712d includes one or more features described with reference to audio notification 712b.

[0183] In addition, such as Figure 7BAs shown, in response to detecting a request from a third user 406 to join a multi-user communication session including a first user 402, a third corresponding electronic device 101c-1 presents a user interface element 714b (e.g., a visual notification) instructing the first user 402 to request the third user 406 to join the multi-user communication session with the first user 402, and an audio notification 714c instructing the first user 402 to request the third user 406 to join the multi-user communication session with the first user 402. The user interface element 714b includes one or more features of the user interface element 714a. The third corresponding electronic device 101c-1 includes one or more features described with reference to the second corresponding electronic device 101b-1. For example, the third corresponding electronic device 101c-1 may optionally include one or more features described with reference to electronic devices 160 / 260. Furthermore, unlike the second corresponding electronic device 101b-1, the third corresponding electronic device 101c-1 presents an audio notification 714c instructing the first user 402 to request the third user 406 (“Charlie”) to join a multi-user communication session with the first user 402. This audio notification includes one or more features described with reference to audio notifications 712d and 712b. In some examples, both the third electronic device 101c and the third corresponding electronic device 101c-1 present an audio notification instructing the third user 406 (“Charlie”) to join a multi-user communication session with the first user 402 because the request from the first electronic device 101a is a second-type request. That is, as described above, the second-type request only enables the third user 406 to join a first-type multi-user communication session, but does not enable the third user 406 to join a second-type multi-user communication session, regardless of the electronic device used by the third user 406 to join. Therefore, in some examples, different electronic devices associated with the non-co-located receiver present a notification instructing the first user 402 to request the co-located receiver to join a multi-user communication session with the first user 402, and to initiate the same type of multi-user communication session (such as the first type of multi-user communication session described herein), regardless of the specific electronic device used by the non-co-located receiver to accept the request. It should be noted that the first electronic device 101a may optionally include one or more features described with reference to electronic devices 101 / 201B / 201B, the second electronic device 101b may optionally include one or more features described with reference to electronic devices 101 / 201B / 201B, and the third electronic device 101c may optionally include one or more features described with reference to electronic devices 101 / 201B / 201B.

[0184] In addition, Figure 7B In this process, the third electronic device 101c detects user input directed at the user interface element 712c. For example, in Figure 7BIn the process, when the third user 406's hand 406a performs an air pinch gesture, the third electronic device 101c detects the third user 406's attention 704c (e.g., gaze) pointing at the user interface element 712c. In response to detecting user input, the third electronic device 101c may optionally send an instruction to the first electronic device 101a indicating that the third user 406 has accepted the request for the first user 402 to join a multi-user communication session. Alternatively, in Figure 7B In this context, the third corresponding electronic device 101c-1 corresponds to the user interface element 714b on its display (e.g., a touch-sensitive display). Figure 7B The user interface element 714b) displayed in the image detects contact 717a of hand 406a at the selected location, which corresponds to the third user 406 accepting a request to join a multi-user communication session with the first user 402, while the third electronic device 101c does not detect input corresponding to accepting the request. In response, the third corresponding electronic device 101c-1 sends an indication to the first electronic device 101a that the third user 406 has accepted the request to join the multi-user communication session with the first user 402. In some examples, regardless of the electronic device used by the third user 406 to join, the first electronic device 101a and the third electronic device 101c establish a first type of multi-user communication session in response to the first electronic device 101a detecting the indication.

[0185] Figure 7C and Figure 7D An example is illustrated whereby a first electronic device 101a initiates a multi-user communication session (e.g., a hybrid multi-user communication session including a first type of multi-user communication session and a second type of multi-user communication session) in response to an indication that a second user 404 from a second electronic device 101b has accepted a request to join a multi-user communication session with a first user 402, and an indication that a third user 406 from an electronic device associated with a third user 406 has accepted a request to join a multi-user communication session with a first user 402. For example, in Figure 7B In this context, the second electronic device 101b optionally detects input corresponding to the second user 404 accepting a request to join a multi-user communication session with the first user 402, while the second corresponding electronic device 101b-1 does not detect input corresponding to accepting the request. For example, in Figure 7B In the process, when the first user 402's hand 402a performs an air pinch gesture, the second electronic device 101b detects input including the first user 402's attention 704b (e.g., gaze) pointing at the user interface element 712a. In response to the second electronic device 101b detecting... Figure 7BUpon receiving the input from the second electronic device 101b, the second electronic device 101b sends an indication to the first electronic device 101a that the second user 404 has accepted the request to join the multi-user communication session with the first user 402. In response to detecting the acceptance indication from the second electronic device 101b, the first electronic device 101a initiates a process of establishing a shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b, such as... Figure 7C The glyph 720 is shown in the text, and is described as follows.

[0186] In some examples, the process of establishing a shared spatial coordinate system for the physical environment 400 between the first electronic device 101a and the second electronic device 101b includes aligning first map data of the physical environment 400 determined by the first electronic device 101a (e.g., by the viewpoint of the first electronic device 101a relative to the first electronic device 101a) with second map data of the physical environment 400 determined by the second electronic device 101b (e.g., by the viewpoint of the second electronic device 101b relative to the second electronic device 101b) to establish a shared spatial coordinate system for the physical environment 400 between the first electronic device 101a and the second electronic device 101b. In some examples, the process of establishing a shared spatial coordinate system for the physical environment 400 between the first electronic device 101a and the second electronic device 101b includes detecting image data of the physical environment 400 via one or more sensors of the first electronic device 101a and one or more sensors of the second electronic device 101b, and then merging and / or aligning the image data to create and / or determine a shared spatial coordinate system for the physical environment 400. For example, a first electronic device 101a sends image data of the physical environment 400 captured by one or more sensors of the first electronic device 101b to a second electronic device 101b, and the second electronic device 101b sends image data of the physical environment 400 captured by one or more sensors of the second electronic device 101b to the first electronic device 101a. In some examples, establishing a shared spatial coordinate system for the physical environment 400 includes the above references. Figure 4C The boxes (e.g., operations) 414, 416, 418, and / or 420 describe one or more characteristics. If a shared spatial coordinate system of the physical environment 400 is successfully established between the first electronic device 101a and the second electronic device 101b in the physical environment 400, then the first electronic device 101a and the second electronic device 101b establish a second type of multi-user communication session with each other while in a first type of multi-user communication session with a third electronic device 101c, such as... Figure 7D As shown.

[0187] exist Figure 7DIn this process, a second type of multi-user communication session is established between the first electronic device 101a and the second electronic device 101b (e.g., between the first user 402 and the second user 404), as shown in font 722a. Simultaneously, a first type of multi-user communication session is established, including the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c, as shown in font 722b. Therefore, from... Figures 7A to 7B to Figures 7C to 7D The first electronic device 101a establishes a second type of multi-user communication session with the second user 404 in response to an indication that the second user 404 has accepted a request from the second electronic device 101b to join a multi-user communication session with the first user 402, and establishes a first type of multi-user communication session with the third user 406 in response to an indication that the third user 406 has accepted a request to join a multi-user communication session with the first user 402. Figure 7D The first type of multi-user communication session includes a first user 402, a second user 404, and a third user 406. For example, in Figure 7D In a first type of multi-user communication session, first user 402 and second user 404 are in a first spatial group, and third user 406 is in a spatial group different from the first spatial group. For example, as Figure 7D As shown, using the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b, the first electronic device 101a presents a visual representation of the third user 406 (e.g., avatar 724), and the second electronic device 101b presents a visual representation of the third user 406 (e.g., avatar 724) at the same corresponding location in the physical environment 400.

[0188] Figure 7D-1 An example is illustrated where a first electronic device 101a and a second electronic device 101b, during a multi-user communication session between them, use a shared spatial coordinate system established between the first electronic device 101a and the second electronic device 101b in a physical environment 400 to present a user interface 434a of a photo application (e.g., sharing virtual content) at a similar (e.g., identical) corresponding location in the physical environment 400, as indicated by glyphs 722a / 722b. In some examples, the user interface 434a of the photo application includes one or more features of the user interface 434. It should be noted that the user interface 434a of the photo application is representative; for example, alternatively, a user interface of a shared messaging application, a content playback application (such as a video playback application or a music playback application), or another type of application's user interface could be used. Additionally, Figure 7D-1An example is illustrated where, during a multi-user communication session between a first electronic device 101a and a second electronic device 101b, the shared spatial coordinate system of the physical environment 400 established between the first electronic device 101a and the second electronic device 101b is used to present the avatar 724 of a third user 406 at a similar (e.g., identical) corresponding location in the physical environment 400, as indicated by glyphs 722a / 722b. Therefore, co-located electronic devices in a multi-user communication session can use the shared spatial coordinate system of the physical environment 400 established between the co-located electronic devices to align the presentation of shared content in the multi-user communication session.

[0189] Figure 7B and Figure 7E An example is illustrated whereby a first electronic device 101a initiates a first type of multi-user communication session in response to an indication that a second user 404 from a second corresponding electronic device 101b-1 has accepted a request to join a multi-user communication session with a first user 402, and an indication that a third user 406 from an electronic device associated with a third user 406 has accepted a request to join a multi-user communication session with a first user 402. For example, in Figure 7B Alternatively, the second corresponding electronic device 101b-1 detects hand contact 717b at a location on its display (e.g., a touch-sensitive display) corresponding to the selection of user interface element 714a. This corresponds to the second user 404 accepting a request to join a multi-user communication session with the first user 402, while the second electronic device 101b does not detect input corresponding to the acceptance request. In response, the second corresponding electronic device 101b-1 sends an indication to the first electronic device 101a that the second user 404 has accepted the request to join the multi-user communication session with the first user 402. In response to detecting the acceptance indication from the second corresponding electronic device 101b-1, the first electronic device 101a initiates a first type of multi-user communication session between the first electronic device 101a and the second electronic device 101b (e.g., between the first user 402 and the second user 404), even if the first user 402 and the second user 404 are physically together in the environment 400 (e.g., juxtaposed). Figure 7C As shown. Similarly, in response to an indication that a request from a third user 406 to join a multi-user communication session with the first user 402 has been accepted, the first electronic device 101a initiates a first type of multi-user communication session between the first electronic device 101a and the third electronic device 101c (e.g., between the first user 402 and the third user 406). Therefore, from Figures 7A to 7B to Figure 7EThe first electronic device 101a establishes a first type of multi-user communication session with the second user 404 in response to an indication that the second user 404 has accepted a request from the second corresponding electronic device 101b-1 to join a multi-user communication session with the first user 402, and establishes a first type of multi-user communication session with the first user 402 in response to an indication that the third user 406 has accepted a request to join a multi-user communication session with the first user 402. Figure 7E The first type of multi-user communication session includes a first user 402, a second user 404, and a third user 406. Therefore, in Figure 7E In this context, a first type of multi-user communication session is established between the first electronic device 101a, the second electronic device 101b, and the third electronic device 101c (e.g., between the first user 402, the second user 404, and the third user 406). For example, in Figure 7E In this context, the first user 402, the second user 404, and the third user 406 are in the same space group, such as reference... Figure 3 As described. In addition, when in a multi-user communication session with the first user 402 and the second user 404, the first electronic device 101a simultaneously presents a visual representation 725 (e.g., two-dimensional tiles) of the second user 404, optionally including a video feed from a camera of the second corresponding electronic device 101b-1, and an avatar 724 of the third user 406.

[0190] In addition, such as Figure 7E As shown, the second corresponding electronic device 101b-1 presents a visual representation of other users in a first type of multi-user communication session. For example, in Figure 7E In the middle, the second corresponding electronic device 101b-1 presents the visual representation 725a of the first user 402 and the visual representation 725b of the third user 406.

[0191] Figure 7E-1 This illustrates how a first electronic device 101a and a second electronic device 101b present a user interface 434a (e.g., sharing virtual content) at different locations within the physical environment 400 when they are in a first type of multi-user communication session (e.g., and not in a second type of multi-user communication session) as indicated by glyph 722a. In some examples, the first electronic device 101a and the second electronic device 101b present the user interface 434a at different locations within the physical environment 400 because a shared spatial coordinate system for the physical environment 400 has not yet been established between the first electronic device 101a and the second electronic device 101b. Additionally, Figure 7E-1An example is illustrated where a first electronic device 101a and a second electronic device 101b, when in a first type of multi-user communication session (e.g., and not in a second type of multi-user communication session) as indicated by glyph 722a, present an avatar 724 of a third user 406 at different locations within physical environment 400. In some examples, the first electronic device 101a and the second electronic device 101b present the avatar 724 of the third user 406 at different locations within physical environment 400 because a shared spatial coordinate system of physical environment 400 has not yet been established between the first electronic device 101a and the second electronic device 101b. Therefore, non-coordinated electronic devices in a multi-user communication session (e.g., electronic devices in which a shared spatial coordinate system of the physical environment in which the electronic devices coexist has not yet been established) present shared content of the multi-user communication session without spatial alignment between the presentations of the shared content, which reduces the processing power associated with presenting shared content of a multi-user communication session between non-coordinated electronic devices.

[0192] Figure 8 A flowchart illustrating, according to some examples, of a method 800 for initiating a multi-user communication session of a different type with a second user of a second electronic device based on the type of electronic device from which a second user of a second electronic device accepts a request to join a multi-user communication session with a first user of a first electronic device. In some examples, method 800 begins at a first electronic device communicating with one or more first displays and one or more first input devices. In some examples, the first electronic device includes... Figures 7A to 7E-1 One or more characteristics of the first electronic device 101a. One or more examples of method 800 are referenced above. Figures 7A to 7E-1 One or more of these are illustrated and / or described. It should be understood that method 800 is an example and more, fewer, or different operations may be performed in the same or different order. Additionally, the operations in method 800 described below may optionally be performed by running an information processing device such as a general-purpose processor (e.g., as per [reference to...]). Figures 2A to 2C (as described) or one or more functional modules in a dedicated chip and / or through Figures 2A to 2C It is implemented using other components.

[0193] Therefore, based on the above description, some examples of this disclosure relate to a method (e.g., Figure 8 Method 800). In some examples, method 800 is performed at a first electronic device communicating with one or more first displays and one or more first input devices. In some examples, a first user of the first electronic device and a second user of a second electronic device are juxtaposed in a physical environment, such as reference... Figure 7AThe first user 402 of the first electronic device 101a and the second user 404 of the second electronic device 101b are juxtaposed in physical environment 400 as described. Method 800 includes sending (802) a request for the second user to join a multi-user communication session including the first user, such as referring to... Figure 7A The method 800 is described as follows: First electronic device 101a sends a request for second user 404 to join a multi-user communication session with first user 402 in response to input pointing to option 702. Method 800 includes, after sending the request, receiving (804) an indication that the second user has accepted the request for the second user to join a multi-user communication session including the first user, such as referring to... Figure 7B The method 800 is described as follows: First electronic device 101a receives from second electronic device 101b an indication that second user 404 has accepted a request from second electronic device 101b to join a multi-user communication session with first user 402 in response to input to user interface element 712a. Method 800 includes, in response to receiving the indication (806), adding (806a) the second user of the second electronic device to the multi-user communication session including the first user, based on determining that the second user has accepted the request from (e.g., via) the second electronic device and that the second electronic device is a first type of electronic device, including making the multi-user communication session a first type of communication session, such as reference... Figures 7A to 7B to Figures 7C to 7D The method 800, described in the section on adding a second user 404 to a second type of multi-user communication session, includes, in response to receiving an instruction (806), adding (806b) the second user of the second electronic device to the multi-user communication session including the first user, based on determining that the second user accepts a request from (e.g., via) the second electronic device and that the second electronic device is a second type of electronic device different from the first type of electronic device, and that the second user accepts the request from the second electronic device. This includes making the multi-user communication session a second type of communication session different from the first type of communication session, such as reference to... Figures 7A to 7B to Figure 7E The first electronic device 101a adds the second user 404 to the first type of multi-user communication session as described.

[0194] Additionally or alternatively, in some examples, the second user of a second electronic device in the physical environment is also a user of a third electronic device in the physical environment, distinct from the second electronic device, such as references. Figure 7AThe second user 404 is described as a user of the second electronic device 101b and a user of the second corresponding electronic device 101b-1. Additionally or alternatively, in some examples, sending a request for the second user to join a multi-user communication session including the first user includes: sending a request to the second electronic device via the second electronic device to the second electronic device to join a multi-user communication session including the first user, such as referring to... Figure 7A The first electronic device 101a, as described in the example, sends a request to the second electronic device 101b in response to input; and sends a request to the third electronic device via the third electronic device for the second user to join a multi-user communication session including the first user, such as references. Figure 7A The method is described as a first electronic device 101a that sends a request to a second corresponding electronic device 101b-1 in response to input. Additionally or alternatively, in some examples, method 800 includes adding a second user to a multi-user communication session including the first user based on determining that the second user accepts a request from (e.g., via) a third electronic device and that the third electronic device is a second type of electronic device. This includes making the multi-user communication session a second type of communication session, as described with reference to the method described where second user 404 accepts a request to join a multi-user communication session with first user 402 via the second corresponding electronic device 101b-1.

[0195] Additionally or alternatively, in some examples, the second user of the second electronic device in the physical environment is also a user of a third electronic device in the physical environment, different from the second electronic device, wherein the third electronic device is an electronic device of a different type from the second electronic device and is either a first-type electronic device or a second-type electronic device, such as references. Figure 7A The second user 404 is described as a user of the second electronic device 101b and a user of the second corresponding electronic device 101b-1. Additionally or alternatively, in some examples, sending a request for the second user to join a multi-user communication session including the first user includes sending a request to the second electronic device for the second user to join a multi-user communication session including the first user, including causing the second electronic device to present a first notification to the second user for notifying the second user of the request to join a multi-user communication session including the first user, such as... Figure 7B The second electronic device 101b presents user interface element 712a. Additionally or alternatively, in some examples, sending a request for the second user to join a multi-user communication session including the first user includes sending a request to a third electronic device for the second user to join the multi-user communication session including the first user, thereby causing the third electronic device to present a second notification to the second user for notifying the second user of the request to join the multi-user communication session including the first user, such as... Figure 7B The second corresponding electronic device 101b-1 presents a user interface element 714a (e.g., a notification).

[0196] Additionally or alternatively, in some examples, the first notification is a first type of notification, such as Figure 7B The audio notification 712b in the example, and the second notification is a second type of notification different from the first type of notification, such as Figure 7B User interface element 712a (e.g., visual notification). Additionally or alternatively, in some examples, the second electronic device is an electronic device of the first type, such as... Figure 7B The second electronic device 101b in the first type of notification includes audio, such as... Figure 7B Audio notifications in 712b, and second-type notifications do not include audio, such as Figure 7B User interface element 714a.

[0197] Additionally or alternatively, in some examples, making a multi-user communication session including a first user a second type of communication session includes displaying a virtual representation of the second user on a second electronic device via one or more first displays, such as Figure 7E The visual representation 725 of the second user 404 (“Alice”). Additionally or alternatively, in some examples, making a multi-user communication session including the first user a first type of communication session includes presenting the second user via one or more first displays, without displaying a virtual representation of the second user on a second electronic device, such as a representation of the physical body of the second user 404 shown in the field of view of the display 120a of the first electronic device 101a, while Figure 7D The first electronic device 101a does not present Figure 7E The visual representation of the second user 404 (“Alice”) 725.

[0198] Additionally or alternatively, in some examples, making a multi-user communication session including a first user a second type of communication session includes initiating a process of presenting the detected audio of the second user via one or more first audio output devices of a first electronic device, such as that described herein with reference to a first type of multi-user communication session (e.g., in...). Figure 3 (in Chinese). Additionally or alternatively, in some examples, making a multi-user communication session including a first user a first type of communication session includes abandoning the initiation of the process, including abandoning the presentation of the second user's audio, as described herein with reference to a second type of multi-user communication session (e.g., in...). Figure 4H and / or Figure 7D middle).

[0199] Additionally or alternatively, in some examples, making a multi-user communication session including a first user a first type of communication session includes initiating a process of establishing a shared spatial coordinate system of the physical environment between the first electronic device and the second electronic device, such as referencing Figure 7C The first electronic device 101a and the second electronic device 101b establish a shared spatial coordinate system for the physical environment 400 as described. Additionally or alternatively, in some examples, the shared spatial coordinate system of the physical environment is based on first map data (such as reference data) determined by the viewpoint of the first electronic device relative to the first electronic device. Figure 7C The first electronic device 101a described in the text) and the second map data (such as reference data) determined by the viewpoint of the second electronic device relative to the second electronic device. Figure 7C (As described in the second electronic device 101b). Additionally or alternatively, in some examples, making a multi-user communication session including a first user a second type of communication session includes abandoning the process of initiating the establishment of a shared spatial coordinate system for the physical environment. For example, the first electronic device 101a and the second electronic device 101b do not establish a shared spatial coordinate system from... Figures 7B to 7E The physical environment uses a shared spatial coordinate system of 400, because Figure 7E The multi-user communication session in which the first electronic device 101a and the second electronic device 101b participate is a first type of multi-user communication session (and not a second type of multi-user communication session).

[0200] Additionally or alternatively, in some examples, method 800 includes, after adding a second user of a second electronic device to a multi-user communication session including the first user, adding a third user of a third electronic device to the multi-user communication session including the first and second users, wherein the third user of the third electronic device is not physically co-located with the first user. For example, after detecting that the second user 404 has accepted the request of the first user 402 to join the multi-user communication session including the first user 402, the first electronic device 101a optionally detects Figure 7B In the example, a third user 406 of a third electronic device 101c has accepted an instruction from a first user 402 to join a multi-user communication session including the first user 402. Continuing the example, in response to detecting that a second user 404 has accepted an instruction from a first user 402 to join a multi-user communication session including the first user 402, the first electronic device 101a adds the second user 404 to the multi-user communication session, and in response to detecting that a third user 406 of a third electronic device 101c has accepted an instruction from a first user 402 to join a multi-user communication session including the first user 402, the first electronic device 101a adds the third user 406, optionally after adding the second user 404.

[0201] Additionally or alternatively, in some examples, method 800 includes, after adding a third user of a third electronic device to a multi-user communication session including a first user and a second user, sending to the third electronic device an indication that the first user of the first electronic device and the second user of the second electronic device are in a first space group in the multi-user communication session, the first space group being different from a second space group in which the third electronic device participates in the multi-user communication session, based on determining that the second electronic device is a first type of electronic device. For example, in response to detecting that the first user 402 and the second user 404 are in a space group with... Figure 7D Indications in the second space group that differ from the first space group in a multi-user communication session involving a third electronic device. Figure 7D The third electronic device 101c may optionally present representations (e.g., avatars) of the first user and the second user in a three-dimensional environment of the third electronic device 101c, based on a spatial arrangement of the first user 402 and the second user 404 in the physical environment 400. Additionally or alternatively, in some examples, method 800 includes, after adding a third user of the third electronic device to a multi-user communication session including the first user and the second user, sending an indication to the third electronic device that the first user of the first electronic device and the second user of the second electronic device are in a second spatial group with the third electronic device, based on determining that the second electronic device is a second type of electronic device. For example, in Figure 7E In response to an indication that the first user 402 and the second user 404 are in the same spatial group as the third user 406, Figure 7E The third electronic device 101c may optionally present a representation (e.g., avatar) of the first user 402 and the second user 404 in a spatial arrangement that does not take into account (e.g., not based on or independent of) the spatial arrangement of the first user 402 and the second user 404 in the three-dimensional environment of the third electronic device 101c.

[0202] Additionally or alternatively, in some examples, method 800 includes displaying shared virtual content of the multi-user communication session, including an application's user interface, at a first location in a first three-dimensional environment of a first electronic device via one or more first displays when the multi-user communication session includes a first user and a second user. For example, in Figure 7D-1 In a multi-user communication session including a first electronic device 101a, a user interface 434a displayed in a first three-dimensional environment presented via the first electronic device 101a is shared. Additionally or alternatively, in some examples, based on determining that the second electronic device is a first type of electronic device, the first location is a first shared location in a shared spatial coordinate system of the physical environment between the first and second electronic devices, such as in… Figure 7D-1The first electronic device 101a and the second electronic device 101b use a shared spatial coordinate system of the physical environment 400 between them to present a user interface 434a at the same corresponding physical location within the physical environment 400. Additionally or alternatively, in some examples, depending on whether the second electronic device is determined to be a second type of electronic device, the first location is not a corresponding shared location in the shared spatial coordinate system of the physical environment, such as in... Figure 7E-1 The first electronic device 101a and the second electronic device 101b present the user interface 434a at different corresponding physical locations (e.g., in different ways) in the physical environment 400, optionally because the first electronic device 101a and the second electronic device 101b have not yet been established. Figure 7E-1 The physical environment in the shared spatial coordinate system of 400.

[0203] Additionally or alternatively, in some examples, the first type of electronic device is a head-mounted device, such as a reference device. Figure 7A The first electronic device 101a is described in the text. Additionally or alternatively, in some examples, the second type of electronic device is a telephone, laptop computer, or tablet computer, as described with reference to the second corresponding electronic device 101b-1. In some examples, the second type of electronic device is a handheld or wrist-worn device, such as a watch.

[0204] Some examples of this disclosure relate to an electronic device including: one or more processors; a memory; and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described above. Some examples of this disclosure relate to a non-transitory computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by one or more processors of the electronic device, cause the electronic device to perform any of the methods described above. Some examples of this disclosure relate to an electronic device including one or more processors, a memory, and components for performing any of the methods described above. Some examples of this disclosure relate to an information processing apparatus used in an electronic device, the information processing apparatus including components for performing any of the methods described above.

[0205] Figures 9A to 9FGenerally, examples according to some examples of this disclosure are illustrated, such as a first electronic device detecting and responding to a third user of a third electronic device being juxtaposed with a first user of the first electronic device in a physical environment when in a multi-user communication session, the multi-user communication session including a first user of the first electronic device and a second user of a second electronic device but excluding a third user of the third electronic device, wherein the first user and the second user are juxtaposed in a physical environment, and wherein a shared spatial coordinate system of the physical environment between the first electronic device and the second electronic device has been established.

[0206] exist Figure 9A In this context, the first user 402 of the first electronic device 101a and the second user 404 of the second electronic device 101b are in a second type of multi-user communication session, as indicated by glyph 901a. Figure 9A The second type of multi-user communication session is hosted on a first networking channel (e.g., a first communication protocol, such as Wi-Fi, a first type of Bluetooth communication protocol, and / or another type of networking channel) (e.g., via which communication takes place). For example, communication between a first electronic device 101a and a second electronic device 101b involved in the second type of multi-user communication session is hosted on the first networking channel. For example, the first electronic device 101a and the second electronic device 101b use receivers, transmitters, and / or transceivers of the first electronic device 101a and the second electronic device 101b to detect and respond to communication between them in the second type of multi-user communication session, the receivers, transmitters, and / or transceivers being configured to operate using data using the first networking channel (e.g., data encoded and / or transmitted or received on the first networking channel).

[0207] exist Figure 9A In this context, the first user 402 and the second user 404 are juxtaposed in a physical environment 400, such as a reference in a physical environment (e.g., Figure 4A The first electronic device 101a and the second electronic device 101b are described as being placed side-by-side in a physical environment 400. Additionally, in Figure 9AIn this embodiment, the first electronic device 101a and the second electronic device 101b use a shared spatial coordinate system of the physical environment 400 established between the first electronic device 101a and the second electronic device 101b (e.g., the established shared spatial coordinate system of the physical environment 400 indicated in glyph 901b) to present the first user interface at the same corresponding location in the physical environment 400. In some examples, the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b is established according to the process of establishing the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b described above. For example, in some examples, the process of establishing the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b includes aligning first map data of the physical environment 400 determined by the first electronic device 101a with second map data of the physical environment 400 determined by the second electronic device 101b to establish the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b. In some examples, the process of establishing a shared spatial coordinate system for the physical environment 400 between the first electronic device 101a and the second electronic device 101b includes detecting image data of the physical environment 400 via one or more sensors of the first electronic device 101a and one or more sensors of the second electronic device 101b, and then merging and / or aligning the image data to create and / or determine the shared spatial coordinate system for the physical environment 400. For example, the first electronic device 101a sends image data of the physical environment 400 captured by one or more sensors of the first electronic device 101a to the second electronic device 101b, and the second electronic device 101b sends image data of the physical environment 400 captured by one or more sensors of the second electronic device 101b to the first electronic device 101a.

[0208] Figure 9AThe shared spatial coordinate system of the physical environment 400 is hosted on a second networking channel (e.g., a second communication protocol, such as Wi-Fi, a second type of Bluetooth communication protocol, and / or another type of networking channel) different from the first networking channel (e.g., via communication therein). For example, the shared spatial coordinate system of the physical environment 400 is established and / or updated by the first electronic device 101a and the second electronic device 101b using the second networking channel. For example, communication between the first electronic device 101a and the second electronic device 101b is hosted on the second networking channel, and this communication relates to the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b and / or relates to the process of establishing the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b. For example, the first electronic device 101a and the second electronic device 101b use receivers, transmitters, and / or transceivers of the first electronic device 101a and the second electronic device 101b to detect and respond to communication relating to a shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b and / or communication established therebetween. The receivers, transmitters, and / or transceivers are configured to operate using data using a second networking channel (e.g., data encoded and / or transmitted or received on the second networking channel). Therefore, in some examples, the first electronic device 101a and the second electronic device 101b include communication systems that allow different types of communication between them on different networking channels, such as… Figure 9A The glyphs in the text are shown as 901a / 901b.

[0209] In some examples, Figure 9AIn this embodiment, under one or more sharing conditions, the user interface 434a is shared from the first electronic device 101a to the second electronic device 101b. For example, assuming the first user 402 is capable of editing the user interface 434a, the second user 404 can interact with the user interface 101a by moving the user interface 434a and / or editing the content of the user interface 434a in the three-dimensional environment presented by the second electronic device 101b, wherein either operation will optionally cause a corresponding change in the three-dimensional environment presented via the first electronic device 434a (e.g., a corresponding change in the position of the user interface 434a in the three-dimensional environment presented by the first electronic device 101a and / or a corresponding change in the content of the user interface 434a in the three-dimensional environment presented by the first electronic device 101a). As another example, assuming the first user 402 is capable of viewing the user interface 434a without editing it, the second user 404 can interact with the user interface 434a by viewing its content without the need for editing capabilities. It should be noted that the user interface 434a may optionally be shared from the first electronic device 101a to the second electronic device 101b using the aforementioned first networking channel, because the user interface 434a is shared in a second type of multi-user communication session between the first electronic device 101a and the second electronic device 101b. In some examples, the user interface 434a is shared from the second electronic device 101b to the second electronic device 101b under one or more sharing conditions.

[0210] from Figures 9A to 9B The third user 406 of the third electronic device 101c enters the physical environment 400 where the first user 402 and the second user 404 are located. Figure 9B In this context, the third user 406 does not engage in multi-user communication sessions with the first user 402 and / or the second user 404. For example, in... Figure 9B In this context, the third user 406 is not in a multi-user communication session (e.g., a second type of multi-user communication session) in which the first user 402 and the second user 404 participate, as shown in glyph 901a.

[0211] In addition, Figure 9B In this context, the first electronic device 101a detects (or determines) that the third user 406 and the first user 402 (e.g., the first electronic device 101a and the third electronic device 101c) are juxtaposed in a physical environment 400, such as with reference herein to the first electronic device 101a and the second electronic device 101b in a physical environment (e.g., Figure 4A The physical environment 400) is described in parallel. For example, the first electronic device 101a and the third electronic device 101c are in Figure 9A The first electronic device 101a and the third electronic device 101c are placed side by side because they satisfy the reference... Figure 4AThe criteria for determination are described. For example, the determination of the first electronic device 101a and the third electronic device 101c may optionally be based on the following: Figure 9A The following are considered in the physical environment 400: the distance between the first electronic device 101a and the third electronic device 101c; communication between the first electronic device 101a and the third electronic device 101c; the strength of wireless signals transmitted and detected between the first electronic device 101a and the third electronic device 101c; the first electronic device 101a and the third electronic device 101c being connected to the same network (e.g., a wireless network) in the physical environment 716; visual detection of the first electronic device 101a and the third electronic device 101c in the physical environment 400; the user of another electronic device in the contact list of the electronic device (e.g., the contact list of the third user 406 in the application on the first electronic device 101a and / or the contact list of the first user 402 in the application on the third electronic device 101c); and / or because the first electronic device 101a and the third electronic device 101c are in the same physical room (e.g., physical environment 716). In response to the detection that a third user 406 and a first user 402 are juxtaposed in physical environment 400, the first electronic device 101a sends the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b to the third electronic device 101c, such as... Figure 9C The glyph 901c indicates this. In some examples, the first electronic device 101a uses the aforementioned second networking channel to send the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b to the third electronic device 101c.

[0212] In some examples, such as Figure 9CThe shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b, indicated by the glyph 901c, is transmitted to the third electronic device 101c. This includes transmitting map data of the physical environment 400 captured between the first electronic device 101a and the second electronic device 101b. For example, the map data of the physical environment 400 may optionally be based on first map data determined by the viewpoint of the first electronic device 101a relative to the first electronic device 101a in the physical environment 400 and second map data determined by the viewpoint of the second electronic device 101b relative to the second electronic device 101b in the physical environment 400. For example, the first electronic device 101a may optionally use one or more sensors of the first electronic device 101a to capture portions of the physical environment 400 surrounding the first electronic device 101a in order to determine a map of the physical environment 400 from the viewpoint of the first electronic device 101a in the physical environment 400. Continuing this example, the second electronic device 101b may optionally use one or more sensors of the second electronic device 101b to capture portions of the physical environment 400 surrounding the second electronic device 101b in order to determine a map of the second electronic device 101b from its viewpoint within the physical environment 400. In some examples, the shared spatial coordinate system of the physical environment 400 includes and / or is based on SLAM map data from the first electronic device 101a and / or the second electronic device 101b, as described above. Thus, in some examples, when in a multi-user communication session with the second electronic device 101b but not with the third electronic device 101c, the first electronic device 101a sends to the third electronic device 101c the shared spatial coordinate system of the physical environment 400 utilized between the first electronic device 101a and the second electronic device 101b in the multi-user communication session.

[0213] In some examples, when and / or after sending the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b to the third electronic device 101c, the first electronic device 101a detects and responds to user input requesting the addition of a third user 406 to the multi-user communication session in which the first electronic device 101a and the second electronic device 101b participate, such as... Figure 9D As shown. In Figure 9DIn this process, the first electronic device 101a detects input corresponding to a request to add a third user 406 to a multi-user communication session including the first user 402 and the second user 404. For example, when the first user 402's hand 402a performs an air pinch gesture, the first electronic device 101a detects the first user 402's attention 904a (e.g., gaze) pointing at the user interface element 902a. In response to the detected input, the first electronic device 101a sends an instruction to the third electronic device 101c that the first user 402 has requested the third user 406 to join the multi-user communication session with the first user 402. In response to the detected instruction that the first user 402 has requested the third user 406 to join the multi-user communication session, the third electronic device 101c notifies the third user 406, such as... Figure 9E As shown.

[0214] exist Figure 9E In this process, the third electronic device 101c notifies the third user 406, via a user interface element 902b, that the first user 402 has requested the third user 406 to join a multi-user communication session that includes the first user 402. In some examples, the user interface element 902b provides the third user 406 with the option to accept or reject the request. In some examples, if the third electronic device 101c detects user input from the third user 406 corresponding to a rejection request, such as user input pointing to an option that can be selected to reject the request, then the third electronic device 101c will send an indication to the first electronic device 101a that the third user 406 has rejected the request to join the multi-user communication session. Figure 9E In this process, the third electronic device 101c detects user input from the third user 406 corresponding to the acceptance request. For example, in Figure 9E In the process, when the third user 406's hand 406a performs an air pinch gesture, the third electronic device 101c detects the third user 406's attention 904b (e.g., gaze) towards the user interface element 902b. In response, the third electronic device 101c sends an instruction to the first electronic device 101a that the third user 406 accepts the request from the first user 402 for the third user 406 to join the multi-user communication session.

[0215] It should be noted that the request from first user 402 to third user 406 to join a multi-user communication session including first user 402 is a request from first user 402 to third user 406 to join a second-type multi-user communication session. As described above, a second-type multi-user communication session involves establishing a shared spatial coordinate system of the physical environment of electronic devices participating in the second-type multi-user communication session with each other. In some examples, the process for establishing the shared spatial coordinate system of the physical environment is initiated before the user accepts the request to join the second-type multi-user communication session, such as by sending the shared spatial coordinate system of the physical environment 400 between first electronic device 101a and second electronic device 101b to third electronic device 101c, which is not a party to the second-type multi-user communication session including first user 402 and second user 404, as shown in the reference. Figure 9C The glyph 901c is described in the text. For example, before the first electronic device 101a sends a request for the third user 406 to join a second type of multi-user communication session with the first user 402 and the second user 404, in Figure 9A In a second type of multi-user communication session between a first electronic device 101a and a second electronic device 101b, a shared spatial coordinate system of the physical environment 400, which utilizes and does not include map data captured by a third electronic device 101c, is transmitted from the first electronic device 101a to the third electronic device 101c.

[0216] exist Figure 9FIn this scenario, the first electronic device 101a adds a third user 406 to a second type of multi-user communication session that includes the first user 402 and the second user 404 (e.g., in response to an indication that the third user 406 has accepted a request from the third electronic device 101c). Additionally, the third electronic device 101c displays the user interface 434a at the same corresponding location in the physical environment 400 as the user interface 434a displayed by the first electronic device 101a and the second electronic device 101b, using the shared spatial coordinate system of the physical environment 400. In some examples, when the third electronic device 101c initially displays the user interface 434a, it uses the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b to display the user interface 434a. In some examples, the shared spatial coordinate system of the physical environment 400 sent by the first electronic device 101a to the third electronic device 101c is a map of the physical environment 400, which is the same physical environment 400 where the third user 406 is located, developed only by the first electronic device 101a and the second electronic device 101b (e.g., only by the electronic device as one party to a second type of multi-user communication session before the third user 406 is added to the multi-user communication session). In some examples, when the third electronic device 101c initially displays the user interface 434a, the third electronic device 101c uses the shared spatial coordinate system of the physical environment 400 to display the user interface 434a, which is based not only on map data of the physical environment 400 captured by the sensors of the first electronic device 101a and the second electronic device 101b, but also on map data of the physical environment 400 captured by the sensors of the third electronic device 101c. In some examples, when a third electronic device 101c is added to a multi-user communication session including a first user 402 and a second user 404 (e.g., including a first electronic device 101a and a second electronic device 101b), the third electronic device 101c updates the shared spatial coordinate system of the physical environment 400 to be based not only on map data of the physical environment 400 captured by sensors of the first electronic device 101a and the second electronic device 101b, but also on map data of the physical environment 400 captured by sensors of the third electronic device 101c. For example, the first electronic device 101a is used to display... Figure 9F The shared spatial coordinate system of the physical environment 400 in the user interface 434a is optionally updated with map data from the third electronic device 101c. Similarly, the second electronic device 101b is used to display... Figure 9FThe shared spatial coordinate system of the physical environment 400 in the user interface 434a is optionally updated with map data from a third electronic device 101c. For example, when a third user 406 is added to a multi-user communication session including a first user 402 and a second user 404, the third electronic device 101c sends map data of the physical environment 400 captured by the sensors of the third electronic device 101c to the first electronic device 101a and / or the second electronic device 101b, and this transmission optionally occurs on a second network channel, the same network channel used by the first electronic device 101a and the second electronic device 101b to transmit the shared spatial coordinate system of the physical environment 400 to each other, as described in reference glyph 901b. In some examples, the first electronic device 101a and / or the second electronic device 101b receive and update the shared spatial coordinate system of the physical environment 400, and the first electronic device 101a and / or the second electronic device 101b respectively display the content of the multi-user communication session through this shared spatial coordinate system.

[0217] Figure 10 A flowchart illustrating a method 1000 for detecting and responding to a third user of a third electronic device being juxtaposed with a first user of a first electronic device in a physical environment when in a multi-user communication session, according to some examples of the present disclosure, the multi-user communication session including a first user of a first electronic device and a second user of a second electronic device but excluding a third user, wherein the first user and the second user are juxtaposed in a physical environment, and wherein a shared spatial coordinate system of the physical environment between the first electronic device and the second electronic device has been established.

[0218] In some examples, method 1000 begins at a first electronic device communicating with one or more first displays and one or more first input devices. In some examples, the first electronic device includes... Figures 9A to 9F One or more characteristics of the first electronic device 101a. One or more examples of method 1000 are referenced. Figures 9A to 9F One or more of the methods described below are illustrated and / or described. It should be understood that method 1000 is an example, and more, fewer, or different operations may be performed in the same or different order. Additionally, the operations in method 1000 described below may optionally be performed by running an information processing device such as a general-purpose processor (e.g., as per [reference to...]). Figures 2A to 2C (as described) or one or more functional modules in a dedicated chip and / or through Figures 2A to 2C It is implemented using other components.

[0219] Therefore, based on the above description, some examples of this disclosure relate to a method (e.g., Figure 10Method 1000). In some examples, method 1000 is performed at a first electronic device communicating with one or more first displays and one or more first input devices. In some examples, a first user of the first electronic device and a second user of a second electronic device are juxtaposed in a physical environment, such as reference... Figure 9A The first user 402 of the first electronic device 101a and the second user 404 of the second electronic device 101b are juxtaposed in the physical environment 400 as described.

[0220] Method 1000 includes when (1002) a multi-user communication session (such as a first user of a first electronic device, a second user of a second electronic device, and a third user of a third electronic device) is in progress (e.g., a third user of a third electronic device). Figure 9A In a second type of multi-user communication session (including first user 402 and second user 404 but excluding third user 406 of third electronic device 101c), and when the shared spatial coordinate system of the physical environment between the first electronic device and the second electronic device (such as by...) Figure 9A When the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b, indicated by the glyph 901b, has been established, it is detected (1002a) that the third user of the third electronic device is juxtaposed with the first user of the first electronic device in the physical environment. For example, as referenced Figure 9B and Figure 9C As described, when the first electronic device 101a and the second user 404 are in a second type of multi-user communication session and when a shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b has been established, the first electronic device 101a detects that the third user 406 of the third electronic device 101c is juxtaposed with the first user 402 in the physical environment 400.

[0221] Method 1000 includes, when (1002) in a multi-user communication session including a first user of a first electronic device and a second user of a second electronic device but excluding a third user of a third electronic device, and when a shared spatial coordinate system of the physical environment between the first electronic device and the second electronic device has been established, in response to detecting that the third user of the third electronic device is juxtaposed with the first user of the first electronic device in the physical environment, initiating (1002b) a shared spatial coordinate system of the physical environment with the third electronic device, such as a reference Figure 9C The glyph 901c describes this. In some examples, initiating a shared spatial coordinate system for sharing the physical environment with a third electronic device includes sending the shared spatial coordinate system of the physical environment to the third electronic device, such as a reference... Figure 9CAs described in glyph 901c. In some examples, the shared spatial coordinate system of the physical environment includes (and / or is based on) map data of the physical environment, which is based on first map data determined by the viewpoint of the first electronic device relative to the first electronic device in the physical environment and second map data determined by the viewpoint of the second electronic device relative to the second electronic device in the physical environment, such as reference Figure 9C The glyph 901c in the text describes this.

[0222] Additionally, method 1000 includes sending (1002c) a request to the third electronic device to join the multi-user communication session including the first user of the first electronic device and the second user of the second electronic device, but excluding the third user of the third electronic device, when (1002) the third electronic device is in a multi-user communication session including the first user of the first electronic device and the second user of the second electronic device, and when a shared spatial coordinate system of the physical environment between the first electronic device and the second electronic device has been established, such as referring to Figure 9D The first electronic device 101a is described in response to detecting input directed to user interface element 902a and sending a request.

[0223] Additionally, method 1000 includes, after sending a request to join a multi-user communication session, adding (1004) the third user of the third electronic device to the multi-user communication session that includes the first user of the first electronic device and the second user of the second electronic device, based on determining that the third user has accepted the request, such as from... Figures 9E to 9F The third user 406 of the third electronic device 101c is added to a second type of multi-user communication session that includes the first user 402 and the second user 404.

[0224] Additionally or alternatively, in some examples, multiple communication sessions including a first user and a second user are hosted on a first networking channel, such as by... Figure 9A As indicated by glyph 901a, and the shared spatial coordinate system of the physical environment is established on a second network channel different from the first network channel, such as a reference... Figure 9A As described in glyph 901b. Additionally or alternatively, in some examples, a shared spatial coordinate system of the physical environment is transmitted to a third electronic device using a networking channel different from the first networking channel. For example, using a networking channel different from the first networking channel would correspond to... Figure 9A The data of the shared spatial coordinate system of the physical environment 400 indicated by the glyph 901b is sent to the third electronic device 101c, such as by... Figure 9C The glyph 901c indicates this. In some examples, the network channel different from the first network channel is the second network channel, such as reference... Figure 9AThe second networking channel is described in glyph 901b. In some examples, the second networking channel is a Bluetooth-based networking channel (e.g., Bluetooth Classic (BR / EDR) or Bluetooth Low Energy (BLE), or another type of Bluetooth-based networking channel). In some examples, the second networking channel is Wi-Fi (e.g., utilizing Wi-Fi).

[0225] Additionally or alternatively, in some examples, a multi-user communication session including a first user of a first electronic device and a second user of a second electronic device, but excluding a third user of a third electronic device, also includes a fourth user of a fourth electronic device. Additionally or alternatively, in some examples, the shared spatial coordinate system of the physical environment is based on first map data determined by the viewpoint of the first electronic device relative to the first electronic device in the physical environment, second map data determined by the viewpoint of the second electronic device relative to the second electronic device in the physical environment, and third map data determined by the viewpoint of the fourth electronic device relative to the fourth electronic device in the physical environment. For example, if the first electronic device 101a is juxtaposed with the second electronic device 101b and the fourth electronic device in physical environment 400, and if in Figure 9B The second type of multi-user communication session will include a first electronic device 101a, a second electronic device 101b, and a fourth electronic device. Figure 9C The shared spatial coordinate system of the physical environment 400 sent by the first electronic device 101a to the third electronic device 101c is based on map data from the first electronic device 101a, the second electronic device 101b, and the fourth electronic device.

[0226] Additionally or alternatively, in some examples, method 1000 includes detecting fourth map data from a third electronic device when in a multi-user communication session including a first user of a first electronic device, a second user of a second electronic device, and a third user of a third electronic device, wherein the fourth map data is determined by the viewpoint of the third electronic device relative to the third electronic device in the physical environment. Additionally or alternatively, in some examples, method 1000 includes updating a shared spatial coordinate system of the physical environment in response to detecting the fourth map data from the third electronic device. In some examples, the shared spatial coordinate system of the physical environment is updated based on the first map data determined by the viewpoint of the first electronic device relative to the first electronic device in the physical environment, the second map data determined by the viewpoint of the second electronic device relative to the second electronic device in the physical environment, and the fourth map data determined by the viewpoint of the third electronic device relative to the third electronic device in the physical environment.

[0227] Additionally or alternatively, in some examples, method 1000 includes, after adding a third user of a third electronic device to a multi-user communication session comprising a first user of a first electronic device and a second user of a second electronic device, detecting third map data from the third electronic device, wherein the third map data is determined by the viewpoint of the third electronic device relative to the third electronic device in the physical environment. For example, when in Figure 9E When the third electronic device 101c is in a second type of multi-user communication session with the first user 402 and the second user 404, the first electronic device 101a may optionally detect map data from the third electronic device 101c. Additionally or alternatively, in some examples, method 1000 includes updating a shared spatial coordinate system of the physical environment in response to detecting third map data from the third electronic device. In some examples, the updated shared spatial coordinate system is based on first map data determined by the viewpoint of the first electronic device relative to the first electronic device in the physical environment, second map data determined by the viewpoint of the second electronic device relative to the second electronic device in the physical environment, and third map data determined by the viewpoint of the third electronic device relative to the third electronic device in the physical environment. For example, when... Figure 9E When the exemplified third electronic device 101c is in a second type of multi-user communication session with the first user 402 and the second user 404, the first electronic device 101a may optionally update the shared spatial coordinate system of the physical environment 400, which the first electronic device 101a uses to apply map data detected by the third electronic device 101c. Figure 9E The user interface 434a is displayed at the location shown in the example.

[0228] Additionally or alternatively, in some examples, method 1000 includes, after adding a third user of a third electronic device to a multi-user communication session that includes a first user of a first electronic device and a second user of a second electronic device, detecting an indication that the corresponding user (e.g., the first user, the second user, or the third user) in the multi-user communication session requests to share the user interface of an application in the multi-user communication session. For example, when a third user 406 is added... Figure 9EIn a second type of multi-user communication session including a first user 402 and a second user 404, the user interface 434a may optionally not be shared between electronic devices in the multi-user communication session (e.g., when a third user 406 is added to the multi-user communication session, the user interface 434a may optionally be private to the first electronic device 101a). Continuing this example, the first electronic device 101a may optionally detect input corresponding to a request to share the user interface 434a to the multi-user communication session. For example, the first electronic device 101a may optionally detect the attention (e.g., gaze) of the first user 402 towards a user interface element that can be selected to share the user interface 434a to the multi-user communication session when the first user 402's hand 402a performs an air pinch gesture. In some examples, method 1000 includes displaying the user interface of the application at a first location in a first three-dimensional environment via one or more first displays in response to an indication that a corresponding user has requested to share the user interface of the application in the multi-user communication session, wherein the first location corresponds to a shared location in a shared spatial coordinate system of the physical environment. For example, in response to detecting an input corresponding to a request to share the user interface 434a into a multi-user communication session, as described above, the first electronic device 101a... Figure 9E User interface 434a is displayed at the location illustrated in the example, which is the same location as the display of the third electronic device 101c. Figure 9E The user interface 434a is associated with the same corresponding physical location.

[0229] Additionally or alternatively, in some examples, method 1000 includes, when in a multi-user communication session including a first user of a first electronic device and a second user of a second electronic device but excluding a third user of a third electronic device, displaying shared virtual content of the multi-user communication session at a first location in a first three-dimensional environment via one or more first displays, wherein the first location in the first three-dimensional environment corresponds to a first shared location in a shared spatial coordinate system of the physical environment between the first electronic device and the second electronic device, such as as shown in the figure, using... Figure 9B The glyph 901b indicates the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b, where the first electronic device 101a and the second electronic device 101b correspond to the three-dimensional environment 450A / 450B. Figure 9B The user interface 434a is displayed at the same location in the physical environment 400. Additionally or alternatively, in some examples, method 1000 includes continuing to display shared virtual content, such as by maintaining access from the third user of the third electronic device to the multi-user communication session that includes the first user of the first electronic device and the second user of the second electronic device, after adding the third user of the third electronic device to the multi-user communication session. Figures 9E to 9FThe user interface 434a is shown, and the third electronic device displays the application's user interface at a first location in the corresponding three-dimensional environment of the third electronic device via one or more third displays of the third electronic device, wherein the first location in the corresponding three-dimensional environment corresponds to a first shared location in the shared spatial coordinate system of the physical environment, such as the location in the third three-dimensional environment 450C by the third electronic device 101c corresponding to the same location in the physical environment 400 associated with the display of the user interface 434a via the first electronic device 101a. Figures 9E to 9F The user interface 434a is shown.

[0230] Additionally or alternatively, in some examples, detecting the juxtaposition of a third user of a third electronic device and a first user of a first electronic device in a physical environment includes detecting prior interactions between the first user and the third user in the physical environment. For example, first electronic device 101a determines that first user 402 and third user 406 are in... Figures 9B to 9C The first electronic device 101a determines that the first user 402 and the third user 406 were previously interacting with each other in the physical environment 400. Additionally or alternatively, in some examples, detecting that the first user and the third user had previously interacted in the physical environment includes detecting that the first user and the third user were previously in a corresponding multi-user communication session when they were juxtaposed in the physical environment. For example, the first electronic device 101a determines that the first user 402 and the third user 406 were in... Figures 9B to 9C In the physical environment 400, the first electronic device 101a determines that the first user 402 and the third user 406 were previously parties to a second type of multi-user communication session with each other when they are juxtaposed in the physical environment 400. Additionally or alternatively, in some examples, the first electronic device includes a first head-mounted device, such as... Figure 9A The first electronic device 101a, and the second electronic device include a second head-mounted device, such as Figure 9A The second electronic device 101b, and the third electronic device includes a third head-mounted device, such as Figure 9F The third electronic device 101c in the system.

[0231] Some examples of this disclosure relate to an electronic device including: one or more processors; a memory; and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described above. Some examples of this disclosure relate to a non-transitory computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by one or more processors of the electronic device, cause the electronic device to perform any of the methods described above. Some examples of this disclosure relate to an electronic device including one or more processors, a memory, and components for performing any of the methods described above. Some examples of this disclosure relate to an information processing apparatus used in an electronic device, the information processing apparatus including components for performing any of the methods described above.

[0232] Figures 11A to 11O Examples of first electronic devices performing different position tracking processes together with different electronic devices are generally illustrated according to some examples of this disclosure.

[0233] In some examples, a first electronic device performs a position tracking process together with a second electronic device in response to detecting that the second electronic device is juxtaposed with the first electronic device in a physical environment. In some examples, performing the position tracking process together with the second electronic device includes establishing a shared spatial coordinate system of the physical environment between the first and second electronic devices. In some examples, in response to establishing the shared spatial coordinate system of the physical environment between the first and second electronic devices, the first electronic device determines a first shared reference origin (e.g., a first shared anchor point), and defines a first pose (e.g., positioning and / or orientation) of the first electronic device in the physical environment based on this first shared reference origin, using the shared spatial coordinate system of the physical environment established between the first and second electronic devices. The first shared reference origin of the first electronic device is associated with a reference position and / or orientation in the physical environment. For example, the first electronic device determines its first shared reference origin using the shared spatial coordinate system of the physical environment established between the first and second electronic devices. In some examples, the first electronic device sends its first shared reference origin to the second electronic device. In some examples, the first electronic device sends its first pose to the second electronic device, and the second electronic device determines its pose in its local coordinate system. In some examples, the second electronic device sends its second shared reference origin to the first electronic device. In some examples, the second electronic device sends its second pose to the first electronic device, and the first electronic device determines its pose in its local coordinate system. In some examples, the first electronic device maintains both its first and second shared reference origins even after establishing a shared spatial coordinate system with the second electronic device. In some examples, if the first electronic device establishes a shared spatial coordinate system with a third electronic device after establishing one with the second electronic device, the first electronic device sends its first shared reference origin instead of using the shared spatial coordinate system between the first and third electronic devices to determine its updated first shared reference origin. Additionally, sending the second shared reference origin to the third electronic device allows the third electronic device to determine its pose in its local coordinate system without requiring the second and third electronic devices to directly establish a shared spatial coordinate system with each other, thereby reducing the processing power involved in position tracking performed by the third electronic device to track the position of the second electronic device in its local coordinate system.

[0234] exist Figure 11AIn this context, the first user 402 of the first electronic device 101a is located in the physical environment 400. From... Figures 11A to 11B The second user 404 of the second electronic device 101b enters the physical environment 400 where the first user 402 of the first electronic device 101a is located. From Figures 11B to 11C The first electronic device 101a detects that the second electronic device 101b is juxtaposed with the first electronic device 101a in the physical environment 400 (e.g., the first electronic device 101a detects that the second user 404 of the second electronic device 101b is juxtaposed with the first user 402 of the first electronic device 101a in the physical environment 400), such as by referring to the physical environment of the first electronic device 101a and the second electronic device 101b (e.g., Figure 4A The physical environment 400) is described as juxtaposed (e.g., determined to be juxtaposed). Alternatively, in some examples, from Figures 11B to 11C The second user 404 enters the physical environment 400 with the second electronic device 101b without wearing the second electronic device 101b, and then wears (e.g., puts on) the second electronic device 101b, which optionally activates and / or initiates a process for determining whether the first electronic device 101a and the second electronic device 101b (e.g., the first user 402 of the first electronic device 101a and the second user 404 of the second electronic device 101b) are juxtaposed in the physical environment 400.

[0235] In response to the detection that the first user 402 and the second user 404 are juxtaposed in the physical environment 400, the first electronic device 101a and the second electronic device 101b establish a shared spatial coordinate system for the physical environment 400 (e.g., perform a position tracking process on each other), such as Figure 11C The glyph 1102a indicates this. In some examples, establishing a shared spatial coordinate system for the physical environment 400 between the first electronic device 101a and the second electronic device 101b includes performing one or more of the operations described above (e.g., referring to...). Figure 4C (For example, Figure 4C Boxes 414, 416, 418 and / or 420) and / or references Figure 7C (For example, Figure 7C The character shape in the middle is 720).

[0236] like Figure 11DAs shown, after establishing a shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b, the first electronic device 101a uses the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b to determine a first shared reference origin 1106a of the first electronic device 101a (e.g., a first shared anchor point or a first shared origin of the first electronic device 101a), as indicated by glyph 1102b and as shown in top view 716a. In some examples, the first shared reference origin 1106a of the first electronic device 101a is associated with a position and / or orientation in the physical environment 400. For example, the first shared reference origin 1106a of the first electronic device 101a corresponds to a reference position and / or reference orientation in the physical environment 400. In some examples, the first electronic device 101a defines the attitude (e.g., positioning and / or orientation) of the first electronic device 101a relative to the first shared reference origin 1106a of the first electronic device 101a in the physical environment 400.

[0237] In addition, such as Figure 11D As shown, after establishing a shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b, the second electronic device 101b uses the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b to determine its second shared reference origin 1106b (e.g., a second shared anchor point or a second shared origin of the second electronic device 101b), as indicated by glyph 1102c and shown in top view 716b. In some examples, the second shared reference origin 1106b of the second electronic device 101b is associated with a position and / or orientation in the physical environment 400. For example, the second shared reference origin 1106b of the second electronic device 101b corresponds to a reference position and / or reference orientation in the physical environment 400. In some examples, the second electronic device 101b defines its attitude (e.g., positioning and / or orientation) relative to its second shared reference origin 1106b in the physical environment 400. In some examples, a first shared reference origin 1106a of a first electronic device 101a is associated with a first location and / or orientation in a physical environment 400, and a second shared reference origin 1106b of a second electronic device 101b is associated with a second location and / or orientation in a physical environment 400 that is different from the first location and / or orientation in the physical environment 400.

[0238] exist Figure 11EIn this process, after determining the first shared reference origin 1106a of the first electronic device 101a, the first electronic device 101a sends the first shared reference origin 1106a to the second electronic device 101b, as shown in glyph 1102d. In response to receiving the first shared reference origin 1106a of the first electronic device 101a (e.g., data indicating the first shared reference origin 1106a of the first electronic device 101a), the second electronic device 101b records (e.g., decodes and / or translates) the first shared reference origin 1106a of the first electronic device 101a in its own coordinate system (e.g., a local map of the physical environment 400), as shown in top view 716b. In some examples, the coordinate system of the second electronic device 101b is private to the first electronic device 101a and / or not shared with it. The coordinate system of the second electronic device 101b differs from the shared spatial coordinate system of the physical environment 400 established between the first electronic device 101a and the second electronic device 101b. For example, the second electronic device 101b decodes (e.g., translates) the first shared reference origin 1106a of the first electronic device 101a into the coordinate system of the second electronic device 101b. This first shared reference origin may optionally be encoded in the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b. Therefore, if a shared spatial coordinate system of the physical environment between the first electronic device 101a and the second electronic device 101b is not established after decoding the first shared reference origin 1106a of the first electronic device 101a into the coordinate system of the second electronic device 101b, the second electronic device 101b still knows the position of the first shared reference origin 1106a of the first electronic device 101a within the physical environment 400, because the second electronic device 101b records the position of the first shared reference origin 1106a of the first electronic device 101a in its own coordinate system.

[0239] Similarly, in Figure 11EIn this process, after determining the second shared reference origin 1106b of the second electronic device 101b, the second electronic device 101b sends the second shared reference origin 1106b to the first electronic device 101a, as shown in glyph 1102e. In response to receiving the second shared reference origin 1106b of the second electronic device 101b (e.g., data indicating the second shared reference origin 1106b of the second electronic device 101b), the first electronic device 101a records (e.g., decodes and / or translates) the second shared reference origin 1106b of the second electronic device 101b in its own coordinate system (e.g., a local map of the physical environment 400), as shown in top view 716a. In some examples, the coordinate system of the first electronic device 101a is private to the second electronic device 101b and / or not shared with it. The coordinate system of the first electronic device 101a differs from the shared spatial coordinate system of the physical environment 400 established between the first electronic device 101a and the second electronic device 101b. For example, the first electronic device 101a decodes (e.g., translates) the second shared reference origin 1106b of the second electronic device 101b into the coordinate system of the first electronic device 101a. This second shared reference origin may optionally be encoded in the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b. Therefore, if a shared spatial coordinate system of the physical environment between the first electronic device 101a and the second electronic device 101b is not established after decoding the second shared reference origin 1106b of the second electronic device 101b into the coordinate system of the first electronic device 101a, the first electronic device 101a still knows the position of the second shared reference origin 1106b of the second electronic device 101b within the physical environment 400, because the first electronic device 101a records the position of the first shared reference origin of the second electronic device 101b in its coordinate system.

[0240] In some examples, in reference Figures 11B to 11E As described above, after performing the position tracking process together with the first electronic device 101a (e.g., after establishing a shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b and / or when the shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b is no longer established and / or updated), the second electronic device 101b maintains the first shared reference origin 1106a of the first electronic device 101a (e.g., the positioning and / or orientation in the physical environment 400 corresponding to the first shared reference origin 1106a of the first electronic device 101a). In some examples, in the reference... Figures 11B to 11EAfter the position tracking process is performed together with the second electronic device 101b as described above (e.g., after establishing a shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b and / or when the shared spatial coordinate system of the physical environment between the first electronic device 101a and the second electronic device 101b is no longer established and / or updated), the first electronic device 101a maintains the second shared reference origin 1106b of the second electronic device 101b (e.g., the positioning and / or orientation in the physical environment 400 corresponding to the second shared reference origin 1106b of the second electronic device 101b).

[0241] In addition, Figure 11E In this context, after determining a first shared reference origin 1106a for a first electronic device 101a (e.g., in response to) this first shared reference origin 1106a, the first electronic device 101a uses (e.g., relative to) its first shared reference origin 1106a to calculate a first attitude (e.g., positioning and / or orientation) of itself in the physical environment 400, and then transmits this first attitude to other electronic devices (e.g., other electronic devices in the physical environment 400), such as as shown in glyph 1102f. For example, in Figure 11E In this process, the first electronic device 101a transmits its first pose to the second electronic device 101b. In some examples, even if a shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b is no longer established (e.g., regardless of whether a shared spatial coordinate system of the physical environment 400 between the first electronic device 101a and the second electronic device 101b is still established), the first electronic device 101a also transmits (e.g., broadcasts) its pose relative to a...

Claims

1. A method, the method comprising: At a first electronic device communicating with one or more first displays and one or more first input devices, wherein the first electronic device and a second electronic device are juxtaposed in a physical environment, wherein the first electronic device has a first orientation relative to a first reference origin in the physical environment, and wherein the second electronic device has a second orientation relative to a second reference origin in the physical environment: Before establishing a multi-user communication session between the first electronic device and the second electronic device: The shared reference origin in the physical environment is determined based on the following: First map data determined by the first electronic device; as well as The second map data determined by the second electronic device; After determining the shared reference origin, first information is received from the second electronic device, the first information including: The second attitude of the second electronic device relative to the second reference origin of the second electronic device; and The offset of the second reference origin of the second electronic device relative to the shared reference origin; and User interface elements are displayed at locations based on the first information via one or more first displays, and these user interface elements can be selected to establish the multi-user communication session between the first electronic device and the second electronic device.

2. The method of claim 1, wherein the first electronic device and the second electronic device are juxtaposed based on determining that the second electronic device is in the field of view of the first electronic device in the physical environment.

3. The method of claim 2, wherein the first electronic device and the second electronic device are juxtaposed based on determining that the user of the second electronic device is in the contact list of the first electronic device.

4. The method of claim 1, wherein the first electronic device and the second electronic device are juxtaposed based on determining that the second electronic device is within a signal-based distance range of the first electronic device in the physical environment.

5. The method of claim 1, wherein the first electronic device and the second electronic device are juxtaposed based on determining that the second electronic device is within a threshold distance of the first electronic device.

6. The method of claim 1, wherein determining the shared reference origin is performed according to at least one of the following: The second electronic device is first determined within the signal-based distance range of the first electronic device in the physical environment. The second determination of the second electronic device within the threshold distance of the first electronic device, and The user of the second electronic device is identified in the contact list of the first electronic device.

7. The method of claim 1, wherein the position of displaying the user interface element corresponds to a corresponding position in the physical environment, the corresponding position being located between a first position of the first electronic device in the physical environment and a second position of the second electronic device in the physical environment, as viewed from the first electronic device.

8. The method of claim 1, wherein the second pose of the second electronic device relative to the second reference origin of the second electronic device included in the first information is a first corresponding pose of the second electronic device at a first time, and wherein the method comprises: After receiving the first information from the second electronic device and after displaying the user interface element at the location based on the first information, updated first information is received from the second electronic device, the updated first information including: The attitude of the second electronic device at a second time after the first time, wherein the attitude of the second electronic device at the second time is a second corresponding attitude of the second electronic device relative to the second reference origin of the second electronic device, which is different from the first corresponding attitude; and The offset of the second reference origin of the second electronic device relative to the shared reference origin; and The user interface elements are displayed at corresponding locations based on the updated first information via the one or more first displays.

9. The method of claim 1, wherein determining the shared reference origin comprises: Identify multiple potential shared reference origins; as well as The shared reference origin is selected from the plurality of potential shared reference origins.

10. The method of claim 1, wherein the method is part of a first position detection process for determining the position of the second electronic device and is not part of a second position detection process for determining the position of the second electronic device, and wherein the method comprises: Input corresponding to the selection of the user interface element is detected via the one or more first input devices; as well as In response to detecting the input corresponding to the selection of the user interface element, a second position detection process for determining the position of the second electronic device is executed, without executing the first position detection process for determining the position of the second electronic device.

11. The method of claim 1, wherein the second electronic device communicates with one or more second displays, and wherein the method comprises: When the user interface element is displayed at the location based on the first information, input corresponding to the selection of the user interface element is detected via the one or more first input devices; In response to detecting the input corresponding to the selection of the user interface element, the multi-user communication session is established between the first electronic device and the second electronic device; as well as When the multi-user communication session between the first electronic device and the second electronic device is active, shared virtual content is displayed at a corresponding location in the physical environment via the one or more first displays, wherein the shared virtual content is also displayed at the corresponding location in the physical environment via the one or more second displays.

12. The method of claim 1, wherein the first electronic device is also juxtaposed with the third electronic device in the physical environment, wherein the third electronic device has a third pose relative to a third reference origin in the physical environment, and wherein the method comprises: When a multi-user communication session has not yet been established between the first electronic device and the third electronic device: The corresponding shared reference origin in the physical environment is determined based on the following: The first map data detected by the first electronic device; as well as Third map data received from the third electronic device; After determining the corresponding shared reference origin, corresponding information is received from the third electronic device, the corresponding information including: The third posture of the third electronic device relative to the third reference origin of the third electronic device; and The corresponding offset of the third reference origin of the third electronic device relative to the corresponding shared reference origin; and A corresponding user interface element is displayed at a corresponding location based on the corresponding information via one or more first displays, and the corresponding user interface element can be selected to establish the multi-user communication session between the first electronic device and the third electronic device.

13. The method according to claim 1, wherein the method comprises: When the user interface element is displayed, the selection of the user interface element is detected via the one or more first input devices; as well as In response to detecting the selection of the user interface element, a multi-user communication session is established between the first electronic device and the second electronic device, wherein the multi-user communication session between the first electronic device and the second electronic device is a first multi-user communication session between the first electronic device and the second electronic device.

14. The method according to claim 13, wherein the method comprises: When in the first multi-user communication session with the second electronic device, the first anchor point of the coordinate system of the physical environment of the first electronic device is marked by the indicator of the second user of the second electronic device; as well as After marking the first anchor point of the coordinate system of the physical environment of the first electronic device with the indicator of the second user of the second electronic device, the first multi-user communication session is stopped.

15. The method of claim 14, wherein the method comprises: After the first multi-user communication session is stopped: Based on the determination that the first electronic device and the second electronic device are juxtaposed in the physical environment after the first multi-user communication session is stopped, the first process is executed; as well as Based on the determination that the first electronic device and the second electronic device are not juxtaposed in the physical environment after the first multi-user communication session is terminated, a second process, different from the first process, is executed.

16. The method of claim 15, wherein determining that the first electronic device and the second electronic device are juxtaposed in the physical environment after the first multi-user communication session is terminated includes one or more of the following: Determine the contact list of the second user on the first electronic device after the first multi-user communication session is terminated; Determine that after the first multi-user communication session is terminated, the second electronic device is within signal-based distance range of the first electronic device in the physical environment; It is determined that the elapsed time since the first multi-user communication session ended is less than a threshold time. as well as It is determined that after the first multi-user communication session is terminated, the first anchor point of the coordinate system of the physical environment of the first electronic device is still marked with the indicator of the second user of the second electronic device.

17. The method of claim 16, wherein: Determining that the first anchor point in the coordinate system of the physical environment of the first electronic device still bears the indicator of the second user of the second electronic device after the first multi-user communication session has ended includes determining that the amount of time elapsed since the first multi-user communication session ended is less than the threshold amount of time; and Determining that the first anchor point of the coordinate system of the physical environment of the first electronic device still does not bear the indicator of the second user of the second electronic device after the first multi-user communication session is terminated includes determining that the amount of time elapsed since the first multi-user communication session was terminated is not less than the threshold amount of time.

18. The method of claim 15, wherein: After the first multi-user communication session is stopped: The first electronic device and the second electronic device are juxtaposed in the physical environment; The first electronic device has a first corresponding attitude relative to a first corresponding reference origin in the physical environment; and The second electronic device has a second corresponding attitude relative to a second corresponding reference origin in the physical environment; and The first process includes: Before establishing a second multi-user communication session between the first electronic device and the second electronic device: The corresponding shared reference origin in the physical environment is determined based on the following: The first corresponding map data determined by the first electronic device; and The second corresponding map data determined by the second electronic device; After determining the corresponding shared reference origin, first corresponding information is received from the second electronic device, the first corresponding information including: The second corresponding attitude of the second electronic device relative to the second corresponding reference origin of the second electronic device; and The corresponding offset of the second corresponding reference origin of the second electronic device relative to the corresponding shared reference origin; and The user interface element is displayed at a corresponding location based on the first corresponding information via one or more first displays, and the user interface element can be selected to establish a second multi-user communication session between the first electronic device and the second electronic device.

19. A first electronic device, the first electronic device comprising: One or more processors; as well as A memory, wherein the one or more processors are configured to execute one or more programs stored in the memory, and wherein the one or more programs include instructions for performing the method according to any one of claims 1 to 18.

20. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by one or more processors of a first electronic device, cause the first electronic device to perform the method according to any one of claims 1 to 18.

Images