Physical companion device for use with an extended reality system

Abstract

Implementations of the subject technology provide a small, portable physical object for use in an extended reality system. The system can include a device that allows a user / wearer to interact with a virtual representation of content, such as stored data and / or applications overlaid on the user's physical environment. The object has a unique identifier that can be obtained by the device or another device. The object can be a non-active device without any internal circuitry, a passive device with circuitry activated by another device, or an active device with its own processing circuitry and / or a display housed in the body of the object. The object can be associated with content displayed by the device to provide a user with a physical object that can be moved or manipulated to move, modify, transport, or store the content generated, stored, and / or displayed in the extended reality environment.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 011,980, filed April 17, 2020, entitled “Physical Companion Devices For Use With Computer-Generated Reality Systems,” the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] This instruction manual pertains to extended reality environments. Background Technology

[0004] Augmented reality (AR) technology aims to bridge the gap between the virtual and physical environments by providing an augmented physical environment enhanced by electronic information. Therefore, the electronic information appears as part of the physical environment as the user perceives it when viewing it via AR. However, the electronic information does not persist in the physical environment when the user is not viewing it via AR. Attached Figure Description

[0005] Some features of this subject matter are shown in the appended claims. However, for illustrative purposes, several embodiments of this subject matter are illustrated in the following figures.

[0006] Figure 1 An exemplary system architecture including various electronic devices capable of implementing the system of this subject matter, according to one or more embodiments, is shown.

[0007] Figure 2 A block diagram illustrating exemplary features of an electronic device and its accessories according to one or more embodiments is shown.

[0008] Figure 3 An example of the physical environment of an electronic device and one or more accessory devices according to embodiments of the subject matter is shown.

[0009] Figure 4 Aspects of an accessory device associated with stored content according to one or more embodiments of the present subject matter are shown.

[0010] Figure 5 Aspects of an accessory device already associated with stored content, according to one or more embodiments of the present subject matter, are shown.

[0011] Figure 6 This paper illustrates various aspects of an accessory device already associated with stored content, which enters the view of an extended reality device according to one or more embodiments of the subject matter.

[0012] Figure 7 A portable device with a unique identifier is shown according to an embodiment of the present subject matter.

[0013] Figure 8 An accessory device having a portable body and communication circuitry according to an embodiment of the subject matter is shown.

[0014] Figure 9 An accessory device having a portable body and processing circuitry according to an embodiment of the present subject art is shown.

[0015] Figure 10 An accessory device having a portable body and a display mounted to the portable body, according to an embodiment of the present subject matter, is shown.

[0016] Figures 11 to 13 Other ways of associating accessories with the content of an extended reality system according to one or more embodiments of the subject matter are shown.

[0017] Figure 14 An accessory device is shown that provides another electronic device with access to content according to one or more embodiments of the subject matter.

[0018] Figure 15 A flowchart illustrating an exemplary process for associating content from an extended reality system with an accessory device, according to an embodiment of the present subject matter, is shown.

[0019] Figure 16 An electronic system is shown that can be used to implement one or more embodiments of the subject matter. Detailed Implementation

[0020] The specific embodiments shown below are intended to describe various configurations of the subject matter and are not intended to represent the only configuration in which the subject matter can be practiced. The accompanying drawings are incorporated herein and form part of the specific embodiments. The specific embodiments include particular details intended to provide a thorough understanding of the subject matter. However, the subject matter is not limited to the specific details set forth herein, but may be practiced in one or more other embodiments. In one or more embodiments, structures and components are shown in block diagram form to avoid obscuring the concepts of the subject matter.

[0021] A physical environment refers to the physical world that people can sense and / or interact with without the aid of electronic devices. A physical environment can include physical features such as physical surfaces or physical objects. For example, a physical environment corresponds to a physical park that includes physical trees, physical buildings, and physical people. People can directly sense and / or interact with a physical environment through senses such as sight, touch, hearing, taste, and smell. Conversely, an extended reality (XR) environment refers to a fully or partially simulated environment that people sense and / or interact with via electronic devices. For example, an XR environment can include augmented reality (AR) content, mixed reality (MR) content, virtual reality (VR) content, etc. In the case of an XR system, a subset of a person's physical motion or a representation thereof is tracked, and in response, one or more features of one or more virtual objects simulated in the XR system are adjusted in a manner consistent with at least one physical law. For example, an XR system can detect head movement and, in response, adjust the graphical content and sound field presented to the person in a manner similar to how such views and sounds change in a physical environment. For example, an XR system can detect movement of electronic devices (e.g., mobile phones, tablets, laptops, etc.) that present the XR environment, and in response, adjust the graphical content and sound field presented to the user in a manner similar to how such views and sounds would change in a physical environment. In some cases (e.g., for accessibility reasons), an XR system may adjust the characteristics of the graphical content in the XR environment in response to representations of physical motion (e.g., voice commands).

[0022] Many different types of electronic systems enable people to sense and / or interact with a variety of XR environments. Examples include head-mounted systems, projection-based systems, head-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays shaped like lenses designed to be placed on a person's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without haptic feedback), smartphones, tablets, and desktop / laptop computers. Head-mounted systems may have an integrated opaque display and one or more speakers. Alternatively, head-mounted systems may be configured to receive external opaque displays (e.g., smartphones). Head-mounted systems may incorporate one or more imaging sensors for capturing images or video of the physical environment, and / or one or more microphones for capturing audio of the physical environment. Head-mounted systems may have transparent or semi-transparent displays instead of opaque displays. Transparent or semi-transparent displays may have a medium through which light representing the image is directed to the person's eyes. The display can utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser scanning light source, or any combination of these technologies. The medium can be an optical waveguide, holographic medium, optical combiner, optical reflector, or any combination thereof. In some embodiments, a transparent or translucent display can be configured to selectively become opaque. Projection-based systems can employ retinal projection techniques that project graphic images onto the human retina. Projection systems can also be configured to project virtual objects onto a physical environment, such as as holograms or on a physical surface.

[0023] The embodiments of the subject matter described herein provide extended reality systems to utilize content generated and / or stored within an XR environment, which can be associated with one or more physical companion devices. The systems and methods disclosed herein associate system-generated and / or stored content with physical companion devices such that the association persists even when the XR components of the system are idle or powered off. In this way, users of an XR system can be provided with a portable companion device (sometimes referred to herein as a “stone” or “go stone”), which the user can carry with them to deliver content from the XR environment within the physical world. The associated content can later be accessed by the same XR system when the XR system detects the companion device, and / or by other devices and / or XR systems when those devices and / or systems detect the companion device. In this way, users of an XR system can be provided with the ability to carry content (such as stacks of digital photos, active documents, or application status and / or digital currency (as an example)) within the physical world for later access by various devices or systems.

[0024] In various implementations, the accessory device may be an passive device having a unique identifier that can be detected by a camera of another device, a passive device having a stored unique identifier that can be transmitted using wireless power from another device, or an active device having processing circuitry and / or memory for processing the identifier, content, etc., and storing and / or transmitting them.

[0025] Figure 1 An exemplary system architecture 100 comprising various electronic devices capable of implementing the present subject matter, according to one or more embodiments, is shown. However, not all depicted components are usable in all embodiments, and one or more embodiments may include additional or different components compared to those shown in the figures. Variations in the arrangement and type of these components may be made without departing from the spirit or scope of the claims set forth herein. Additional components, different components, or fewer components may be provided.

[0026] System architecture 100 includes electronic devices 105, handheld electronic devices 104, electronic devices 110, electronic devices 115, supporting equipment 160, and a server 120. For illustrative purposes, system architecture 100 is described in... Figure 1 The system is shown to include electronic device 105, handheld electronic device 104, electronic device 110, electronic device 115, accessory device 160, and server 120; however, system architecture 100 may include any number of electronic devices, accessory devices, and any number of servers or a data center including multiple servers.

[0027] Electronic device 105 may be a smartphone, tablet, or head-mounted portable system (e.g., a head-mounted display device that can be worn by the user) that includes a display system capable of presenting a visualization of an extended reality environment to the user. Electronic device 105 may be powered by a battery and / or another power source. In one example, the display system of electronic device 105 provides the user with a stereoscopic presentation of an extended reality environment, enabling the provision of a three-dimensional visual display rendering a specific scene. In one or more embodiments, instead of utilizing electronic device 105 to access the extended reality environment or otherwise, the user may use a handheld electronic device 104, such as a tablet, watch, mobile device, etc.

[0028] Electronic device 105 may include one or more cameras, such as camera 150 (e.g., visible light camera, infrared camera, etc.). Furthermore, electronic device 105 may include various sensors 152, including but not limited to cameras, image sensors, touch sensors, microphones, inertial measurement units (IMUs), heart rate sensors, temperature sensors, lidar sensors, radar sensors, sonar sensors, GPS sensors, Wi-Fi sensors, near-field communication sensors, radio frequency sensors, etc. Additionally, electronic device 105 may include hardware elements capable of receiving user input, such as hardware buttons or switches. User input detected by such sensors and / or hardware elements corresponds to various input modalities for initiating the generation of supplementary virtual content within a given extended reality environment. For example, such input modalities may include, but are not limited to, face tracking, eye tracking (e.g., gaze direction), hand tracking, posture tracking, biometric readings (e.g., heart rate, pulse, pupil dilation, respiration, temperature, electroencephalogram, olfaction), speech or audio recognition (e.g., specific hot words), and activation of buttons or switches, etc. Electronic device 105 can also detect the presence of people or objects in the scene and / or the occurrence of events to initiate the provision of supplementary virtual content within the extended reality environment.

[0029] Electronic device 105 may be communicatively coupled to infrastructure devices, such as electronic device 110 and / or electronic device 115. Generally, such infrastructure devices may include more computing resources and / or available power compared to electronic device 105. In one example, electronic device 105 may operate in various modes. For example, electronic device 105 may operate in an independent mode, independent of any infrastructure device. When electronic device 105 operates in independent mode, the number of input modes may be constrained by the power and / or processing limitations of electronic device 105, such as the device's available battery power. In response to power limitations, electronic device 105 may deactivate certain sensors within the device itself to conserve battery power and / or free up processing resources.

[0030] Electronic device 105 can also operate in a wireless connection mode (e.g., connected to a base device via a wireless connection), thereby working in conjunction with a given base device. Electronic device 105 can also operate in a connection mode where electronic device 105 is physically connected to a base device (e.g., via a cable or some other physical connector), and can utilize power resources provided by the base device (e.g., when the base device is charging electronic device 105 while physically connected).

[0031] When electronic device 105 operates in wireless or connected mode, at least a portion of processing user input and / or rendering the extended reality environment can be offloaded to the base device, thereby reducing the processing burden on electronic device 105. For example, in one embodiment, electronic device 105 works in conjunction with electronic device 110 or 115 to generate an extended reality environment comprising physical and / or virtual objects that enable various forms of interaction (e.g., visual, auditory, and / or physical or tactile interaction) between the user and the generated extended reality environment in real time. In one example, electronic device 105 provides a rendering of a scene corresponding to the extended reality environment, which can be perceived and interacted with by the user in real time. Additionally, as part of presenting the rendered scene, electronic device 105 may provide the user with auditory and / or tactile or haptic feedback. The content of a given rendered scene may depend on available processing power, network availability and capacity, available battery power, and the current system workload.

[0032] In one implementation, electronic device 105 detects accessory device 160 in the vicinity of electronic device 105 (e.g., within the range of one or more sensors 152 and / or within the field of view of one or more cameras such as camera 150 of electronic device 105). Electronic device 105 may detect a unique identifier of accessory device 160 (e.g., using camera 150, sensor 152, and / or communication circuitry within electronic device 105). Electronic device 105 may determine (e.g., via communication with memory at accessory device 160, server 120, electronic device 105, and / or any other device) whether any content stored at accessory device 160, server 120, electronic device 105, and / or any other device is associated with accessory device 160.

[0033] For example, if the accessory device 160 is a passive device, the electronic device 105 can obtain the identifier of the accessory device 160 and obtain content previously associated with the accessory device 160, which is stored in the electronic device 105, the server 120, and / or another device using that identifier. In other examples, a key (e.g., a decryption key) for accessing the content can be obtained from the accessory device 160, and / or the key can be obtained and / or derived based on the identifier of the accessory device 160. In another example, the electronic device 105 can obtain content stored at the accessory device 160. In other examples, the electronic device 105 can assign and / or transfer content not yet associated with the accessory device to the accessory device 160.

[0034] Network 106 can communicatively (directly or indirectly) couple electronic devices 105, 110, 160 and / or 115 to each other device and / or server 120. In one or more embodiments, network 106 may be an interconnection network of devices that may include the Internet or be communicatively coupled to the Internet.

[0035] Electronic device 110 may include a touchscreen and may be, for example, a smartphone including a touchscreen, a portable computing device such as a laptop computer including a touchscreen, a companion device including a touchscreen (e.g., a digital camera, headphones), a tablet device including a touchscreen, a wearable device including a touchscreen (e.g., a watch, wristband, etc.), any other suitable device including, for example, a touchscreen, or any electronic device with a touchpad. In one or more embodiments, electronic device 110 may not include a touchscreen but may support touchscreen-like gestures, such as in extended reality environments. In one or more embodiments, electronic device 110 may include a touchpad. Figure 1 In this context, by way of example, electronic device 110 is depicted as a mobile smartphone device with a touchscreen. In one or more embodiments, electronic device 110, handheld electronic device 104, and / or electronic device 105 may be and / or may include all or part of the electronic device described below with respect to an electronic system, which is described below with respect to an electronic system. Figure 16 In one or more embodiments, the electronic device 110 may be another device, such as an Internet Protocol (IP) camera, a tablet computer, or an accessory such as an electronic stylus.

[0036] Electronic device 115 may be, for example, a desktop computer, portable computing devices such as laptops, smartphones, accessories (e.g., digital cameras, headphones), tablet devices, wearable devices such as watches, wristbands, etc. Figure 1 In this context, by way of example, electronic device 115 is described as a desktop computer. Electronic device 115 may be and / or may include, as described below relative to... Figure 16 The electronic system under discussion, in whole or in part.

[0037] Server 120 may form all or part of a computer network or server group 130, such as in a cloud computing or data center implementation. For example, server 120 stores data and software and includes specific hardware (e.g., processors, graphics processors, and other dedicated or custom processors) for rendering and generating extended reality content such as graphics, images, videos, audio, and multimedia files. In one implementation, server 120 may function as a cloud storage server that stores any of the aforementioned extended reality content generated by the aforementioned devices and / or server 120.

[0038] Figure 2 Block diagrams are shown of various components that may be included in electronic device 105 and accessory device 160 according to various aspects of this disclosure. Figure 2 As shown, electronic device 105 may include one or more cameras, such as camera 150, for capturing images of the physical environment surrounding the electronic device, and one or more sensors 152 for obtaining environmental information associated with the physical environment surrounding electronic device 105. Sensors 152 may include depth sensors (e.g., time-of-flight sensors, infrared sensors, radar, sonar, lasers, etc.), one or more microphones, and / or other types of sensors for sensing the physical environment. In some embodiments, electronic device 105 also includes communication circuitry 208 for communicating with companion device 160, electronic device 110, electronic device 115, server 120, and / or other devices and / or systems. Communication circuitry 208 may include radio frequency (RF) communication circuitry for detecting radio frequency identification (RFID) tags, Bluetooth Low Energy (BLE) communication circuitry, other near field communication (NFC) circuitry, WiFi communication circuitry, cellular communication circuitry, and / or other wired and / or wireless communication circuitry.

[0039] As shown in the figure, electronic device 105 includes processing circuitry 204 (e.g., one or more processors and / or integrated circuits) and memory 206. Memory 206 may store instructions or code for execution by processing circuitry 204. Memory 206 may store (e.g., temporarily or permanently) content generated and / or otherwise acquired by electronic device 105. In some operational scenarios, memory 206 may store one or more ancillary devices (such as...) associated with portions of the content stored in memory 206. Figure 2 The identifier of the supporting equipment (160).

[0040] Processing circuitry 204 can generate display content for display by display 200. The display content may include images of portions of the physical environment and / or computer-generated content, such as virtual reality content, augmented reality content, mixed reality content, etc. The display content may include one or more indicators of content stored in memory 206. For example, when a user opens a digital photograph stored in memory 206 via electronic device 105, display 200 may display a copy of the image, a thumbnail of the image, etc. As described herein, display 200 of electronic device 105 may be an opaque display providing a transparent video view (e.g., using an image from camera 150) of the physical environment surrounding electronic device 105, or may include a transparent or semi-transparent display allowing the user of electronic device 105 to directly view portions of the physical environment via the display. Display 200 may overlay computer-generated (CG) content onto the user's view of portions of the physical environment via or through the display.

[0041] To allow electronic device 105 and / or another device to associate content with accessory device 160, accessory device 160 includes an identifier 210 that is unique to the accessory device and accessible by electronic device 105 and / or the other device. As an example, identifier 210 may be a code printed or attached to the physical body of the accessory device. The code (such as an alphanumeric code, an image-based code, a quick-response (QR) code, or generally any code) may be obtained and / or derived from an image captured by camera 150 of electronic device 105 and / or the other device to identify accessory device 160. In one or more embodiments, the code may be detected and / or obtained using a specific image sensor, such as an infrared (IR) sensor.

[0042] In some embodiments, the accessory device 160 is a passive device that includes a passive identifier 210 and has no internal circuitry. In this example, the identifier 210 can be identified by a light-based sensor, such as a camera, optical sensor, IR sensor, or another sensor capable of identifying the identifier 210 using light reflected or emitted by the identifier 210 at or near the surface of the accessory device. However, in other embodiments, the accessory device 160 may be a passive device having communication circuitry 212 activated by another device, such as communication circuitry 208 of electronic device 105, or it may be an active device having its own processing circuitry 216, memory 214, battery 220, and / or a display 218 (e.g., an electronic ink display or other display) housed in the body of the accessory device. In embodiments where the accessory device 160 includes a display 218, the display 218 may be an electronic ink display, allowing the accessory device 160 to operate with very low power. In embodiments where the accessory device 160 includes processing circuitry 216 and / or communication circuitry 212, the identifier 210 may be stored in memory 214, rather than being visible on the outside of the device body. The information stored in memory 214 may be encrypted and / or password protected.

[0043] Figure 3 An example of a physical environment in which electronic device 105 and multiple supporting devices 160 are arranged is shown. Figure 3 In the example, user 101 wears electronic device 105 in physical environment 300. Figure 3In the example, physical environment 300 includes physical object 306, a portion of which can be viewed by user 101 via display 200 (e.g., based on images from one or more cameras such as camera 150, which are provided to display 200 in an opaque implementation or directly through display 200 in a transparent or semi-transparent implementation), and the portion of the physical object can be viewed directly by the user without using any technology (if it would not otherwise be obstructed from view). Figure 3 In this example, computer-generated content is displayed by monitor 200 (e.g., overlaid on or in front of a portion of the physical environment 300). In this example, monitor 200 displays application windows 304 (e.g., file manager application windows, browser windows, social media application windows, content editor application windows, or any other application user interface). In this example, the user has also created a photo stack 308 (e.g., pulling digital images into a virtual stack from a photo library accessible from electronic device 105).

[0044] like Figure 3 As indicated, each accessory device 160 may be a small (e.g., coin-sized) physical object set up for use with an XR system that includes XR devices (such as electronic devices 105) that allow wearers or users 101 to interact with virtual representations and / or applications of content such as stored data (e.g., photos, videos, audio files, text files).

[0045] like Figure 3 As shown in the example, application window 304 and photo stack 308 each have a position indicator 310 displayed by display 200, which has an approximate size and shape of one of the companion devices 160. In one example, position indicator 310 is provided with an illustrated icon 312 that visually links the position indicator to photo stack 308 (e.g., digital images). However, Figure 3 Another example is shown in which a position indicator 310 is provided without an illustrated icon, indicating the proximity of the application window 304 and the association between the position indicator 310 and the application window 304. The position indicator 310 indicates the location where the accessory device 160 can be placed to associate the accessory device with the corresponding content displayed on the display 200.

[0046] Figure 4This illustration depicts a scenario where one of the accessory devices 160 has been physically moved (e.g., by user 101) to a physical location within the physical environment 300, corresponding to the location indicated by the position indicator 310 on which the photo stack 308 is displayed by the display 200. In this scenario, when the electronic device 105 detects (e.g., using camera 150 and / or sensor 152) that the accessory device 160 is at the location corresponding to the position indicator 310, the electronic device 105 associates the accessory device 160 with the photo stack 308.

[0047] When the companion device 160 is associated with specific content (e.g., photo stack 308 in this example), the companion device 160 may store that content (e.g., in memory 214), the companion device 160 may store a link or key to content stored on another device (e.g., in the cloud at server 120), or (e.g., if the companion device 160 is a passive or non-active device) may store the association at electronic device 105 and / or in the cloud (e.g., by storing a unique identifier for the stone in conjunction with the stored content).

[0048] The association generated by the electronics 105 between the accessory device 160 and the photo stack 308 is persistently stored at the electronics 105, in the cloud (e.g., at one or more servers 120), and / or at the accessory device 160. In this way, the photo stack 308 can be transported with the accessory device 160 even when the accessory device 160 is no longer in the field of view of the camera 150 and / or within proximity to the sensor 152 and / or communication circuitry 208 of the electronics 105.

[0049] exist Figure 3 and Figure 4 In the example, a device such as electronic device 105, configured to display computer-generated content overlaid on a view of the physical environment 300, is disposed in an extended reality system. This extended display system also includes an accessory device comprising a portable unit and a unique identifier obtainable by the device. As shown, the device can display an indicator of stored content (e.g., a photo stack 308 or the status of an application corresponding to application window 304) overlaid on the view of the physical environment. The device can detect the accessory device 160 in the physical environment 300 and, in response to a user request corresponding to the displayed indicator (e.g., in this example, to move the accessory device to location indicator 310), associate the unique identifier of the accessory device with information associated with the stored content.

[0050] Figure 5This illustrates a scenario where the accessory device 160, which was previously associated with the photo stack 308, has been removed from the portion of the physical environment 300 visible through or via the display 200. In this example, the photo stack 308 is no longer displayed on the display 200, but remains persistently stored at the accessory device 160.

[0051] Figure 6 This illustrates how, at a later time, when the electronic device 105 detects the companion device 160 again at any location within the field of view corresponding to the display 200, an indicator of the photo set 308 (or the photo set 308 itself) can be displayed at that location. For example, a camera or another sensor of the electronic device 105 can obtain the identifier of the companion device 160 when (or before) the companion device 160 enters the field of view corresponding to the display 200, and based on the identity of the companion device 160, obtain the photo stack, photo list, and thumbnails of the photo stack (e.g., from local storage at the companion device 105, from the companion device 160, or from a remote storage device such as server 120), and display an indicator of the photo stack 308 and / or the photo stack 308 itself at or near the location where the companion device 160 enters the field of view corresponding to the display 200.

[0052] In addition, and in conjunction with the following text Figure 14 In more detail, if the companion device 160 is later brought into proximity to another device (e.g., the user's laptop or electronic device 105 or other devices registered to the user or another user), the companion device 160 may provide the other device with access to the photo set 308 (e.g., and / or any other content already associated with the companion device 160).

[0053] It should be understood that the combination Figures 3 to 6 The description of the content associated with accessory device 160 is an example of photo stacking, which is merely illustrative, and accessory device 160 can be associated with any other content in an XR environment. As another example, if the user of electronic device 105 is operating an application (such as a social media application) associated with application window 304, electronic device 105 may display a location indicator 310 associated with that application, and accessory device 160 or another accessory device 160 may be placed at the location indicator to associate accessory device 160 with the social media application (e.g., with the application's open user interface, with the application's current state, or with any other aspect of the application).

[0054] Other examples of content from the electronic device 105 that can be associated with the companion device 160 include, but are not limited to, information for tethering an application to the companion device 160, information for associating digital assistant actions with the companion device 160, and / or information for associating functions with the companion device 160. In one example, the electronic device 105 may associate a specific function of a particular application or website with the companion device 160. For example, the companion device 160 may be associated with an ordering or purchasing function of an e-commerce application. In this way, a user of the electronic device 105 can tap a physical object in their physical environment (i.e., the companion device 160 associated with that function) to place an order or make a purchase. Therefore, in some implementations, the companion device 160 may provide a haptic object that a user can physically interact with while in an XR environment, and various different functions can be assigned to the haptic object. This provides users with a more intuitive way to interact with the XR environment compared to systems where tactile stimulation is unavailable when interacting with CG content in the XR environment (e.g., in VR, MR, or AR systems that only offer gesture or voice control) or where the controller must be constantly held by the user to control the XR environment.

[0055] In yet another example, a set of supporting devices 160 can be used to physically sort and / or arrange digital content, such as search results obtained using electronic device 105. In this example, when a search (e.g., a local storage search, a remote storage search, or a network search such as an internet search) is performed by electronic device 105 and multiple supporting devices 160 are simultaneously near electronic device 105, each supporting device 160 can be associated with one of the multiple supporting devices 160 to allow the user to physically move and / or select supporting device 160 to sort and / or select search results.

[0056] Figures 7 to 10 Various embodiments of the supporting equipment 160 according to various aspects of this disclosure are shown. Figure 7 In this example, the accessory device 160 is implemented as a passive device having a physical body 400 (e.g., a body formed of plastic, rubber, metal, glass, and / or combinations of these and / or other materials) with an identifier 210 on its outer surface. In this example, the identifier 210 is implemented as a Quick Response (QR) code on the outer surface of the physical body 400, which can be recognized by, for example, a camera of another device (such as the camera of electronic device 105 or another electronic device). The QR code may be printed, etched, or attached to the outer surface of the physical body 400 (as an example).

[0057] Figure 8Another exemplary embodiment of the accessory device 160 is shown, wherein the communication circuitry 212 is disposed within the physical body 400. For example, the physical body 400 may form a housing of the accessory device 160 having an internal cavity in which the communication circuitry 212 is disposed. In this example, the communication circuitry 212 may be a passive communication circuitry (such as an NFC tag or RFID tag) that uses inductive power from another device to transmit an identifier of the accessory device 160, or it may be an active (e.g., locally powered) communication circuitry 212 that can be activated to transmit an identifier.

[0058] Figure 9 Another example of an accessory device 160 in one embodiment is shown, wherein the accessory device is an active device with additional circuitry (such as processing circuitry 216 and / or memory 214 and communication circuitry 212 disposed within physical body 400). In this example, memory 214 may be used to store an identifier of accessory device 160, an identifier of content stored at another device or server and associated with accessory device 160, encrypted information for accessing content stored at another device or server and associated with accessory device 160, and / or for storing content at accessory device 160. The encrypted information may be, for example, a key and / or information from which the key can be derived and / or obtained, and / or the encrypted information may be authorization information (such as information corresponding to a software license) authorizing a proximity device to access one or more applications, features, and / or functions provided by a software license.

[0059] like Figure 10 As shown, the accessory device 160 may also be implemented with a display 218, which can be operated (e.g., by processing circuitry 216 within the physical body 400 and / or by processing circuitry of another device such as electronic device 105) to display an indicator 1000 of content associated with the accessory device 160 (e.g., and stored at the accessory device 160, electronic device 105, and / or a remote server such as server 120). Figure 10 In one example, display 218 shows the text “My Photos” to indicate that the user’s digital photos have been associated with companion device 160. In other examples, display 218 may display thumbnails of one or more photos, or other indicators of other content already associated with companion device.

[0060] As in Figure 10In the examples, providing an accessory device 160 with a display 218 facilitates the use of multiple accessory devices 160. For example, a user could carry several accessory devices 160 in their pocket, which they could later easily sort through to access photos, applications, ongoing editing projects (e.g., word processing documents or presentations), currency, and / or other content already associated with the accessory device 160. In some embodiments, the display 218 may provide a limited-function user interface for the accessory device 160 (e.g., to allow the user to delete content or associate it with content (e.g., by pressing and holding the display or a button set on the physical body 400), or to enter a password to access associated content). In any of the various embodiments discussed herein, the accessory device 160 may be provided without any user input components such as buttons, touch interfaces, or any physical connectors (e.g., such that content associated with the accessory device can only be accessed by another device and is not physically coupled or connected to other devices). In some embodiments, including those in which the accessory device 160 includes processing circuitry and memory, the accessory device 160 may have limited functionality (e.g., the ability to store associated content or information for accessing associated content, but not the ability to perform user interaction, process data, execute applications, etc.).

[0061] In the various examples described herein, the physical body 400 of the accessory device 160 may be small enough that the accessory device 160 can be carried, for example, in the palm of a user's hand, or that several accessories 160 can be carried in the user's hand or in a user's pocket. For example, the physical body may have a maximum size of less than two inches, less than one inch, or less than half an inch. The physical body may have, for example, a size of less than two inches, less than one inch, or less than half an inch. Figures 7 to 10 The example may have a circular shape (e.g., a spherical shape or an elliptical shape) or may have another shape such as a straight line. The physical body 400 may be smaller than the size of a smartphone or smartwatch and may simulate the size and / or shape of a coin (e.g., a twenty-five coin, a five-cent coin, a ten-cent coin, etc.).

[0062] In combination Figures 3 to 6 In the example discussed, content accessible to the electronic device 105 is associated with the accessory device 160 by placing the accessory device 160 at a location in physical space corresponding to a clearly visible display location of the location indicator 310. However, it should be understood that content may be associated with the accessory device 160 in other ways.

[0063] Figure 11 An example is shown in which an accessory device 160, which is not associated with any content, is visible in the field of view corresponding to the area of ​​the display 200, and in which the photo stack 308 and the application window 304 have associated position indicators 310 (which should be understood to be optional).

[0064] Figure 12 An additional example is shown in which the photo stack 308 is associated with the companion device 160 by placing the companion device 160 at a location in the physical environment 300 corresponding to the display representation of the photo stack 308 on the display 200. In this example, a visual indicator 1200 (e.g., a dashed box around the photo stack or any other visual highlight or indicator) is displayed by the display 200 to indicate to the user that the photo stack 308 is associated with the companion device 160. In some embodiments, options for approving or rejecting the association may be presented to the user.

[0065] Figure 13 This illustrates yet another example of how content itself (e.g., in this example, the displayed photo stack 308) can be moved to or toward the location of the accessory device 160 (as indicated by arrow 1300) to associate content with the accessory device. For example, user 101 can provide gesture input to electronic device 105 by virtually grabbing or squeezing the displayed photo stack and dragging the displayed stack to the location of accessory device 160, or by throwing or swiping the photo stack onto accessory device 160.

[0066] As discussed herein, providing an accessory device 160 that can be associated with content from an extended reality system can facilitate the transfer of content to another device and / or access to the content by another device.

[0067] Figure 14 An example of another electronic device 1400 implemented as a laptop computer is shown, which can interact with an accessory device 160 to access content associated with the accessory device 160. For example... Figure 14 As shown, electronic device 1400 can obtain the identifier of accessory device 160 (e.g., using camera 1404 to capture an image of the code on the outer surface of the physical body 400 of accessory device 160, such as...). Figure 7 In one example, NFC communication with the communication circuitry of the companion device 160 may be used to obtain an identifier from the companion device's communication circuitry. The electronic device 1400 can then use the identifier of the companion device 160 to obtain associated content from the companion device itself or from another source (e.g., from electronic device 105 or from server 120). In some operational scenarios, the electronic device 1400 may also use the identity of the companion device 160 to directly obtain a key to the content associated with the companion device 160 from the companion device 160 or from another source. In another example, the electronic device 1400 may directly obtain the content associated with the companion device 160 from the companion device 160 itself (e.g., if the associated content is stored in the companion device's memory 214).

[0068] exist Figure 14 In the example, electronic device 1400 has already used companion device 160 to obtain the photo stack 308 and the content for displaying application window 304, because they were previously displayed by electronic device 105 and the same content is displayed on monitor 1402. Figure 14 As shown, the electronic device 1400 may also display content 1406 generated and / or obtained by the electronic device 1400 independently of the companion device 160. In some embodiments, the content 1406 may also be associated with the companion device 160 later (e.g., for later use with the electronic device 105 or for transfer to another user or another device).

[0069] Figure 15 A flowchart illustrating an exemplary process 1500 for associating content with an accessory device according to an embodiment of the subject matter is shown. For illustrative purposes, this document primarily refers to... Figure 1 , Figure 2 and Figures 7 to 10 The process 1500 is described using electronic equipment 105 and auxiliary equipment 160. However, the process 1500 is not limited to... Figure 1 , Figure 2 and Figures 7 to 10 The electronic device 105 and the supporting device 160, and one or more blocks (or operations) of process 1500 may be performed by one or more other components of other suitable devices (including electronic device 110, electronic device 115, and / or server 120). Further for illustrative purposes, some blocks of process 1500 are described herein as occurring sequentially or linearly. However, multiple blocks of process 1500 may occur in parallel. Furthermore, the blocks of process 1500 need not be performed in the order shown, and / or one or more blocks of process 1500 need not be performed and / or may be replaced by other operations.

[0070] like Figure 15 As shown, at box 1502, a device (such as electronic device 105) displays an indicator of content (e.g., photo stack 308, application window 304, other computer-generated content, etc.) overlaid on a view of the physical environment. In various examples, the content may be stored at the device or at a remote device (such as another device or server such as server 120).

[0071] At box 1504, the device obtains an identifier, such as identifier 210 of a companion device (such as companion device 160) in the physical environment. The device can do this by capturing an image of the portable subject of the companion device (e.g., using camera 150) or by receiving wireless communication from the companion device (e.g., from...). Figure 2 The identifier is obtained by using communication circuits 212 to 208.

[0072] At box 1506, in response to a user request, the device may associate the accessory device identifier with information associated with the content. The device may also determine (e.g., using camera 150 and / or sensor 152) the location of the accessory device 160 in the physical environment. An indicator displaying content overlaid on a view of the physical environment at box 1502 may include an indicator displaying content overlaid at another location in the physical environment, and a user request may include the user changing the location of the accessory device or another location of the indicator (e.g., by placing the accessory device at a location associated with the displayed indicator of the stored content, or by moving the displayed indicator of the stored content to or toward the location of the accessory device, as described above in conjunction with examples). Figures 3 to 6 and Figures 11 to 13 (As described).

[0073] Associating the identifier of the accessory device 160 with information associated with the content may include storing the accessory device identifier and the content-associated information at a remote server (such as server 120) separate from the accessory device and the device, storing the content at the accessory device, and / or storing the accessory device identifier in conjunction with the content at an electronic device 105. In one example, associating the accessory device with the content may include storing the accessory device identifier associated with the stored content at the device. In some examples, the content may be transferred to the accessory device, and the accessory device may store the stored content at its location. In some examples, the device may provide the accessory device with a key to the stored content and / or information from which keys can be derived. The key may be a decryption key, an authorization key, or other keys that facilitate access to and / or authorization of encrypted and / or unencrypted content stored at the accessory device, the device, or a remote server (such as server 120). The content may remain associated with the accessory device identifier even after the accessory device is removed from the device's view.

[0074] In embodiments where the accessory includes a display (such as an electronic ink display mounted on a portable body of the accessory), the accessory may further include a processor (such as a processor for operating the electronic ink display) Figure 2 The processor of the processing circuit 204. After the content has been associated with the accessory device, the processing circuit can operate the electronic ink display to display another indicator of the stored content associated with the unique identifier (e.g., such as...). Figure 10 (As shown).

[0075] As mentioned above, for example Figure 5 and Figure 6As described, the device can detect the companion device at a new location in the physical environment and display an indicator of stored content on a view of the physical environment overlaid at the new location (e.g., in response to the detection of the companion device using the device's sensors and / or camera and based on the new location of the detected companion device).

[0076] As mentioned above, for example Figure 14 As described above, when the accessory device is near an auxiliary device separate from the accessory device and the device, the accessory device can provide the auxiliary device (such as electronic device 1400) with access to the content.

[0077] As described above, one aspect of this technology is the collection and use of data obtainable from specific and legitimate sources to associate content with portable companion devices. This disclosure envisions that, in some instances, the collected data may include personal information data that uniquely identifies or can be used to identify a specific person. Such personal information data may include video data, three-dimensional geometric data, demographic data, location-based data, online identifiers, telephone numbers, email addresses, home addresses, biometric data or records related to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other personal information.

[0078] This disclosure recognizes that the use of such personal information data in the techniques of this invention can be used to benefit users. For example, personal information data can be used to associate content (such as CG content) with portable physical devices.

[0079] This disclosure assumes that entities responsible for collecting, analyzing, disclosing, transmitting, storing, or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, it is expected that such entities will implement and consistently apply privacy practices generally recognized as meeting or exceeding industry or governmental requirements for protecting user privacy. Such information regarding the use of personal data should be prominently displayed and easily accessible to the user, and should be updated as the collection and / or use of data changes. Users' personal information should be collected only for lawful use. Furthermore, such collection / sharing should only occur after receiving user consent or other lawful grounds provided for in applicable law. In addition, such entities should consider taking any necessary steps to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and processes. Additionally, such entities may be subject to third-party evaluations to demonstrate their compliance with widely accepted privacy policies and practices. Furthermore, policies and practices should be tailored to the specific types of personal information data collected and / or accessed, and made applicable to applicable laws and standards, including specific jurisdictional considerations that may be used to impose higher standards. For example, in the United States, the collection or access to certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while health data in other countries may be subject to other regulations and policies and should be handled accordingly.

[0080] Regardless of the foregoing, this disclosure also anticipates implementation schemes for users to selectively block the use or access to personal information data. That is, this disclosure anticipates providing hardware and / or software components to prevent or block access to such personal information data. For example, when the displayed content is associated with a physical companion device, the subject matter technology can be configured to allow users to opt-in or opt-out during or at any time after registering for the service, participating in the collection and / or sharing of personal information data. In addition to providing opt-in and opt-out options, this disclosure envisions providing notifications related to access to or use of personal information. For example, users may be notified when downloading an application that their personal information data will be accessed, and then reminded again just before the application accesses the personal information data.

[0081] Furthermore, the purpose of this disclosure is to manage and process personal information data to minimize the risk of unintentional or unauthorized access or use. Once data is no longer needed, this risk can be minimized by limiting data collection and deleting data. Additionally, and where applicable, including in certain health-related applications, data deidentification can be used to protect user privacy. Deidentification can be facilitated, where appropriate, by removing identifiers, controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than the address level or at a level insufficient for facial recognition), controlling how data is stored (e.g., aggregating data among users), and / or other methods such as differentiated privacy.

[0082] Therefore, while this disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, it is also contemplated that various embodiments can be implemented without access to such personal information data. That is, various embodiments of the present invention will not be rendered inoperable due to the absence of all or part of such personal information data.

[0083] Figure 16 An electronic system 1600 is shown that can be used to implement one or more embodiments of the subject matter technology. The electronic system 1600 may be... Figure 1 The illustrated electronic device 105, handheld electronic device 104, electronic device 110, electronic device 115, and / or server 120 are shown and / or may be part of them. The electronic system 1600 may include various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1600 includes a bus 1608, one or more processing units 1612, system memory 1604 (and / or cache), ROM 1610, persistent storage device 1602, input device interface 1614, output device interface 1606, and one or more network interfaces 1616, or subsets and variations thereof.

[0084] Bus 1608 generally represents all system, peripheral, and chipset buses that communicatively connect numerous internal devices of electronic system 1600. In one or more embodiments, bus 1608 communicatively connects the one or more processing units 1612 to ROM 1610, system memory 1604, and persistent storage device 1602. The one or more processing units 1612 retrieve instructions to be executed and data to be processed from these various memory units in order to perform the processes disclosed in this paper. In different embodiments, the one or more processing units 1612 may be a single processor or a multi-core processor.

[0085] ROM 1610 stores static data and instructions required by the one or more processing units 1612 and other modules of the electronic system 1600. Alternatively, persistent storage device 1602 can be a read-write memory device. Persistent storage device 1602 can be a non-volatile memory cell that stores instructions and data even when the electronic system 1600 is powered off. In one or more embodiments, mass storage devices (such as magnetic disks or optical disks and their corresponding disk drives) can be used as persistent storage device 1602.

[0086] In one or more embodiments, a removable storage device (such as a floppy disk, flash drive, and corresponding disk drive) may be used as persistent storage device 1602. Like persistent storage device 1602, system memory 1604 may be a read-write memory device. However, unlike persistent storage device 1602, system memory 1604 may be volatile read-write memory, such as random access memory. System memory 1604 may store any instructions and data that one or more processing units 1612 may need during operation. In one or more embodiments, the processes disclosed in this subject matter are stored in system memory 1604, persistent storage device 1602, and / or ROM 1610 (each implemented as a non-transitory computer-readable medium). The one or more processing units 1612 retrieve instructions to be executed and data to be processed from these various memory units to perform the processes of one or more embodiments.

[0087] Bus 1608 is also connected to input device interface 1614 and output device interface 1606. Input device interface 1614 enables a user to transmit information and select commands to electronic system 1600. Input devices that can be used with input device interface 1614 may include, for example, an alphanumeric keypad and a pointing device (also known as a "cursor control device"). Output device interface 1606 enables, for example, the display of images generated by electronic system 1600. Output devices that can be used with output device interface 1606 may include, for example, printers and display devices such as liquid crystal displays (LCDs), light-emitting diode (LED) displays, organic light-emitting diode (OLED) displays, flexible displays, flat panel displays, solid-state displays, projectors, or any other device for outputting information. One or more embodiments may include a device that acts as both an input device and an output device, such as a touchscreen. In these embodiments, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, voice, or tactile input.

[0088] Finally, as Figure 16As shown, bus 1608 also couples electronic system 1600 to one or more networks and / or to one or more network nodes via one or more network interfaces 1616, such as Figure 1 The electronic device 110 shown is an example. Thus, the electronic system 1600 may be part of a computer network (such as a LAN, wide area network (“WAN”), or intranet), or may be part of a network of networks (such as the Internet). Any or all components of the electronic system 1600 may be used in conjunction with the subject matter disclosed herein.

[0089] These functions can be implemented in computer software, firmware, or hardware. The technology can be implemented using one or more computer program products. Programmable processors and computers can be included in or packaged as mobile devices. The process and logic flow can be executed by one or more programmable processors and one or more programmable logic circuits. General-purpose and special-purpose computing devices, as well as storage devices, can be interconnected via communication networks.

[0090] Some implementations include electronic components, such as microprocessors, storage devices, and memories, that store computer program instructions in a machine-readable or computer-readable medium (also referred to as a computer-readable storage medium, machine-readable medium, or machine-readable storage medium). Examples of such computer-readable media include RAM, ROM, read-only optical disc (CD-ROM), recordable optical disc (CD-R), rewritable optical disc (CD-RW), read-only digital versatile optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini-SD card, micro-SD card, etc.), magnetic and / or solid-state hard disk drives, read-only and recordable... Disks, high-density optical discs, any other optical or magnetic media, and floppy disks. Computer-readable media may store computer programs that can be executed by at least one processing unit and include a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as machine code generated by a compiler, and files that include higher-level code that can be executed by a computer, electronic components, or microprocessor using an interpreter.

[0091] While the above discussion primarily concerns microprocessors or multi-core processors that execute software, some implementations utilize one or more integrated circuits such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some implementations, such integrated circuits execute instructions stored on the circuit itself.

[0092] As used in this specification and any claim of this application, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms exclude persons or groups of persons. For the purposes of this specification, the terms "displayed" or "being displayed" mean displayed on an electronic device. As used in this specification and any claim of this application, the terms "computer-readable medium" and "computer-readable media" are entirely limited to tangible, touchable objects that store information in a form readable by a computer. These terms do not include any wireless signals, wired download signals, or any other transient signals.

[0093] To provide user interaction, embodiments of the subject matter described herein can be implemented on a computer having a display device for displaying information to the user and a keyboard and pointing device that the user can use to provide input to the computer. The display device is, for example, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, and the pointing device is, for example, a mouse or trackball. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including acoustic, voice, or tactile input. Furthermore, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser in response to a request received from a web browser on the user's client device.

[0094] Implementations of the subject matter described herein can be implemented in a computing system that includes back-end components, such as a data server, or middleware components, such as an application server, or front-end components, such as a client computer with a graphical user interface or web browser through which a user can interact with implementations of the subject matter described herein, or any combination of one or more such back-end components, middleware components, or front-end components. Components of the system can be interconnected via digital data communication of any form or medium, such as a communication network. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), interconnected networks (e.g., the Internet) and peer-to-peer networks (e.g., self-organizing peer-to-peer networks).

[0095] A computing system may include clients and servers. Clients and servers are generally geographically isolated and can interact via a communication network. The client-server relationship is established by means of computer programs running on respective computers and having a client-server relationship with each other. In some implementations, the server transmits data (e.g., HTML pages) to the client device (e.g., to display data to a user interacting with the client device and to receive user input from the user interacting with the client device). Data generated at the client device (e.g., the result of user interaction) can be received from the client device at the server.

[0096] According to various aspects of this disclosure, a system is provided comprising: a device configured to display computer-generated content overlaid on a view of a physical environment; and an accessory device. The accessory device includes: a portable unit; and a unique identifier obtainable by the device. The device is configured to display an indicator of stored content overlaid on the view of the physical environment; detect the accessory device in the physical environment; and, in response to a user request corresponding to the displayed indicator, associate the unique identifier of the accessory device with information associated with the stored content.

[0097] According to various aspects of this disclosure, a method is provided, the method comprising: displaying an indicator of content overlaid on a view of a physical environment via a device; obtaining an identifier of an accessory device in the physical environment via the device; and, in response to a user request, associating the identifier of the accessory device with information associated with the content.

[0098] According to various aspects of this disclosure, an accessory for an extended reality system is provided, the accessory comprising: a portable body; a communication circuit disposed within the portable body; and a processing circuit configured to associate the accessory with content displayed on another device within the extended reality system, in response to information received from the other device via the communication circuit and at least in part based on the location of the accessory in the physical environment of the accessory and the other device.

[0099] Those skilled in the art will recognize that the various exemplary blocks, modules, elements, components, methods, and algorithms described herein can be implemented as electronic hardware, computer software, or a combination of both. To illustrate this interchangeability between hardware and software, the various exemplary blocks, modules, elements, components, methods, and algorithms have been generally described above in terms of functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The functionality can be implemented differently for each specific application. Various components and blocks can be arranged differently (e.g., in different orders or divided in different ways) without departing from the scope of the subject matter.

[0100] It should be understood that the specific order or hierarchical structure of the steps in the process disclosed in this invention is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchical structure of the steps in the process can be rearranged. Some steps in the process can be performed simultaneously. The appended method claims present elements of various steps in the sample order and are not intended to limit one to the specific order or hierarchical structure presented.

[0101] The preceding description is provided to enable those skilled in the art to practice the various aspects described herein. The preceding description provides various examples of the subject matter, and the subject matter is not limited to these examples. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, this claim is not intended to be limited to the aspects shown herein, but rather to be consistent with the language of the claim, wherein reference to elements in the singular form is not intended to mean “only one,” but rather “one or more,” unless specifically indicated. Unless otherwise specifically stated, the term “some” means one or more. Male pronouns (e.g., his) include female and neutral (e.g., her and its), and vice versa. Titles and subtitles (if any) are for convenience only and do not limit the invention described herein.

[0102] As used herein, the term "website" can include any aspect of a website, including one or more web pages, one or more servers used to host or store web-related content, etc. Therefore, the term "website" is used interchangeably with the terms "web page" and "server." The predicates "configured to," "capable of operating," and "programmed to" do not imply any specific tangible or intangible modification to a particular subject but are intended to be used interchangeably. For example, a component or a processor configured to monitor and control operations may also mean that the processor is programmed to monitor and control operations or that the processor is capable of operating to monitor and control operations. Similarly, a processor configured to execute code can be interpreted as a processor programmed to execute code or capable of operating to execute code.

[0103] As used herein, the term "automatic" can include execution by a computer or machine without user intervention; for example, by instructions in response to a predicate action of a computer or machine or other initiating mechanism. The word "example" is used herein to mean "serving as an example or illustration." No aspect or design described herein as an "example" is necessarily to be construed as preferred or advantageous over other aspects or designs.

[0104] Phrases such as "aspect" do not imply that this aspect is essential to the subject matter or that this aspect applies to all configurations of the subject matter. Disclosures relating to an aspect may apply to all configurations, or one or more configurations. An aspect may provide one or more examples. Phrases such as "aspect" may refer to one or more aspects, or vice versa. Phrases such as "implementation" do not imply that this implementation is essential to the subject matter or that this implementation applies to all configurations of the subject matter. Disclosures relating to an implementation may apply to all implementations, or one or more implementations. An implementation may provide one or more examples. Phrases such as "implementation" may refer to one or more implementations, or vice versa. Phrases such as "configuration" do not imply that this configuration is essential to the subject matter or that this configuration applies to all configurations of the subject matter. Disclosures relating to a configuration may apply to all configurations or one or more configurations. A configuration may provide one or more examples. Phrases such as "configuration" may refer to one or more configurations, or vice versa.

Claims

1. A system comprising: A device for an extended reality (XR) system, the device being configured to display computer-generated content overlaid on a view of the physical environment; and Supporting equipment, the supporting equipment includes: Portable main unit; and A unique identifier, obtainable by the device, wherein the accessory device is a physical device associated with content generated within the XR environment created by the XR system, wherein the association persists when components of the XR system are idle or powered off, wherein the accessory device stores password information to access content stored on another device or server associated with the accessory device and / or stores encrypted content on the accessory device. The device is configured as follows: An indicator that displays the stored content overlaid on the view of the physical environment; Detecting the supporting equipment in the physical environment; and In response to a user request corresponding to the displayed indicator, the unique identifier of the accessory device is associated with information related to the stored content.

2. The system of claim 1, wherein the user request includes placing the accessory device at a location associated with a displayed indicator of the stored content, or moving the stored content to or toward the location of the accessory device.

3. The system of claim 1, wherein the accessory further includes an electronic ink display mounted to the portable body, and wherein the accessory further includes a processor configured to operate the electronic ink display to display another indicator of the stored content associated with the unique identifier.

4. The system of claim 1, wherein the device is configured to obtain the unique identifier by capturing an image of the portable subject of the accessory device or by receiving wireless communication from the accessory device.

5. The system of claim 1, wherein the device is configured to associate the accessory device with the storage content by storing the unique identifier of the accessory device in association with the storage content at the device.

6. The system of claim 1, wherein the stored content is stored at the device or a remote device, and wherein the device is further configured to cause the stored content to be transmitted to the accessory device, and wherein the accessory device is configured to store the stored content at the accessory device.

7. The system of claim 1, wherein the device is configured to persistently associate the stored content with the unique identifier of the accessory device after the accessory device is removed from the field of view of the device's camera.

8. The system of claim 7, wherein the device is further configured to detect the companion device at a new location in the physical environment and display the indicator of the stored content on the view of the physical environment overlaid at the new location.

9. The system of claim 1, wherein when the supporting device approaches an additional device separate from the supporting device and the device, the supporting device is further configured to provide the additional device with access to the stored content.

10. The system of claim 1, wherein the device is configured to provide a key, wherein the key is a decryption key that facilitates access to and / or authorizes access to and / or authorization of the encrypted content stored at the associated device.

11. The system of claim 1, wherein the password information is authorization information corresponding to a software license, the software license authorizing a nearby device to access one or more applications, features and / or functions provided by the software license.

12. A method comprising: An indicator that displays stored content overlaid on a view of the physical environment via a device of an extended reality XR system, the device being configured to display computer-generated content overlaid on the view of the physical environment; The system detects supporting devices in the physical environment, the supporting devices including a portable unit and a unique identifier that can be obtained by the device, wherein the supporting device is a physical device associated with content generated within the XR environment created by the XR system, wherein the association continues when the components of the XR system are idle or powered off, wherein the supporting device stores password information to access content stored on another device or server associated with the supporting device and / or stores encrypted content on the supporting device. as well as In response to a user request corresponding to the displayed indicator, the unique identifier of the accessory device is associated with information related to the stored content.

13. The method of claim 12, wherein associating the unique identifier of the accessory device with the information associated with the content comprises storing the unique identifier of the accessory device and the information associated with the content at a remote server separate from the accessory device and the device.

14. The method of claim 13, wherein the device is configured to persistently associate the stored content with the unique identifier of the accessory device after the accessory device is removed from the field of view of the device's camera, and wherein the device is configured to detect the accessory device at a new location in the physical environment and display the indicator of the stored content overlaid on the view of the physical environment at the new location.

15. The method of claim 13, wherein the device is configured to provide a key, wherein the key includes a decryption key that facilitates access to and / or authorizes access to and / or authorization of the encrypted content stored at the accompanying device.

16. An accessory for an extended reality system, the accessory comprising: Portable main unit; A unique identifier that can be obtained by another device of the extended reality XR system, wherein the other device is configured to display computer-generated content overlaid on a view of the physical environment; A communication circuit is disposed within the portable main body; and A processing circuit configured to associate the unique identifier of the companion device with content generated within the XR environment created by the XR system, in response to information received from the other device via the communication circuit and based at least in part on the other device detecting the location of the companion device in the physical environment of the companion device and the other device, and a user request corresponding to an indicator of the content displayed on the other device overlaid on the physical environment, wherein the association persists when components of the extended reality system are idle or powered off, and wherein the companion device stores password information to access content stored at the other device or server associated with the companion device and / or stores encrypted content at the companion device.

17. The accessory device of claim 16, further comprising a memory configured to store the unique identifier of the accessory device, wherein the processing circuitry is configured to associate the accessory device with the content in such a way as: Receive via the communication circuit a key configured to provide access to the content; and The key is stored in the memory.

18. The accessory device of claim 16, wherein the content includes at least one of a file stored on the other device, the status of an application on the other device, or information for performing the functions of the other device.

19. The accessory device of claim 16, further comprising a display mounted to the accessory device, the display being configured to display an indicator of the content associated with the accessory device after the content is associated with the accessory device.

20. The accessory device according to claim 19, wherein the display comprises an electronic ink display.

Images