Utilizing cloud anchors in authentication

Abstract

Systems and methods for authenticating a device are described. The systems and methods can include detecting, by a sensor on a wearable device, at least one cloud anchor, the at least one cloud anchor including an identifier associated with a network and configured for a physical environment. In response to detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor, triggering extraction of the identifier from the at least one cloud anchor. The systems and methods can also include joining the wearable device to the network based on a received authentication corresponding to the extracted identifier.

Description

Technical Field

[0001] This instruction manual generally covers certified equipment. Background Technology

[0002] Mobile devices can be used to control one or more electronic devices in buildings such as homes and offices. Such buildings typically include numerous devices that can be remotely accessed and / or controlled. For example, light bulbs can be switched or otherwise adjusted by a mobile device using various types of wireless communication. In some examples, such mobile devices may include cameras for capturing image data and image recognition software for processing the image data to detect devices in the building, which can be within the field of view of the camera onboard the mobile device. Summary of the Invention

[0003] A system of one or more computers can be configured to perform a specific operation or action by means of software, firmware, hardware, or a combination thereof installed on the system, which, in operation, causes the system to perform the action. One or more computer programs can be configured to perform a specific operation or action by means of instructions that, when executed by a data processing device, cause the device to perform an action.

[0004] In a first general aspect, a system and method are described for detecting at least one cloud anchor by sensors on a wearable device, the at least one cloud anchor including an identifier associated with a network and configured for a physical environment. In response to detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device and that the wearable device can access the at least one cloud anchor, the system and method may include: triggering the extraction of an identifier from the at least one cloud anchor, and adding the wearable device to the network based on received authentication corresponding to the extracted identifier.

[0005] These and other aspects may individually or in combination include one or more of the following. For example, the system and method may further include aspects such that the identifier includes an SSID and login credentials, the network is a Wi-Fi network, the identifier is associated with at least one controllable device that provides access to the network, and the authentication is received from the at least one controllable device.

[0006] In some implementations, the identifier is associated with a 3D map of the augmented reality (AR) environment, the 3D map including visual data mapped to the physical environment stored by at least one controllable device within the physical environment, and detecting that the wearable device can access the at least one cloud anchor includes determining whether the wearable device provides data from sensors to match at least a portion of the visual data mapped to the physical environment. In some implementations, the at least one controllable device is configured to deauthenticate the wearable device from the network in response to detecting that the wearable device's location is no longer within a threshold distance. In some implementations, the location is determined based on detecting that the at least one cloud anchor is within the wearable device's field of view. Additionally, the method of claim 1 may include one or more (e.g., all) (or any combination thereof) of the following features of claims 6 to 10, as described below.

[0007] In a second general aspect, systems and methods are described for receiving a request for access to a controllable device at a wearable device communicatively coupled to a controllable device in a physical environment, the request being determined to be received from an unauthenticated user or device. The systems and methods may further include: determining the posture of the wearable device within the physical environment; and rendering a prompt in the wearable device to select whether to grant or deny access to the controllable device by the unauthenticated user or device. In response to receiving permission to access the controllable device, the systems and methods may include authenticating the unauthenticated user or device to access the controllable device from a region of the physical environment defined based on the determined posture of the wearable device, and executing the request.

[0008] These and other aspects may individually or in combination include one or more of the following. For example, the area may be defined within a spherical image of the physical environment, based on a 3D map of at least one cloud anchor associated with the physical environment. In some implementations, the system and method may include: rendering another cue in a wearable device to trigger a user of the wearable device to look in a direction corresponding to the area; capturing an image of the area, the image capturing a portion of the user or device requesting authentication; and storing the image by the wearable device in a database of authenticated devices or users.

[0009] In some implementations, the system and method may include: receiving a second request to access a controllable device; rendering a prompt in a wearable device to trigger a user of the wearable device to capture an image associated with the second request; receiving a command for capturing the image associated with the second request; and comparing the second captured image with images in a database. In response to determining that the second captured image matches at least a portion of at least one image in the database, the system and method may further include triggering the execution of the second request.

[0010] In some implementations, the request is an auditory request spoken by a user in a physical environment. The wearable device determines the direction in which the auditory request is received and determines a region of the physical environment based on the determined direction of the auditory request. Furthermore, the method of claim 6 may include one or more (e.g., all) (or any combination thereof) of the following features of claims 1 to 5.

[0011] Implementations of the described technology may include hardware, methods or processes, or computer software on a computer-accessible medium. Details of one or more embodiments are set forth in the accompanying drawings and the following description. Other features will be apparent from the specification, drawings, and claims. Attached Figure Description

[0012] Figure 1 An augmented reality (AR) system for authenticating wearable devices is illustrated according to embodiments described throughout this disclosure.

[0013] Figure 2 A block diagram of an example system for authenticating devices in physical and / or AR environments, according to embodiments described throughout this disclosure, is shown.

[0014] Figures 3A-3B An example of an electronic wearable device according to embodiments described throughout this disclosure is shown.

[0015] Figure 4 This is a flowchart illustrating an example of a network joining process according to an embodiment described throughout this disclosure.

[0016] Figure 5 This is a flowchart illustrating an example of a process for authenticating a controllable device according to embodiments described throughout this disclosure.

[0017] Figure 6 Examples of computer devices and mobile computer devices that can be used with the technologies described herein are shown.

[0018] The same reference numerals in the various figures indicate the same elements. Detailed Implementation

[0019] This document describes examples related to arbitrary authentication for augmented reality (AR) wearable electronic devices (i.e., wearable devices). The authentication process may include an image-based assessment of the environment surrounding the wearable device. The image-based assessment may take into account aspects of the wearable device's location and / or the location of the electronic device the wearable device seeks to control or authenticate it. In some implementations, the image-based assessment may take into account the location of cloud anchors associated with the environment.

[0020] A device seeking authentication (e.g., a wearable device) can capture location aspects of its environment and transmit these aspects (and / or other data) to another electronic device in the environment. This other electronic device in the environment can evaluate the location information (and / or other data received from the wearable device) to determine whether to authenticate the wearable device and / or whether to provide specific data and / or services to the wearable device via the other electronic device.

[0021] In some implementations, the wearable device described herein can be arbitrarily authenticated by one or more other devices in a physical, virtual, or AR environment. For example, an electronic assistant-type device (e.g., a home assistant, a car assistant) can be used to authenticate other devices in a shared space and those sharing specific settings and resources associated with the electronic assistant. In some implementations, other controllable devices such as electronic thermostats, electronic doorbell devices, televisions, security devices, appliances, or other electronic devices that can be viewed by the wearable device can be used to authenticate the wearable device and / or other devices in the shared space.

[0022] Conventional systems typically lack a mechanism to authenticate requests from devices or users via an assistant device. In particular, conventional systems can receive and execute arbitrary requests (e.g., controlling lights, controlling music, etc.) in shared settings (e.g., dormitories, hotels, office buildings, etc.) and execute such requests without the permission of the device owner.

[0023] The systems and methods described herein can provide technology that, for example, grants permission for a guest-owned device. Such technology can provide advantages to device owners by enabling them to assign control over such arbitrary requests to allow specific users to execute requests while disallowing or preventing other users from executing requests without the device owner's permission. The technology can provide a way to grant permission to devices making arbitrary requests (i.e., devices configured to receive keywords such as "Ok computer…", auditory requests, device control requests, etc.). Permission can be granted by the device owner and is not solicited upon receiving instructions (e.g., gestures, selections, auditory responses, etc.) from the device owner to allow arbitrary requests from specific devices and / or users. For example, the device owner can gesture toward the device location or user location associated with the request. In some embodiments, user interface content can be displayed to the device owner in a wearable device display to request authentication of the requesting user and / or approval for controllable device use.

[0024] In some implementations, the wearable devices described herein can utilize AR content (e.g., cloud anchors) to be authenticated for access to services offered by a network and / or device characteristics (e.g., access to a Wi-Fi network, access to an AR session, access to devices in the environment, etc.). For example, the systems and methods described herein can provide techniques for authenticating guest devices to access services in the environment. For instance, conventional authentication of a Wi-Fi network / service may involve the user having to request a password from another user, manually select a network, and manually enter a password. The systems and techniques described herein can utilize virtual cloud anchors (i.e., cloud anchors) to arbitrarily authenticate guest devices to a Wi-Fi network (or other services outside such a network) associated with an environment (e.g., a hotel, home, property, office building, etc.). Such authentication can be provided based on permissions configured by the service and / or network owner.

[0025] Typically, this disclosure relates to the use of wearable devices that can capture visual data to identify three-dimensional (3D) map information (e.g., 3D meshes, 3D feature maps, cloud anchors, etc.) or two-dimensional (e.g., 2D) information about a particular physical environment to determine the location of a controllable device in a physical space at least partially represented by the visual data and / or gain access to that controllable device. Such information may include any number of visual positioning data regarding separate controllable devices visible or detectable by the wearable device. This information may be locally stored or remotely stored on a server computer. In some embodiments, this information may be used to recover 2D information. In some embodiments, this information may be used to detect three-degree-of-freedom (DoF) positioning of the controllable device. In some embodiments, this information may be used to detect six-DoF positioning of the controllable device. In some embodiments, this information may be used to detect five-DoF positioning of the controllable device. In some embodiments, this information may be used to recover 2D and / or 5-DoF map data.

[0026] Once the location is determined and / or other user input is authenticated, the wearable device can be authenticated to the controllable device and begin controlling the controllable device. User interface (UI) content and / or audio signals can be provided to the wearable device to allow control and / or feedback from the user of the wearable device. In some implementations, the UI content can provide the user of the wearable device with one or more controls to control the controllable device and / or additional information about the controllable device.

[0027] The techniques described herein offer the advantage of conserving battery power on wearable devices by using cloud anchors to authenticate them to Wi-Fi networks. Such techniques ensure that wearable devices do not need to utilize cellular services, which can deplete electronic device batteries faster than Wi-Fi services. Alternatively, the techniques described herein can utilize cloud anchor information to authenticate devices, provide access to Wi-Fi or other networks, and enable controllable devices associated with physical and / or AR environments.

[0028] Figure 1 An augmented reality (AR) system 100 for authenticating wearable devices to other electronic devices, according to embodiments described throughout this disclosure, is illustrated. A user can enter a physical space such as environment 102 and may carry one or more computing systems. For example, a first user 104 is shown seated and associated with (e.g., near) computing system 106-1 and wearing wearable device 104-1. Computing system 106-1 is a mobile device. Wearable device 104-1 is a pair of AR glasses. Similarly, a second user 106 can enter environment 102. The second user may also have any number of electronic devices. In this example, user 106 is holding mobile device 106-2 and wearing wearable device 104-2. Environment 102 may include any number of computing systems. Similarly, environment 102 may include any number of wearable devices. For example, an additional wearable device 104-3 represents a ring controller that can interact with other devices in environment 102. Similarly, an additional wearable device 104-4 represents a smartwatch that can interact with other devices in environment 102.

[0029] In some embodiments, computing system 106-1 may be communicatively coupled to wearable device 104-1. In some embodiments, wearable device 104-1 may be a standalone device that is not configured, connected, or communicating with another device associated with user 104. In some embodiments, computing system 106-2 may be communicatively coupled to wearable device 104-2. In some embodiments, wearable device 104-2 may be a standalone device that is not configured, connected, or communicating with another device associated with user 106.

[0030] In some implementations, computing systems 106-1 and 106-2 are auxiliary (i.e., optional) and not used for interaction within environment 102. For example, wearable device 104-1 can operate independently to experience networks, services, or other controllable devices within environment 102. Similarly, wearable device 104-2 can operate without computing system 106-2. Therefore, wearable devices 104-1 and 104-2 can operate independently. In some implementations, wearable device 104-2 can request access to services, networks, etc., via wearable device 104-1.

[0031] Users 104 and 106 associated with one or more of computing systems 106-1, 106-2 and / or wearable devices 104-1, 104-2 may wish to control one or more controllable devices in environment 102. As shown in environment 102, a first controllable device 108-1 is a digital assistant device, and a second controllable device 108-2 is a Wi-Fi router. Environment 102 may include any number of controllable devices 108. To control such a controllable device 108, it may involve evaluating and obtaining authentication from the owner of the particular controllable device 108. Computing systems 106-1, 106-2 and / or wearable devices 104-1, 104-2 can be authenticated using the systems and techniques described herein. Authentication can provide such systems and devices with access to services, networks, controllable devices, and / or permissions associated with such controllable devices. For example, system 100 and environment 102 may be configured with a cloud anchor 110, which can be detected and accessed to obtain information that can be used to authenticate wearable devices 104-1 and 104-2 to obtain access to specific services, networks, and / or controllable devices. In some implementations, a wearable device (e.g., owner AR glasses 104-1) can provide authentication for other wearable devices (e.g., guest AR glasses 104-2).

[0032] As used herein, a cloud anchor (e.g., a virtual cloud anchor) can refer to a set of one or more visual feature points included in a 3D map. A cloud anchor can be associated with a persistent set of visual features representing a physical object in the physical world. In the AR system described herein, a wearable device can receive digital information about the user's environment and can create one or more cloud anchors, such as cloud anchor 110, based on this digital information, where each cloud anchor 110 can represent a portion of a physical space within a building, such as a room (or a portion thereof). The user can add virtual objects to a virtual scene associated with a room and can then link (or pin) the virtual objects to locations within cloud anchor 110.

[0033] Cloud anchor 110 can be shared with other users who can join the space and view and interact with content or devices owned by the master user (or another user). In some examples, to allow other users to join the master's environment, the master user can locate the identifier of a specific cloud anchor and provide the identifier to another user, who can then manually enter the identifier to join the AR scene. In some implementations, the cloud anchor can be configured by the master to automatically authenticate specific devices to access objects, devices, services, and / or systems associated with (or communicatively coupled to access) the cloud anchor. Each device associated with environment 102 (e.g., computing systems 106-1, 106-2 and wearable devices 104-1, 104-2) can access cloud anchor 110 via an image sensor associated with such a device.

[0034] In some implementations, the wearable device can move through the physical environment, collecting data related to feature points of any number of cloud anchors 110. At certain times, combinations of feature points can be provided as specific identifiers used to gain access to a network or device. The identifier may include one or more feature points associated with one or more cloud anchors 110. For example, if the wearable device 104-2 can generate an identifier associated with a specific cloud anchor corresponding to or configured with a controllable device, then the wearable device 104-2 can then be authenticated to gain access to the network or device.

[0035] In operation, wearable device 104-1 can represent a master (i.e., the owner of controllable device 108). User 104 (i.e., master / owner) can have configured controllable device 108-1 (digital assistant) and / or controllable device 108-2 (e.g., Wi-Fi router) to provide arbitrary cloud anchor-based authentication to guest users when verifying that guest users can access cloud anchor 110, and to provide such authentication to the corresponding wearable device 104-2, wearable device 104-3, wearable device 104-4, computing system 106-1, controllable device 108-1, or controllable device 108-2.

[0036] In some implementations, cloud anchor 110 may be associated with any number of identifiers. For example, cloud anchor 110 may include a location identifier indicating the location of a physical or virtual object associated with a 3D map of a particular environment. Cloud anchor 110 may alternatively or additionally include identifiers indicating services, passwords, and / or other indicators for accessing services, networks, functions, features, etc.

[0037] In a non-limiting example, system 100 may provide network access to wearable devices. For example, user 106 may wear wearable device 104-2 (e.g., AR glasses) to walk into lobby environment 102. Wearable device 104-2 may include at least one image sensor capable of detecting at least one cloud anchor (e.g., cloud anchor 110). Cloud anchor 110 may include an identifier associated with at least one controllable device (e.g., controllable device 108-2 representing a Wi-Fi network router). Controllable device 108-2 may be configured for physical environment 102 to provide Wi-Fi service to any wearable device, such as detecting cloud anchor 110. For example, in response to detecting that the location of at least one controllable device (e.g., Wi-Fi router 108-2) is within a threshold distance of wearable device 104-2, wearable device 104-2 may trigger the extraction of an identifier from said at least one cloud anchor 110. For example, when moving within a predefined threshold distance of device 108-2 and / or when moving within the viewing distance of device 108-2 (i.e., wearable device 104-2 may have cloud anchor 110 within its field of view, detectable distance, and / or another visible and detectable range of device 108-2), the wearable device can automatically request login credentials (e.g., SSID, username, and password) from device 108-2. In some implementations, the threshold distance may correspond to a threshold time, during which time is determined that time has elapsed and device 104-2 can no longer view the specific cloud anchor based on the detection that a specific wearable device 104-2 can no longer access the specific cloud anchor 110. At this time, Wi-Fi can be deauthenticated on wearable device 104-2.

[0038] In some implementations, the threshold distance may relate to a full or partial line of sight of a particular device. For example, the threshold distance may relate to ensuring that the wearable device (e.g., wearable device 104-2) has a line of sight to the controllable device 108-2. Another example of the threshold distance may relate to ensuring that the wearable device 104-2 has, for example, at least a partial view (i.e., partial line of sight) of the particular controllable device within environment 102.

[0039] Upon request for identifier information, wearable device 104-2 can provide such information to at least one controllable device in a sequence identified by device 104-2, enabling authentication to occur. For example, upon providing identifier information, wearable device 104-2 can join the network based on the received authentication of the provided identifier. Device 108-2 can authenticate, or user 104 can authenticate via another device or wearable device associated with environment 102 and / or controllable device 108-2. For example, at least one controllable device can be an electronic assistant (e.g., controllable device 108-1) configured with controllable device 108-2 (i.e., a network Wi-Fi router) providing the network.

[0040] In another non-limiting example, system 100 can authenticate another user or device to use, control, or otherwise access a specific controllable device associated with environment 102. For example, user 106 may represent a guest and have access to the lobby (i.e., the physical environment) of environment 102. User 106 may be associated with one or more electronic devices (as shown using devices 104-2 and 106-2). However, a user may not have any devices on her and may not be associated with any device in environment 102. Therefore, the system described herein can authenticate devices and / or can authenticate a specific user based on audio (or visual) requests received from such a user.

[0041] In this example, user 104 may represent the owner of environment 102 and controllable devices within environment 102 (e.g., controllable devices 108-1 and 108-2). User 104 may be wearing wearable device 104-1 (e.g., AR glasses), which can be used to authenticate other users to devices in environment 102. For example, user 106 may enter environment 102 and may, for example, make a request to access controllable device 1081. This request may be made via device 104-2, device 106-2, or another device associated with user 106. In some implementations, the request may instead include auditory utterances or visual cues performed by user 106.

[0042] User 106 can execute requests to use, control, or otherwise access an electronic assistant (such as controllable device 108-1). In operation, system 100 can receive requests at wearable device 104-1 (or device 104-3 or device 104-4). One or all wearable devices 104-1, 104-3, or 104-4 can be communicatively coupled to controllable device 108-1, such that any of devices 104-1, 104-3, or 104-4 can authenticate access to controllable device 108-1. In this example, the request is determined by wearable device 104-1, for example, having been received from an unauthenticated user or device (e.g., from user 106 or from unauthorized computing device 106-2 or wearable device 104-2).

[0043] When an unauthorized user or device receives a request, wearable device 104-1 can determine its own posture. This posture can be that of wearable device 104-1 owned by its owner and physically present in the environment. Wearable device 104-1 can then trigger (on its associated display) the rendering of cue 113 to allow or deny access to controllable device 108-1 by an unauthorized user 106 or device (device 106-2 or device 104-2). Cue 113 describes the issue and available controls, but other cueing is, of course, possible. For example, additional cue 115 could instruct user 104 to look towards a door in environment 102 to receive an authentication request. User 104 could perform the posture of looking towards the door of the room to receive (i.e., be offered) such an authentication request.

[0044] In some implementations, user 104 of wearable device 104-1 may perform gesture 112 within the field of view of a camera associated with wearable device 104-1, for example, to indicate approval to device 104-1 (i.e., AR glasses). Such approval may then enable authentication of user 106 or device 104-2 or 106-2, which may allow the request to be executed on controllable device 108-1. For example, if the request comes from device 106-2 to play music from a selected playlist or the request is an audible request 114 to play music, authentication may enable device 108-1 to play music at least in part based on the approval associated with wearable device 104-1, as indicated by the indicator representing request 116.

[0045] In response to receiving permission to access controllable device 108-1 or in response to a gesture performed by user 104 that is visible to device 104-1, wearable device 104-1 can authenticate an unauthenticated user or device to access controllable device 108-1 from a region of the physical environment, and can then execute that request. The region of the physical environment can be defined based on the determined posture of the wearable device. For example, the region can be determined using an onboard camera of wearable device 104-1 or an audio sensor associated with device 104-1. In this example, region A is an example of a region defined based on the location of user 106 (or a device owned by user 106). Other defined regions and sub-regions within environment 102 are, of course, possible.

[0046] Figure 2 A block diagram of an example system 200 for authenticating devices in physical and / or AR environments, according to embodiments described throughout this disclosure, is shown. System 200 includes at least a first wearable device 104-1, such as Figure 1As described herein. Device 104-1 can receive requests for access to controllable device 108. In some embodiments, server computer 202 can be used to access, generate, view, and use cloud anchors. In some embodiments, wearable computing device 104-1 can perform and store operations described regarding server computer 202, thereby eliminating the need for use of server computer 202 utilizing the techniques described herein.

[0047] In some examples, wearable device 104-1 includes one or more computing devices, wherein at least one of the devices is a display device capable of being worn on or near a person's skin. In some examples, wearable device 104-1 is a wearable device or includes wearable devices. Wearable devices may include head-mounted display (HMD) devices, such as optical head-mounted display (OHMD) devices, transparent head-up display (HUD) devices, augmented reality (AR) devices, or other devices such as goggles or headphones with sensors, displays, and computing capabilities. In some examples, wearable devices include AR glasses (e.g., smart glasses). AR glasses are optical head-mounted display devices designed in the shape of a pair of glasses.

[0048] In some examples, wearable device 104-1 includes a wearable device (e.g., AR glasses) and a computing device (e.g., a mobile computing device, such as a smartphone, tablet, laptop, or another wearable device such as a smartwatch). Wearable device 104-1 can connect to the computing device wirelessly via a short-range connection (e.g., Bluetooth or Near Field Communication (NFC)) or an internet connection (e.g., Wi-Fi or a mobile network). In some examples, some components of wearable device 104-1 are included in the wearable device, and some components of wearable device 104-1 are included in the computing device.

[0049] Wearable device 104-1 includes one or more processors 204, which may be formed in a substrate configured to execute one or more machine-executable instructions or software, firmware, or a combination thereof. The processor 204 may be semiconductor-based—that is, the processor may include semiconductor materials capable of performing digital logic. Such a processor 204 may include a CPU, GPU, and / or DSP, to name just a few examples.

[0050] Wearable device 104-1 may also include one or more memory devices 206. Memory device 206 may include any type of storage means for storing information in a format readable and / or executable by processor 204. Memory device 206 may store applications and modules that perform certain operations when executed by processor 204. In some examples, applications and modules may be stored in external storage devices and loaded into memory device 206. Wearable device 104-1 includes one or more antennas (not shown) configured to communicate with other computing devices.

[0051] Wearable device 104-1 includes a display 208. Display 208 may include a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, an electrophoretic display (EPD), or a micro-projection display using an LED light source. In some examples, display 208 is projected onto the user's field of view. In some examples, in the case of AR glasses, display 208 may provide a transparent or semi-transparent display, allowing the user wearing the glasses to see not only the image provided by display 208 but also information within the field of view of the AR glasses located behind the projected image.

[0052] Wearable device 104-1 includes a sensor system 210. Sensor system 210 includes an image sensor 212 configured to acquire image data. In some examples, sensor system 210 includes multiple image sensors 212. The image sensors 212 can capture photographs and record video. Sensor system 210 may include an inertial measurement unit (IMU) 214. IMU 214 can detect motion, movement, and / or acceleration of wearable device 104-1. IMU 214 may include various different types of sensors, such as, for example, accelerometers, gyroscopes, magnetometers, and other such sensors. Sensor system 210 includes an audio sensor 216 configured to detect audio received by wearable device 104-1. Sensor system 210 may include other types of sensors, such as light sensors, distance and / or proximity sensors, contact sensors such as capacitive sensors, timers, and / or other sensors and / or different combinations of sensors. The sensor system 210 can be used to obtain information associated with cloud anchor data 218 using cloud anchor application 220 and identifier 222 associated with 3D map 230 associated with such cloud anchor.

[0053] Identifier 222 may include information associated with one or more cloud anchors 110. This information may include location data corresponding to computing devices, wearable devices, virtual objects, and / or physical objects associated with the physical environment. In some embodiments, the information may include configuration data for joining a network, AR sessions, obtaining device control, etc. In some embodiments, the information included in identifier 222 may include image data, such as images stored for authentication purposes, which may be stored in image authentication database 242.

[0054] System 200 includes a Cloud Anchor application 220 that can be used to interact with Cloud Anchor for the purpose of authenticating specific locations and features associated with devices and images captured by such a device. For example, if wearable computing device 104-1 receives a request to authenticate a specific user or device, system 200 can be accessed to determine the location of one or more controllable devices 108 in environment 102. A user operating wearable computing device 104-1 can point the image sensor 212 of sensor system 210 at a specific other user or device (e.g., to controllable 108-1) and move wearable device 104-1 around to map the environment from different perspectives and positions. Such a map can be stored in a server map database 234 or stored on the device as Cloud Anchor data 218.

[0055] In operation, the cloud anchor application 220 receives data from the sensor system 210 and generates cloud anchor data 218, which includes visual data 224, device posture 226, and anchor posture 228. The wearable device 104-1 can generate or access a 3D map 230 based on the cloud anchor data 218. In some embodiments, the wearable device 104-1 may alternatively access the cloud anchor data 218 at a server computer 202. The server computer 202 may include a visual positioning data service 232 to generate the cloud anchor data 218 and generate the map 230 of the map database 234. In this example, the map 230 may be generated by a 3D map generator 236 associated with a specific physical and / or virtual environment.

[0056] Wearable device 104-1 can determine attitude and / or positioning 240 based on onboard camera 238 or image sensor 212. Positioning 240 can be 3D positioning of controllable device 108-1, where 3D positioning 240 is the location (and optionally, orientation) of controllable device 108-1 in the 3D physical environment in which the user (and / or wearable device 104-1) is operating within it. In some examples, positioning includes three-DoF positioning 240a of the controllable device or wearable device. In some examples, positioning 240 includes six-DoF positioning 240b of the controllable device or wearable device. In some embodiments, positioning 240 includes five-DoF positioning 240c of the controllable device or wearable device.

[0057] Wearable device 104-1 also includes a gesture detector 244. The gesture detector 244 can access sensor system 210 to detect and translate received gestures captured by camera 238 and / or image sensor 212 of device 104-1. In some embodiments, gesture detector 244 can identify gestures and translate them into inputs for accessing and / or controlling specific controllable devices. In some embodiments, gesture detector 244 receives gestures provided by a user and interprets patterns in the gestures to translate those patterns into inputs for controllable devices, such as in environment 102. In some embodiments, camera 238 may represent a low-resolution camera on wearable device 104-4, which is embedded on the edge of a smartwatch that can also track, view, and translate captured image data from cloud anchor 110 for authentication purposes.

[0058] The wearable device 104-1 also includes a control system 246, which includes various control system devices to facilitate the operation of the wearable device 104-1. The control system 246 may include a processor 314 operatively coupled to components of the wearable device 104-1.

[0059] Wearable device 104-1 also includes a communication module 248. Communication module 248 enables wearable device 104-1 to communicate to exchange information with another computing device and to authenticate other devices within range of device 104-1 or other identifiable elements in the environment. For example, wearable device 104-1 can be operatively coupled to another computing device to facilitate communication via, for example, a wired connection, a wireless connection via, for example, Wi-Fi or Bluetooth, or other types of connections.

[0060] In some embodiments, wearable device 104-1 is configured to communicate with server computer 202 via network 250. Server computer 202 may represent one or more computing devices in the form of multiple different devices, such as a standard server, a group of such servers, or a rack server system. In some embodiments, server computer 202 is a single system sharing components, such as a processor and memory. Network 250 may include the Internet and / or other types of data networks, such as a local area network (LAN), wide area network (WAN), cellular network, satellite network, or other types of data networks. Network 250 may also include any number of computing devices (e.g., computers, servers, routers, network switches, etc.). These are configured to receive and / or transmit data within network 250.

[0061] Server computer 202 includes one or more processors (not shown) that may be formed in a substrate configured to execute one or more machine-executable instructions or software, firmware, or a combination thereof. The processor may be semiconductor-based—that is, the processor may include semiconductor material capable of performing digital logic. Server computer 202 includes one or more memory devices (not shown). The memory devices may include main memory storing information in a format that can be read and / or executed by the processor.

[0062] In some examples, server computer 202 is configured to perform visual positioning data service 232. Visual positioning data service 232 may be an augmented reality (AR) collaboration service that allows users to create cloud anchors (e.g., 3D map 230) for creating multiplayer or collaborative AR experiences that users can share. For example, users can configure virtual content and / or controllable devices 108 in which other users can view, authenticate, and / or otherwise interact from different locations within a shared physical environment.

[0063] For example, a user can create a local cloud anchor (e.g., a 3D map 230) within their environment. During hosting, the wearable device 104-1 can upload data to generate the 3D map 230 at the visual positioning data service 232, which returns a unique identifier 222 for accessing a specific controllable device. This unique identifier 222 can be distributed to other users to join the same AR environment, or in some implementations to access the controllable device after authentication using the identifier 222 associated with the cloud anchor data 218.

[0064] although Figure 2 A single wearable device 104-1 and two controllable devices 108 are shown, but the embodiments described herein can cover any number of such systems (e.g., more than two). Although Figure 1 Showing the ratio Figure 2 The system depicted has fewer equipment components, but Figure 1 The device described herein may include Figure 2 Any or all components of the system (or may have access to them).

[0065] Figures 3A-3B Various views of examples of AR wearable devices according to embodiments described throughout this disclosure are shown. Figure 3A This is a front view of an example of a wearable device according to embodiments described throughout this disclosure. In this example, the wearable device is an AR glasses 300A (e.g., Figure 1 (Wearable electronic device 104-1 in the system). Typically, AR glasses 300A may include any or all components of system 200. AR glasses 300A may also be referred to as smart glasses, which means an optical head-mounted display device designed in the shape of a pair of glasses. For example, smart glasses are glasses that add information (e.g., a projection display) to what the wearer sees through the glasses.

[0066] Although the AR glasses 300A are shown as a wearable electronic device described herein, other types of wearable devices are possible. For example, wearable devices may include head-mounted display (HMD) devices, such as optical head-mounted display (OHMD) devices, transparent head-up display (HUD) devices, augmented reality (AR) devices, or other devices such as goggles or headphones with sensors, displays, and computing capabilities. In some examples, wearable devices may be watches, mobile devices, pieces of jewelry, ring controllers, or other wearable controllers.

[0067] like Figure 3A As shown, the AR glasses 300A includes a frame 302, in which a display device 304 is coupled (or in the glass portion of the frame 302). The AR glasses 300A also includes an audio output device 306, an illumination device 308, a sensing system 310, a control system 312, at least one processor 314, and a camera 316.

[0068] Display device 304 may include a near-eye display, such as those using bird-spot or waveguide optics. For example, such an optical design can project light from a display source onto a portion of a teleprompter glass that acts as a beamsplitter positioned at a 45-degree angle. The beamsplitter can allow for both reflection and transmission values, allowing light from the display source to be partially reflected while the remaining light is transmitted through. Such an optical design allows a user to see both physical items in the world adjacent to the digital image generated by the display (e.g., UI elements, virtual content, etc.). In some embodiments, waveguide optics may be used to depict content on display device 304 of AR glasses 300A.

[0069] An audio output device 306 (e.g., one or more speakers) may be coupled to the frame 302. The sensing system 310 may include various sensing devices, and the control system 312 includes various control system devices to facilitate the operation of the AR glasses 300A. The control system 312 may include a processor 314 operatively coupled to components of the control system 312.

[0070] Camera 316 may be capable of capturing still and / or moving images. In some embodiments, camera 316 may be a depth camera that can collect data related to the distance of an external object from camera 316. In some embodiments, camera 316 may be a point-tracking camera that can, for example, detect and follow one or more optical markers on an external device, such as, for example, an input device on a screen or an optical marker on a finger. In some embodiments, AR glasses 300A may include an illumination device 308 that can, for example, selectively operate using camera 316 to detect objects (e.g., virtual and physical) in the field of view of camera 316.

[0071] AR glasses 300A may include a communication module (e.g., communication module 248) that communicates with processor 314 and control system 312. The communication module can provide communication between devices housed within AR glasses 300A and with external devices such as controllers, mobile devices, and / or other computing devices. The communication module can enable AR glasses 300A to communicate with another computing device to exchange information and to authenticate other devices within range of AR glasses 300A or other identifiable elements in the environment. For example, AR glasses 300A may be operatively coupled to another computing device to facilitate communication via, for example, a wired connection, a wireless connection via, for example, Wi-Fi or Bluetooth, or other types of connections.

[0072] Figure 3BThis is a rear view 300B of AR glasses 300A according to embodiments described throughout this disclosure. AR glasses 300B may be... Figure 1 Examples of wearable devices 104-1 or 104-2 100. AR glasses 300B are glasses that add information (e.g., a projected display 320) to what the wearer sees through the glasses. In some examples, instead of projected information, the display 320 is a microdisplay within a lens. In some examples, AR glasses 300B (e.g., lenses or goggles) are visual aids that include a lens 322 (e.g., a glass or hard plastic lens) mounted in a frame 302, which typically uses a bridge 324 on the nose and arched members 326 (e.g., temples or temple pieces) resting on the ears to hold the lens 322 in front of a person's eyes.

[0073] Figure 4 This is a flowchart illustrating an example of a computer-implemented process 400 for joining a network according to embodiments described throughout this disclosure. Typically, process 400 utilizes the systems and algorithms described herein to enable a user device (e.g., a wearable device) to scan a physical environment to identify multiple cloud anchors in the environment, retrieve relevant AR content using the cloud anchors, and identify and / or retrieve network credentials using the cloud anchors. This allows the user device to join a network in a pre-recorded environment based on the use of the identified network credentials.

[0074] Process 400 may utilize a computing system having at least one processing device and a memory storing instructions, which, when executed, cause the processing device to perform the plurality of operations and computer-implemented steps described in the claims. Typically, system 100, system 200, and / or system 600 may be used in the description and execution of process 400.

[0075] At block 402, process 400 includes the detection by sensors on a wearable device of at least one cloud anchor that includes an identifier associated with a network and configured for a physical environment. For example, the image sensor 212 or audio sensor 216 of a wearable device 104-2 of a user 106 entering environment 102 can detect a cloud anchor 110 having an identifier 222 associated with a network (e.g., network 250) in physical environment 202. As described throughout this disclosure, cloud anchor 110 can be configured for a physical environment.

[0076] In some implementations, the identifier may include an SSID and login credentials, for example, if the network is a Wi-Fi network for a router device (e.g., controllable device 108-2). In some implementations, the identifier is associated with at least one controllable device, which is a device that provides access to the network. For example, the controllable device may be device 108-2 (i.e., a network router). In some implementations, the controllable device may instead be electronic assistant device 108-1, which may be configured to control a second controllable device 108-2 that provides access to the network (i.e., via a router).

[0077] At block 404, process 400 includes: triggering the extraction of an identifier from the at least one cloud anchor in response to detecting that the location associated with the at least one cloud anchor is within a threshold distance of the wearable device and that the wearable device can access the at least one cloud anchor. For example, in response to detecting that the location of cloud anchor 110 is within a threshold distance of wearable device 104-2 and that wearable device 104-2 can access the at least one cloud anchor 110, identifier 222 can be extracted from cloud anchor 110 and, for example, provided to one or more devices associated with a network, or to one or more devices associated with physical environment 102 or another environment communicatively coupled to physical environment 102 (e.g., controllable device 108-2). In some embodiments, proximity to a particular location is determined based on detecting that cloud anchor 110 is within the field of view of wearable device 104-2 or another device within environment 102.

[0078] In some implementations, the cloud anchor providing access to the network is an integrated component providing network access. For example, one or more cloud anchors (e.g., cloud anchor 110) can be configured to allow and disallow specific network access. In some implementations, cloud anchor 110 may include, for example, information that can be used as a gateway to connect to networks provided by controllable devices 108-1 and / or 108-2 and / or other controllable devices available within environment 102 and / or accessible to wearable device 104-2. This information may include usernames, passwords, location data used to verify device location via the cloud anchor, or other data stored at the cloud anchor in environment 102 that can be matched with visual data captured by wearable device 104-2.

[0079] In some implementations, the cloud anchor may not be directly associated with a controllable device, but may instead be configured as a resource to verify, for example, the identity of a user or device, the location of a user or device, or other details of environment 102, before generating a response to a request to authenticate a specific network or device in the opposing environment 102.

[0080] In some implementations, identifier 222 is associated with a 3D map 230 of the augmented reality (AR) environment 102. The 3D map 230 may include visual data 224 mapped to the physical environment 102 and stored at the at least one controllable device 108-2 within the physical environment 102. In some implementations, detecting that the wearable device 104-2 can access the at least one cloud anchor 110 includes determining whether the wearable device 104-2 provides data from sensor 212 to match at least a portion of the visual data 224 mapped to the physical environment 102.

[0081] At box 406, process 400 includes adding wearable device 104-2 to the network based on received authentication of the provided identifier. For example, authentication may be generated and sent to wearable device 104-2 by a cloud anchor. In some implementations, authentication may be generated and sent to wearable device 104-2 by controllable device 108-2. For example, upon receiving the correct identifier 222, wearable device 104-2 can be authenticated to controllable device 108-2, which may trigger wearable device 104-2 to automatically join the network provided by controllable device 108-2.

[0082] In some embodiments, the at least one controllable device 108-2 is configured to deauthenticate wearable device 104-2 from the network in response to detecting that the location of the wearable device is no longer within a threshold distance. In some embodiments, the threshold distance may correspond to a threshold time, during which time is determined that time has elapsed and device 104-2 can no longer view the specific cloud anchor 110 based on the detection that the specific wearable device 104-2 can no longer access the specific cloud anchor. At this time, Wi-Fi can be deauthenticated on wearable device 104-2. In some embodiments, the cloud anchor may be configured to trigger the deauthentication of a specific device based on detecting that such a device is outside the threshold distance.

[0083] In some implementations, process 400 includes the detection by sensors on a wearable device of at least one cloud anchor including an identifier associated with at least one controllable device configured for a physical environment. For example, the image sensor 212 or audio sensor 216 of a wearable device 104-2 of a user 106 entering environment 102 can detect a cloud anchor 110 having an identifier 222 associated with a controllable device 108-2 (e.g., a network router) in physical environment 102.

[0084] In some implementations, process 400 includes triggering the extraction of an identifier from the at least one cloud anchor and providing the identifier to the at least one controllable device in response to detecting that the location of the at least one controllable device is within a threshold distance of the wearable device and that the wearable device can access the at least one cloud anchor. For example, in response to detecting that the location of the at least one controllable device 108-2 is within a threshold distance of the wearable device 104-2 and that the wearable device 104-2 can access the at least one cloud anchor 110, identifier 222 can be extracted from the cloud anchor 110 and provided to the controllable device 108-2.

[0085] In some implementations, the identifier may include an SSID and login credentials, for example, if the network is a Wi-Fi network of a router (e.g., controllable device 108-2). The at least one controllable device may be a device providing access to the network. For example, the controllable device may be device 108-2 (i.e., the network router). In some implementations, the controllable device may instead be electronic assistant device 108-1, which may be configured to control a second controllable device 108-2 (i.e., the router) providing network access. In some implementations, the location is determined based on detecting that the cloud anchor 110 is within the field of view of the wearable device 104-2.

[0086] In some implementations, the controllable device that provides access to the network is an integrated component that provides access to the network. For example, if the controllable device is router controllable device 108-2, then device 108-2 can control access to the network and / or control access to other devices (e.g., 104-1, 104-2, 104-3, 106-1, 106-2, 108-1) that can access the network via router controllable device 108-2.

[0087] In some implementations, the cloud anchor 110 may include information that can be used as a gateway to connect to a network provided by controllable devices 108-1 and / or 108-2 and / or other controllable devices available within environment 102 and / or accessible to wearable device 104-2. This information may include a username, password, location data used to verify the device's location via the cloud anchor, or other data stored at the cloud anchor in environment 102 that can match visual data captured by wearable device 104-2.

[0088] In some implementations, the cloud anchor may not be directly associated with a controllable device, but may instead be configured as a resource for verifying the identity, location, etc. of a user or device, for example, before generating a response to a request to authenticate a specific network or device in the opposing environment 102.

[0089] In some implementations, identifier 222 is associated with a 3D map 230 of the augmented reality (AR) environment 102. The 3D map 230 may include visual data 224 mapped to the physical environment 102 and stored at the at least one controllable device 108-2 within the physical environment 102. In some implementations, detecting that the wearable device 104-2 can access the at least one cloud anchor 110 includes determining whether the wearable device 104-2 provides data from sensor 212 to match at least a portion of the visual data 224 mapped to the physical environment 102.

[0090] In some implementations, process 400 includes authentication at controllable device 108-2 based on the received identifier, adding wearable device 104-2 to the network. For example, upon receiving the correct identifier 222, wearable device 104-2 can be authenticated to controllable device 108-2, which can trigger wearable device 104-2 to automatically join the network provided by controllable device 108-2.

[0091] In some implementations, the at least one controllable device 108-2 is configured to deauthenticate wearable device 104-2 from the network in response to detecting that the location of the wearable device is no longer within a threshold distance. In some implementations, the threshold distance may correspond to a threshold time, during which time is determined that time has elapsed and device 104-2 can no longer view the specific cloud anchor 110 based on the detection that the specific wearable device 104-2 can no longer access the specific cloud anchor. At this time, Wi-Fi can be deauthenticated on wearable device 104-2.

[0092] Figure 5This is a flowchart illustrating an example of a computer-implemented process 500 for authenticating a controllable device according to embodiments described throughout this disclosure. In this example, a wearable device 104-1 may be communicatively coupled to a controllable device 108-1 (e.g., an electronic assistant device) that controls resources (e.g., music) accessible to an authenticated user. Typically, process 500 utilizes the systems and algorithms described herein to detect a request from an unknown speaker to controllable device 108-1, which may be configured to be operated by an authenticated device and / or user. Wearable device 104-1 may initiate the authentication process by determining its location (e.g., determining orientation, posture, etc.) with respect to the physical environment 102. A microphone associated with wearable device 104-1 may be used to determine the orientation of the speaker making the auditory request. Device owner 104 wearing wearable device 104-1 is prompted to grant or deny the request. In some embodiments, device owner 104 is prompted to look toward the speaker making the request (or in the direction of the speaker). If the device owner 104 grants the request, the area of ​​the room (e.g., described by the area of ​​the sphere) is marked as authenticated.

[0093] In some implementations, process 500 may also capture an image of the speaker's face to generate and store an embedding (e.g., a high-dimensional feature vector). The embedding does not represent a real-world identity, but rather represents the speaker's appearance (or parts thereof) that can be associated with a specific physical environment. In some implementations, the embedding is combined with an identity associated with controllable device 108-1 so that device 108-1 can be efficiently controlled by a user. For example, the identity may be combined with a speaker's playlist, and thus playing music (e.g., an indicator representing request 116) may trigger the playback of such a playlist.

[0094] In some implementations, the speaker's (e.g., the requester's) facial features can be compared with previously authorized faces stored at the wearable device or at a server computing system accessible to the wearable device. If the device owner 104 approves the received request, subsequent requests from the same speaker can be authenticated by that area of ​​the room from which the request originated (using the same location / pose as described above or using stored facial features).

[0095] Process 500 may utilize a computing system having at least one processing device and a memory storing instructions, which, when executed, cause the processing device to perform the plurality of operations and computer-implemented steps described in the claims. Typically, system 100, system 200, and / or system 600 may be used in the description and execution of process 500.

[0096] At block 502, process 500 includes receiving a request for access to a controllable device at a wearable device communicatively coupled to a controllable device in a physical environment. For example, wearable device 104-1 may receive a request from user 106, as indicated by audible request 114 in physical environment 102. In some embodiments, the request may instead be received from wearable device 104-2 or device 106-2. Wearable device 104-1 may use sensor system 210 and / or cloud anchor application 220 to determine that the request was received from an unauthenticated user or device (i.e., user 106 or device 104-2, 106-2). If it is determined that the request was received from an unidentified or unauthorized user, process 500 may initiate an authentication check process.

[0097] At box 504, process 500 includes determining the pose of the wearable device within its physical environment. For example, wearable device 104-1 may determine its own position 240 in environment 102 (e.g., 3DoF pose 240a, 6DoF pose 240b, or 5DoF pose 240c). The pose may be used in the event of authentication.

[0098] At box 506, process 500 includes rendering a prompt in the wearable device to select whether to grant or deny access to the controlled device by an unauthorized user or device. For example, a prompt may be generated by wearable device 104-1 to present information to user 104. The information (e.g., prompts 113, 115) may include cloud anchor data, UI prompts approving user and / or device access, or other data assisting user 104 in authorizing and / or otherwise authenticating access to one or more controlled devices. Other data may also be considered, including but not limited to image capture data, feature points associated with the cloud anchor, or other identifiable information from the scene. Such information can be used to generate specific prompts for the user.

[0099] At box 508, process 500 includes authenticating an unauthenticated user (e.g., user 106) or device (e.g., device 104-2 or device 106-2) from the area of ​​physical environment 102 (e.g., Figure 1Region A) accesses a controllable device (e.g., device 108-1, device 108-2) and executes request 116 (i.e., play music). For example, in response to receiving permission to access the controllable device (e.g., gesture 112 and / or input at prompt 113, etc.), process 500 may include authenticating an unauthenticated user or device to access the controllable device from a region of the physical environment. In response to receiving permission to access the controllable device, process 500 may also include executing the received request. The region (e.g., region A) may be defined based on the determined pose of wearable device 104-1. For example, the region may be defined within a spherical image of physical environment 102, wherein the spherical image is based on a 3D map 230 of at least one cloud anchor 110 associated with physical environment 102.

[0100] In some implementations, process 500 may further include rendering, for example, another prompt (e.g., prompt 115) in wearable device 104-1 to trigger the user of the wearable device to look in the direction corresponding to an area (e.g., area A). In response to the user 104 looking in the direction of area A, wearable device 104-1 may capture an image of the area, which may capture all or part of the user 106 (or device 106-2 or 104-2) requesting authentication. The image may be stored by wearable device 104-2 in a database of authenticated devices or users. In some implementations, for example, the image may instead be stored at server computer 202. Such an image may be used for future authentication requests.

[0101] For example, wearable device 104-1 may receive a second request to access controllable device 108-1. Wearable device 104-1 may then render a prompt to trigger user 104 to capture an image associated with the second request. Wearable device 104-1 may receive a command (from user 104) to capture the image associated with the second request and may do so. For example, wearable device 104-1 may compare the second captured image with images in database 242. In response to determining that the second captured image matches at least a portion of at least one image in database 242, wearable device 104-1 may trigger controllable device 108-1 to execute the second request.

[0102] In some implementations, the request is an auditory request spoken by user 106 within physical environment 102. Wearable device 104-1 can determine the direction in which the auditory request is received and can determine the area of ​​the physical environment based on the determined direction of the auditory request.

[0103] Figure 6Examples of computer devices 600 and 650 that can be used with the techniques described herein are shown. Computing device 600 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, tablet computers, workstations, personal digital assistants, smart devices, appliances, electronic sensor-based devices, televisions, servers, blade servers, mainframes, and other suitable computing devices. Computing device 650 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the inventions described and / or claimed in this document.

[0104] Computing device 600 includes a processor 602, a memory 604, a storage device 606, a high-speed interface 608 connected to the memory 604 and a high-speed expansion port 610, and a low-speed interface 612 connected to a low-speed bus 614 and the storage device 606. The processor 602 may be a semiconductor-based processor. The memory 604 may be a semiconductor-based memory. Each of components 602, 604, 606, 608, 610, and 612 is interconnected using various buses and may be suitably mounted on a common motherboard or otherwise mounted. The processor 602 can process instructions for execution within computing device 600, including instructions stored in the memory 604 or on the storage device 606, to display graphical information of a GUI on an external input / output device such as a display 616 coupled to the high-speed interface 608. In other embodiments, multiple processors and / or multiple buses, as well as multiple memories and memory types, may be suitably used. In addition, multiple computing devices 600 can be connected, each of which provides a portion of the necessary operation (e.g., as a server library, a set of blade servers, or a multiprocessor system).

[0105] Memory 604 stores information within computing device 600. In one embodiment, memory 604 is one or more volatile memory cells. In another embodiment, memory 604 is one or more non-volatile memory cells. Memory 604 may also be another form of computer-readable medium, such as a magnetic disk or optical disk. Typically, computer-readable media can be non-transitory computer-readable media.

[0106] Storage device 606 provides large-capacity storage for computing device 600. In one embodiment, storage device 606 may be or contain computer-readable media, such as floppy disk devices, hard disk devices, optical disk devices, magnetic tape devices, flash memory or other similar solid-state storage devices or device arrays, including devices in storage area networks or other configurations. The computer program product may be tangibly embodied in an information carrier. The computer program product may also contain instructions that, when executed, perform one or more methods and / or computer-implemented methods, such as those described above. The information carrier is a computer or machine-readable medium, such as memory 604, storage device 606, or memory on processor 602.

[0107] High-speed controller 608 manages bandwidth-intensive operations for computing device 600, while low-speed controller 612 manages lower bandwidth-intensive operations. This functional allocation is merely exemplary. In one embodiment, high-speed controller 608 is coupled to memory 604, display 616 (e.g., via a graphics processor or accelerator), and high-speed expansion port 610, which can accept various expansion cards (not shown). In this embodiment, low-speed controller 612 is coupled to storage device 606 and low-speed expansion port 614. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, Wireless Ethernet), can be coupled to one or more input / output devices, such as keyboards, pointing devices, scanners, or networking devices such as switches or routers, for example, via a network adapter.

[0108] Computing device 600 can be implemented in a variety of different forms, as shown in the figure. For example, it can be implemented as a standard server 620, or multiple times in a group of such servers. It can also be implemented as part of a rack server system 624. Alternatively, it can be implemented in a computer such as a laptop computer 622. Alternatively, components from computing device 600 can be combined with other components in a mobile device (not shown), such as device 650. Each of such devices can contain one or more of computing devices 600, 650, and the entire system can consist of multiple computing devices 600, 650 communicating with each other.

[0109] Computing device 650 includes processor 652, memory 664, input / output devices such as display 654, communication interface 666 and transceiver 668, and other components. Device 650 may also be provided with storage devices, such as microdrives or other devices, to provide additional storage. Each of components 650, 652, 664, 654, 666, and 668 is interconnected using various buses, and some of the components may be suitably mounted on a common motherboard or otherwise installed.

[0110] Processor 652 can execute instructions within computing device 650, including instructions stored in memory 664. The processor can be implemented as a chip set including discrete and multiple analog and digital processor chips. For example, the processor can provide coordination for other components of device 650, such as control of the user interface, applications running by device 650, and wireless communication of device 650.

[0111] Processor 652 can communicate with the user through control interface 658 and display interface 656 coupled to display 654. Display 654 can be, for example, a TFT LCD (Thin Film Transistor Liquid Crystal Display) or OLED (Organic Light Emitting Diode) display or other suitable display technology. Display interface 656 can include suitable circuitry for driving display 654 to present graphics and other information to the user. Control interface 658 can receive commands from the user and translate them for submission to processor 652. Additionally, an external interface 662 can be provided to communicate with processor 652 to enable near-field communication between device 650 and other devices. For example, in some embodiments, external interface 662 can provide wired communication, or in other embodiments, wireless communication, and multiple interfaces may also be used.

[0112] Memory 664 stores information within computing device 650. Memory 664 can be implemented as one or more computer-readable media, one or more volatile memory cells, or one or more non-volatile memory cells. Extended memory 674 can also be provided and connected to device 650 via extended interface 672, which may include, for example, a SIMM (Single In-line Memory Module) card interface. Such extended memory 674 can provide additional storage space for device 650, or it can also store applications or other information for device 650. Specifically, extended memory 674 may include instructions for performing or supplementing the above processes, and may also include security information. Therefore, for example, extended memory 674 can be provided as a security module of device 650 and can be programmed with instructions allowing secure use of device 650. Furthermore, security applications can be provided via a SIMM card along with additional information, such as placing identification information on the SIMM card in an unbreakable manner.

[0113] The memory may include, for example, flash memory and / or NVRAM memory, as discussed below. In one embodiment, the computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine-readable medium that can be received, for example, via transceiver 668 or external interface 662, such as memory 664, extended memory 674, or memory on processor 652.

[0114] Device 650 can communicate wirelessly via communication interface 666, which may include digital signal processing circuitry if necessary. Communication interface 666 can provide communication under various modes or protocols, such as GSM voice calls, SMS, EMS or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS. This communication can occur, for example, via radio frequency transceiver 668. Additionally, short-range communication can occur, such as using Bluetooth, Wi-Fi, or other transceivers (not shown). Furthermore, GPS (Global Positioning System) receiver module 670 can provide additional navigation and location-related wireless data to device 650, which can be appropriately used by applications running on device 650.

[0115] Device 650 can also communicate audibly using audio codec 660, which can receive spoken information from a user and convert it into usable digital information. Audio codec 660 can also generate audible sounds for the user, for example, through a speaker in the mobile phone of device 650. Such sounds can include sounds from voice phone calls, recorded sounds (e.g., voice messages, music files, etc.), and sounds generated by applications operating on device 650.

[0116] The computing device 650 can be implemented in a variety of different forms, as shown in the figure. For example, it can be implemented as a cellular phone 680. It can also be implemented as part of a smartphone 682, a personal digital assistant, or other similar mobile device.

[0117] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, specially designed ASICs (Application-Specific Integrated Circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs executable and / or interpretable on a programmable system, the programmable system including at least one programmable processor, which may be dedicated or general-purpose, coupled to receive and send data and instructions from and to a storage system, at least one input device, and at least one output device.

[0118] These computer programs (also referred to as modules, programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0119] To provide interaction with the user, the systems and techniques described herein can be implemented on a computer having a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor or LED (light-emitting diode)) and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user. For example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0120] The systems and technologies described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or middleware components (e.g., an application server), or front-end components (e.g., a client computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), and the Internet.

[0121] A computing system may include clients and servers. Clients and servers are typically geographically separated and usually interact through a communication network. The client-server relationship arises from computer programs running on the respective computers and having a client-server relationship with each other.

[0122] In some embodiments, Figure 6 The computing device depicted may include sensors that interface with a virtual reality or head-mounted device (VR head-mounted device / AR head-mounted device / HMD device 690). For example, in Figure 6 One or more sensors included in the computing device 650 or other computing device depicted may provide input to the AR / VR headset 690, or generally to the AR / VR space. Sensors may include, but are not limited to, touchscreens, accelerometers, gyroscopes, pressure sensors, biometric sensors, temperature sensors, humidity sensors, and ambient light sensors. The computing device 650 may use the sensors to determine the absolute position of the computing device in the AR / VR space and / or detected rotation, which can then be used as input to the AR / VR space. For example, the computing device 650 may be incorporated into the AR / VR space as a virtual object, such as a controller, laser pointer, keyboard, weapon, etc. Positioning of the computing device / virtual object by the user when incorporated into the AR / VR space allows the user to position the computing device to view the virtual object in some way within the AR / VR space.

[0123] In some embodiments, one or more input devices included on or connected to computing device 650 can be used as input to an AR / VR space. Input devices may include, but are not limited to, touchscreens, keyboards, one or more buttons, trackpads, touchpads, pointing devices, mice, trackballs, joysticks, cameras, microphones, headphones or earphones with input capabilities, game controllers, or other connectable input devices. When the computing device is incorporated into an AR / VR space, a user interacting with the input devices included on computing device 650 can induce specific actions within the AR / VR space.

[0124] In some embodiments, one or more output devices included on computing device 650 can provide output and / or feedback to a user of AR / VR headset 690 in an AR / VR space. The output and feedback can be visual, tactile, or audio. The output and / or feedback may include, but is not limited to, rendering an AR / VR space or virtual environment, vibration, turning on and off or flashing and / or blinking one or more lights or flashlights, emitting alarms, playing ringtones, playing songs, and playing audio files. Output devices may include, but are not limited to, vibration motors, vibration coils, piezoelectric devices, electrostatic devices, light-emitting diodes (LEDs), flashlights, and speakers.

[0125] In some embodiments, computing device 650 may be placed within AR / VR headset 690 to create an AR / VR system. AR / VR headset 690 may include one or more positioning elements that allow for proper positioning of computing device 650, such as smartphone 682, within AR / VR headset 690. In such embodiments, the display of smartphone 682 may render stereoscopic images representing AR / VR space or virtual environment.

[0126] In some embodiments, computing device 650 may represent another object in a computer-generated 3D environment. User interactions with computing device 650 (e.g., rotation, shaking, touching a touchscreen, swiping a finger across a touchscreen) can be interpreted as interactions with an object in the AR / VR space. As an example only, the computing device could be a laser pointer. In such an example, computing device 650 represents a virtual laser pointer in a computer-generated 3D environment. When a user manipulates computing device 650, the user in the AR / VR space perceives the laser pointer as moving. The user receives feedback from interactions with computing device 650 in the AR / VR environment on computing device 650 or on AR / VR headset 690.

[0127] In some embodiments, computing device 650 may include a touchscreen. For example, a user may interact with the touchscreen in a specific manner that mimics what is happening on the touchscreen based on what is happening in AR / VR space. For instance, a user may use pinch gestures to zoom in on content displayed on the touchscreen. This pinch gesture on the touchscreen can cause information provided in AR / VR space to be zoomed in or out. In another example, the computing device may be rendered as a virtual book in a computer-generated 3D environment. In AR / VR space, pages of the book can be displayed, and a user's finger swiping across the touchscreen can be interpreted as turning / flipping pages of the virtual book. As each page is turned / flipped, in addition to seeing the page content change, audio feedback, such as the sound of pages turning in a book, can be provided to the user.

[0128] In some embodiments, one or more input devices other than a computing device (e.g., a mouse, a keyboard) may be rendered in a computer-generated 3D environment. The rendered input devices (e.g., a rendered mouse, a rendered keyboard) may be used as if rendered in AR / VR space to control objects in AR / VR space.

[0129] Several embodiments have been described. However, it will be understood that various modifications can be made without departing from the spirit and scope of the invention.

[0130] Furthermore, the logical flow depicted in the accompanying drawings does not require the specific order or sequence shown to achieve the desired result. Additionally, other steps may be provided, or steps may be eliminated from the described flow, and other components may be added to or removed from the described system. Therefore, other embodiments are within the scope of the appended claims.

[0131] Further, the above description provides users with controls that allow them to choose whether and when the systems, programs, devices, networks, or features described herein can enable the collection of user information (e.g., information about the user's social networks, social actions or activities, occupation, user preferences, or the user's current location), and whether the user sends content or communications from the server. Additionally, some data may be processed in one or more ways before being stored or used, such that user information is removed. For example, a user's identity may be processed so that no user information can be identified against the user, or the user's geographic location may be generalized (e.g., to the city, zip code, or state level) when location information is available, making it impossible to determine the user's specific location. Therefore, users can have control over what information is collected, how that information is used, and what information is provided to the user.

[0132] Computer systems (e.g., computing devices) can be configured to communicate wirelessly with a network server via a communication link established with the network using any known wireless communication technologies and protocols, including radio frequency (RF), microwave frequency (MWF), and / or infrared frequency (IRF) wireless communication technologies and protocols suitable for communication over a network.

[0133] According to various aspects of this disclosure, embodiments of the various techniques described herein can be implemented in digital electronic circuit systems or in computer hardware, firmware, software, or combinations thereof. Embodiments can be implemented as computer program products (e.g., computer programs tangibly embodied in an information carrier, machine-readable storage device, computer-readable medium, or tangible computer-readable medium) for processing by or controlling the operation of a data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). In some embodiments, a tangible computer-readable storage medium can be configured to store instructions that, when executed, cause a processor to perform a process. Computer programs such as those described above can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for a computing environment. Computer programs can be deployed for processing on a single computer or on multiple computers distributed across a site or interconnected via a communication network.

[0134] The specific structural and functional details disclosed herein are merely for the purpose of describing exemplary embodiments. However, exemplary embodiments may be embodied in many alternative forms and should not be construed as being limited to the embodiments set forth herein.

[0135] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the embodiments. As used herein, the singular forms “a,” “an,” and “the / said” are also intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms “comprises,” “comprising,” “includes,” and / or “including” specify the presence of the said features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0136] It will be understood that when an element is referred to as being “coupled,” “connected,” or “in response to” another element or “on”, it can be directly coupled, connected, or in response to another element or on another element, or there may be intermediate elements present. Conversely, when an element is referred to as being “directly coupled,” “directly connected,” or “directly in response to” another element or “directly on”, there are no intermediate elements present. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items.

[0137] Spatial relative terms, such as “beneath,” “below,” “lower,” “above,” “upper,” etc., may be used herein for ease of description to describe an element or feature in relation to another element or feature shown in the accompanying drawings. It will be understood that spatial relative terms are intended to cover different orientations of the device in use or operation other than those depicted in the accompanying drawings. For example, if the device in the drawings is flipped, an element described as “below” or “beneath” to another element or feature would then be oriented as “above” to that element or feature. Thus, the term “below” can encompass both above and below orientations. The device may be oriented in other ways (rotated 70 degrees or rotated in other orientations), and the spatial relative descriptors used herein may be interpreted accordingly.

[0138] This document describes exemplary embodiments of the concept with reference to cross-sectional views of idealized embodiments (and intermediate structures) as example examples. As such, variations in the illustrated shapes due to manufacturing techniques and / or tolerances are expected. Therefore, the described exemplary embodiments of the concept should not be construed as limited to the specific shapes of the areas shown herein, but rather include deviations in shape due to manufacturing processes, for example. Thus, the areas shown in the figures are schematic in nature, and their shapes are not intended to represent the actual shapes of areas of the device, nor are they intended to limit the scope of the exemplary embodiments.

[0139] It will be understood that although the terms “first,” “second,” etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. Therefore, without departing from the teachings of this embodiment, a “first” element may be referred to as a “second” element.

[0140] Unless otherwise defined, the terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which these concepts belong. It will be further understood that terms such as those defined in common dictionaries should be interpreted as having the same meaning as they have in the relevant technical and / or specification context, and will not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0141] While certain features of the described embodiments have been shown as described herein, many modifications, substitutions, alterations, and equivalents will now occur to those skilled in the art. Therefore, it will be understood that the appended claims are intended to cover such modifications and alterations falling within the scope of the embodiments. It should be understood that they have been presented by way of example only and not as limitation, and various changes in form and detail are possible. Any part of the apparatus and / or methods described herein can be combined in any combination, except for mutually exclusive combinations. The embodiments described herein may include various combinations and / or sub-combinations of the functions, components, and / or features of the different embodiments described.

Claims

1. A computer-implemented method comprising: detecting, by a sensor on a wearable device, at least one cloud anchor, the at least one cloud anchor comprising an identifier associated with a network and configured for a physical environment, wherein the at least one cloud anchor comprises a set of visual features mapped to the physical environment; in response to detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device and that the wearable device has access to the at least one cloud anchor based on matching at least a portion of the set of visual features to data from the sensor, triggering extraction of the identifier from the at least one cloud anchor; and based on a received authentication corresponding to the extracted identifier, connecting the wearable device to the network.

2. The computer-implemented method of claim 1, wherein: the identifier comprises a SSID and login credentials; the network is a Wi-Fi network; the identifier is associated with at least one controllable device, the at least one controllable device being a device that provides access to the network; and the authentication is received from the at least one controllable device.

3. The computer-implemented method of claim 1, wherein: the set of visual features is included in a map that is mapped to the physical environment stored by at least one controllable device within the physical environment.

4. The computer-implemented method of claim 2, wherein, the at least one controllable device is configured to deauthenticate the wearable device from the network in response to detecting that the location of the wearable device is no longer within the threshold distance.

5. The computer-implemented method of any one of claims 1 to 4, wherein, the location is determined based on detecting that the at least one cloud anchor is within a field of view of the wearable device.

6. A computing system comprising: a wearable device comprising a sensor; at least one processing device; and a memory storing instructions that, when executed, cause the system to perform operations comprising: detecting, by the sensor of the wearable device, at least one cloud anchor, the at least one cloud anchor comprising an identifier associated with a network and configured for a physical environment, wherein the at least one cloud anchor comprises a set of visual features mapped to the physical environment; in response to detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device and that the wearable device has access to the at least one cloud anchor based on matching at least a portion of the set of visual features to data from the sensor, triggering extraction of the identifier from the at least one cloud anchor; and based on a received authentication corresponding to the extracted identifier, connecting the wearable device to the network.

7. The computing system of claim 6, wherein: the identifier comprises a SSID and login credentials; the identifier is associated with at least one controllable device, the at least one controllable device being a device that provides access to the network; and the authentication is received from the at least one controllable device. the at least one controllable device is configured to deauthenticate the wearable device from the network in response to detecting that the location of the wearable device is no longer within the threshold distance.

8. The computing system of claim 7, wherein, ​ 9. The computing system of claim 6, wherein: the set of visual features is included in a map that is mapped to the physical environment stored by at least one controllable device within the physical environment.

10. The computing system of any one of claims 6 to 9, wherein, the location is determined based on detecting that the at least one cloud anchor is within a field of view of the wearable device.

11. A non-transitory machine-readable medium having instructions stored thereon that, when executed by a processor, cause a computing device to: detecting, by a sensor on the wearable device, at least one cloud anchor, the at least one cloud anchor comprising an identifier associated with a network and configured for a physical environment, wherein, the at least one cloud anchor includes a set of visual features mapped to the physical environment; in response to detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor based on data from the sensor matching at least a portion of the set of visual features, trigger extraction of the identifier from the at least one cloud anchor; and connect the wearable device to the network based on a received authentication corresponding to the extracted identifier.

12. The machine-readable medium of claim 11, wherein: the identifier includes a SSID and login credentials; the identifier is associated with at least one controllable device that is a device that provides access to the network; and the authentication is received from the at least one controllable device.

13. The machine readable medium of claim 12, wherein, the at least one controllable device is configured to deauthenticate the wearable device from the network in response to detecting that the location of the wearable device is no longer within the threshold distance.

14. The machine-readable medium of claim 11, wherein: the set of visual features is included in a map that is mapped to the physical environment stored by at least one controllable device within the physical environment.

15. The machine readable medium of any one of claims 11 to 14, wherein, the location is determined based on detecting that the at least one cloud anchor is within a field of view of the wearable device.

Images