Using Cloud Anchors for Authentication

Abstract

A system and method for authenticating a device are described. The system and method may include a sensor on a wearable device detecting at least one cloud anchor including an identifier associated with a network and configured for a physical environment. In response to detecting a location associated with the at least one cloud anchor being within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor, triggering extraction of an identifier from the at least one cloud anchor. The system and method may further include joining the wearable device to a network based on a received authentication corresponding to the extracted identifier.

Description

[Technical field]

[0001] The present specification generally relates to authentication devices. [Background technology]

[0002] background Mobile devices may be used to control one or more electronic devices in buildings, such as homes and offices. Such buildings often include devices that can be accessed and / or controlled remotely. For example, light bulbs may be switched or adjusted by the mobile device using various types of wireless communications. In some examples, such mobile devices may include a camera for capturing image data and image recognition software for processing the image data to detect devices in the building that may be within the field of view of the camera mounted on the mobile device. Summary of the Invention [Means for solving the problem]

[0003] overview One or more computer systems may be configured to perform particular operations or actions by having software, firmware, hardware, or a combination thereof installed on the system that, when operated, causes the system to perform the actions. One or more computer programs may be configured to perform particular operations or actions by containing instructions that, when executed by a data processing device, cause the device to perform the actions.

[0004] In a first general aspect, a system and method are described for a sensor on a wearable device to detect at least one cloud anchor including an identifier associated with a network and configured for a physical environment. The system and method may include triggering extraction of an identifier from the at least one cloud anchor in response to detecting a location associated with the at least one cloud anchor being within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor, and joining the wearable device to the network based on a received authentication corresponding to the extracted identifier.

[0005] These and other aspects, alone or in combination, may include one or more of the following. For example, the system and method may further include the aspect that the identifier includes an SSID and login credentials, the network is a Wi-Fi network, the identifier is associated with at least one controllable device that is a device that provides access to the network, and the authentication is received from the at least one controllable device.

[0006] In some implementations, the identifier is associated with a 3D map of an Augmented Reality (AR) environment, the 3D map including visual data mapped to the physical environment stored by at least one controllable device in the physical environment, and detecting that the wearable device has access to the at least one cloud anchor includes determining whether the wearable device provides data from a sensor that matches at least a portion of the visual data mapped to the physical environment. In some implementations, the at least one controllable device is configured to de-authenticate the wearable device from the network in response to detecting that a location of the wearable device is no longer within a threshold distance. In some implementations, the location is determined based on detecting that at least one cloud anchor is within a field of view of the wearable device. Furthermore, the method of claim 1 above may include one or more (e.g., all) of the following features (or any combination thereof) of claims 6-10 described below.

[0007] In a second general aspect, a system and method are described for receiving a request to access a controllable device at a wearable device communicatively coupled to the controllable device in a physical environment, the request being determined to be received from an unauthenticated user or device. The system and method may further include determining a posture of the wearable device within the physical environment and rendering a prompt at the wearable device to select whether to allow or deny access to the controllable device by the unauthenticated user or device. In response to receiving authorization to access the controllable device, the system and method may include authenticating the unauthenticated user or device and executing the request to access the controllable device from an area of ​​the physical environment defined based on the determined posture of the wearable device.

[0008] These and other aspects, alone or in combination, may include one or more of the following: For example, the region may be defined within a spherical image of the physical environment, the spherical image being based on a 3D map of at least one cloud anchor associated with the physical environment. In some implementations, the system and method may include rendering another prompt at the wearable device to trigger a user of the wearable device to look in a direction corresponding to the region, capturing an image of the region capturing a portion of the user or device requesting authentication, and the wearable device storing the image in a database of authenticated devices or users.

[0009] In some implementations, the systems and methods include receiving a second request to access the controllable device, rendering a prompt at the wearable device to trigger a user of the wearable device to capture an image associated with the second request, receiving a command to capture the image associated with the second request, and comparing the captured second image with images in the database. The systems and methods may further include triggering execution of the second request in response to determining at least a portion of a match of the captured second image with at least one of the images in the database.

[0010] In some implementations, the request is an auditory request spoken by the user in the physical environment, the wearable device determines a direction from which the auditory request is received, and the region of the physical environment is determined based on the determined direction from which the auditory request is received. Further, the method as claimed in claim 6 above may include one or more (e.g., all) of the following features (or any combination thereof) of claims 1-5 above.

[0011] Implementations of the described techniques may include hardware, methods or processes, or computer software on a computer-accessible medium. Details of one or more implementations are set forth in the accompanying drawings and the following description. Other features will be apparent from the description and drawings, and from the claims. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 illustrates an augmented reality (AR) system for authenticating a wearable device, according to an implementation described throughout this disclosure. [Diagram 2] FIG. 1 is a block diagram illustrating an example system for authenticating a device in a physical and / or AR environment, in accordance with implementations described throughout this disclosure. [Figure 3A] A diagram showing an example of an electronic wearable device according to implementations described throughout this disclosure. [Figure 3B] A diagram showing an example of an electronic wearable device according to implementations described throughout this disclosure. [Figure 4] 4 is a flowchart illustrating an example process for joining a network, according to implementations described throughout this disclosure. [Diagram 5] 4 is a flowchart illustrating an example process for authenticating a controllable device, according to implementations described throughout this disclosure. [Figure 6] FIG. 1 illustrates an example of a computing device and a mobile computing device that can be used with the techniques described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Like reference numbers in the various drawings indicate like elements. Detailed Description This specification describes examples related to the general authentication of an augmented reality (AR) wearable electronic device (i.e., a wearable device). The authentication process may include an image-based evaluation of the environment surrounding the wearable device. The image-based evaluation may take into account positional aspects of the wearable device and / or electronic devices that the wearable device is trying to control or authenticate. In some implementations, the image-based evaluation may take into account the location of cloud anchors relative to the environment.

[0014] A device seeking authentication (e.g., a wearable device) may capture positional aspects of the environment and transmit such aspects (and / or other data) to other electronic devices in the environment. The other electronic devices in the environment may evaluate the positional aspects (and / or other data received from the wearable device) to determine whether to authenticate the wearable device and / or provide certain data and / or services to the wearable device via the other electronic device(s).

[0015] In some implementations, the wearable device described herein may be authenticated by one or more other devices in general in a physical, virtual, or AR environment. For example, an electronic assistant type device (e.g., home assistant, car assistant) can be used to authenticate other devices in a shared space as well as share certain settings and resources related to the electronic assistant. In some implementations, other controllable devices such as electronic thermostats, electronic doorbell devices, televisions, security devices, home appliances, or other electronic devices visible by the wearable device can be used to authenticate the wearable device and / or other devices in the shared space.

[0016] Conventional systems typically do not have a mechanism for authenticating requests from a device or user via an assistant device. In particular, conventional systems may receive and execute routine requests (e.g., control lights, control music, etc.) in shared settings (e.g., dorm rooms, hotels, office buildings, etc.) and execute such requests without the permission of the device owner.

[0017] Systems and methods described herein may provide techniques for authenticating such authorization by, for example, a guest-owned device. Such techniques may provide benefits to device owners by allowing them to assign control of such routine requests to allow certain users to perform the requests while prohibiting or blocking other users from performing the requests without the device owner's permission. These techniques may provide a way to set permissions for routine request devices (i.e., devices configured to receive keywords (e.g., an auditory request "OK, computer...", a device control request, etc.)). Permissions may be enabled by the device owner or may be bypassed upon receiving an indication (e.g., a gesture, selection, voice response, etc.) from the device owner to allow routine requests from a particular device and / or user. For example, the device owner may gesture toward a device location or user location associated with the request. In some implementations, user interface content may be displayed to the device owner on the wearable device display to request authentication of the requesting user and / or controllable device use approval.

[0018] In some implementations, the wearable devices described herein may leverage AR content (e.g., cloud anchors) to be authenticated to gain access to network-provided services and / or device functions (e.g., access to a Wi-Fi network, access to an AR session, access to devices in an environment, etc.). For example, the systems and techniques described herein may provide techniques for authenticating guest devices to access services in an environment. For example, traditional authentication to a Wi-Fi network / service involves a user who may have to request a password from another user, manually select a network, and manually enter a password. The systems and techniques described herein may leverage virtual cloud anchors (i.e., cloud anchors) to generically authenticate guest devices to Wi-Fi networks (or other services away from such networks) associated with an environment (e.g., a hotel, a home, an estate, an office building, etc.). Such authentication may be provided according to permissions set by the owner of the service and / or network.

[0019] In general, the present disclosure relates to the use of a wearable device that may capture visual data to identify three-dimensional (3D) map information (e.g., 3D meshes, 3D feature maps, cloud anchors, etc.) or two-dimensional (e.g., 2D) information about a particular physical environment based on the visual data to determine a location of the controllable device within a physical space represented at least in part by the visual data and / or to gain access to the controllable device. Such information may include any amount of visual positioning data about another controllable device that can be seen or detected by the wearable device. The information may be stored locally or remotely on a server computer. In some implementations, the information may be used to recover 2D information. In some implementations, the information may be used to detect a three degree of freedom (DoF) position of the controllable device. In some implementations, the information may be used to detect a 6DoF position of the controllable device. In some implementations, the information may be used to detect a 5DoF position of the controllable device. In some implementations, the information may be used to recover 2D and / or 5DoF map data.

[0020] Upon determining the location and / or authenticating analysis of other user input, the wearable device may be able to authenticate to the controllable device and begin controlling the controllable device. User interface (UI) content and / or audio signals may be provided to the wearable device to enable control and / or feedback to a user of the wearable device. In some implementations, the UI content may provide the user of the wearable device with one or more controls for controlling the controllable device and / or additional information regarding the controllable device.

[0021] The techniques described herein provide the advantage of preserving battery power for a wearable device by using a cloud anchor to authenticate the wearable device to a Wi-Fi network. Such techniques ensure that the wearable device does not need to utilize cellular services that may drain the electronic device's battery more quickly than utilizing Wi-Fi services. Instead, the techniques described herein may utilize cloud anchor information to authenticate the device and provide access to Wi-Fi or other networks to enable control of controllable devices associated with a physical and / or AR environment.

[0022] FIG. 1 illustrates an augmented reality (AR) system 100 for authenticating a wearable device to other electronic devices, according to implementations described throughout this disclosure. A user may enter a physical space, such as an environment 102, carrying one or more computing systems. For example, a first user 104 is shown seated, may be associated with (e.g., or may be located nearby) a computing system 106-1, and may wear a wearable device 104-1. The computing system 106-1 is a mobile device. The wearable device 104-1 is a pair of AR glasses. Similarly, a second user 106 may enter the environment 102. The second user may also have any number of electronic devices. In this example, the user 106 holds a mobile device 106-2 and wears a wearable device 104-2. Any number of computing systems may be included in the environment 102. Similarly, any number of wearable devices may be included in the environment 102. For example, additional wearable device 104-3 represents a ring controller that may interact with other devices in environment 102. Similarly, additional wearable device 104-4 represents a smart watch that may interact with other devices in environment 102.

[0023] In some implementations, the computing system 106-1 may be communicatively coupled to the wearable device 104-1. In some implementations, the wearable device 104-1 may be a standalone device that is not configured, connected, or in communication with another device associated with the user 104. In some implementations, the computing system 106-2 may be communicatively coupled to the wearable device 104-2. In some implementations, the wearable device 104-2 may be a standalone device that is not configured, connected, or in communication with another device associated with the user 106.

[0024] In some implementations, computing systems 106-1 and 106-2 are auxiliary (i.e., optional) and are not used for interaction within environment 102. For example, wearable device 104-1 may function standalone to experience networks, services, or other controllable devices within environment 102. Similarly, wearable device 104-2 may operate without computing system 106-2. Thus, wearable devices 104-1 and 104-2 may operate standalone. In some implementations, wearable device 104-2 may request access to services, networks, etc. via wearable device 104-1.

[0025] A user 104, 106 associated with one or more of the computing systems 106-1, 106-2 and / or wearable devices 104-1, 104-2 may desire to control one or more controllable devices in the environment 102. As shown in the environment 102, a first controllable device 108-1 is a digital assistant device and a second controllable device 108-2 is a Wi-Fi router. Any number of controllable devices 108 may be included in the environment 102. Controlling such controllable devices 108 may include evaluating and obtaining authorization from an owner of a particular controllable device 108. The computing systems 106-1, 106-2 and / or wearable devices 104-1, 104-2 may be authenticated by the systems and techniques described herein. Authentication may provide such systems and devices with access to services, networks, controllable devices, and / or permissions associated with such controllable devices. For example, the system 100 and environment 102 may be configured with cloud anchors 110 that may be detected and accessed to obtain information that may be used to authenticate the wearable devices 104-1, 104-2 to gain access to particular services, networks, and / or controllable devices. In some implementations, one wearable device (e.g., the host AR glasses 104-1) may provide authentication for another wearable device (e.g., the guest AR glasses 104-2).

[0026] As used herein, a cloud anchor (e.g., a virtual cloud anchor) may refer to one or more sets of visual feature points included in a 3D map. A cloud anchor may be associated with a set of persistent visual features that represent a physical object in the physical world. In the AR system described herein, a wearable device may receive digital information about a host user's environment and may create one or more cloud anchors, such as cloud anchor(s) 110, based on the digital information, where each cloud anchor 110 may represent a portion of a physical space, such as a room (or part thereof) in a building. The host user may add virtual objects to a virtual scene associated with the room, and the virtual objects may then be linked (or pinned) to a location in the cloud anchor 110.

[0027] The cloud anchor 110 may be shared with other users, who may join the space and view and interact with content or devices owned by the host user (or another user). In some examples, to allow other users to join the host's environment, the host user may locate an identifier for a particular cloud anchor and provide the identifier to the other users, who may manually enter the identifier to join the AR scene. In some implementations, the cloud anchor may be configured by the host to automatically authenticate a particular device to access objects, devices, services, and / or systems associated with (or communicatively coupled to access) the cloud anchor. Each device associated with the environment 102 (e.g., computing systems 106-1, 106-2 and wearable devices 104-1, 104-2) can access the cloud anchor 110 via image sensors associated with such devices.

[0028] In some implementations, a guest's wearable device may move through a physical environment collecting data associated with feature points of any number of cloud anchors 110. At some point, a combination of feature points may be provided as a specific identifier for gaining access to a network or device. The identifier may include one or more feature points associated with one or more cloud anchors 110. For example, if the wearable device 104-2 can generate an identifier associated with a particular cloud anchor that corresponds to or is configured with a controllable device, the wearable device 104-2 may then be authenticated to gain access to the network or device.

[0029] In operation, wearable device 104-1 may represent a host (i.e., the owner of controllable device 108). User 104 (i.e., the host / owner) may have controllable device 108-1 and / or controllable device 108-2 configured to provide general cloud anchor-based authentication to a guest user upon verifying that the guest user has access to cloud anchor 110 and provide such verification to wearable device 104-2, wearable device 104-3, wearable device 104-4, computing system 106-1, controllable device 108-1 (a digital assistant), or controllable device 108-2 (e.g., a Wi-Fi router), respectively.

[0030] In some implementations, cloud anchor 110 may be associated with any number of identifiers. For example, cloud anchor 110 may include a location identifier indicating the location of a physical or virtual object associated with a 3D map of a particular environment. Cloud anchor 110 may alternatively or additionally include identifiers indicating services, passwords, and / or other indicia for accessing services, networks, functions, features, etc.

[0031] In one non-limiting example, the system 100 may provide access to a network to a wearable device. For example, a user 106 may walk into a lobby environment 102 wearing a wearable device 104-2 (e.g., AR glasses). The wearable device 104-2 may include at least one image sensor that may detect at least one cloud anchor (e.g., cloud anchor 110). The cloud anchor 110 may include an identifier associated with at least one controllable device (e.g., controllable device 108-2 representing a Wi-Fi network router). The controllable device 108-2 may be configured to provide Wi-Fi service to any wearable device for which the physical environment 102 detects, for example, the cloud anchor 110. For example, in response to detecting a location of at least one controllable device (e.g., Wi-Fi router 108-2) to be within a threshold distance of the wearable device 104-2, the wearable device 104-2 may trigger extraction of an identifier from the at least one cloud anchor 110. For example, the wearable device may automatically request login credentials (e.g., SSID, username and password) from the device 108-2 when it moves within a predefined threshold distance of the device 108-2 and / or if it moves within viewing distance of the device 108-2 (i.e., the wearable device 104-2 may have a cloud anchor 110 within the field of view, detectable distance, and / or another visually detectable range of the device 108-2). In some implementations, the threshold distance may correspond to a threshold time that elapses after which it is determined that the device 104-2 can no longer see a particular cloud anchor based on detecting that the particular wearable device 104-2 no longer has access to the particular cloud anchor 110. At such time, Wi-Fi may be de-authenticated at the wearable device 104-2.

[0032] In some implementations, the threshold distance may relate to a full or partial line of sight of a particular device. For example, the threshold distance may relate to ensuring a wearable device (e.g., wearable device 104-2) has line of sight to controllable device 108-2. Another example of the threshold distance may relate to, for example, ensuring that wearable device 104-2 has at least a partial view (i.e., partial line of sight) of a particular controllable device in environment 102.

[0033] Upon requesting identifier information, the wearable device 104-2 may provide the information to at least one controllable device in a sequence recognized by the device 104-2 such that authentication may occur. For example, upon providing identifier information, the wearable device 104-2 may join the network based on a received authentication of the provided identifier. The device 108-2 may authenticate or the user 104 may authenticate via another device or wearable device associated with the environment 102 and / or the controllable device 108-2. For example, the at least one controllable device may be an electronic assistant (e.g., the controllable device 108-1) configured with the controllable device 108-2 (i.e., a network Wi-Fi router) providing the network.

[0034] In another non-limiting example, the system 100 may authenticate another user or device to use, control, or otherwise access a particular controllable device associated with the environment 102. For example, a user 106 may represent a guest and may enter the lobby (i.e., physical environment) of the environment 102. The user 106 may be associated with one or more electronic devices (as shown by devices 104-2 and 106-2). However, the user may not possess any devices and may not be associated with any devices in the environment 102. Thus, the system described herein may authenticate to a device and / or authenticate a particular user based on a voice (or visual) request received from such user.

[0035] In this example, user 104 may represent environment 102 and a host of controllable devices within environment 102 (e.g., controllable device 108-1 and controllable device 108-2). User 104 may wear wearable device 104-1 (e.g., AR glasses) that may be used to authenticate other users to devices within environment 102. For example, user 106 may enter environment 102 and provide a request to access, for example, controllable device 1081. The request may be made via device 104-2, device 106-2, or another device associated with user 106. In some implementations, the request may instead include an audio utterance or a visual indicator performed by user 106.

[0036] A user 106 may execute a request to use, control, or otherwise access an electronic assistant (such as controllable device 108-1). In operation, system 100 may receive a request at wearable device 104-1 (or device 104-3 or device 104-4). One or all of wearable devices 104-1, 104-3, 104-4 may be communicatively coupled to controllable device 108-1 such that any of devices 104-1, 104-3, 104-4 may be authorized to access controllable device 108-1. In this example, the request is determined, for example, by wearable device 104-1 to have been received from an unauthorized user or device (e.g., originating from user 106 or received from unauthorized computing device 106-2 or wearable device 104-2).

[0037] Upon receiving a request from an unauthorized user or device, the wearable device 104-1 may determine a pose of itself. The pose may be that of the wearable device 104-1 owned by the host and in the physical environment. The wearable device 104-1 may then trigger the rendering (on a display associated with itself) of a prompt 113 to select whether to allow or deny the unauthorized user 106 or device (device 106-2 or device 104-2) access to the controllable device 108-1. The prompt 113 depicts a question and a selectable control, although other prompts are of course possible. For example, an additional prompt 115 instructs the user 104 to look at a door in the environment 102 to receive an authentication request. The user 104 may perform a gesture of looking at the door of a room to receive (i.e., to be served with) such an authentication request.

[0038] In some implementations, a user 104 of the wearable device 104-1 may perform a gesture 112 within the view of a camera associated with the wearable device 104-1, for example, to indicate approval for the device 104-1 (i.e., the AR glasses). Such approval may then allow authentication of the user 106 or the device 104-2 or 106-2, thereby allowing a request to be performed at the controllable device 108-1. For example, if the request was from the device 106-2 to play music from a selected playlist, or if the request was an audible request 114 to play music, authentication would allow the device 108-1 to play music, as indicated by an indicator representing the request 116, based at least in part on the approval associated with the wearable device 104-1.

[0039] In response to receiving permission to access the controllable device 108-1 or in response to a gesture visible by the device 104-1 being performed by the user 104, the wearable device 104-1 may authenticate the unauthorized user or device to access the controllable device 108-1 from a region of the physical environment and then execute the request. The region of the physical environment may be defined based on a determined pose of the wearable device. For example, the region may be determined using an on-board camera of the wearable device 104-1 or an audio sensor associated with the device 104-1. In this example, region A is one example of a region defined based on the position of the user 106 (or a device owned by the user 106). Of course, other defined regions and sub-regions within the environment 102 are possible.

[0040] 2 is a block diagram illustrating an example system 200 for authenticating a device in a physical and / or AR environment, according to implementations described throughout this disclosure. The system 200 includes at least a first wearable device 104-1, as described in FIG. 1. The device 104-1 may receive a request to access a controllable device 108. In some implementations, a server computer 202 may be utilized to access, generate, view, and use cloud anchors. In some implementations, the wearable computing device 104-1 may perform and store operations described with respect to the server computer 202, obviating the use of the server computer 202 in the techniques described herein.

[0041] In some examples, the wearable device 104-1 includes one or more computing devices, and at least one of the devices is a display device that can be worn on or in close proximity to a person's skin. In some examples, the wearable device 104-1 is or includes a wearable device. The wearable device may include a head-mounted display (HMD) device, such as an optical head-mounted display (OHMD) device, a transparent heads-up display (HUD) device, an augmented reality (AR) device, or other devices such as goggles or headsets that have sensors, displays, and computing capabilities. In some examples, the wearable device includes AR glasses (e.g., smart glasses). AR glasses are optical head-mounted display devices designed in the shape of glasses.

[0042] In some examples, the wearable device 104-1 includes a wearable device (e.g., AR glasses) and a computing device (e.g., a mobile computing device such as a smartphone, tablet, laptop, or another wearable device such as a smartwatch). The wearable device 104-1 may be connected to the computing device via a wireless connection, such as a short-range connection (e.g., a Bluetooth® connection or a near-field communication (NFC) connection) or an Internet connection (e.g., Wi-Fi or a mobile network). In some examples, some of the components of the wearable device 104-1 are included in the wearable device and some of the components of the wearable device 104-1 are included in the computing device.

[0043] The wearable device 104-1 includes one or more processors 204, which may be formed on a substrate configured to execute one or more machine-executable instructions or software, firmware, or a combination thereof. The processors 204 may be semiconductor-based, i.e., the processors may include semiconductor material capable of executing digital logic. Such processors 204 may include a CPU, a GPU, and / or a DSP, to name just a few examples.

[0044] The wearable device 104-1 may also include one or more memory devices 206. The memory device 206 may include any type of storage device that stores information in a format that can be read and / or executed by the processor(s) 204. The memory device 206 may store applications and modules that perform certain operations when executed by the processor(s) 204. In some examples, the applications and modules may be stored in an external storage device and loaded into the memory device 206. The wearable device 104-1 includes one or more antennas (not shown) configured to communicate with other computing devices.

[0045] The wearable device 104-1 includes a display 208. The display 208 may include a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting display (OLED), an electro-phoretic display (EPD), or a micro-projection display employing an LED light source. In some examples, the display 208 is projected into the user's field of view. In some examples, in the case of AR glasses, the display 208 may provide a transparent or translucent display so that a user wearing the glasses can see both the image provided by the display 208 and information located in the field of view of the AR glasses behind the projected image.

[0046] The wearable device 104-1 includes a sensor system 210. The sensor system 210 includes an image sensor 212 configured to acquire image data. In some examples, the sensor system 210 includes multiple image sensors 212. The image sensor 212 is capable of taking pictures and recording videos. The sensor system 210 may include an inertial measurement unit (IMU) 214. The IMU 214 may detect motion, movement, and / or acceleration of the wearable device 104-1. The IMU 214 may include a variety of different types of sensors, such as, for example, accelerometers, gyroscopes, magnetometers, and other such sensors. The sensor system 210 includes an audio sensor 216 configured to detect audio received by the wearable device 104-1. The sensor system 210 may include other types of sensors, such as light sensors, distance and / or proximity sensors, contact sensors such as capacitance sensors, timers, other sensors, and / or different combination(s) of sensors. The sensor system 210 may be used to obtain information related to cloud anchor data 218 using a cloud anchor application 220, as well as identifiers 222 associated with a 3D map 230 related to such cloud anchor.

[0047] The identifier 222 may include information related to one or more cloud anchors 110. The information may include location data corresponding to a computing device, a wearable device, a virtual object, and / or a physical object associated with a physical environment. In some implementations, the information may include configuration data for networking, joining an AR session, gaining device control, etc. In some implementations, the information included in the identifier 222 may include image data, such as images stored for authentication purposes, such as may be stored in the image authentication database 242.

[0048] The system 200 includes a cloud anchor application 220 that may be used to interact with a cloud anchor for purposes of authenticating specific locations and features associated with devices and images captured by such devices. For example, when the wearable computing device 104-1 receives a request to authenticate a particular user or device, the system 200 may be accessed to determine the location of one or more controllable devices 108 within the environment 102. A user operating the wearable computing device 104-1 may aim the image sensor 212 of the sensor system 210 at a particular other user or device (e.g., at the controllable device 108-1) and move the wearable device 104-1 to map the environment from different viewing angles and positions. Such maps may be stored as cloud anchor data 218 in the map database 234 of the server or on the device.

[0049] During operation, the cloud anchor application 220 receives data from the sensor system 210 and generates cloud anchor data 218 including visual data 224, device pose(s) 226, and anchor pose(s) 228. The wearable device 104-1 may generate or access a 3D map 230 based on the cloud anchor data 218. In some implementations, the wearable device 104-1 may instead access the cloud anchor data 218 at the server computer 202. The server computer 202 may include a visual positioning data service 232 for generating the cloud anchor data 218 and generating the map 230 for the map database 234. The map 230 may be generated by a 3D map generator 236, which in this example is associated with a particular physical and / or virtual environment.

[0050] The wearable device 104-1 may determine an attitude and / or position 240 based on the on-board camera 238 or image sensor 212. The position 240 may be a 3D position of the controllable device 108-1, which is the position (and, optionally, the orientation) of the controllable device 108-1 in a 3D physical environment in which the user (and / or the wearable device 104-1) is operating. In some examples, the position includes a 3DoF position 240a of the controllable device or wearable device. In some examples, the position 240 includes a 6DoF position 240b of the controllable device or wearable device. In some implementations, the position 240 includes a 5DoF position 240c of the controllable device or wearable device.

[0051] The wearable device 104-1 also includes a gesture detector 244. The gesture detector 244 may access the sensor system 210 to detect and translate received gestures captured by the camera 238 and / or image sensor 212 of the device 104-1. In some implementations, the gesture detector 244 may recognize gestures and translate them into inputs for accessing and / or controlling a particular controllable device. In some implementations, the gesture detector 244 receives user-provided gestures, interprets gesture patterns, and translates the patterns into inputs for, for example, controllable devices in the environment 102. In some implementations, the camera 238 may represent a low-resolution camera of the wearable device 104-4, where the camera is embedded in the edge of a smartwatch, and is capable of tracking, viewing, and interpreting image data captured from the cloud anchor 110 for authentication purposes.

[0052] The wearable device 104-1 also includes a control system 246 that includes various control system devices for facilitating operation of the wearable device 104-1. The control system 246 may include a processor 314 operably coupled to the components of the wearable device 104-1.

[0053] The wearable device 104-1 also includes a communications module 248. The communications module 248 may enable the wearable device 104-1 to communicate to exchange information with another computing device and to authenticate other devices within range of the device 104-1 or other identifiable elements in the environment. For example, the wearable device 104-1 may be operatively coupled to another computing device to facilitate communication, e.g., via a wired connection, a wireless connection, e.g., via Wi-Fi or Bluetooth, or other type of connection.

[0054] In some implementations, the wearable device 104-1 is configured to communicate with a server computer 202 over a network 250. The server computer 202 may represent one or more computing devices in the form of a number of different devices, such as, for example, a standard server, a group of such servers, or a rack server system. In some implementations, the server computer 202 is a single system that shares components such as a processor and memory. The network 250 may include the Internet and / or a local area network (LAN), a wide area network (WAN), a cellular network, a satellite network, or other types of data networks. The network 250 may also include any number of computing devices (e.g., computers, servers, routers, network switches, etc.) configured to receive and / or transmit data within the network 250.

[0055] The server computer 202 includes one or more processors (not shown), which may be formed on a substrate configured to execute one or more machine-executable instructions or software, firmware, or a combination thereof. The processor(s) may be semiconductor-based, i.e., the processor may include semiconductor material capable of executing digital logic. The server computer 202 includes one or more memory devices (not shown). The memory devices may include a main memory that stores information in a format that can be read and / or executed by the processor.

[0056] In some examples, the server computer 202 is configured to execute a visual positioning data service 232. The visual positioning data service 232 can be an augmented reality (AR) collaboration service that enables users to create cloud anchors (e.g., 3D maps 230) for creating multiplayer or collaborative AR experiences that users can share. For example, users can configure virtual content and / or controllable devices 108 that other users can view, authenticate, and / or otherwise interact with from different positions within a shared physical environment.

[0057] For example, a user may create a local cloud anchor (e.g., 3D map 230) in their environment. During hosting, the wearable device 104-1 can upload data to generate the 3D map 230 in the visual positioning data service 232, which returns a unique identifier 222 for accessing the particular controllable device. The unique identifier 222 can be distributed to other users to join the same AR environment or, in some implementations, to access the controllable device after authenticating using the identifier 222 associated with the cloud anchor data 218.

[0058] 2 shows a single wearable device 104-1 and two controllable devices 108, the implementations described herein may encompass any number of such systems (e.g., three or more). Although FIG. 1 shows fewer device elements than the system shown in FIG. 2, the device shown in FIG. 1 may include (or have access to) any or all of the elements of the system in FIG. 2.

[0059] 3A-3B are various diagrams illustrating an example of an AR wearable device according to implementations described throughout this disclosure. FIG. 3A is a front view illustrating an example of a wearable device according to implementations described throughout this disclosure. In this example, the wearable device is AR glasses 300A (e.g., wearable electronic device 104-1 of FIG. 1). In general, the AR glasses 300A may include any or all components of system 200. The AR glasses 300A may be illustrated as smart glasses, which represent an optical head-mounted display device designed in the shape of glasses. For example, smart glasses are glasses that add information (e.g., project a display) next to what the wearer sees through the glasses.

[0060] Although AR glasses 300A are shown as the wearable electronic device described herein, other types of wearable devices are possible. For example, the wearable device may include a head-mounted display (HMD) device, such as an optical head-mounted display (OHMD) device, a transparent head-up display (HUD) device, an augmented reality (AR) device, or other devices such as goggles or headsets having sensors, displays, and computing capabilities. In some examples, the wearable device may be a watch, a mobile device, jewelry, a ring controller, or other wearable controller.

[0061] 3A , the AR glasses 300A include a frame 302 having a display device 304 coupled within the frame 302 (or within a glass portion of the frame 302). The AR glasses 300A also include an audio output device 306, an illumination device 308, a sensing system 310, a control system 312, at least one processor 314, and a camera 316.

[0062] The display device 304 may include a see-through near-eye display, such as one that uses birdbath optics or waveguide optics. For example, such an optical design may project light from a display source onto a portion of teleprompter glass that acts as a beam splitter placed at a 45-degree angle. The beam splitter may allow for reflection and transmission values ​​that partially reflect the light from the display source while transmitting the remaining light. Such an optical design may enable the user to see both physical items in the world next to the digital images (e.g., UI elements, virtual content, etc.) generated by the display. In some implementations, waveguide optics may be used to represent content on the display device 304 of the AR glasses 300A.

[0063] An audio output device 306 (e.g., one or more speakers) may be coupled to the frame 302. The sensing system 310 may include various sensing devices and a control system 312 including various control system devices to facilitate operation of the AR glasses 300A. The control system 312 may include a processor 314 operably coupled to the components of the control system 312.

[0064] The camera 316 may be capable of capturing still and / or video images. In some implementations, the camera 316 may be a depth camera capable of collecting data related to the distance of an external object from the camera 316. In some implementations, the camera 316 may be a point tracking camera capable of detecting and tracking one or more optical markers on an external device, such as, for example, an optical marker on an input device or a finger on a screen. In some implementations, the AR glasses 300A may include an illumination device 308 selectively operable, for example, with the camera 316, for detection of objects (e.g., virtual and physical) within the field of view of the camera 316. The illumination device 308 may be selectively operable, for example, with the camera 316, for detection of objects within the field of view of the camera 316.

[0065] The AR glasses 300A may include a communication module (e.g., the communication module 248) that communicates with the processor 314 and the control system 312. The communication module may enable communication between devices housed within the AR glasses 300A as well as communication with external devices, such as, for example, a controller, a mobile device, and / or other computing devices. The communication module may enable the AR glasses 300A to communicate to exchange information with another computing device and to authenticate other devices within range of the AR glasses 300A or other identifiable elements in the environment. For example, the AR glasses 300A may be operably coupled to another computing device to facilitate communication, for example, via a wired connection, a wireless connection, for example via Wi-Fi or Bluetooth, or other type of connection.

[0066] FIG. 3B is a rear view 300B showing AR glasses 300A according to implementations described throughout this disclosure. The AR glasses 300B may be an example of the wearable device 104-1 or 104-2 100 of FIG. 1. The AR glasses 300B are glasses that add information (e.g., project a display 320) next to what the wearer sees through the glasses. In some examples, instead of projecting information, the display 320 is an in-lens microdisplay. In some examples, the AR glasses 300B (e.g., eyeglasses or spectacles) are visual aids that include lenses 322 (e.g., glass or hard plastic lenses) attached to a frame 302 that holds the glasses in front of a person's eyes, typically utilizing a bridge 324 over the nose and temples 326 (e.g., temples or temple pieces) that rest on the ears.

[0067] 4 is a flow chart illustrating an example of a computer-implemented process 400 for coupling to a network, according to implementations described throughout this disclosure. In general, process 400 utilizes the systems and algorithms described herein to enable a user device (e.g., a wearable device) to scan a physical environment to identify a number of cloud anchors in the environment, obtain relevant AR content using the cloud anchors, and identify and / or obtain network credentials using the cloud anchors that may enable the user device to join a pre-provisioned network in the environment based on use of the identified network credentials.

[0068] Process 400 may utilize a computing system comprising at least one processing device and a memory storing instructions that, when executed, cause the processing device to perform the operations and computer-implemented steps recited in the claims. In general, systems 100, 200 and / or system 600 may be used in describing and implementing process 400.

[0069] At block 402, process 400 includes a sensor on a wearable device detecting at least one cloud anchor that includes an identifier associated with a network and that is configured for the physical environment. For example, an image sensor 212 or an audio sensor 216 of a wearable device 104-2 of a user 106 entering an environment 102 may detect a cloud anchor 110 having an identifier 222 associated with a network (e.g., network 250) in the physical environment 202. The cloud anchor 110 may be configured for the physical environment as described throughout this disclosure.

[0070] In some implementations, the identifier may include, for example, an SSID and login credentials if the network is a Wi-Fi network for a router device (e.g., controllable device 108-2). In some implementations, the identifier is associated with at least one controllable device that is a device that provides access to the network. For example, the controllable device may be device 108-2 (i.e., a network router). In some implementations, the controllable device may instead be electronic assistant device 108-1, which may be configured to control a second controllable device 108-2 that provides the network (i.e., via a router).

[0071] At block 404, process 400 includes triggering extraction of an identifier from at least one cloud anchor in response to detecting a location associated with the at least one cloud anchor within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor. For example, in response to detecting a location of the cloud anchor 110 within a threshold distance of the wearable device 104-2 and detecting that the wearable device 104-2 has access to the at least one cloud anchor 110, the identifier 222 may be extracted from the cloud anchor 110 and provided, for example, to one or more devices associated with the network or to one or more devices associated with the physical environment 102 or another environment communicatively coupled to the physical environment 102 (e.g., controllable device 108-2). In some implementations, proximity to a particular location is determined based on detecting that the cloud anchor 110 is within a field of view of the wearable device 104-2 or another device in the environment 102.

[0072] In some implementations, the cloud anchor providing access to the network is an essential component of providing the network. For example, one or more cloud anchors (e.g., cloud anchor 110) may be configured to allow and disallow certain network access. In some implementations, cloud anchor 110 may contain information usable as a gateway to connect to networks provided by controllable device 108-1 and / or 108-2 and / or other controllable devices available in environment 102 and / or other controllable devices accessible to wearable device 104-2, for example. The information may include username, password, location data to identify the device via the cloud anchor, or other data stored in the cloud anchor of environment 102 that may match visual data captured by wearable device 104-2.

[0073] In some implementations, a cloud anchor may not be directly associated with a controllable device, but instead may be configured as a resource for verifying, for example, a user or device ID, a user or device location, or other environment 102 details, before generating a response to an authentication request to a particular network or device within the environment 102.

[0074] In some implementations, the identifier 222 is associated with a 3D map 230 of the augmented reality (AR) environment 102. The 3D map 230 may include visual data 224 mapped to the physical environment 102 and stored on at least one controllable device 108-2 within the physical environment 102. In some implementations, detecting that the wearable device 104-2 has access to the at least one cloud anchor 110 includes determining whether the wearable device 104-2 provides data from a sensor 212 that matches at least a portion of the visual data 224 mapped to the physical environment 102.

[0075] At block 406, the process 400 includes coupling the wearable device 104-2 to the network based on the received authentication of the provided identifier. For example, the authentication may be generated by the cloud anchor and sent to the wearable device 104-2. In some implementations, the authentication may be generated by the controllable device 108-2 and sent to the wearable device 104-2. For example, upon receiving the correct identifier 222, the wearable device 104-2 may be authenticated to the controllable device 108-2 and trigger automatic joining of the wearable device 104-2 to the network provided by the controllable device 108-2.

[0076] In some implementations, at least one controllable device 108-2 is configured to de-authenticate the wearable device 104-2 from the network in response to detecting that the location of the wearable device is no longer within a threshold distance. In some implementations, the threshold distance may correspond to a threshold time after which it is determined that the device 104-2 can no longer see a particular cloud anchor based on detecting that the particular wearable device 104-2 is no longer accessible to the particular cloud anchor 110. At such time, Wi-Fi may be de-authenticated on the wearable device 104-2. In some implementations, the cloud anchor may be configured to trigger de-authentication of a particular device based on detecting that such device is outside the threshold distance.

[0077] In some implementations, the process 400 includes a sensor on the wearable device detecting at least one cloud anchor that includes an identifier associated with at least one controllable device configured for the physical environment. For example, an image sensor 212 or an audio sensor 216 of a wearable device 104-2 of a user 106 entering an environment 102 may detect a cloud anchor 110 having an identifier 222 associated with a controllable device 108-2 (e.g., a network router) in the physical environment 102.

[0078] In some implementations, the process 400 includes, in response to detecting a location of the at least one controllable device within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor, triggering extraction of an identifier from the at least one cloud anchor and providing the identifier to the at least one controllable device. For example, in response to detecting a location of the at least one controllable device 108-2 within a threshold distance of the wearable device 104-2 and detecting that the wearable device 104-2 has access to the at least one cloud anchor 110, the identifier 222 may be extracted from the cloud anchor 110 and provided to the controllable device 108-2.

[0079] In some implementations, the identifier may include, for example, an SSID and login credentials if the network is a Wi-Fi network for a router (e.g., controllable device 108-2). At least one controllable device may be a device that provides access to the network. For example, the controllable device may be device 108-2 (i.e., a network router). In some implementations, the controllable device may instead be electronic assistant device 108-1 that may be configured to control a second controllable device 108-2 that provides the network (i.e., a router). In some implementations, the location is determined based on detecting that the cloud anchor 110 is within the field of view of the wearable device 104-2.

[0080] In some implementations, the controllable device that provides access to a network is an integral component of providing the network. For example, if the controllable device is a router controllable device 108-2, the device 108-2 may control access to the network and / or control access for other devices (e.g., 104-1, 104-2, 104-3, 106-1, 106-2, 108-1) that may access the network via the router controllable device 108-2.

[0081] In some implementations, the cloud anchor 110 may contain information that can be used as a gateway to connect to networks provided by, for example, the controllable devices 108-1 and / or 108-2 and / or other controllable devices available in the environment 102 and / or devices accessible to the wearable device 104-2. The information may include usernames, passwords, location data for validating the device via the cloud anchor, or other data stored in the cloud anchor of the environment 102 that may match visual data captured by the wearable device 104-2.

[0082] In some implementations, a cloud anchor may not be directly associated with a controllable device, but instead may be configured as a resource that verifies the identity of a user or device, the location of a user or device, etc., before generating a response to an authentication request for a particular network or device within the environment 102, for example.

[0083] In some implementations, the identifier 222 is associated with a 3D map 230 of the augmented reality (AR) environment 102. The 3D map 230 may include visual data 224 mapped to the physical environment 102 and stored on at least one controllable device 108-2 within the physical environment 102. In some implementations, detecting that the wearable device 104-2 has access to the at least one cloud anchor 110 includes determining whether the wearable device 104-2 provides data from a sensor 212 that matches at least a portion of the visual data 224 mapped to the physical environment 102.

[0084] In some implementations, the process 400 includes coupling the wearable device 104-2 to the network based on a received authentication of an identifier provided at the controllable device 108-2. For example, upon receiving the correct identifier 222, the wearable device 104-2 may be authenticated to the controllable device 108-2, triggering automatic joining of the wearable device 104-2 to the network provided by the controllable device 108-2.

[0085] In some implementations, at least one controllable device 108-2 is configured to deauthorize the wearable device 104-2 from the network in response to detecting that the location of the wearable device is no longer within a threshold distance. In some implementations, the threshold distance may correspond to a threshold time after which it is determined that the device 104-2 can no longer see a particular cloud anchor based on detecting that the particular wearable device 104-2 no longer has access to the particular cloud anchor 110. At such time, Wi-Fi may be deauthorized at the wearable device 104-2.

[0086] FIG. 5 is a flow chart illustrating an example of a computer-implemented process 500 for authenticating to a controllable device according to implementations described throughout this disclosure. In this example, a wearable device 104-1 may be communicatively coupled to a controllable device 108-1 (e.g., an electronic assistant device) that controls a resource (e.g., music) accessible by an authenticated user. In general, the process 500 utilizes the systems and algorithms described herein to detect that an unknown speaker has provided a request for the controllable device 108-1, which may be configured to be operated by an authenticated device and / or user. The wearable device 104-1 may begin the authentication process by localizing (e.g., determining position, pose, etc.) with respect to the physical environment 102. A microphone associated with the wearable device 104-1 may be used to determine the direction of the speaker providing the auditory request. The device owner 104 wearing the wearable device 104-1 is prompted to allow or deny the request. In some implementations, the device owner 104 is prompted to look at (or look in the direction of) the speaker providing the request. If the device owner 104 grants the request, the region of the room (e.g., as described by the region of the sphere) is marked as authenticated.

[0087] In some implementations, the process 500 may capture an image of the speaker's face to generate and store an embedding (e.g., a high-dimensional feature vector). The embedding does not represent a real-world identity, but rather represents the speaker's appearance (or partial appearance) that can be associated with a particular physical environment. In some implementations, the embedding is tied to an ID associated with the controllable device 108-1 to efficiently control the device 108-1 by a user. For example, the ID may be tied to a speaker's playlist, such that playing music (e.g., an indicator representing a request 116) may trigger the playback of such a playlist.

[0088] In some implementations, the facial features of a speaker (e.g., a requester) may be compared to previously authorized faces stored on the wearable device or stored on a server computing system accessible to the wearable device. If the device owner 104 authorizes the received request, subsequent requests by the same speaker may be authenticated by the area of ​​the room from which the request came (using the same localization / pose as described above or using stored facial features).

[0089] Process 500 may utilize a computing system comprising at least one processing device and a memory storing instructions that, when executed, cause the processing device to perform the operations and computer-implemented steps recited in the claims. In general, systems 100, 200 and / or system 600 may be used in describing and implementing process 500.

[0090] At block 502, process 500 includes receiving a request to access a controllable device at a wearable device communicatively coupled to the controllable device in the physical environment. For example, wearable device 104-1 may receive a request from user 106 as indicated by an audible request 114 in physical environment 102. In some implementations, the request may instead be received from wearable device 104-2 or device 106-2. Wearable device 104-1 may determine, using sensor system 210 and / or cloud anchor application 220, that the request is received from an unauthorized user or device (i.e., user 106 or device 104-2, 106-2). If it is determined that the request is received from an unauthorized or unauthenticated user, process 500 may initiate an authentication check process.

[0091] At block 504, the process 500 includes determining a posture of the wearable device within the physical environment. For example, the wearable device 104-1 may determine its own position 240 (e.g., the 3DoF posture 240a or the 6DoF posture 240b or the 5DoF posture 240c) within the environment 102. This posture may be used when authentication occurs.

[0092] At block 506, the process 500 includes rendering a prompt at the wearable device to select to allow or deny access to the controllable devices by an unauthenticated user or device. For example, the prompt may be generated by the wearable device 104-1 to present information to the user 104. The information (e.g., prompts 113, 115) may include cloud anchor data, UI prompts to authorize the user and / or device, or other data to assist the user 104 in authorizing and / or otherwise authenticating access to one or more controllable devices. Other data may also be considered, including, but not limited to, image capture data, feature points associated with the cloud anchor, or other identifiable information from the scene. Such information may be used to generate a specific prompt for the user.

[0093] At block 508, the process 500 includes authenticating an unauthenticated user (e.g., user 106) or device (e.g., device 104-2 or device 106-2) to access a controllable device (e.g., device 108-1, device 108-2) from a region of the physical environment 102 (e.g., region A in FIG. 1 ) and executing the request 116 (i.e., playing music). For example, the process 500 may include authenticating an unauthenticated user or device to access a controllable device from a region of the physical environment in response to receiving authorization to access the controllable device (e.g., input at gesture 112 and / or prompt 113, etc.). The process 500 may also include executing the received request in response to receiving authorization to access the controllable device. The region (e.g., region A) may be defined based on a determined pose of the wearable device 104-1. For example, the region may be defined within a spherical image of the physical environment 102 , the spherical image being based on a 3D map 230 of at least one cloud anchor 110 associated with the physical environment 102 .

[0094] In some implementations, the process 500 may also include rendering another prompt (e.g., prompt 115) at the wearable device 104-1, for example, to trigger a user of the wearable device to look in a direction corresponding to the region (e.g., region A). In response to the user 104 looking in the direction of region A, the wearable device 104-1 may capture an image of the region that may capture all or a portion of the user 106 (or device 106-2 or 104-2) requesting authentication. The image may be stored by the wearable device 104-2 in a database of authenticated devices or users. In some implementations, the image may instead be stored, for example, in the server computer 202. Such an image may be used for future authentication requests.

[0095] For example, a second request to access the controllable device 108-1 may be received by the wearable device 104-1. The wearable device 104-1 may then render a prompt that triggers the user 104 of the wearable device 104-1 to capture an image associated with the second request. The wearable device 104-1 may receive and execute a command (from the user 104) to capture an image associated with the second request. The wearable device 104-1 may, for example, compare the second captured image with images in the database 242. In response to determining at least a portion of a match of the second captured image to at least one of the images in the database 242, the wearable device 104-1 may trigger the controllable device 108-1 to execute the second request.

[0096] In some implementations, the request is an auditory request spoken by a user 106 in the physical environment 102. The wearable device 104-1 may determine a direction from which the auditory request is received and may determine an area of ​​the physical environment based on the determined direction from which the auditory request is received.

[0097] 6 illustrates an example of a computing device 600 and a mobile computing device 650 that may be used with the techniques described herein. The computing device 600 is intended to represent various forms of digital computers, such as laptops, desktops, tablets, workstations, personal digital assistants, smart devices, appliances, electronic sensor-based devices, televisions, servers, blade servers, mainframes, and other suitable computing devices. The computing device 650 is intended to represent various forms of mobile devices, such as personal digital assistants, mobile phones, smartphones, and other similar computing devices. The components, their connections and relationships, and their functions illustrated herein are intended to be exemplary only and are not intended to limit the implementation of the invention described and / or claimed herein.

[0098] Computing device 600 includes a processor 602, a memory 604, a storage device 606, a high-speed interface 608 connecting to memory 604 and a high-speed expansion port 610, and a low-speed interface 612 connecting to a low-speed bus 614 and storage device 606. Processor 602 may be a semiconductor-based processor. Memory 604 may be a semiconductor-based memory. Each of components 602, 604, 606, 608, 610, and 612 may be interconnected using various buses, mounted on a common motherboard, or otherwise mounted as appropriate. Processor 602 is capable of processing instructions for execution within computing device 600, including instructions stored in memory 604 or storage device 606, to display graphical information for a GUI on an external input / output device, such as a display 616 coupled to high-speed interface 608. In other implementations, multiple processors and / or multiple buses may be used, along with multiple memories and types of memories, as appropriate. Also, multiple computing devices 600 may be connected, each providing a portion of the required operations (eg, as a bank of servers, a group of blade servers, or a multi-processor system).

[0099] The memory 604 stores information within the computing device 600. In one implementation, the memory 604 is one or more volatile memory units. In another implementation, the memory 604 is one or more non-volatile memory units. The memory 604 may also be another form of computer-readable medium, such as a magnetic disk or optical disk. In general, the computer-readable medium may be a non-transitory computer-readable medium.

[0100] The storage device 606 can provide mass storage for the computing device 600. In one implementation, the storage device 606 can be a floppy disk device, a hard disk device, an optical disk device, a tape device, or the like, or an array of devices including flash memory or other similar solid-state memory devices, or devices in a storage area network or other configuration. The computer program product is tangibly embodied on an information carrier. The computer program product may also include instructions that, when executed, perform one or more methods and / or computer-implemented methods as described above. The information carrier is a computer-readable or machine-readable medium, such as the memory 604, the storage device 606, or a memory on the processor 602.

[0101] The high-speed controller 608 manages the bandwidth-intensive operations of the computing device 600, and the low-speed controller 612 manages the less bandwidth-intensive operations. Such an allocation of functions is merely exemplary. In one implementation, the high-speed controller 608 is coupled to the memory 604, the display 616 (e.g., via a graphics processor or accelerator), and a high-speed expansion port 610 that can accept various expansion cards (not shown). In this implementation, the low-speed controller 612 is coupled to the storage device 606 and the low-speed expansion port 614. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), may be coupled to one or more input / output devices, such as a keyboard, pointing device, scanner, or a network device, such as a switch or router, for example, via a network adapter.

[0102] Computing device 600 may be implemented in many different forms, as shown. For example, it may be implemented as a standard server 620, or multiple times in a group of such servers. It may also be implemented as part of a rack server system 624. It may also be implemented in a computer, such as a laptop computer 622. Or, components from computing device 600 may be combined with other components in a mobile device (not shown), such as device 650. Each such device may include one or more of computing devices 600, 650, and the entire system may be made up of multiple computing devices 600, 650 in communication with each other.

[0103] Computing device 650 includes components such as a processor 652, memory 664, input / output devices such as a display 654, a communication interface 666, and a transceiver 668. Device 650 may also include a storage device, such as a microdrive or other device, to provide additional storage. Each of components 650, 652, 664, 654, 666, and 668 are interconnected using various buses, and some of the components may be implemented on a common motherboard or in other manners as appropriate.

[0104] The processor 652 can execute instructions in the computing device 650, including instructions stored in the memory 664. The processor may be implemented as a chipset of chips including separate analog and digital processors. The processor may be provided for coordination of other components of the device 650, such as, for example, control of a user interface, applications executed by the device 650, and wireless communication by the device 650.

[0105] The processor 652 may communicate with a user via a control interface 658 and a display interface 656 coupled to a display 654. The display 654 may be, for example, a thin-film-transistor liquid crystal display (TFT LCD) or an organic light-emitting diode (OLED) display, or other suitable display technology. The display interface 656 may include appropriate circuitry for driving the display 654 to present graphical and other information to the user. The control interface 658 may receive commands from a user and translate the commands for submission to the processor 652. Additionally, an external interface 662 may be provided in communication with the processor 652 to enable short-range communication of the device 650 with other devices. The external interface 662 may, for example, enable wired communication in some implementations or wireless communication in other implementations, and multiple interfaces may be used.

[0106] The memory 664 stores information within the computing device 650. The memory 664 may be implemented as one or more of one or more computer readable media, one or more volatile memory units, or one or more non-volatile memory units. An expansion memory 674 may also be provided and connected to the device 650 through an expansion interface 672, which may include, for example, a Single In Line Memory Module (SIMM) card interface. Such expansion memory 674 may provide extra storage space for the device 650 or may store applications or other information for the device 650. Specifically, the expansion memory 674 may include instructions to perform or supplement the processes described above and may include secure information. Thus, for example, the expansion memory 674 may be provided as a security module for the device 650 and may be programmed with instructions that enable secure use of the device 650. Additionally, secure applications may be provided via a SIMM card along with additional information, such as placing identifying information on the SIMM card in an unhackable manner.

[0107] The memory may include, for example, flash memory and / or NVRAM memory, as described below. In one implementation, the computer program product is tangibly embodied on an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier may be a computer-readable or machine-readable medium, such as memory 664, expansion memory 674, or memory on the processor 652, and may be received, for example, via the transceiver 668 or the external interface 662.

[0108] Device 650 may communicate wirelessly via communication interface 666, which may include digital signal processing circuitry as necessary. Communication interface 666 may enable communication in a variety of modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS. Such communication may occur, for example, via radio frequency transceiver 668. Additionally, short-range communication may occur, such as using Bluetooth, Wi-Fi, or other such transceivers (not shown). Additionally, a Global Positioning System (GPS) receiver module 670 may provide device 650 with additional navigation and location-related wireless data that may be used as appropriate by applications executing on device 650.

[0109] Device 650 may also communicate audibly using audio codec 660, which may receive voice information from a user and convert it into usable digital information. Audio codec 660 may likewise generate audible sounds for the user, such as through a speaker in a handset of device 650. Such sounds may include sounds from a voice telephone call, may include recorded sounds (e.g., voice messages, music files, etc.), and may include sounds generated by applications running on device 650.

[0110] The computing device 650 may be implemented in a number of different forms as shown, for example as a mobile phone 680, or as part of a smartphone 682, personal digital assistant, or other similar mobile device.

[0111] Various implementations of the systems and techniques described herein may be realized in digital electronic circuitry, integrated circuits, specially designed application specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs executable and / or interpretable on a programmable system including at least one programmable processor, which may be special purpose or general purpose, coupled to transmit and receive data and instructions from a storage system, at least one input device, and at least one output device.

[0112] These computer programs (also known as modules, programs, software, software applications or codes) contain machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus and / or device (e.g., magnetic disks, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0113] To enable user interaction, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor or light emitting diode (LED)) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) for enabling the user to provide input to the computer. Other types of devices can be used to provide user interaction as well. For example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback). Also, input from the user can be received in any form, including acoustic, speech, or tactile input.

[0114] The systems and techniques described herein may include back-end components (e.g., as a data server), or may include middleware components (e.g., an application server), or may include front-end components (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or may be implemented in any combination of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

[0115] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The client-server relationship arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0116] In some embodiments, the computing device shown in FIG. 6 may include sensors that interface with a virtual reality or headset (VR headset / AR headset / HMD device 690). For example, one or more sensors included in computing device 650 or other computing devices shown in FIG. 6 may provide input to AR / VR headset 690 or generally provide input to the AR / VR space. The sensors may include, but are not limited to, a touch screen, an accelerometer, a gyroscope, a pressure sensor, a biometric sensor, a temperature sensor, a humidity sensor, and an ambient light sensor. Computing device 650 may use the sensors to determine an absolute position and / or a detected rotation of the computing device in the AR / VR space, which may then be used as input to the AR / VR space. For example, computing device 650 may be embedded in the AR / VR space as a virtual object such as a controller, a laser pointer, a keyboard, a weapon, etc. Positioning of the computing device / virtual object by the user when embedded in the AR / VR space allows the user to position the computing device to view the virtual object in a particular manner in the AR / VR space.

[0117] In some embodiments, one or more input devices included in or connected to computing device 650 can be used as input to the AR / VR space. The input devices can include, but are not limited to, a touch screen, a keyboard, one or more buttons, a trackpad, a touchpad, a pointing device, a mouse, a trackball, a joystick, a camera, a microphone, an earphone or earbuds with input capabilities, a game controller, or other connectable input devices. When the computing device is integrated into the AR / VR space, a user interacting with an input device included in computing device 650 can cause certain actions to occur in the AR / VR space.

[0118] In some embodiments, one or more output devices included in computing device 650 can provide output and / or feedback to a user of AR / VR headset 690 in the AR / VR space. The output and feedback can be visual, auditory, or audio. The output and / or feedback can include, but is not limited to, rendering the AR / VR space or virtual environment, vibration, turning one or more lights or strobes on and off or blinking and / or flashing, sounding an alarm, playing a chime, playing a song, and playing an audio file. Output devices can include, but are not limited to, vibration motors, vibration coils, piezoelectric devices, electrostatic devices, light emitting diodes (LEDs), strobes, and speakers.

[0119] In some embodiments, the computing device 650 can be placed within an AR / VR headset 690 to create an AR / VR system. The AR / VR headset 690 can include one or more positioning elements that allow a computing device 650, such as a smartphone 682, to be placed in an appropriate position within the AR / VR headset 690. In such embodiments, a display of the smartphone 682 can render stereoscopic images that represent an AR / VR space or virtual environment.

[0120] In some embodiments, the computing device 650 may appear as another object in the computer-generated 3D environment. A user's interaction with the computing device 650 (e.g., rotating, shaking, touching a touchscreen, swiping a finger across a touchscreen) may be interpreted as an interaction with an object in the AR / VR space. By way of example only, the computing device may be a laser pointer. In such an example, the computing device 650 appears as a virtual laser pointer in the computer-generated 3D environment. As the user manipulates the computing device 650, the user in the AR / VR space sees the movement of the laser pointer. The user receives feedback from the interaction with the computing device 650 in the AR / VR environment on the computing device 650 or on the AR / VR headset 690.

[0121] In some embodiments, the computing device 650 may include a touch screen. For example, a user may interact with the touch screen in a particular manner that may mimic what happens on the touch screen as what happens in the AR / VR space. For example, a user may use a pinch type of motion to zoom the content displayed on the touch screen. This pinch type motion on the touch screen may cause the information provided in the AR / VR space to zoom. In another example, the computing device may be rendered as a virtual book in a computer-generated 3D environment. In the AR / VR space, pages of the virtual book may be displayed and a swipe of the user's finger across the touch screen may be interpreted as turning / flipping pages of the virtual book. As each page is turned / flipped, in addition to seeing the content of the page change, the user may be provided with audio feedback, such as the sound of turning a page in a book.

[0122] In some embodiments, one or more input devices (e.g., mouse, keyboard) in addition to the computing device may be rendered in the computer-generated 3D environment. The rendered input devices (e.g., rendered mouse, rendered keyboard) may be used as rendered in the AR / VR space to control objects within the AR / VR space.

[0123] While a number of embodiments have been described, it will be understood that various modifications can be made without departing from the spirit and scope of the invention.

[0124] Also, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Further, other steps may be provided or removed from the described flows, and other components may be added or removed from the described systems. Accordingly, other embodiments are within the scope of the following claims.

[0125] In addition to the above, a user may be provided with control over both whether the systems, programs, devices, networks, or features described herein enable the collection of user information (e.g., information about the user's social network, social actions, or activities, occupation, user preferences, or the user's current location) and whether content or communications are sent to the user from the server. Additionally, certain data may be processed in one or more ways to remove user information before being stored or used. For example, the user's identity may be processed such that user information cannot be determined about the user, or the user's geographic location may be generalized where location information is obtained (e.g., to the city, zip code, or state level) such that the user's specific location cannot be determined. In this manner, the user may control what information is collected about the user, how that information is used, and what information is provided to the user.

[0126] A computer system (e.g., a computing device) may be configured to wirelessly communicate with the network server over a communications link established with the network server using any known wireless communication technologies and protocols, including radio frequency (RF), microwave frequency (MWF), and / or infrared frequency (IRF) wireless communication technologies and protocols suitable for communication over the network.

[0127] According to aspects of the disclosure, the various techniques described herein may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations thereof. Implementations may be implemented as a computer program product (e.g., an information carrier, a machine-readable storage device, a computer-readable medium, a computer program tangibly embodied in a tangible computer-readable medium) for processing by or controlling the operation of a data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). In some implementations, a tangible computer-readable storage medium may be configured to store instructions that, when executed, cause a processor to perform a process. Computer programs such as the computer programs described above may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, such as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0128] The specific structural and functional details disclosed herein are merely representative for the purposes of describing example embodiments, however, example embodiments may be embodied in many alternative forms and should not be construed as being limited to only the embodiments set forth herein.

[0129] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the embodiments. As used herein, the singular indefinite and definite articles are intended to include the plural, unless the context clearly indicates otherwise. It will be further understood that as used herein, the words "comprise" and / or "comprise" specify the presence of stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0130] When an element is referred to as being "coupled," "connected," or "responsive to" or "on" another element, it will be understood that it can be directly coupled, connected, or responsive to the other element, or that intervening elements may also be present. In contrast, when an element is referred to as being "directly coupled," "directly connected," or "directly responsive to" or "directly on" another element, there are no intervening elements present. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0131] Spatially relative language such as "below," "below," "below," "above," "above" and the like may be used herein to describe one element or feature in relation to another element or feature as shown for ease of description. It will be understood that the spatially relative language is intended to encompass different orientations of the device in use or operation in addition to the orientation shown in the figures. For example, if the device in the figures is turned over, an element described as "below" or "below" the other element or feature would be oriented "above" the other element or feature. Thus, the language "below" can encompass both an orientation above and below. The device may be otherwise oriented (rotated 70 degrees or at other orientations) and the spatially relative descriptors used herein may be interpreted accordingly.

[0132] Exemplary embodiments of the present concepts are described herein with reference to cross-sectional illustrations that are schematic illustrations of idealized embodiments (and intermediate structures) of the exemplary embodiments. As such, variations from the shapes of the illustrations are expected as a result, for example, of manufacturing techniques and / or tolerances. As such, exemplary embodiments of the described concepts should not be construed as limited to the particular shapes of regions illustrated herein, but should be construed to include, for example, deviations in shapes that result from manufacturing. Thus, the regions illustrated in the figures are schematic in nature, and their shapes are not intended to represent the actual shapes of regions of a device, and are not intended to limit the scope of the exemplary embodiments.

[0133] It will be understood that terms such as "first", "second" and the like may be used herein to describe various elements, but these elements should not be limited by these terms. These terms are used only to distinguish one element from another. Thus, a "first" element could be termed a "second" element without departing from the teachings of the present embodiment.

[0134] Unless otherwise defined, the terms used herein (including technical and scientific terms) have the same meaning as those commonly understood by those skilled in the art to which these concepts belong. Furthermore, terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and / or this specification, and will not be interpreted in an idealized or overly formal sense unless expressly defined as such in this specification.

[0135] Although certain features of the described implementations have been shown as described herein, those skilled in the art will recognize many modifications, substitutions, changes, and equivalents. It is therefore to be understood that the claims are intended to cover such modifications and changes as fall within the scope of the implementations. It is to be understood that these are presented only as examples and not as limitations, and that various changes in form and details may be made. Any part of the apparatus and / or method described herein may be combined in any combination except mutually exclusive combinations. The implementations described herein may include various combinations and / or subcombinations of the functions, components, and / or features of the different implementations described.

Claims

1. A sensor on the wearable device detects at least one cloud anchor that includes an identifier associated with the network and that is configured for the physical environment; triggering an extraction of the identifier from the at least one cloud anchor in response to detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor; and joining the wearable device to the network based on a received authentication corresponding to the extracted identifier.

2. The identifier includes an SSID and login credentials; the network is a Wi-Fi network; the identifier is associated with at least one controllable device, the controllable device being a device that provides access to the network; The computer-implemented method of claim 1 , wherein the authentication is received from the at least one controllable device.

3. the identifier is associated with a 3D map of an augmented reality (AR) environment, the 3D map including visual data mapped to the physical environment stored by at least one controllable device within the physical environment; 3. The computer-implemented method of claim 1 or 2, wherein detecting that the wearable device has access to the at least one cloud anchor includes determining whether the wearable device provides data from the sensor that matches at least a portion of the visual data mapped to the physical environment.

4. 3. The computer-implemented method of claim 2, wherein the at least one controllable device is configured to de-authorize the wearable device from the network in response to detecting that the position of the wearable device is no longer within the threshold distance.

5. 5. The computer-implemented method of claim 1, wherein the location is determined based on detecting the at least one cloud anchor within a field of view of the wearable device.

6. The method includes receiving, at the wearable device communicatively coupled to the controllable device in the physical environment, a request to access the controllable device, wherein the request is determined to be received from an unauthenticated user or device, the method further comprising: Determining a posture of the wearable device within the physical environment; and Rendering a prompt on the wearable device to select whether to allow or deny access to the controllable device by the unauthorized user or device; In response to receiving authorization to access the controllable device, authenticating the unauthenticated user or device to access the controllable device from a region of the physical environment defined based on the determined pose of the wearable device; and executing said request.

7. 7. The computer-implemented method of claim 6, wherein the region is defined within a spherical image of the physical environment, the spherical image being based on a 3D map of at least one cloud anchor associated with the physical environment.

8. Rendering, at the wearable device, another prompt to trigger a user of the wearable device to look in a direction corresponding to the region; and capturing an image of the area capturing a portion of the user or device requesting authentication; The computer-implemented method of claim 6 or 7, further comprising the wearable device storing the image in a database of authenticated devices or users.

9. receiving a second request to access the controllable device; Rendering, at the wearable device, a prompt to trigger the user of the wearable device to capture a second image associated with the second request; and receiving a command to capture the second image associated with the second request; comparing the captured second image with the images in the database; 10. The computer-implemented method of claim 8, further comprising triggering execution of the second request in response to determining at least a portion of a match of the captured second image with at least one of the images in the database.

10. the request is an auditory request spoken by a user in the physical environment; The wearable device determines a direction from which the auditory request is received; and The computer-implemented method of any one of claims 6 to 9, wherein the region of the physical environment is determined based on the determined direction from which the auditory request is received.

11. 1. A computing system comprising: A wearable device including a sensor; at least one processing device; and a memory storing instructions for causing the system to perform operations, the operations comprising: The sensor of the wearable device detects at least one cloud anchor that includes an identifier associated with a network and that is configured for a physical environment; triggering an extraction of the identifier from the at least one cloud anchor in response to detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor; and joining the wearable device to the network based on a received authentication corresponding to the extracted identifier.

12. The identifier includes an SSID and login credentials; the identifier is associated with at least one controllable device, the controllable device being a device that provides access to the network; The computing system of claim 11 , wherein the authentication is received from the at least one controllable device.

13. The computing system of claim 12 , wherein the at least one controllable device is configured to de-authorize the wearable device from the network in response to detecting that the position of the wearable device is no longer within the threshold distance.

14. the identifier is associated with a 3D map of an augmented reality (AR) environment, the 3D map including visual data mapped to the physical environment stored by at least one controllable device within the physical environment; 14. The computing system of claim 11, wherein detecting that the wearable device has access to the at least one cloud anchor comprises determining whether the wearable device provides data from the sensor that matches at least a portion of the visual data mapped to the physical environment.

15. The computing system of claim 11 , wherein the location is determined based on detecting the at least one cloud anchor within a field of view of the wearable device.

16. A program storing instructions that, when executed by a processor, cause a computing device to: detecting, by a sensor on the wearable device, at least one cloud anchor that includes an identifier associated with the network and that is configured for the physical environment; triggering an extraction of the identifier from the at least one cloud anchor in response to detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor; A program that joins the wearable device to the network based on a received authentication corresponding to the extracted identifier.

17. The identifier includes an SSID and login credentials; the identifier is associated with at least one controllable device, the controllable device being a device that provides access to the network; The method of claim 16 , wherein the authentication is received from the at least one controllable device.

18. 20. The program of claim 17, wherein the at least one controllable device is configured to de-authorize the wearable device from the network in response to detecting that the location of the wearable device is no longer within the threshold distance.

19. the identifier is associated with a 3D map of an augmented reality (AR) environment, the 3D map including visual data mapped to the physical environment stored by at least one controllable device within the physical environment; The program of any one of claims 16 to 18, wherein detecting that the wearable device has access to the at least one cloud anchor comprises determining whether the wearable device provides data from the sensor that matches at least a portion of the visual data mapped to the physical environment.

20. The program of any one of claims 16 to 19, wherein the location is determined based on detecting that the at least one cloud anchor is within a field of view of the wearable device.

Images