Leveraging cloud anchors in authentication

Abstract

To provide systems and methods for authenticating a guest device to access services in an environment.SOLUTION: The method comprises detecting, by a sensor on a wearable device, at least one cloud anchor that includes an identifier associated with a network and configured for a physical environment; detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device; detecting that the wearable device has access to the at least one cloud anchor; in response thereto, triggering extraction of the identifier from the at least one cloud anchor; and joining the wearable device to the network based on a received authentication corresponding to the extracted identifier.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] TECHNICAL FIELD This specification relates generally to authentication devices. [Background technology]

[0002] background Mobile devices may be used to control one or more electronic devices in buildings, such as homes and offices. Such buildings often include devices that can be accessed and / or controlled remotely. For example, light bulbs may be switched or adjusted by the mobile device using various types of wireless communications. In some examples, such mobile devices may include a camera for capturing image data and image recognition software for processing the image data to detect devices in the building that may be within the field of view of the camera mounted on the mobile device. Summary of the Invention [Means for solving the problem]

[0003] overview One or more computer systems may be configured to perform particular operations or actions by having software, firmware, hardware, or a combination thereof installed on the system that, when run, causes the system to perform the actions. One or more computer programs may be configured to perform particular operations or actions by containing instructions that, when executed by a data processing device, cause the device to perform the actions.

[0004] In a first general aspect, a system and method are described for detecting at least one cloud anchor configured for a physical environment, where a sensor on a wearable device includes an identifier associated with a network. The system and method may include, in response to detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor, triggering extraction of an identifier from the at least one cloud anchor, and joining the wearable device to the network based on received authentication corresponding to the extracted identifier.

[0005] These and other aspects, alone or in combination, may include one or more of the following. For example, the system and method may further include the aspect that the identifier includes an SSID and login credentials, the network is a Wi-Fi network, the identifier is associated with at least one controllable device that is a device that provides access to the network, and the authentication is received from the at least one controllable device.

[0006] In some implementations, the identifier may be a 3D representation of an augmented reality (AR) environment. The 3D map is associated with a map, the 3D map including visual data mapped to the physical environment stored by at least one controllable device in the physical environment, and detecting that the wearable device has access to the at least one cloud anchor includes determining whether the wearable device provides data from a sensor that matches at least a portion of the visual data mapped to the physical environment. In some implementations, the at least one controllable device detects that the position of the wearable device is no longer within the threshold. and configured to de-authenticate the wearable device from the network in response to detecting that the wearable device is not within a distance of the cloud anchor. In some implementations, the location is determined based on detecting that at least one cloud anchor is within a field of view of the wearable device. Furthermore, the method of claim 1 above may include one or more (e.g., all) of the following features (or any combination thereof) of claims 6-10 described below.

[0007] In a second general aspect, systems and methods are described for receiving a request to access a controllable device at a wearable device communicatively coupled to the controllable device in a physical environment, where the request is determined to be received from an unauthorized user or device. The systems and methods may further include determining a posture of the wearable device within the physical environment and rendering a prompt at the wearable device for selecting whether to allow or deny access to the controllable device by the unauthorized user or device. In response to receiving permission to access the controllable device, the systems and methods may include authenticating the unauthorized user or device and fulfilling the request to access the controllable device from a region of the physical environment defined based on the determined posture of the wearable device.

[0008] These and other aspects, alone or in combination, may include one or more of the following: For example, the region may be defined within a spherical image of the physical environment, the spherical image being based on a 3D map of at least one cloud anchor associated with the physical environment. In some implementations, systems and methods may include rendering, at the wearable device, another prompt to trigger a user of the wearable device to look in a direction corresponding to the region; capturing an image of the region capturing a portion of the user or device requesting authentication; and storing, by the wearable device, the image in a database of authenticated devices or users.

[0009] In some implementations, the systems and methods include receiving a second request to access the controllable device, rendering, at the wearable device, a prompt to trigger a user of the wearable device to capture an image associated with the second request, receiving a command to capture the image associated with the second request, and comparing the captured second image with images in a database. The systems and methods may further include triggering execution of the second request in response to determining at least a portion of a match of the captured second image with at least one of the images in the database.

[0010] In some implementations, the request is an auditory request spoken by the user in the physical environment, the wearable device determines a direction from which the auditory request is received, and the region of the physical environment is determined based on the determined direction from which the auditory request is received. Further, the method of claim 6 above may include one or more (e.g., all) of the following features (or any combination thereof) of claims 1-5 above.

[0011] Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium. Details of one or more implementations are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 illustrates an augmented reality (AR) system for authenticating a wearable device, according to an implementation described throughout this disclosure. [Figure 2] FIG. 1 is a block diagram illustrating an example system for authenticating a device in a physical and / or AR environment, according to implementations described throughout this disclosure. [Figure 3A]FIG. 1 illustrates an example of an electronic wearable device, according to implementations described throughout this disclosure. [Figure 3B] FIG. 1 illustrates an example of an electronic wearable device, according to implementations described throughout this disclosure. [Figure 4] 1 is a flowchart illustrating an example process for joining a network, according to implementations described throughout this disclosure. [Figure 5] 10 is a flowchart illustrating an example process for authenticating a controllable device, according to implementations described throughout this disclosure. [Figure 6] FIG. 1 illustrates an example of a computing device and a mobile computing device that can be used with the techniques described herein. DETAILED DESCRIPTION OF THE INVENTION

[0013] Like reference symbols in the various drawings indicate like elements. Detailed Description This specification describes examples related to general authentication of augmented reality (AR) wearable electronic devices (i.e., wearable devices). The authentication process may include an image-based assessment of the environment surrounding the wearable device. The image-based assessment may consider positional aspects of the wearable device and / or positional aspects of electronic devices that the wearable device attempts to control or authenticate. In some implementations, the image-based assessment may consider the location of cloud anchors relative to the environment.

[0014] A device seeking authentication (e.g., a wearable device) may capture locational aspects of the environment and transmit such aspects (and / or other data) to other electronic devices in the environment. The other electronic devices in the environment may evaluate the locational aspects (and / or other data received from the wearable device) to determine whether to authenticate the wearable device and / or whether to provide particular data and / or services to the wearable device via the other electronic device(s).

[0015] In some implementations, a wearable device described herein may be generally authenticated by one or more other devices in a physical, virtual, or AR environment. For example, an electronic assistant-type device (e.g., a home assistant, a car assistant) can be used to authenticate other devices in a shared space as well as share certain settings and resources related to the electronic assistant. In some implementations, other controllable devices, such as an electronic thermostat, an electronic doorbell device, a television, a security device, a home appliance, or other electronic device visible by the wearable device, can be used to authenticate the wearable device and / or other devices in the shared space.

[0016] Conventional systems typically do not have a mechanism for authenticating requests from devices or users via assistant devices. In particular, conventional systems may receive and execute routine requests (e.g., control lights, control music, etc.) in shared settings (e.g., dorm rooms, hotels, office buildings, etc.) and execute such requests without the permission of the device owner.

[0017] The systems and methods described herein may provide techniques for authenticating such authorization by, for example, guest-owned devices. Such techniques may allow a device owner to authorize certain users to perform requests while prohibiting or blocking other users from performing requests without the device owner's permission. These techniques may provide benefits to device owners by allowing them to assign control of requests. These techniques may provide a way to set permissions for routine request devices (i.e., devices configured to receive keywords (e.g., an auditory request such as "OK, computer..."), device control requests, etc.). Permissions may be enabled by the device owner and may be bypassed upon receiving an indication from the device owner (e.g., a gesture, a selection, a voice response, etc.) to allow routine requests from a particular device and / or user. For example, the device owner may gesture toward a device or user location associated with the request. In some implementations, user interface content may be displayed to the device owner on the wearable device display to request authentication of the requesting user and / or authorization to use the controllable device.

[0018] In some implementations, wearable devices described herein may leverage AR content (e.g., cloud anchors) to be authenticated to obtain access to network-offered services and / or device functions (e.g., access to a Wi-Fi network, access to an AR session, access to devices in an environment, etc.). For example, the systems and techniques described herein may provide techniques for authenticating guest devices to access services in an environment. For example, traditional authentication to a Wi-Fi network / service involves a user potentially having to request a password from another user, manually select a network, and manually enter a password. The systems and techniques described herein may leverage virtual cloud anchors (i.e., cloud anchors) to generically authenticate guest devices to Wi-Fi networks (or other services away from such networks) associated with an environment (e.g., a hotel, a home, an estate, an office building, etc.). Such authentication may be provided according to permissions set by the owner of the service and / or network.

[0019] In general, the present disclosure relates to the use of a wearable device that may incorporate visual data to identify three-dimensional (3D) map information (e.g., a 3D mesh, a 3D feature map, a cloud anchor, etc.) or two-dimensional (e.g., 2D) information about a particular physical environment based on the visual data, in order to determine the location of the controllable device within a physical space represented at least in part by the visual data and / or to gain access to the controllable device. Such information may include any amount of visual positioning data about another controllable device that can be seen or detected by the wearable device. The information may be stored locally or remotely on a server computer. In some implementations, the information may be used to recover 2D information. In some implementations, the information may be used to determine the three degrees of freedom (DOF) of the controllable device. F) may be used to detect a position. In some implementations, the information may be used to detect a 6DoF position of the controllable device. In some implementations, the information may be used to detect a 5DoF position of the controllable device. In some implementations, the information may be used to recover 2D and / or 5DoF map data.

[0020] Authentication analysis of the location determination and / or other user input allows the wearable device to authenticate to the controllable device and begin controlling the controllable device. User interface (UI) content and / or audio signals may be provided to the wearable device to enable control and / or feedback to the user of the wearable device. In some implementations, the UI content may provide the user of the wearable device with one or more controls for controlling the controllable device and / or additional information about the controllable device.

[0021] The techniques described herein provide the advantage of preserving battery power for a wearable device by using a cloud anchor to authenticate the wearable device to a Wi-Fi network. Such techniques ensure that the wearable device does not need to use cellular services, which may drain the electronic device's battery more quickly than using Wi-Fi services. Instead, the techniques described herein may utilize cloud anchor information to authenticate the device, provide access to Wi-Fi or other networks, and enable control of controllable devices associated with a physical and / or AR environment.

[0022] FIG. 1 illustrates an augmented reality (AR) system 100 for authenticating a wearable device to other electronic devices, according to implementations described throughout this disclosure. Users may enter a physical space, such as environment 102, and may carry one or more computing systems. For example, a first user 104 is shown seated, may be associated with (e.g., or may be located nearby) computing system 106-1, and may wear a wearable device 104-1. Computing system 106-1 is a mobile device. Wearable device 104-1 is a pair of AR glasses. Similarly, a second user 106 may enter environment 102. The second user may also have any number of electronic devices. In this example, user 106 holds mobile device 106-2 and wears wearable device 104-2. Any number of computing systems may be included in environment 102. Similarly, any number of wearable devices may be included in environment 102. For example, additional wearable device 104-3 represents a ring controller that may interact with other devices in environment 102. Similarly, additional wearable device 104-4 represents a smartwatch that may interact with other devices in environment 102.

[0023] In some implementations, computing system 106-1 may be communicatively coupled to wearable device 104-1. In some implementations, wearable device 104-1 may be a standalone device that is not configured, connected, or in communication with another device associated with user 104. In some implementations, computing system 106-2 may be communicatively coupled to wearable device 104-2. In some implementations, wearable device 104-2 may be a standalone device that is not configured, connected, or in communication with another device associated with user 106.

[0024] In some implementations, computing systems 106-1 and 106-2 are auxiliary (i.e., optional) and are not used for interaction within environment 102. For example, wearable device 104-1 may function standalone to experience networks, services, or other controllable devices within environment 102. Similarly, wearable device 104-2 may operate without computing system 106-2. Thus, wearable devices 104-1 and 104-2 may operate standalone. In some implementations, wearable device 104-2 may request access to services, networks, etc. via wearable device 104-1.

[0025] A user 104, 106 associated with one or more of the computing systems 106-1, 106-2 and / or wearable devices 104-1, 104-2 may desire to control one or more controllable devices in the environment 102. As shown in the environment 102, a first controllable device 108-1 is a digital assistant device and a second controllable device 108-2 is a Wi-Fi router. Any number of controllable devices 108 may be included in the environment 102. Such controllable devices 108 may be Controlling may include evaluating and obtaining authorization from the owner of a particular controllable device 108. The computing systems 106-1, 106-2 and / or the wearable devices 104-1, 104-2 may be authenticated by the systems and techniques described herein. Authentication may provide such systems and devices with access to services, networks, controllable devices, and / or permissions associated with such controllable devices. For example, the system 100 and environment 102 may be configured with cloud anchors 110 that can be detected and accessed to obtain information that can be used to authenticate the wearable devices 104-1, 104-2 to obtain access to particular services, networks, and / or controllable devices. In some implementations, one wearable device (e.g., the host AR glasses 104-1) may provide authentication for another wearable device (e.g., the guest AR glasses 104-2).

[0026] As used herein, a cloud anchor (e.g., a virtual cloud anchor) may refer to one or more sets of visual feature points included in a 3D map. A cloud anchor may be associated with a set of persistent visual features that represent physical objects in the physical world. In the AR system described herein, a wearable device may receive digital information about a host user's environment and create one or more cloud anchors, such as cloud anchor(s) 110, based on the digital information, where each cloud anchor 110 may represent a portion of a physical space, such as a room (or portion thereof) in a building. The host user may add virtual objects to a virtual scene associated with the room and can then link (or anchor) the virtual objects to positions in the cloud anchors 110.

[0027] The cloud anchor 110 may be shared with other users, allowing them to join the space and view and interact with content or devices owned by the host user (or another user). In some examples, to allow other users to participate in the host's environment, the host user may locate an identifier for a particular cloud anchor and provide the identifier to the other users, or the users may manually enter the identifier to participate in the AR scene. In some implementations, the cloud anchor may be configured by the host to automatically authenticate a particular device to access objects, devices, services, and / or systems associated with (or communicatively coupled to access) the cloud anchor. Each device associated with the environment 102 (e.g., computing systems 106-1, 106-2 and wearable devices 104-1, 104-2) can access the cloud anchor 110 via image sensors associated with such devices.

[0028] In some implementations, a guest's wearable device may travel through a physical environment collecting data related to feature points of any number of cloud anchors 110. At some point, a combination of feature points may be provided as a specific identifier for gaining access to a network or device. The identifier may include one or more feature points associated with one or more cloud anchors 110. For example, if wearable device 104-2 can generate an identifier associated with a particular cloud anchor that corresponds to or is configured with a controllable device, wearable device 104-2 may then be authenticated to gain access to the network or device.

[0029] In operation, wearable device 104-1 may represent a host (i.e., the owner of controllable device 108). User 104 (i.e., the host / owner) may provide general cloud anchor-based authentication to a guest user upon verifying that the guest user has access to cloud anchor 110, and may transmit such verification to wearable device 104. -2, wearable device 104-3, wearable device 104-4, computing system 106-1, controllable device 108-1 (a digital assistant), or controllable device 108-2 (e.g., a Wi-Fi router), respectively.

[0030] In some implementations, cloud anchor 110 may be associated with any number of identifiers. For example, cloud anchor 110 may include a location identifier that indicates the location of a physical or virtual object associated with a 3D map of a particular environment. Cloud anchor 110 may alternatively or additionally include an identifier that indicates a service, password, and / or other indicator for accessing a service, network, function, feature, etc.

[0031] In one non-limiting example, system 100 may provide access to a network to a wearable device. For example, user 106 may walk into lobby environment 102 wearing wearable device 104-2 (e.g., AR glasses). Wearable device 104-2 may include at least one image sensor that may detect at least one cloud anchor (e.g., cloud anchor 110). Cloud anchor 110 may include an identifier associated with at least one controllable device (e.g., controllable device 108-2 representing a Wi-Fi network router). Controllable device 108-2 may be configured such that physical environment 102, for example, provides Wi-Fi service to any wearable device that detects cloud anchor 110. For example, in response to detecting that the location of at least one controllable device (e.g., Wi-Fi router 108-2) is within a threshold distance of wearable device 104-2, wearable device 104-2 may trigger extraction of the identifier from at least one cloud anchor 110. For example, a wearable device may automatically request login credentials (e.g., SSID, username, and password) from device 108-2 when it moves within a predefined threshold distance of device 108-2 and / or when it moves within viewing distance of device 108-2 (i.e., wearable device 104-2 may have cloud anchor 110 within its field of view, detectable distance, and / or another visually detectable range). In some implementations, the threshold distance may correspond to a threshold time after which it is determined that device 104-2 can no longer see a particular cloud anchor, based on detecting that the particular wearable device 104-2 no longer has access to the particular cloud anchor 110. At such time, Wi-Fi authentication may be de-authenticated at wearable device 104-2.

[0032] In some implementations, the threshold distance may relate to a full or partial line of sight of a particular device. For example, the threshold distance may relate to ensuring a wearable device (e.g., wearable device 104-2) has line of sight to controllable device 108-2. Another example of the threshold distance may relate, for example, to ensuring that wearable device 104-2 has at least a partial view (i.e., partial line of sight) of a particular controllable device in environment 102.

[0033] Upon requesting identifier information, the wearable device 104-2 may provide the information to at least one controllable device in a sequence recognized by the device 104-2 such that authentication can occur. For example, upon providing identifier information, the wearable device 104-2 may join the network based on the received authentication of the provided identifier. The device 108-2 may authenticate via another device or wearable device associated with the environment 102 and / or the controllable device 108-2, The user 104 may authenticate. For example, at least one controllable device may be an electronic assistant (e.g., controllable device 108-1) configured with controllable device 108-2 (i.e., a network Wi-Fi router) providing a network.

[0034] In another non-limiting example, system 100 may authenticate another user or device to use, control, or otherwise access a particular controllable device associated with environment 102. For example, user 106 may represent a guest and enter the lobby (i.e., physical environment) of environment 102. User 106 may be associated with one or more electronic devices (such as those shown by devices 104-2 and 106-2). However, the user may not possess any devices and may not be associated with any devices in environment 102. Thus, the system described herein may authenticate to a device and / or authenticate a particular user based on an audio (or visual) request received from such user.

[0035] In this example, user 104 may represent environment 102 and a host of controllable devices within environment 102 (e.g., controllable device 108-1 and controllable device 108-2). User 104 may wear wearable device 104-1 (e.g., AR glasses) that may be used to authenticate other users to devices within environment 102. For example, user 106 may enter environment 102 and provide a request to access, for example, controllable device 1081. The request may be made via device 104-2, device 106-2, or another device associated with user 106. In some implementations, the request may instead include an audio utterance or a visual indicator performed by user 106.

[0036] User 106 may execute a request to use, control, or otherwise access an electronic assistant (such as controllable device 108-1). During operation, system 100 may receive the request at wearable device 104-1 (or device 104-3 or device 104-4). One or all of wearable devices 104-1, 104-3, 104-4 may be communicatively coupled to controllable device 108-1 such that any of devices 104-1, 104-3, 104-4 can be authenticated for access to controllable device 108-1. In this example, the request is determined, for example, by wearable device 104-1 to have been received from an unauthorized user or device (e.g., originating from user 106 or received from unauthorized computing device 106-2 or wearable device 104-2).

[0037] Upon receiving a request from an unauthorized user or device, wearable device 104-1 may determine its own pose. The pose may be that of wearable device 104-1 owned by the host and within the physical environment. Wearable device 104-1 may then trigger the rendering (on a display associated with itself) of prompt 113 to select whether to allow or deny access to controllable device 108-1 by the unauthorized user 106 or device (device 106-2 or device 104-2). Prompt 113 depicts a question and a selectable control, although other prompts are certainly possible. For example, additional prompt 115 instructs user 104 to look at a door in environment 102 to receive an authentication request. User 104 may perform a gesture of looking at a room door to receive (i.e., provide) such an authentication request.

[0038] In some implementations, the user 104 of the wearable device 104-1 may, for example, use the wearable device 104-1 to indicate approval for the device 104-1 (i.e., the AR glasses). A gesture 112 may be performed within the view of a camera associated with device 104-1. Such approval may then allow authentication of user 106 or device 104-2 or 106-2, thereby allowing the request to be executed at controllable device 108-1. For example, if the request was from device 106-2 to play music from a selected playlist, or if the request was an audible request 114 to play music, authentication would allow device 108-1 to play the music, as indicated by the indicator representing request 116, based at least in part on the approval associated with wearable device 104-1.

[0039] In response to receiving permission to access controllable device 108-1 or in response to a gesture visible by device 104 being performed by user 104, wearable device 104-1 may authenticate the unauthorized user or device to access controllable device 108-1 from a region of the physical environment and then execute the request. The region of the physical environment may be defined based on a determined pose of the wearable device. For example, the region may be determined using an onboard camera of wearable device 104-1 or an audio sensor associated with device 104-1. In this example, region A is an example of a region defined based on the location of user 106 (or a device owned by user 106). Of course, other defined regions and sub-regions within environment 102 are possible.

[0040] 2 is a block diagram illustrating an example system 200 for authenticating devices in a physical and / or AR environment, according to implementations described throughout this disclosure. The system 200 includes at least a first wearable device 104-1, as described in FIG. 1. The device 104-1 may receive a request to access a controllable device 108. In some implementations, a server computer 202 may be utilized to access, generate, view, and use cloud anchors. In some implementations, the wearable computing device 104-1 may perform and store operations described with respect to the server computer 202, eliminating the use of the server computer 202 in the techniques described herein.

[0041] In some examples, the wearable device 104-1 includes one or more computing devices, at least one of which is a display device that can be worn on or near a person's skin. In some examples, the wearable device 104-1 is or includes a wearable device. The wearable device may include a head-mounted display (HMD) device, such as an optical head-mounted display (OHMD) device, a transparent heads-up display (HUD) device, an augmented reality (AR) device, or other devices such as goggles or a headset having sensors, a display, and computing capabilities. In some examples, the wearable device includes AR glasses (e.g., smart glasses). AR glasses are optical head-mounted display devices designed in the shape of glasses.

[0042] In some examples, wearable device 104-1 includes a wearable device (e.g., AR glasses) and a computing device (e.g., a mobile computing device such as a smartphone, tablet, laptop, or another wearable device such as a smartwatch). Wearable device 104-1 may be connected to the computing device via a wireless connection, such as a short-range connection (e.g., a Bluetooth® connection or a near-field communication (NFC) connection) or an internet connection (e.g., Wi-Fi or a mobile network). In some examples, the components of wearable device 104-1 Some are included in the wearable device, and some of the components of wearable device 104-1 are included in the computing device.

[0043] The wearable device 104-1 includes one or more processors 204, which may be formed on a substrate configured to execute one or more machine-executable instructions or software, firmware, or a combination thereof. The processors 204 may be semiconductor-based, i.e., the processors may include semiconductor material capable of executing digital logic. Such processors 204 may include a CPU, a GPU, and / or a DSP, to name just a few examples.

[0044] The wearable device 104-1 may also include one or more memory devices 206. The memory device 206 may include any type of storage device that stores information in a format that can be read and / or executed by the processor(s) 204. The memory device 206 may store applications and modules that perform certain operations when executed by the processor(s) 204. In some examples, the applications and modules may be stored on an external storage device and loaded into the memory device 206. The wearable device 104-1 includes one or more antennas (not shown) configured to communicate with other computing devices.

[0045] Wearable device 104-1 includes a display 208. Display 208 may be a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting display, or the like. The display 208 may include an organic light emitting diode (OLED), an electrophoretic display (EPD), or a micro-projection display employing an LED light source. In some examples, the display 208 is projected into the user's field of view. In some examples, in the case of AR glasses, the display 208 may provide a transparent or translucent display so that a user wearing the glasses can see both the image provided by the display 208 and information located in the field of view of the AR glasses behind the projected image.

[0046] The wearable device 104-1 includes a sensor system 210. The sensor system 210 includes an image sensor 212 configured to acquire image data. In some examples, the sensor system 210 includes multiple image sensors 212. The image sensor 212 is capable of taking pictures and recording video. The sensor system 210 may include an inertial measurement unit (IMU) 214. The IMU 214 is configured to acquire image data. The IMU 214 may detect movement, movement, and / or acceleration of the wearable device 104-1. The IMU 214 may include a variety of different types of sensors, such as, for example, an accelerometer, a gyroscope, a magnetometer, and other such sensors. The sensor system 210 includes an audio sensor 216 configured to detect audio received by the wearable device 104-1. The sensor system 210 may include other types of sensors, such as light sensors, distance and / or proximity sensors, contact sensors such as capacitance sensors, timers, other sensors, and / or different combination(s) of sensors. The sensor system 210 may be used to obtain information related to cloud anchor data 218 using a cloud anchor application 220, as well as an identifier 222 associated with a 3D map 230 associated with such cloud anchor.

[0047] The identifiers 222 may include information related to one or more cloud anchors 110. The information may include location data corresponding to computing devices, wearable devices, virtual objects, and / or physical objects associated with the physical environment. In some implementations, the information may include information related to the network, participation in an AR session, device control, and the like. In some implementations, the information included in identifier 222 may include image data, such as an image stored for authentication purposes, such as may be stored in image authentication database 242.

[0048] System 200 includes a cloud anchor application 220 that can be used to interact with a cloud anchor for the purpose of authenticating specific locations and features associated with devices and images captured by such devices. For example, when wearable computing device 104-1 receives a request to authenticate a particular user or device, system 200 can be accessed to determine the location of one or more controllable devices 108 within environment 102. A user operating wearable computing device 104-1 can point image sensor 212 of sensor system 210 at a particular other user or device (e.g., at controllable device 108-1) and move wearable device 104-1 to map the environment from different viewing angles and positions. Such maps can be stored as cloud anchor data 218 in a map database 234 on the server or on the device.

[0049] During operation, the cloud anchor application 220 receives data from the sensor system 210 and generates cloud anchor data 218 including visual data 224, device pose(s) 226, and anchor pose(s) 228. The wearable device 104-1 may generate or access a 3D map 230 based on the cloud anchor data 218. In some implementations, the wearable device 104-1 may instead access the cloud anchor data 218 at the server computer 202. The server computer 202 may include a visual positioning data service 232 for generating the cloud anchor data 218 and generating the map 230 for the map database 234. The map 230 may be generated by a 3D map generator 236, in this example, associated with a particular physical and / or virtual environment.

[0050] The wearable device 104-1 may determine an attitude and / or position 240 based on the onboard camera 238 or image sensor 212. The position 240 may be a 3D position of the controllable device 108-1, which is the position (and optionally, orientation) of the controllable device 108-1 in a 3D physical environment in which the user (and / or wearable device 104-1) is operating. In some examples, the position includes a 3DoF position 240a of the controllable or wearable device. In some examples, the position 240 includes a 6DoF position 240b of the controllable or wearable device. In some implementations, the position 240 includes a 5DoF position 240c of the controllable or wearable device.

[0051] Wearable device 104-1 also includes a gesture detector 244. Gesture detector 244 may access sensor system 210 to detect and interpret received gestures captured by camera 238 and / or image sensor 212 of device 104-1. In some implementations, gesture detector 244 may recognize gestures and translate them into inputs for accessing and / or controlling particular controllable devices. In some implementations, gesture detector 244 receives user-provided gestures, interprets gesture patterns, and translates the patterns into inputs for, for example, controllable devices in environment 102. In some implementations, camera 238 may represent a low-resolution camera of wearable device 104-4, where the camera is embedded in the rim of a smartwatch, capable of tracking, viewing, and interpreting image data captured from cloud anchor 110 for authentication purposes.

[0052] In addition, the wearable device 104-1 controls the operation of the wearable device 104-1. The control system 246 may include a processor 314 operably coupled to the components of the wearable device 104-1.

[0053] The wearable device 104-1 also includes a communications module 248. The communications module 248 may enable the wearable device 104-1 to communicate with another computing device to exchange information and authenticate other devices within range of the device 104-1 or other identifiable elements in the environment. For example, the wearable device 104-1 may be operably coupled to another computing device to facilitate communication, e.g., via a wired connection, a wireless connection, e.g., via Wi-Fi or Bluetooth, or other type of connection.

[0054] In some implementations, wearable device 104-1 is configured to communicate with server computer 202 over network 250. Server computer 202 may represent one or more computing devices in the form of a number of different devices, such as, for example, a standard server, a group of such servers, or a rack server system. In some implementations, server computer 202 is a single system that shares components such as processor and memory. Network 250 may be the Internet and / or a local area network (LAN), a wide area network (WAN), a cellular network, , a satellite network, or other type of data network. Network 250 may also include any number of computing devices (e.g., computers, servers, routers, network switches, etc.) configured to receive and / or transmit data within network 250.

[0055] The server computer 202 includes one or more processors (not shown), which may be formed on a substrate configured to execute one or more machine-executable instructions or software, firmware, or a combination thereof. The processor(s) may be semiconductor-based, i.e., the processor may include semiconductor material capable of executing digital logic. The server computer 202 includes one or more memory devices (not shown). The memory devices may include a main memory that stores information in a format that can be read and / or executed by the processor.

[0056] In some examples, the server computer 202 is configured to execute a visual positioning data service 232. The visual positioning data service 232 may be an augmented reality (AR) collaboration service that enables users to create cloud anchors (e.g., 3D maps 230) for creating multiplayer or collaborative AR experiences that users can share. For example, users may configure virtual content and / or controllable devices 108 that other users can view, authenticate, and / or otherwise interact with from different positions within a shared physical environment.

[0057] For example, a user may create a local cloud anchor (e.g., 3D map 230) in their environment. During hosting, wearable device 104-1 can upload data to generate 3D map 230 at visual positioning data service 232, which returns a unique identifier 222 for accessing the particular controllable device. The unique identifier 222 can be distributed to other users to join the same AR environment or, in some implementations, to access the controllable device after authenticating using the identifier 222 associated with cloud anchor data 218.

[0058] 2 shows a single wearable device 104-1 and two controllable devices 108, implementations described herein may encompass any number of such systems (e.g., three or more). While FIG. 1 shows fewer device elements than the system shown in FIG. 2, the device shown in FIG. 1 may include (or have access to) any or all of the elements of the system in FIG. 2.

[0059] 3A-3B are various diagrams illustrating an example of an AR wearable device according to implementations described throughout this disclosure. FIG. 3A is a front view illustrating an example of a wearable device according to implementations described throughout this disclosure. In this example, the wearable device is AR glasses 300A (e.g., wearable electronic device 104-1 of FIG. 1). In general, AR glasses 300A may include any or all components of system 200. AR glasses 300A may be illustrated as smart glasses, which represent an optical head-mounted display device designed in the shape of glasses. For example, smart glasses are glasses that add information (e.g., project a display) alongside what the wearer sees through the glasses.

[0060] Although AR glasses 300A are shown as the wearable electronic device described herein, other types of wearable devices are possible. For example, the wearable device may include a head-mounted display (HMD) device, such as an optical head-mounted display (OHMD) device, a transparent head-up display (HUD) device, an augmented reality (AR) device, or other devices such as goggles or a headset having sensors, a display, and computing capabilities. In some examples, the wearable device may be a watch, a mobile device, jewelry, a ring controller, or other wearable controller.

[0061] 3A , the AR glasses 300A include a frame 302 having a display device 304 coupled within the frame 302 (or within a glass portion of the frame 302). The AR glasses 300A also include an audio output device 306, an illumination device 308, a sensing system 310, a control system 312, at least one processor 314, and a camera 316.

[0062] The display device 304 may include a see-through near-eye display, such as one that uses birdbath or waveguide optics. For example, such an optical design may project light from a display source onto a portion of teleprompter glass that acts as a beam splitter positioned at a 45-degree angle. The beam splitter may allow for reflection and transmission values ​​that partially reflect light from the display source while transmitting the remaining light. Such an optical design may enable a user to see both physical items in the world next to the digital images (e.g., UI elements, virtual content, etc.) generated by the display. In some implementations, waveguide optics may be used to present content on the display device 304 of the AR glasses 300A.

[0063] An audio output device 306 (e.g., one or more speakers) may be coupled to the frame 302. The sensing system 310 may include various sensing devices and a control system 312 including various control system devices to facilitate operation of the AR glasses 300A. The control system 312 may include a processor 314 operably coupled to the components of the control system 312.

[0064] The camera 316 may be capable of capturing still and / or video images. In some implementations, the camera 316 may be a depth camera that can collect data related to the distance of external objects from the camera 316. In some implementations, the camera 316 may be, for example, a point-tracking camera capable of detecting and tracking one or more optical markers on an external device, such as an optical marker on an input device or a finger on a screen. In some implementations, the AR glasses 300A may include an illumination device 308 selectively operable, for example, with the camera 316, for detection of objects (e.g., virtual and physical) within the field of view of the camera 316. The illumination device 308 may selectively operate, for example, with the camera 316, for detection of objects within the field of view of the camera 316.

[0065] The AR glasses 300A may include a communications module (e.g., communications module 248) that communicates with the processor 314 and the control system 312. The communications module may enable communications between devices housed within the AR glasses 300A as well as communications with external devices, such as, for example, a controller, a mobile device, and / or other computing devices. The communications module may enable the AR glasses 300A to communicate to exchange information with another computing device and to authenticate other devices within range of the AR glasses 300A or other identifiable elements in the environment. For example, the AR glasses 300A may be operably coupled to another computing device to facilitate communications, for example, via a wired connection, a wireless connection, for example, via Wi-Fi or Bluetooth, or other type of connection.

[0066] FIG. 3B is a back view 300B of AR glasses 300A, according to implementations described throughout this disclosure. AR glasses 300B may be an example of wearable device 104-1 or 104-2 100 of FIG. 1. AR glasses 300B are glasses that add information (e.g., project a display 320) alongside what the wearer sees through the glasses. In some examples, instead of projecting information, display 320 is an in-lens microdisplay. In some examples, AR glasses 300B (e.g., eyeglasses or or spectacles) are attached to a frame 302 that holds the eyeglasses in front of a person's eyes. A visual aid that includes lenses 322 (e.g., glass or hard plastic lenses) and typically utilizes a bridge 324 over the nose and temples 326 (e.g., temples or temple pieces) that rest on the ears.

[0067] 4 is a flowchart illustrating an example computer-implemented process 400 for joining a network, according to implementations described throughout this disclosure. Generally, process 400 utilizes the systems and algorithms described herein to enable a user device (e.g., a wearable device) to scan a physical environment to identify multiple cloud anchors within the environment, use the cloud anchors to obtain relevant AR content, and identify and / or obtain network credentials using the cloud anchors, which may enable the user device to join a network in a pre-provisioned environment based on use of the identified network credentials.

[0068] Process 400 may utilize a computing system including at least one processing device and a memory storing instructions that, when executed, cause the processing device to perform the operations and computer-implemented steps set forth in the claims. In general, systems 100, 200 and / or system 600 may be used in describing and implementing process 400.

[0069] At block 402, process 400 includes a sensor on a wearable device detecting at least one cloud anchor that includes an identifier associated with a network and configured for the physical environment. For example, an image sensor 212 or an audio sensor 216 of a wearable device 104-2 of a user 106 entering the environment 102 detects an identifier associated with a network (e.g., network 250) in the physical environment 202. 222. Cloud anchor 110 may be configured for a physical environment as described throughout this disclosure.

[0070] In some implementations, the identifier may include, for example, an SSID and login credentials if the network is a Wi-Fi network for a router device (e.g., controllable device 108-2). In some implementations, the identifier is associated with at least one controllable device that is a device that provides access to the network. For example, the controllable device may be device 108-2 (i.e., a network router). In some implementations, the controllable device may instead be electronic assistant device 108-1, which can be configured to control a second controllable device 108-2 that provides the network (i.e., via the router).

[0071] At block 404, process 400 includes triggering extraction of an identifier from at least one cloud anchor in response to detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor. For example, in response to detecting that a location of cloud anchor 110 is within a threshold distance of wearable device 104-2 and detecting that the wearable device has access to the at least one cloud anchor 110, identifier 222 may be extracted from cloud anchor 110 and provided, for example, to one or more devices associated with the network or one or more devices associated with physical environment 102 or another environment communicatively coupled to physical environment 102 (e.g., controllable device 108-2). In some implementations, proximity to a particular location is determined based on detecting that cloud anchor 110 is within a field of view of wearable device 104-2 or another device in environment 102.

[0072] In some implementations, a cloud anchor providing access to a network is an essential component of providing the network. For example, one or more cloud anchors (e.g., cloud anchor 110) may be configured to allow and disallow specific network access. In some implementations, cloud anchor 110 may contain information usable as a gateway to connect to networks provided by controllable device 108-1 and / or 108-2 and / or other controllable devices available in environment 102 and / or accessible to wearable device 104-2, for example. The information may include a username, password, location data for verifying the device via the cloud anchor, or other data stored in the cloud anchor of environment 102 that may match visual data captured by wearable device 104-2.

[0073] In some implementations, the cloud anchor may not be directly associated with a controllable device, but instead may be configured as a resource for verifying, for example, user or device ID, user or device location, or other environment 102 details before generating a response to an authentication request to a particular network or device within the environment 102.

[0074] In some implementations, the identifier 222 is associated with a 3D map 230 of the augmented reality (AR) environment 102. The 3D map 230 may include visual data 224 mapped to the physical environment 102 and stored on at least one controllable device 108-2 within the physical environment 102. In some implementations, detecting that the wearable device 104-2 has access to at least one cloud anchor 110 may include detecting that the wearable device 104-2 has access to at least one cloud anchor 110. This includes determining whether the Able device 104-2 provides data from the sensor 212 that matches at least a portion of the visual data 224 mapped to the physical environment 102.

[0075] At block 406, process 400 includes coupling wearable device 104-2 to the network based on the received authentication of the provided identifier. For example, the authentication may be generated by the cloud anchor and sent to wearable device 104-2. In some implementations, the authentication may be generated by controllable device 108-2 and sent to wearable device 104-2. For example, upon receiving the correct identifier 222, wearable device 104-2 may be authenticated to controllable device 108-2 and trigger automatic joining of wearable device 104-2 to the network provided by controllable device 108-2.

[0076] In some implementations, at least one controllable device 108-2 is configured to deauthenticate the wearable device 104-2 from the network in response to detecting that the wearable device's location is no longer within a threshold distance. In some implementations, the threshold distance may correspond to a threshold time after which it is determined that the device 104-2 can no longer see a particular cloud anchor 110, based on detecting that the particular wearable device 104-2 no longer has access to the particular cloud anchor 110. At such time, Wi-Fi may be deauthenticated on the wearable device 104-2. In some implementations, the cloud anchor may be configured to trigger deauthentication of a particular device based on detecting that such device is outside the threshold distance.

[0077] In some implementations, process 400 includes a sensor on the wearable device detecting at least one cloud anchor that includes an identifier associated with at least one controllable device configured for the physical environment. For example, an image sensor 212 or an audio sensor 216 of a wearable device 104-2 of a user 106 entering the environment 102 may detect a cloud anchor 110 having an identifier 222 associated with a controllable device 108-2 (e.g., a network router) in the physical environment 102.

[0078] In some implementations, the process 400 includes, in response to detecting that the position of the at least one controllable device is within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor, triggering extraction of an identifier from the at least one cloud anchor and providing the identifier to the at least one controllable device. For example, in response to detecting that the position of the at least one controllable device 108-2 is within a threshold distance of the wearable device 104-2 and detecting that the wearable device 104-2 has access to the at least one cloud anchor 110, the identifier 222 may be extracted from the cloud anchor 110 and provided to the controllable device 108-2.

[0079] In some implementations, the identifier may include, for example, an SSID and login credentials if the network is a Wi-Fi network for a router (e.g., controllable device 108-2). At least one controllable device may be a device that provides access to the network. For example, the controllable device may be device 108-2 (i.e., a network router). In some implementations, the controllable device may instead be electronic assistant device 108-1 that may be configured to control a second controllable device 108-2 that provides the network (i.e., a router). In some implementations, the location is determined based on detecting that a cloud anchor 110 is within the field of view of wearable device 104-2.

[0080] In some implementations, a controllable device that provides access to a network is an integral component of the network. For example, if the controllable device is router controllable device 108-2, device 108-2 may control access to the network and / or control access for other devices (e.g., 104-1, 104-2, 104-3, 106-1, 106-2, 108-1) that may access the network through router controllable device 108-2.

[0081] In some implementations, cloud anchor 110 may include information that can be used as a gateway to connect to networks provided by controllable device 108-1 and / or 108-2 and / or other controllable devices available in environment 102 and / or devices accessible by wearable device 104-2, for example. The information may include usernames, passwords, location data for validating devices via the cloud anchor, or other data stored in the cloud anchor of environment 102 that can be matched with visual data captured by wearable device 104-2.

[0082] In some implementations, the cloud anchor may not be directly associated with a controllable device, but instead may be configured as a resource that verifies, for example, the identity of a user or device, the location of a user or device, etc., before generating a response to an authentication request for a particular network or device within the environment 102.

[0083] In some implementations, the identifier 222 is associated with a 3D map 230 of the augmented reality (AR) environment 102. The 3D map 230 may include visual data 224 mapped to the physical environment 102 and stored on at least one controllable device 108-2 within the physical environment 102. In some implementations, detecting that the wearable device 104-2 has access to the at least one cloud anchor 110 includes determining whether the wearable device 104-2 provides data from a sensor 212 that matches at least a portion of the visual data 224 mapped to the physical environment 102.

[0084] In some implementations, the process 400 includes coupling the wearable device 104-2 to the network based on received authentication of an identifier provided at the controllable device 108-2. For example, upon receiving the correct identifier 222, the wearable device 104-2 may be authenticated to the controllable device 108-2, triggering automatic joining of the wearable device 104-2 to the network provided by the controllable device 108-2.

[0085] In some implementations, at least one controllable device 108-2 is configured to deauthorize the wearable device 104-2 from the network in response to detecting that the wearable device's location is no longer within a threshold distance. In some implementations, the threshold distance may correspond to a threshold time after which it is determined that the particular wearable device 104-2 can no longer see a particular cloud anchor 110, based on detecting that the particular wearable device 104-2 no longer has access to the particular cloud anchor 110. At such time, Wi-Fi may be deauthorized on the wearable device 104-2.

[0086] 5 is a flowchart illustrating an example of a computer-implemented process 500 for authenticating to a controllable device, according to implementations described throughout this disclosure. In this example, wearable device 104-1 is accessed by an authenticated user. The controllable device 104-1 may be communicatively coupled to a controllable device 108-1 (e.g., an electronic assistant device) that controls an audio resource (e.g., music). Generally, the process 500 utilizes the systems and algorithms described herein to detect that an unknown speaker has provided a request for the controllable device 108-1, which may be configured to be operated by an authenticated device and / or user. The wearable device 104-1 may begin the authentication process by localizing (e.g., determining position, pose, etc.) with respect to the physical environment 102. A microphone associated with the wearable device 104-1 may be used to determine the direction of the speaker providing the auditory request. The device owner 104 wearing the wearable device 104-1 is prompted to accept or reject the request. In some implementations, the device owner 104 is prompted to look at (or look in the direction of) the speaker providing the request. If the device owner 104 accepts the request, an area of ​​the room (e.g., described by the area of ​​the sphere) is marked as authenticated.

[0087] In some implementations, process 500 may capture an image of a speaker's face to generate and store an embedding (e.g., a high-dimensional feature vector). The embedding does not represent a real-world identity, but rather represents the speaker's appearance (or partial appearance) that can be associated with a particular physical environment. In some implementations, the embedding is tied to an ID associated with controllable device 108-1 to efficiently control device 108-1 by a user. For example, the ID may be tied to a speaker's playlist, such that playing music (e.g., an indicator representing request 116) may trigger the playback of such playlist.

[0088] In some implementations, the speaker's (e.g., requester's) facial features may be compared to previously authorized faces stored on the wearable device or on a server computing system accessible to the wearable device. If the device owner 104 authorizes the received request, subsequent requests by the same speaker may be authenticated by the area of ​​the room from which the request came (using the same localization / pose as described above or using stored facial features).

[0089] Process 500 may utilize a computing system including at least one processing device and a memory storing instructions that, when executed, cause the processing device to perform the operations and computer-implemented steps set forth in the claims. In general, systems 100, 200 and / or system 600 may be used in describing and implementing process 500.

[0090] At block 502, process 500 includes receiving a request to access a controllable device at a wearable device communicatively coupled to the controllable device in the physical environment. For example, wearable device 104-1 may receive the request from user 106 as indicated by an audible request 114 in physical environment 102. In some implementations, the request may instead be received from wearable device 104-2 or device 106-2. Wearable device 104-1 may determine, using sensor system 210 and / or cloud anchor application 220, that the request is received from an unauthorized user or device (i.e., user 106 or device 104-2, 106-2). If it is determined that the request is received from an unauthorized user, process 500 may initiate an authentication check process.

[0091] At block 504, the process 500 includes determining a posture of the wearable device within the physical environment. For example, the wearable device 104-1 determines its position 240 (e.g., a 3DoF posture 240a or a 6DoF posture 240b) within the environment 102. b or 5DoF pose 240c), which can be used when authentication occurs.

[0092] At block 506, process 500 includes rendering a prompt at the wearable device to select whether to allow or deny access to the controllable devices by an unauthorized user or device. For example, the prompt may be generated by wearable device 104-1 to present information to user 104. The information (e.g., prompts 113, 115) may include cloud anchor data, UI prompts for authorizing the user and / or device, or other data to assist user 104 in authorizing and / or otherwise authenticating access to one or more controllable devices. Other data may also be considered, including, but not limited to, image capture data, feature points associated with the cloud anchor, or other identifiable information from the scene. Such information may be used to generate a specific prompt for the user.

[0093] At block 508, process 500 includes authenticating an unauthenticated user (e.g., user 106) or device (e.g., device 104-2 or device 106-2) to access a controllable device (e.g., device 108-1, device 108-2) from a region of physical environment 102 (e.g., region A in FIG. 1 ) and performing request 116 (i.e., playing music). For example, process 500 may include authenticating an unauthenticated user or device to access a controllable device from a region of the physical environment in response to receiving permission to access the controllable device (e.g., gesture 112 and / or input at prompt 113, etc.). Process 500 may also include performing the received request in response to receiving permission to access the controllable device. The region (e.g., region A) may be defined based on a determined pose of wearable device 104-1. For example, the region may be defined within a spherical image of the physical environment 102 , where the spherical image is based on a 3D map 230 of at least one cloud anchor 110 associated with the physical environment 102 .

[0094] In some implementations, process 500 may also include rendering another prompt (e.g., prompt 115) at wearable device 104-1, for example, to trigger a user of the wearable device to look in a direction corresponding to the region (e.g., region A). In response to user 104 looking in the direction of region A, wearable device 104-1 may capture an image of the region, which may capture all or a portion of user 106 (or device 106-2 or 104-2) requesting authentication. The image may be stored by wearable device 104-2 in a database of authenticated devices or users. In some implementations, the image may instead be stored, for example, in server computer 202. Such an image may be used for future authentication requests.

[0095] For example, a second request to access controllable device 108-1 may be received by wearable device 104-1. Wearable device 104-1 may then render a prompt that triggers a user 104 of wearable device 104-1 to capture an image associated with the second request. Wearable device 104-1 may receive and execute a command (from user 104) to capture an image associated with the second request. Wearable device 104-1 may, for example, compare the second captured image with images in database 242. In response to determining at least a portion of a match of the second captured image to at least one of the images in database 242, wearable device 104-1 may trigger controllable device 108-1 to execute the second request.

[0096] In some implementations, the request is an auditory request spoken by user 106 in physical environment 102. Wearable device 104-1 may determine the direction from which the auditory request is received and may determine the region of the physical environment based on the determined direction from which the auditory request is received.

[0097] 6 illustrates an example of a computer device 600 and a mobile computer device 650 that may be used with the techniques described herein. Computing device 600 is intended to represent various forms of digital computers, such as laptops, desktops, tablets, workstations, personal digital assistants, smart devices, appliances, electronic sensor-based devices, televisions, servers, blade servers, mainframes, and other suitable computing devices. Computing device 650 is intended to represent various forms of mobile devices, such as personal digital assistants, mobile phones, smartphones, and other similar computing devices. The components, their connections and relationships, and their functions illustrated herein are intended to be exemplary only and are not intended to limit implementations of the inventions described and / or claimed herein.

[0098] Computing device 600 includes a processor 602, memory 604, a storage device 606, a high-speed interface 608 connecting to memory 604 and a high-speed expansion port 610, and a low-speed interface 612 connecting to a low-speed bus 614 and storage device 606. Processor 602 may be a semiconductor-based processor. Memory 604 may be semiconductor-based memory. Each of components 602, 604, 606, 608, 610, and 612 may be interconnected using various buses, mounted on a common motherboard, or mounted in other manners as appropriate. Processor 602 is capable of processing instructions for execution within computing device 600, including instructions stored in memory 604 or storage device 606, to display graphical information for a GUI on an external input / output device, such as a display 616 coupled to high-speed interface 608. In other implementations, multiple processors and / or multiple buses may be used, along with multiple memories and types of memory, as appropriate. Also, multiple computing devices 600 may be connected, each providing a portion of the required operations (eg, as a bank of servers, a group of blade servers, or a multi-processor system).

[0099] The memory 604 stores information within the computing device 600. In one implementation, the memory 604 is one or more volatile memory units. In another implementation, the memory 604 is one or more non-volatile memory units. The memory 604 may also be another form of computer-readable medium, such as a magnetic disk or an optical disk. In general, the computer-readable medium may be a non-transitory computer-readable medium.

[0100] The storage device 606 can provide mass storage for the computing device 600. In one implementation, the storage device 606 can be a floppy disk device, a hard disk device, an optical disk device, a tape device, or the like, or an array of devices including flash memory or other similar solid-state memory devices, or devices in a storage area network or other configuration. A computer program product is tangibly embodied on an information carrier. The computer program product can also include instructions that, when executed, perform one or more methods and / or computer-implemented methods, such as those described above. The information carrier can be implemented by the memory 604, the storage device 606, or a processor. The storage device 602 may be a computer-readable or machine-readable medium, such as a memory on the storage device 602.

[0101] The high-speed controller 608 manages bandwidth-intensive operations of the computing device 600, and the low-speed controller 612 manages less bandwidth-intensive operations. This allocation of functionality is merely exemplary. In one implementation, the high-speed controller 608 is coupled to the memory 604, the display 616 (e.g., via a graphics processor or accelerator), and the high-speed expansion port 610, which can accept various expansion cards (not shown). In this implementation, the low-speed controller 612 is coupled to the storage device 606 and the low-speed expansion port 614. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), may be coupled to one or more input / output devices, such as a keyboard, pointing device, scanner, etc., or to a network device, such as a switch or router, for example, via a network adapter.

[0102] Computing device 600, as shown, may be implemented in many different forms. For example, it may be implemented as a standard server 620, or multiple times in a group of such servers. It may also be implemented as part of a rack server system 624. It may also be implemented in a computer such as a laptop computer 622. Or, components from computing device 600 may be combined with other components in a mobile device (not shown), such as device 650. Each such device may include one or more of computing devices 600, 650, and the entire system may be made up of multiple computing devices 600, 650 in communication with each other.

[0103] Computing device 650 includes components such as a processor 652, memory 664, input / output devices such as a display 654, a communication interface 666, and a transceiver 668. Device 650 may also include a storage device such as a microdrive or other device to provide additional storage. Each of components 650, 652, 664, 654, 666, and 668 are interconnected using various buses, and some of the components may be implemented on a common motherboard or in other manners as appropriate.

[0104] The processor 652 can execute instructions within the computing device 650, including instructions stored in the memory 664. The processor may be implemented as a chipset of chips including separate analog and digital processors. The processor may be provided for coordination of other components of the device 650, such as control of a user interface, applications executed by the device 650, and wireless communication by the device 650, for example.

[0105] The processor 652 may communicate with a user via a control interface 658 and a display interface 656 coupled to a display 654. The display 654 may be, for example, a thin-film-transistor liquid crystal display (TFT LCD) or an organic light-emitting diode (OLED) display. or other suitable display technology. Display interface 656 may include appropriate circuitry for driving display 654 to present graphical and other information to a user. Control interface 658 may receive commands from a user and translate the commands for submission to processor 652. Additionally, an external interface 662 may be provided in communication with processor 652 to enable short-range communication of device 650 with other devices. External interface 662 may, for example, enable wired communication in some implementations or wireless communication in other implementations. Multiple interfaces may be used.

[0106] Memory 664 stores information within computing device 650. Memory 664 may be implemented as one or more computer-readable media, one or more volatile memory units, or one or more nonvolatile memory units. Expansion memory 674 may also be provided and connected to device 650 through expansion interface 672, which may include, for example, a Single In Line Memory Module (SIMM) card interface. Such expansion memory 674 may provide extra storage space for device 650 or may store applications or other information for device 650. Specifically, expansion memory 674 may include instructions for performing or supplementing the processes described above and may include secure information. Thus, for example, expansion memory 674 may be provided as a security module for device 650 and may be programmed with instructions that enable secure use of device 650. Furthermore, secure applications may be provided via a SIMM card, along with additional information, such as placing identifying information on the SIMM card in an unhackable manner.

[0107] The memory may include, for example, flash memory and / or NVRAM memory, as described below. In one implementation, a computer program product is tangibly embodied on an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier may be a computer-readable or machine-readable medium, such as memory 664, expansion memory 674, or memory on processor 652, and may be received, for example, via transceiver 668 or external interface 662.

[0108] Device 650 may communicate wirelessly via a communications interface 666, which may include digital signal processing circuitry as needed. Communications interface 666 may enable communication in a variety of modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS. Such communications may occur, for example, via a radio frequency transceiver 668. Additionally, short-range communications may occur, such as using Bluetooth, Wi-Fi, or other such transceivers (not shown). Additionally, a Global Positioning System (GPS) receiver module 670 may be included in the device. The wireless communication system may provide the device 650 with additional navigation and location-related wireless data that may be used by applications running on the device 650 as appropriate.

[0109] Device 650 may also communicate audibly using audio codec 660, which may receive voice information from a user and convert it into usable digital information. Audio codec 660 may also generate audible sounds for the user, such as through a speaker in a handset of device 650. Such sounds may include sounds from a voice telephone call, recorded sounds (e.g., voice messages, music files, etc.), and sounds generated by applications running on device 650.

[0110] The computing device 650 may be implemented in a number of different forms, as shown, including as a mobile phone 680, a smartphone 682, a personal digital assistant, or part of another similar mobile device.

[0111] Various implementations of the systems and techniques described herein may be implemented using digital electronic circuitry, integrated circuits, or other electronic devices. circuits, specially designed application specific integrated circuits (ASICs), computer hardware, firmware, software, and These various implementations may include implementation in one or more computer programs executable and / or interpretable on a programmable system including at least one programmable processor, which may be special purpose or general purpose, coupled to transmit and receive data and instructions from a storage system, at least one input device, and at least one output device.

[0112] These computer programs (also known as modules, programs, software, software applications, or code) contain machine instructions for a programmable processor and may be implemented in a high-level procedural and / or object-oriented programming language and / or in an assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., magnetic disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0113] To enable user interaction, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor or light emitting diode (LED)) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) for enabling the user to provide input to the computer. Other types of devices can be used to provide user interaction as well. For example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback). Also, input from the user can be received in any form, including acoustic, speech, or tactile input.

[0114] The systems and techniques described herein may include back-end components (e.g., as data servers), or include middleware components (e.g., application servers), or include front-end components (e.g., client computers having a graphical user interface or web browser through which a user can interact with an implementation of the systems and techniques described herein), or may be implemented in any combination of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

[0115] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The client-server relationship arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0116] In some embodiments, the computing device shown in FIG. 6 may be a virtual reality or The computing device 650 may include sensors that interface with the AR / VR headset 690 or headset (VR headset / AR headset / HMD device 690). For example, one or more sensors included in the computing device 650 or other computing devices shown in FIG. 6 can provide input to the AR / VR headset 690 or generally to the AR / VR space. The sensors can include, but are not limited to, a touchscreen, an accelerometer, a gyroscope, a pressure sensor, a biometric sensor, a temperature sensor, a humidity sensor, and an ambient light sensor. The computing device 650 can use the sensors to determine the absolute position and / or detected rotation of the computing device in the AR / VR space, which can then be used as input to the AR / VR space. For example, the computing device 650 may be incorporated into the AR / VR space as a virtual object such as a controller, laser pointer, keyboard, weapon, etc. The user's positioning of the computing device / virtual object when incorporated into the AR / VR space allows the user to position the computing device to view the virtual object in a particular manner in the AR / VR space.

[0117] In some embodiments, one or more input devices included in or connected to computing device 650 can be used as input to the AR / VR space. The input devices can include, but are not limited to, a touchscreen, a keyboard, one or more buttons, a trackpad, a touchpad, a pointing device, a mouse, a trackball, a joystick, a camera, a microphone, an earphone or earbuds with input capabilities, a game controller, or other connectable input devices. When the computing device is integrated into the AR / VR space, a user interacting with an input device included in computing device 650 can cause certain actions to occur in the AR / VR space.

[0118] In some embodiments, one or more output devices included in computing device 650 can provide output and / or feedback to a user of AR / VR headset 690 in the AR / VR space. The output and feedback can be visual, auditory, or audio. The output and / or feedback can include, but is not limited to, rendering the AR / VR space or virtual environment, vibration, turning one or more lights or strobes on and off or blinking and / or flashing, sounding an alarm, playing a chime, playing a song, and playing an audio file. Output devices can include, but are not limited to, vibration motors, vibration coils, piezoelectric devices, electrostatic devices, light-emitting diodes (LEDs), strobes, and speakers.

[0119] In some embodiments, the computing device 650 can be placed within an AR / VR headset 690 to create an AR / VR system. The AR / VR headset 690 can include one or more positioning elements that allow the computing device 650, such as the smartphone 682, to be placed in an appropriate position within the AR / VR headset 690. In such embodiments, the display of the smartphone 682 can render stereoscopic images representing the AR / VR space or virtual environment.

[0120] In some embodiments, the computing device 650 may appear as another object in a computer-generated 3D environment. A user's interaction with the computing device 650 (e.g., rotating, shaking, touching the touchscreen, swiping a finger across the touchscreen) may be interpreted as an interaction with an object in the AR / VR space. By way of example only, the computing device may be a laser pointer. In such an example, the computing device 650 may appear as a computer The computing device 650 appears as a virtual laser pointer in the computer-generated 3D environment. As the user manipulates the computing device 650, the user in the AR / VR space sees the movement of the laser pointer. The user receives feedback from their interaction with the computing device 650 in the AR / VR environment on the computing device 650 or on the AR / VR headset 690.

[0121] In some embodiments, the computing device 650 may include a touchscreen. For example, a user may interact with the touchscreen in a particular manner that can mimic what occurs on the touchscreen as occurring within the AR / VR space. For example, a user may use a pinch-type motion to zoom content displayed on the touchscreen. This pinch-type motion on the touchscreen may zoom information provided in the AR / VR space. In another example, the computing device may be rendered as a virtual book in a computer-generated 3D environment. In the AR / VR space, pages of the virtual book may be displayed, and swiping of the user's finger across the touchscreen may be interpreted as turning / flipping pages of the virtual book. As each page is turned / flipped, in addition to seeing the content of the page change, the user may be provided with audio feedback, such as the sound of turning pages in a book.

[0122] In some embodiments, one or more input devices (e.g., a mouse, a keyboard) in addition to a computing device can be rendered in the computer-generated 3D environment, and the rendered input devices (e.g., a rendered mouse, a rendered keyboard) can be used as rendered in the AR / VR space to control objects within the AR / VR space.

[0123] While a number of embodiments have been described, it will be understood that various modifications can be made without departing from the spirit and scope of the invention.

[0124] Also, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Further, other steps may be provided or removed from the described flows, and other components may be added or removed from the described systems. Accordingly, other embodiments are within the scope of the following claims.

[0125] In addition to the above, user-selectable controls are provided for both whether the systems, programs, devices, networks, or functionality described herein enable the collection of user information (e.g., information about the user's social networks, social actions, or activities, occupation, user preferences, or the user's current location) and whether content or communications are sent to the user from the server. Furthermore, certain data may be processed in one or more ways to remove user information before being stored or used. For example, a user's identity may be processed so that user information cannot be determined about that user, or the user's geographic location may be generalized where location information is obtained (e.g., to the city, zip code, or state level) so that the user's specific location cannot be determined. In this way, users can control what information is collected about them, how that information is used, and what information is provided to them.

[0126] The computer system (e.g., computing device) may utilize radio frequency (RF), microwave frequency (MWF), and / or infrared frequency (IRF) signals adapted for communication over the network. ) may be configured to communicate wirelessly with the network server via a communications link established with the network server using any known wireless communications technology and protocol.

[0127] In accordance with aspects of the present disclosure, the various techniques described herein may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations thereof. Implementations may also be implemented as a computer program product (e.g., an information carrier, machine-readable storage device, computer-readable medium, computer program tangibly embodied in a tangible computer-readable medium) for processing by or controlling the operation of a data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). In some implementations, a tangible computer-readable storage medium may be configured to store instructions that, when executed, cause a processor to perform a process. Computer programs such as those described above can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, such as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0128] The specific structural and functional details disclosed herein are merely representative for purposes of describing example embodiments, however, example embodiments may be embodied in many alternative forms and should not be construed as limited to only the embodiments set forth herein.

[0129] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the embodiments. As used herein, the singular indefinite and definite articles are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that as used herein, the words "comprise" and / or "comprises" specify the presence of stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0130] When an element is referred to as being "coupled," "connected," or "responsive to," or "on" another element, it will be understood that it can be directly coupled, connected, or responsive to the other element, or that intervening elements may also be present. In contrast, when an element is referred to as being "directly coupled," "directly connected," or "directly responsive to," or "directly on," another element, there are no intervening elements present. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0131] Spatially relative language such as "below," "below," "below," "above," "above" and the like may be used herein for ease of description to describe one element or feature in relation to other elements or features, as shown. It will be understood that the spatially relative language is intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as "below" or "below" other elements or features would then be oriented "above" the other elements or features. Thus, the language "below" can encompass both an orientation of above and below. The device may be oriented in other ways (rotated 70 degrees or at other orientations) and the spatially relative descriptors used herein may be interpreted accordingly.

[0132] Exemplary embodiments of the present concepts are described herein with reference to cross-sectional illustrations that are schematic illustrations of idealized embodiments (and intermediate structures) of the exemplary embodiments. As such, variations from the shapes of the illustrations are to be expected as a result, for example, of manufacturing techniques and / or tolerances. As such, exemplary embodiments of the described concepts should not be construed as limited to the particular shapes of regions illustrated herein but should be construed to include, for example, deviations in shapes that result from manufacturing. Accordingly, the regions illustrated in the figures are schematic in nature and their shapes are not intended to represent the actual shape of a region of a device and are not intended to limit the scope of the exemplary embodiments.

[0133] It will be understood that terms such as "first," "second," and the like may be used herein to describe various elements, but these elements should not be limited by these terms. These terms are used only to distinguish one element from another. Thus, a "first" element could be termed a "second" element without departing from the teachings of the present embodiments.

[0134] Unless otherwise defined, the terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which these concepts belong. Furthermore, terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and / or this specification, and will not be interpreted in an idealized or overly formal sense unless expressly defined as such in this specification.

[0135] While certain features of the described implementations have been shown as described herein, many modifications, substitutions, changes, and equivalents will occur to those skilled in the art. It is therefore to be understood that the claims are intended to cover such modifications and variations as fall within the scope of the implementations. It should be understood that these have been presented by way of example only, and not limitation, and that various changes in form and details may be made. Any portions of the apparatus and / or methods described herein may be combined in any combination except mutually exclusive combinations. The implementations described herein may include various combinations and / or subcombinations of the functions, components, and / or features of the different implementations described.

Claims

1. a sensor on the wearable device detecting at least one cloud anchor that includes an identifier associated with the network and that is configured for the physical environment; triggering extraction of the identifier from the at least one cloud anchor in response to detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor; and joining the wearable device to the network based on a received authentication corresponding to the extracted identifier.

2. The identifier includes an SSID and login credentials; the network is a Wi-Fi network; the identifier is associated with at least one controllable device that is a device that provides access to the network; The computer-implemented method of claim 1 , wherein the authentication is received from the at least one controllable device.

3. the identifier is associated with a 3D map of an augmented reality (AR) environment, the 3D map including visual data mapped to the physical environment stored by at least one controllable device within the physical environment; 3. The computer-implemented method of claim 1 or 2, wherein detecting that the wearable device has access to the at least one cloud anchor comprises determining whether the wearable device provides data from the sensors that matches at least a portion of the visual data mapped to the physical environment.

4. 3. The computer-implemented method of claim 2, wherein the at least one controllable device is configured to de-authenticate the wearable device from the network in response to detecting that the location of the wearable device is no longer within the threshold distance.

5. 5. The computer-implemented method of claim 1, wherein the location is determined based on detecting the at least one cloud anchor within a field of view of the wearable device.

6. 1. A computer-implemented method comprising: receiving, at a wearable device communicatively coupled to a controllable device in a physical environment, a request to access the controllable device, wherein the request is determined to be received from an unauthorized user or device, the method further comprising: determining a posture of the wearable device within the physical environment; Rendering a prompt at the wearable device to select whether to allow or deny access to the controllable device by the unauthorized user or device; In response to receiving authorization to access the controllable device, authenticating the unauthenticated user or device to access the controllable device from a region of the physical environment defined based on the determined pose of the wearable device; and executing said request.

7. 7. The computer-implemented method of claim 6, wherein the region is defined within a spherical image of the physical environment, the spherical image being based on a 3D map of at least one cloud anchor associated with the physical environment.

8. Rendering, at the wearable device, another prompt to trigger a user of the wearable device to look in a direction corresponding to the region; and capturing an image of the area capturing a portion of the user or device requesting authentication; The computer-implemented method of claim 6 or 7, further comprising the wearable device storing the image in a database of authenticated devices or users.

9. receiving a second request to access the controllable device; Rendering, at the wearable device, a prompt to trigger the user of the wearable device to capture an image associated with the second request; and receiving a command to capture the image associated with the second request; comparing the captured second image with the images in the database; 10. The computer-implemented method of claim 8, further comprising: triggering execution of the second request in response to determining at least a portion of a match of the captured second image with at least one of the images in the database.

10. the request is an auditory request spoken by a user in the physical environment; The wearable device determines a direction from which the auditory request is received; The computer-implemented method of any one of claims 6 to 9, wherein the region of the physical environment is determined based on the determined direction from which the auditory request is received.

11. 1. A computing system comprising: a wearable device including a sensor; at least one processing device; a memory storing instructions for causing the system to perform operations, the operations comprising: The sensor of the wearable device detects at least one cloud anchor that includes an identifier associated with a network and that is configured for a physical environment; triggering extraction of the identifier from the at least one cloud anchor in response to detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor; and joining the wearable device to the network based on a received authentication corresponding to the extracted identifier.

12. The identifier includes an SSID and login credentials; the identifier is associated with at least one controllable device that is a device that provides access to the network; The computing system of claim 11 , wherein the authentication is received from the at least one controllable device.

13. The at least one controllable device may, in response to detecting that the position of the wearable device is no longer within the threshold distance, remove the wearable device from the network. The computing system of claim 12 , configured to deauthorize a trusted device.

14. the identifier is associated with a 3D map of an augmented reality (AR) environment, the 3D map including visual data mapped to the physical environment stored by at least one controllable device within the physical environment; 14. The computing system of claim 11, wherein detecting that the wearable device has access to the at least one cloud anchor comprises determining whether the wearable device provides data from the sensors that matches at least a portion of the visual data mapped to the physical environment.

15. The computing system of claim 11 , wherein the location is determined based on detecting the at least one cloud anchor within a field of view of the wearable device.

16. A non-transitory machine-readable medium storing instructions that, when executed by a processor, cause a computing device to: detecting, by a sensor on the wearable device, at least one cloud anchor that includes an identifier associated with the network and that is configured for the physical environment; triggering extraction of the identifier from the at least one cloud anchor in response to detecting that a location associated with the at least one cloud anchor is within a threshold distance of the wearable device and detecting that the wearable device has access to the at least one cloud anchor; A machine-readable medium that joins the wearable device to the network based on a received authentication corresponding to the extracted identifier.

17. The identifier includes an SSID and login credentials; the identifier is associated with at least one controllable device that is a device that provides access to the network; 17. The machine-readable medium of claim 16, wherein the authentication is received from the at least one controllable device.

18. 20. The machine-readable medium of claim 17, wherein the at least one controllable device is configured to de-authorize the wearable device from the network in response to detecting that the location of the wearable device is no longer within the threshold distance.

19. the identifier is associated with a 3D map of an augmented reality (AR) environment, the 3D map including visual data mapped to the physical environment stored by at least one controllable device within the physical environment; 19. The machine-readable medium of claim 16, wherein detecting that the wearable device has access to the at least one cloud anchor comprises determining whether the wearable device provides data from the sensors that matches at least a portion of the visual data mapped to the physical environment.

20. 20. The machine-readable medium of claim 16, wherein the location is determined based on detecting the at least one cloud anchor within a field of view of the wearable device.

Images