Distributed device location seeking

Abstract

In one embodiment, an electronic device includes a communication radio, a memory to store instructions, one or more processors to execute the instructions, where the instructions cause the one or more processors to: receive a request for location information for a target accessory device associated with an owner electronic device, wherein the request comprises a data packet having at least one public key; receiving a beacon signal from a wireless device and determining at least one proximity value to the wireless device from the beacon signal; performing a comparison between a key for the wireless device from the received beacon signal and the at least one public key; and selectively transmitting location information to the owner electronic device based on a result of the comparison, wherein the location information includes the at least one proximity value and information about a defined space within the location environment.

Description

[0001] Related patent applications

[0002] This application claims the benefit of priority to U.S. patent application No. 18 / 346,600, filed on July 3, 2023, entitled “Distributed Device Location Finding,” and U.S. Provisional Application No. 63 / 376,649, filed on September 22, 2022, entitled “Distributed Device Location Finding,” each of which is incorporated herein by reference. Technical Field

[0003]

[0013] Embodiments described herein relate to using a device locator service to locate a device.

[0004] Background Information

[0005] Previous device locator services did not provide meaningful assistance in locating devices when a connection between devices had not yet been established. In particular, before a wireless connection between devices was established, a user would wander around a location, walking from room to room, trying to establish a wireless connection to locate a target device. Therefore, there is a need to provide an improved locator service. Summary of the invention

[0006] Various embodiments of a machine learning system will be described herein. Other features and benefits will be apparent from the accompanying drawings and from the detailed description. In one embodiment, an electronic device includes a communication radio device, a memory for storing instructions, and one or more processors for executing the instructions, wherein the instructions cause the one or more processors to: receive a request for location information for a target accessory device associated with an owner electronic device, and the request includes a data packet having at least one public key; receive a beacon signal from a wireless device and determine at least one proximity value to the wireless device based on the beacon signal; perform a comparison between a key for the wireless device from the received beacon signal and the at least one public key; and selectively send location information to the owner electronic device based on the comparison result, wherein the location information includes the at least one proximity value and information about a defined space within a location environment.

[0007] In one or more embodiments, the one or more processors are used to execute the instruction of the electronic device, wherein the instruction further causes the one or more processors to: determine that the received beacon signal does not contain the at least one public key; and send location information to the target electronic device indicating that the electronic device is not within the wireless communication range. In one or more embodiments, the one or more processors are used to execute the instruction of the electronic device, and the instruction further causes the one or more processors to: receive from the owner electronic device an indication that the at least one proximity value for the electronic device is the highest value compared to at least one other proximity value from a group of electronic devices. In one or more embodiments, the one or more processors are used to execute the instruction of the electronic device, wherein the instruction further causes the one or more processors to: establish a radio connection with the target accessory device. In one or more embodiments, the wireless controller of the electronic device includes a ranging sensor and the one or more processors of the device execute the instructions, the instructions further causing the one or more processors to: determine a range relative to the target accessory device and a direction to the target accessory device via the ranging sensor of the wireless controller during a two-way ranging operation; determine a target position estimate for the target accessory device relative to the electronic device based on the range relative to the target accessory device and the direction to the target wireless device; and send at least one of the target position estimate for the target accessory device, a defined space identifier for the electronic device, or an indicator of the direction to the target accessory device to the owner electronic device. In one or more embodiments, the one or more processors are used to execute the instructions of the electronic device, wherein the instructions further cause the one or more processors to: receive a request from the target accessory device, the request including at least one of a request to play audio, a request to send a target position of the target accessory device to the owner electronic device, or a request to send location information for the electronic device to the owner electronic device. In one or more embodiments, the one or more processors are used to execute the instructions of the electronic device, wherein the instructions further cause the one or more processors to: send a request to the target accessory device to increase a beacon transmission rate. In one or more embodiments, the one or more processors are used to execute the instructions of the electronic device, wherein the instructions further cause the one or more processors, wherein one or more wireless ranging operations include ranging operations performed via an ultra-wideband radio device. In one or more embodiments, the one or more processors are used to execute the instructions of the electronic device, wherein the instructions further cause the one or more processors to: present a signal strength proximity view and a ranging view, including: presenting at least one indicator of a proximity value from the electronic device to the target wireless device and an indicator for a direction to the target wireless device along a trajectory in a user interface. In one or more embodiments, the electronic device is a voice-activated virtual assistant.In one or more embodiments, the proximity value is a signal strength measurement. In one or more embodiments, the one or more processors are used to execute the instructions of the electronic device, wherein the instructions further cause the one or more processors to: detect whether an occupant is adjacent to the defined space for the electronic device, and adjust playback of content requested by the owner electronic device.

[0008] In one embodiment, a method executed by one or more processors of an electronic device to perform the following operations: sending a beacon signal having at least one public key from a wireless device; performing a two-way ranging operation with a smart home device, wherein the smart home device is selected by an owner electronic device to communicate with the wireless device. In one or more embodiments, the method also provides sending audio content for playback to the smart home device. In one or more embodiments, the method also provides stopping the two-way ranging operation with the smart home device; and performing a two-way ranging operation with the owner electronic device.

[0009] In one embodiment, a method executed by one or more processors of an electronic device to perform the following operations: sending a request for location information about a target accessory device associated with an owner electronic device to one or more smart-home devices, wherein the request includes a data packet having at least one public key; receiving one or more proximity values ​​from the one or more smart-home devices; and presenting location information from at least one of the one or more smart-home devices via a user interface, the at least one smart-home device being selected based on a comparison between the proximity values. In one or more embodiments, the method further includes: selectively presenting at least one of a ranging view, a signal strength proximity view, or a ranging and signal strength proximity view within the user interface, wherein presenting the ranging view includes: determining a range relative to a target accessory device and a direction to the target accessory device via a ranging sensor of a wireless controller during a two-way ranging operation; determining a target location estimate for the target accessory device relative to the owner electronic device based on the range relative to the target accessory device and the direction to the target accessory device; and presenting the ranging view includes displaying at least one of an indicator for the target location estimate for the target accessory device and the direction to the target accessory device.

[0010] In one embodiment, an electronic device includes: a communication radio, a communication radio, a speaker, a memory for storing instructions, and one or more processors for executing the instructions, wherein the instructions cause the one or more processors to: receive a request for location information for a target accessory device associated with an owner electronic device, wherein the request includes a data packet having at least one public key; receive a beacon signal from a wireless device and determine at least one signal strength value based on the beacon signal; determine that the received beacon signal for the wireless device contains the at least one public key; estimate at least one proximity value to the target accessory device based on the at least one signal strength value; send the at least one proximity value to the owner electronic device; receive a request from the owner electronic device to play content for the target accessory device, wherein the owner device selects the electronic device to play content based on the proximity value; and adjust playback of the content based on an occupancy value for a defined space, the electronic device being designated as being located adjacent to the defined space. In one or more embodiments, the electronic device also provides for adjusting playback, including reducing the volume of the content based on the occupancy value. In one or more embodiments, the electronic device also provides for adjusting playback, including receiving wireless communication from the owner electronic device and detecting an increase in the occupancy value for the defined space; and increasing the volume of the playback for the content. In one or more embodiments, the electronic device also provides for adjusting playback, including detecting an increase in the occupancy value for the defined space based on at least one of the received audio or sensor data; and decreasing the volume of the playback for the content. In one or more embodiments, the electronic device also provides for adjusting playback, including the instructions causing the one or more processors to: receive an indication from another smart home device that the owner electronic device is proximate to the defined space for the electronic device; and increase the volume of the playback for the content.

[0011] In the specification and the drawings, many specific details are described to provide a comprehensive understanding of various embodiments. However, in some instances, well-known or conventional details are not described in order to provide a concise discussion of the embodiments. In addition, "one embodiment" or "embodiment" or "some embodiments" mentioned in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment. The phrase "embodiment" appears in various positions in this specification and does not necessarily refer to the same embodiment. It should be noted that the flowcharts or operations described herein may vary without departing from the embodiments described herein. For example, operations may be performed in parallel, simultaneously, or in an order different from the illustrated order. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1is a block diagram of a network operating environment for an electronic device according to an embodiment.

[0013] Figure 2 A system for locating a wireless accessory device according to an embodiment is illustrated.

[0014] Figure 3 A system for pairing and locating a wireless accessory according to an embodiment is illustrated.

[0015] Figure 4 is a flow chart illustrating a method for use with a device locator system according to an embodiment.

[0016] Figure 5 is a sequence diagram illustrating a method for presenting a device locator user interface for use with the device locator system described herein, according to an embodiment.

[0017] Figure 6 A device locator system for enabling device location services for a target accessory device according to an embodiment is illustrated.

[0018] Figure 7 A device locator system for enabling location services for a target accessory device according to an embodiment is illustrated.

[0019] Figure 8 A flow chart for a method for enabling location services for a target accessory device according to an embodiment is illustrated.

[0020] Fig. 9 A flow chart for a method for enabling location services for a target accessory device according to an embodiment is illustrated.

[0021] FIG. 10A to FIG. 10B A device locator system for enabling location services for a target accessory device according to an embodiment is illustrated.

[0022] Fig.11 A flow chart for a method for enabling location services for a target accessory device according to an embodiment is illustrated.

[0023] Fig.12 A flow chart for a method for enabling location services for a target accessory device according to an embodiment is illustrated.

[0024] Figures 13 to 15 A flow chart for a method that may be performed by a detector device according to an embodiment is illustrated.

[0025] Figures 16 to 22 A device locator user interface is illustrated according to one or more embodiments.

[0026] Fig.23is a block diagram illustrating an exemplary API architecture that may be used in some implementations.

[0027] Fig.24 is a block diagram of a device architecture for a mobile or embedded device according to an embodiment.

[0028] Fig.25 is a block diagram of a computing system according to an embodiment. DETAILED DESCRIPTION

[0029] The embodiments described herein generally provide a technique for presenting information on a mobile device to guide and / or provide information from smart home devices located in a location environment to assist users in actively participating in locating a target device. The mobile device can use various data sources and / or signals to provide information to assist in locating the target device, including signals received from the target device itself, a detector device, and / or one or more smart home devices. The accessibility of various types of signals and audible sounds provided by the target device and the smart home device can change as the mobile device moves within the location environment of the target device. In some embodiments, the location environment can be composed of multiple defined spaces (e.g., rooms, etc.), which are specified with identifiers to convey the area where the target device and / or smart home device are located within the location environment. The embodiment can utilize one or more smart home devices in a defined space accessible through a network as a distributed collection of search agents to locate the target device within the location environment and provide the mobile device with location information for the target device obtained by the smart home device. In some embodiments, the smart home device can play content or amplify the audible sound provided by the target device to help the user actively participate in locating the target device.

[0030] One or more smart home devices can communicate with other smart home devices and mobile devices to enable the location of the target device. In particular, the smart home device can provide information about the limited spatial location of the smart home device within the location environment and the proximity of the smart home device to the target device. In some embodiments, when the mobile device is not within the range of forming a wireless connection with the target device and / or receiving a beacon signal from the target device, the smart home device can be requested to play content (e.g., audio, video and / or any combination thereof) to assist the mobile device user who is moving around the location environment to try to locate the target device. For example, the smart home device can amplify the sound that the target device itself is playing or is requested to play in order to help the mobile device user locate the room closest to the target device. When the data source and / or signal is accessible to provide location information to the mobile device, various user interface views can be presented in the user interface of the mobile device to represent information from one or more smart home devices and / or target devices.

[0031] In some embodiments, occupancy detection in a defined space (e.g., a room) of a location environment can be used to determine and / or adjust the provision of location services (e.g., playback of content) for locating a target device. Any combination of sensors and / or signals from devices in the location environment can allow for a determination of whether the defined space is occupied. Occupancy detection in a defined space can include using signals, user routines, microphones, machine learning, and / or sensors to determine whether there are occupants within the defined space of the location environment. In one embodiment, if the defined space is occupied, the playback of the content can be adjusted, such as by lowering the volume and / or modifying a flashing light on a smart home device. In another embodiment, the volume of content played by a smart home device can be reduced or increased as the mobile device user moves toward occupying the defined space, depending on the specific implementation.

[0032] Figure 1 1 is a block diagram of a network operating environment 100 for electronic devices according to an embodiment. The network operating environment 100 includes multiple electronic devices, such as accessory devices 101, smart home devices 107, and mobile devices 102. The accessory device 101 can be paired with the mobile device 102. In one embodiment, the mobile accessory device 101 can be a group of accessory devices that can be paired with the mobile device 102, and the group of accessory devices can form a device group. Optionally, the device group with the accessory device 101 can be a box with a wired connection to the accessory device 101 stored therein. In some embodiments, the box itself can also be an accessory device 101 that can be paired with the mobile device 102. By way of example, the accessory devices 101 can collectively be a device such as an Apple or In another embodiment, the accessory device may itself be an Apple In some embodiments, accessory device 101 may not be able to communicate via a wide area network. In other embodiments, devices 101, 102, and 107 may each be an electronic device capable of communicating via a wireless network. Some example mobile electronic devices include, but are not limited to, the following: smartphones, tablet computers, notebook computers, wearable computers (e.g., smart watches or other wearable computing accessories), mobile media players, personal digital assistants, Locator tags, headphones, head-mounted displays, health equipment, speakers, and other similar devices.

[0033] Smart home devices 107 can be devices associated with a defined space within a location environment. In some embodiments, smart home devices 107 can be fixed (e.g., appliances such as thermostats) to a defined space by installing or placing it in a defined space (e.g., a room in a home). Alternatively, smart home devices 107 can be mobile devices associated with a defined space by being placed in a defined space and / or registered as being associated with a defined space. Some example smart home devices include, but are not limited to, the following: smart speaker devices, streaming devices, televisions, refrigerators, thermostats, dishwashers, security systems, Internet access points or devices, Wi-Fi and / or mesh routers, desktop computers, air purifiers, light switches, home hub devices, baby monitors, furniture, entry system devices, smart locks, light bulbs, sprinkler systems, health monitors, and / or any other electronic devices that can be associated with a defined space.

[0034] In some embodiments, smart home devices 107 can be associated with smart home devices 107 by storing a defined space identifier in association with home application 192 and / or home service 194. By way of example, home application 192 and / or home service 194 can store defined space (e.g., hallway, backyard patio, office, bedroom, front yard, etc.) information (e.g., user-defined identifier) ​​within the location environment for smart home devices 107 in a database (e.g., relational database, object-oriented database, file system, etc.). In some embodiments, an owner or administrator for smart home devices 107 can register smart home devices 107 with home application 192 and / or home service 194 by storing location information including a user-defined identifier. In some embodiments, mobile device 102 can be used to discover smart home devices 107 and provide the ability to register (e.g., by storing smart home device information, etc.) smart home devices 107 with home application and / or home service 194. The discovery process can include using mobile device 102 to register smart home devices 107 in a defined space where the smart home devices are detected by mobile device 102. Discovery by mobile device 102 may include, but is not limited to, forming a wireless connection with smart-home device 107, receiving beacon data from smart-home device 107 (e.g., to form a connection, advertisement, etc.), using sensors to capture a representation (e.g., a QR code) for smart-home device 107 with mobile device 102, and / or any other signal that can be used to convey information for smart-home device 107. Upon discovery, mobile device 102 may facilitate registration of smart-home device 107 in a defined space within a location environment determined by mobile device 102. For example, mobile device 102 may discover smart-home device 107 during a setup process for smart-home device 107. During the discovery and / or setup process, mobile device 102 may determine a defined space for smart-home device 107 by querying a user via a user interface of mobile device 102 (e.g., a home application, a smart-home device application, etc.) and / or smart-home device 107 to enter location information for the defined space. Each of electronic devices 101, 107, and 102 may optionally include a user interface, such as user interface 104 of mobile device 102. In other embodiments, the electronic device 101 and / or the smart home device 107 as an accessory device may not have a user interface.

[0035] Electronic devices 101, 102, and 107 may be third-party devices that utilize an application programming interface to access a device locator service. Third-party devices may be provided by different device manufacturers or may be part of different ecosystems (e.g., operating systems) from electronic devices 101, 102, and / or 107. Electronic devices 101, 102, and 107 may communicate via one or more wired and / or wireless networks 110 to perform data communications. For example, a wireless network 112 (e.g., a cellular network, a Wi-Fi network) may communicate with a wide area network 114 (such as the Internet) using a gateway 116. Similarly, an access device 118, such as a mobile hotspot wireless access device, may provide communication access to the wide area network 114. The gateway 116 and the access device 118 may then communicate with the wide area network 114 via a combination of wired networks and / or wireless networks.

[0036] In some implementations, both voice communications and data communications can be established through the wireless network 112 and / or the access device 118. For example, the mobile device 102 can make and receive phone calls (e.g., using the VoIP protocol), send and receive email messages (e.g., using the POP3 protocol), and retrieve electronic documents and / or streams, such as web pages, photos, and videos, through the wireless network 112, the gateway 116, and the wide area network 114 (e.g., using the TCP / IP or UDP protocol). In some implementations, the mobile device 102 can make and receive phone calls, send and receive email messages, and retrieve electronic documents through the access device 118 and the wide area network 114. In some implementations, the mobile device 101 and / or the mobile device 102 can be physically connected to the access device 118 using one or more cables, for example, where the access device 118 is a personal computer. In this configuration, the mobile device 101 or the mobile device 102 can be referred to as a "tethered" device. In one embodiment, mobile device 101 can communicate with mobile device 102 and / or smart home device 107 via wireless peer-to-peer connection 120. Wireless peer-to-peer connection 120 can be used to synchronize data between devices.

[0037] Electronic devices 101, 102, and 107 may communicate with one or more services, such as telephone services 130, instant messaging services 140, media services 150, storage services 160, device locator services 170, certificate authorities 106, and home services 194, via one or more wired and / or wireless networks 110. For example, telephone services 130 may enable telephone communications between mobile devices or between a mobile device and a wired telephone device. Telephone services 130 may route voice over IP (VoIP) calls via wide area network 114, or may access a cellular voice network (e.g., wireless network 112). Instant messaging services 140 may, for example, provide email and / or other instant messaging services. Media services 150 may, for example, provide access to media files, such as song files, audiobooks, movie files, video clips, and other media data. Storage services 160 may provide network storage capabilities to mobile devices 101 and 102 to store documents and media files. Device locator service 170 can enable a user to locate a lost or misplaced device that is connected to one or more wired and / or wireless networks 110 at least at some point. Home service 194 can enable a user to manage smart home devices 107 with a user of home application 192. Other services can also be provided, including software update services for updating operating system software or client software on mobile devices. In one embodiment, instant messaging service 140, media service 150, storage service 160, home service 194, and device locator service 170 can each be associated with a cloud service provider, wherein various services are facilitated via cloud service accounts associated with electronic devices 101, 102, and 107.

[0038] In some embodiments, accessory device 101 and / or device group 101, smart home device 107, and mobile device 102 may register with certificate authority 106. In some embodiments, certificate authority 106 is an entity that issues digital certificates, and the service may be implemented using a group of servers managed by a device manufacturer, a service provider, or a registration service. The certificate provided by certificate authority 106 may verify the validity of verifiable information received about the device, such as a specific manufacturer of the device, a serial number, an identifier or other identifier of the device group, an indicator that the device is part of the device group, and / or any other verifiable information. In some embodiments, a device manufacturer may establish a device group by grouping the serial numbers of accessory devices in the device group. In other embodiments, the certificate may be encrypted by device 101, 107, and / or 102 before being sent to a third party, and when a third party requests verification of information provided by accessory device 101, mobile device 102, smart home device 107, and / or a device within the device group, it may be decrypted at a verification service (e.g., certificate authority or another verification service). In some embodiments, a security token may be provided by accessory device 101 in a pairing request. Additional examples of pairing devices using location services can be found in U.S. patent application Ser. No. 17 / 219,595, filed on March 21, 2021, and entitled “Secure Pairing and Pairing Lock for Accessory Devices,” which is incorporated herein by reference in its entirety.

[0039] Electronic devices 101, 102 and smart home devices 107 may have applications, services and functionality that are locally accessible on devices including location services 180. Specifically, mobile devices 101, 107 and / or 102 may have a device locator application (e.g., a "find my" application) 190 to utilize device locator services 170 and location services 180. Locally accessible data may be stored at known locations 182 and secure or trusted locations 184. In some cases, a machine learning algorithm 186 can be used to identify user routines, known locations 182 and / or trusted locations 184. Although cluster analysis is provided as an example of a machine learning algorithm that can be used, those skilled in the art will recognize that other algorithms can be used to identify potential known or trusted locations. By way of example, cluster data analysis can be used to identify and classify and provide semantic tags for locations or defined spaces, such as locations frequented by users. The secure or trusted location 184 may be explicitly specified or confirmed by the user of the mobile device 102 after data analysis. In addition, user routines may be explicitly defined by the user of the mobile device 102. In other cases, known locations 182, trusted locations 184, and user routines may be catalogued offline and provided by the device locator service 170, a home application or service, or a third party (e.g., a database with map information, a home hub device, etc.).

[0040] On-device heuristics and / or machine learning models can be used to infer the relationship between a user and a location (e.g., including a defined space) based on analysis of locally stored data about frequently visited locations (including locations frequently visited by the user, known locations, routines, and / or any other locations). For example, frequently visited locations such as home, vehicle, workplace, any location frequently visited by a user with a mobile device (e.g., accessory device 101 and mobile device 102), and / or any other location designated by the user as a trusted location 184. Known locations 182 can be commercial locations, public spaces, parks, museums, front yards, backyards, and / or any other location that a user may frequently visit. Boundary information for the corresponding stored locations can be stored together with the classification type of the location and any semantic tags assigned to the location. The stored information may include a defined set of boundaries or radius distances around a point location to allow the creation of a geo-fence for the location. A geo-fence is a virtual perimeter of a real-world geographic area. A global positioning system (GPS) can be used to create a virtual fence around a location and track the physical location of electronic devices 101, 102, and 107 within the geo-fence boundary and the entry and exit of the bounded area.

[0041] The machine learning algorithm 186 may include on-device heuristics, machine learning algorithms, or a combination thereof to analyze and assign labels about the movement or travel of a device that is designated as being in an "in transit" state, a "moving" state, or a "stationed" state in a defined space for a certain period of time. The analysis may be performed using various signals from data sources available to the mobile device 102, including but not limited to the following: sensor data, positioning data, calendar data, transit card usage data, application data, historical data about travel patterns / routines, and / or any other data accessible to the mobile device 102. In some embodiments, the mobile device 102 may be classified with a "stationed" semantic label after remaining within the geographic boundaries of a defined location (e.g., a trusted location 184) for a defined period of time. The positioning data of the mobile device 102 may remain within the boundaries of a geo-fence of a specific location for a certain period of time (e.g., 5 minutes). Sensor data (such as accelerometer data) may indicate that the mobile device 102 is in a stationary state to support the inference that it is in a stationary state.

[0042] Application data may support an inference that the mobile device 102 is in a stationary state, such as the mobile device is at a calendar appointment location. Application data indicating the type of application in use may also provide an inference of a device in a stationary state, such as using media applications and / or smart home devices. Historical data about a user's routines or patterns while traveling can be used to determine whether the mobile device 102 is stationary in a defined space, such as a bedtime routine at a home or hotel location. Mobile devices 102 may be classified as having an "in transit" label based on the user's previous behavior, pattern, or routine and analyzed on the mobile device 102. For example, a user may have a routine of going to work at approximately the same time every day, and if the data on the device supports repeating the pattern, an "in transit" status may be assigned. In the simplest case, the speed at which the mobile device moves or enters and exits a known geographic area (e.g., using a geo-fence) may allow an inference that the mobile device 102 is in transit. If the mobile device 102 is detected as accelerating in a known transportation area (e.g., on a road, highway, train route, etc.), the mobile device 102 may be given an "in transit" status. Similarly, the mobile device 102 may be designated as "in transit" if a transportation application / card is being used / is being utilized.

[0043] Figure 2A system 200 for locating a wireless accessory device 201 according to an embodiment is illustrated. In one embodiment, the wireless accessory device 201 is another embodiment of an accessory device 101 (and optional case) that can be paired as part of a device group. In other embodiments, the accessory device 201 is an individual accessory device that is not part of a device group and is each individually paired with a mobile device 102. Each wireless accessory 201 includes one or more wireless transceivers and can communicate with a companion device (e.g., mobile device 102) directly or indirectly (e.g., through another device or computer) over a wireless network or a peer-to-peer communication link. The accessory device 201 can provide a beacon signal for one or more accessory devices, and / or each accessory device 201 can be independently and separately able to be found by each providing a beacon signal. Some examples of wireless accessory devices 201 include, but are not limited to, wireless earbuds, EarPods, AirPods, input devices, charging devices, cases for accessories, earphones, headphones, fitness equipment, health equipment, display devices, external hard drives, adapters for other wearable devices (e.g., smart watches, fitness bands, optical head mounted displays), speakers, and / or any other mobile device. Paired accessory groups can be devices of the same type (e.g., speakers, AirPods, fitness weights, etc.) or devices of different types (e.g., a smartphone and a credit card reader, etc.).

[0044] Wireless accessory device 201 may also include other wireless devices, such as input devices, including but not limited to credit card reading devices, stylus devices, mice, keyboards, game controllers, and / or remote controls. In one embodiment, wireless accessory 201 also includes a device that is at least temporarily unable to access a wide area network such as the Internet (e.g., Figure 1 The wireless accessory 201 may be a smart phone, tablet computer, laptop computer, smart speaker device, television or television set-top box connected to the wide area network 114 shown in the figure. The wireless accessory 201 may also be any other wireless device, including a beacon or locator tag that can be attached to other devices to allow tracking or locating those devices. In one embodiment, the wireless accessory 201 may be from a device group of accessory devices that are paired with the mobile device 102 and / or the smart home device 107 using a wireless technology standard (such as but not limited to Bluetooth). The wireless accessory 201 may also communicate with the mobile device 102 and / or the smart home device 107 via wireless technology, which includes a specific implementation of any wireless standard and protocol, such as Wi-Fi Direct, Zigbee or AirPlay. Although the supporting device with which the wireless accessory 201 is paired is generally referred to as the mobile device 102, the supporting device is not limited to a mobile device. In some embodiments, the supporting device may also include a laptop or desktop device, and may also include some wearable accessories, such as but not limited to a smart watch device or a wearable display.

[0045] In one embodiment, the wireless accessory 201 can periodically transmit a wireless beacon signal. The wireless accessory 201 can use one of the various wireless technologies described herein (e.g., Bluetooth, Wi-Fi, etc.) to transmit the beacon signal, and in one embodiment, ultra-wideband (UWB) radio technology can also be used to send the beacon. The beacon signal can be transmitted using a single wireless technology, one of multiple optional wireless technologies, or multiple simultaneous wireless technologies. The beacon signal can transmit a beacon identifier that includes information that specifically identifies an individual wireless accessory 201 and / or a group of devices. In one embodiment, the beacon identifier is a public encryption key associated with the device.

[0046] The beacon signal may also convey information about the wireless accessory 201 and / or the smart home device 107, such as device status information and / or verifiable information. The device status information in the beacon signal may include, but is not limited to, the following: beacon type, device classification, battery level, any predefined device status, device status, lost status, alarm status, separated from owner status, near owner status, status of one or more accessory devices in the adjacent device group, wired or wireless connection status, status of physical connection to one or more accessory devices in the device group, pairing status indicating whether the accessory device is paired, pending pairing status, battery life status, charging status, near smart home device status, and / or any other status information. The near owner status may indicate that the wireless accessory 201 has detected that a mobile device 102 associated with the owner of the accessory is nearby. The lost or "separated from owner" status may indicate that the wireless accessory 201 has determined that it has been lost or has been placed in a lost state by the owner of the device. The alarm status may indicate that the wireless accessory 201 is placed in a state where the device should trigger an alarm if the device moves from its current location. The near smart-home device status can indicate that the wireless accessory device 201 is communicating with the smart-home device 107 for the owner device and / or the detector device.

[0047] In some embodiments, verifiable information may include any information that may be needed to establish trust or authorization, that is, the pairing process, setup process, device discovery process and / or search process may continue, wherein the device presents verifiable information. By way of example, verifiable information may include information established by a device manufacturer, such as a serial number or a set of serial numbers in a device group or smart home device. In some embodiments, verifiable information may include the state or status information of the device. Verifiable information may include, but is not limited to, the following: device type, member of a device group, serial number, device group, serial number of other devices in the device group, state or status information, software version and / or any other verifiable information. Verifiable information may be sent to a certificate authority 106 or other confirmation service to verify the received information presented by the device to another device. Verifiable information may be encrypted and / or sent with a token to allow further verification of the device.

[0048] In some embodiments, a beacon signal from wireless accessory 201 can be detected by smart-home device 107. Mobile device 102 can send a request to smart-home device 107 to perform a beacon scan to determine whether smart-home device 107 can detect a beacon signal sent from wireless accessory device 201. Mobile device 102 can communicate with smart-home device 107 directly and / or via a request via device locator server 203. In some embodiments, mobile device 102 can authorize sending a data packet with a request that includes one or more public keys identifying an advertisement from wireless accessory device 201. If smart-home device 107 detects a beacon signal from wireless accessory device 201, smart-home device 107 can communicate to mobile device 102 (e.g., directly or via device locator server 203, etc.) and / or other smart-home devices 107 that the beacon signal was received by smart-home device 107. For example, if a beacon signal from a wireless accessory device 201 includes an advertisement having a key from one or more public keys sent by the mobile device 201 along with the request, the smart home device 107 can respond to the request by sending location information (e.g., ranging data, signal strength information, etc.) for the wireless accessory device 201 to the mobile device 102 and / or the device locator server 203.

[0049] In one embodiment, location information for wireless accessory device 201 received from smart-home device 107 can be transmitted to mobile device 102 and / or device locator server 203 with or without encryption. A received signal strength indication (RSSI) for the beacon signal can be transmitted to mobile device 102 along with the location data for wireless accessory device 201. RSSI data can then be used to determine the distance of wireless accessory 201 from smart-home device 107 and facilitate triangulation on mobile device 102. Location information provided to mobile device 102 can include information about the location of smart-home device 107 within the location environment. Where RSSI data is transmitted in an unencrypted state, in one embodiment, device locator server 203 and / or mobile device 102 can use RSSI information to reduce noise by discarding very weak signals if there are other stronger signals from other smart-home devices 107.

[0050] In some embodiments, smart home device 107 can be used to play audio to help locate wireless accessory device 201. For example, if smart home device 107 has the strongest signal strength among a group of smart devices requested to perform a beacon scan to locate wireless accessory device 201, smart home device 107 can provide a notification to the mobile device user (e.g., play audio, play video, turn on lights, etc.). In some embodiments, wireless accessory device 201 can send a request with content for playback and / or settings for smart home device 107 to perform in order to help mobile device 102 locate wireless accessory 201.

[0051] In some embodiments, the beacon signal can be detected by a detector device 202 that is locally close to the wireless accessory 201 so as to locate the lost wireless accessory 201 using crowdsourcing. In other embodiments, the smart home device 107 can provide additional functionality as a detector device 202. The detector device 202 can be a device similar to the mobile device 102 and can receive and transmit data through the wide area network 114, and the receiving and transmitting are performed using wireless technologies similar to the wireless accessory 201 (e.g., Bluetooth, etc.). Specifically, the detector device 202 can use a wireless protocol to receive data, and the beacon signal is transmitted through the wireless protocol. The detector device 202 can determine the location using one or more location and / or positioning services, including but not limited to satellite positioning services 206 or ground positioning systems using RF signals received from wireless base stations 205 such as Wi-Fi access points or cellular tower transmitters of cellular telephone networks. In one embodiment, the detector device 202 periodically stores its location determined based on one or more location and / or positioning services. The stored location can be associated with a timestamp for which the location was determined. When the probe device 202 receives the beacon signal from the wireless accessory 201, the probe device 202 can transmit the location of the probe device 202 to the device locator server 203 via the wide area network 114. The timestamp used to determine the location of the probe device 202 can be associated with the timestamp of receiving the beacon signal to associate the geographic location with the received beacon signal.

[0052] In the event that the wireless accessory 201 provides a public key within the beacon signal, the detector device 202 can encrypt the determined location data and transmit the encrypted location data to the device locator server 203 via the wide area network 114. In one embodiment, additional data can be encrypted and transmitted with the location data, or transmitted to the device locator server 203 without encryption. For example, the RSSI for the beacon signal can be transmitted with the location data. The RSSI data can then be used to determine the distance of the wireless accessory 201 from the detector device 202 and assist in triangulation on the owner device. In the event that the RSSI data is transmitted in an unencrypted state, in one embodiment, the server can use the RSSI information to reduce noise by discarding very weak signals if other stronger signals are present. In one embodiment, UWB ranging data can also be provided, where such data is available.

[0053] In one embodiment, the detector device 202 and / or the smart home device 107 can behave differently upon receiving a beacon signal from the wireless accessory 201 depending on the device state communicated by the wireless accessory 201. For a standard beacon signal, the detector device 202 can place the encrypted location data in a queue and transmit the location data to the device locator server 203 during a periodic transmission window. However, if the wireless accessory 201 is indicating an alarm state, the detector device 202 can immediately transmit the location data to the device locator server 203. In the event that the smart home device 107 receives an indication that the wireless accessory device 201 is in an alarm state, the smart home device 107 can immediately play the media file on behalf of the wireless accessory device 201. Additionally, if the beacon signal of the wireless accessory 201 indicates that the accessory is near the owner of the accessory, the detector device 202 may not transmit the location data to the device locator server 203. Alternatively, the detector device 202 can delay the transmission of the encrypted location data.

[0054] If the owner of the wireless accessory 201 wishes to locate the wireless accessory device 201, the owner may access a device locator user interface (UI) 204 on the mobile device 102. The device locator UI 204 may be associated with a device locator application that is used to locate electronic devices and accessories registered to a user's online account (such as a cloud service account or another type of online account). The device owner may use the device locator UI 204 to query the device locator server 203 for location data that may have been transmitted to the device locator server by the detector device 202 of the wireless accessory 201. In one embodiment, the mobile device 102 may transmit a public encryption key associated with the wireless accessory 201 to the device locator server 203. The device locator server 203 may then return any stored location data corresponding to the public encryption key. The location data returned to the mobile device 102 may be encrypted data encrypted by the detector device 202 using the public encryption key. The mobile device 102 may decrypt the encrypted location data using an associated private key. The decrypted location data is then processed by the mobile device 102 to determine the most likely location of the wireless accessory 201. In various embodiments, the most likely location of wireless accessory 201 can be determined by triangulation from multiple receive locations and using other data such as beacon signal RSSI associated with each location and timestamp or UWB ranging data included within the location data.

[0055] Figure 3A system 300 for pairing and locating a wireless accessory device according to embodiments described herein is illustrated. In one embodiment, a mobile device 102 (e.g., an example of device 101) of a user of a wireless accessory 201 can present an accessory pairing UI 302 through which the user can pair the mobile device 102 with the wireless accessory 201. During an initial pairing (305) between the mobile device 102 and the wireless accessory 201, a public key exchange (310) can be performed between the mobile device and the wireless accessory 201. In one embodiment, during the public key exchange (310), the mobile device 102 and the wireless accessory 201 exchange public keys of a public key pair generated by the device and the accessory 201. In one embodiment, the public key exchange (310) is a one-way transmission in which the mobile device 102 transmits a public key of a public / private key pair to the wireless accessory 201. Additionally or alternatively, the public key exchange (310) can be a Diffie-Hellman key exchange in which the device and the accessory establish a shared secret between the two parties. In one embodiment, the public key exchange (310) additionally uses elliptic curve cryptography to establish a shared secret. For example, Elliptic Curve Diffie-Hellman (ECDH) can be used to implement the establishment of a public key pair and one or more shared secrets. In one embodiment, the one or more shared secrets include an anti-tracking secret that can be used by wireless accessory 201 to periodically derive additional public keys.

[0056] After the wireless accessory 201 has been paired with the mobile device 102, the wireless accessory 201 can periodically broadcast a beacon signal 301 including device status information and a beacon identifier. In one embodiment, the beacon identifier is a public key derived from a shared secret established during the public key exchange (310). In addition, the wireless accessory 201 can periodically perform a public key derivation (315) to generate a new public key and begin broadcasting the new public key as a beacon identifier. The public key is a K-byte key, where a new K-byte key is generated every M minutes. The values ​​K and M can vary between embodiments. In one embodiment, a 28-byte K value is used. In one embodiment, a 27-byte K value is used. The value K can be determined at least in part based on a beacon length associated with a wireless protocol used to transmit the beacon signal 301. In one embodiment, the beacon signal can transmit a variant of a beacon advertising packet associated with a low-power radio protocol (such as Bluetooth low energy).

[0057] In one embodiment, the value M is 15 minutes, such that a new K-byte key is generated every 15 minutes. The public key can be deterministically derived based on the timestamp and anti-tracking secret generated during the public key exchange 310. The public key derivation (315) process enables the wireless accessory 201 to use different keys over time, thereby preventing a particular key from being associated with a particular device for a long time. The key can be derived based on an anti-tracking secret known only to the mobile device 102 and the wireless accessory 201, thereby allowing the mobile device 102 and only the mobile device to determine which public key the wireless accessory 201 will broadcast at any given timestamp. The anti-tracking secret can be generated along with the ECDH public key and transmitted to the wireless accessory 201. The anti-tracking secret can then be used to enable the wireless accessory 201 to generate a public key sequence P i In one embodiment, the public key sequence P i =λ i P, which defines the scalar or exponential value λ i Group operations with group elements, such as elliptic curve points P. Scalar or exponent value λ = KDF(AT,i), where KDF is a key derivation function, AT is an anti-tracking secret, and i is a counter or timestamp.

[0058] In one embodiment, backtracking resistance can be enabled to protect the anti-tracking secret in the event that the wireless accessory 201 is compromised. When backtracking resistance is enabled, the anti-tracking secret is transmitted to the wireless accessory 201, but the wireless accessory does not retain the anti-tracking secret. Instead, the accessory calculates the value λ i+1 =H(λ i || time), where λ0=AT and H is a cryptographic hash function. The wireless accessory 201 then stores λ i for a given period of time i. If wireless accessory 201 is compromised, only the current and future values ​​of λ for i are exposed i In one embodiment, reverse tracking resistance is achieved by periodically changing λ i The data is written to the non-volatile memory of the wireless accessory 201 for execution.

[0059] In one embodiment, the wireless accessory 201 can transmit a beacon signal 301 every two seconds, but other beacon rates can be used and can vary in certain circumstances. For example, when in a near owner state, the wireless accessory 201 can reduce the beacon rate. The beacon rate can also vary based on events triggered by an accelerometer. For example, when in an alarm state, the wireless accessory 201 can increase the beacon rate, which can be triggered by an accelerometer on the wireless accessory 201.

[0060] If, after transmitting beacon signal 301, wireless accessory 201 receives a reply from a mobile device 102 associated with a user of the accessory indicating that mobile device 102 is within range of the wireless accessory, then wireless accessory 201 can enter the near owner state. Additionally, the amount of data transmitted by beacon signal 301 can be reduced while the wireless accessory is in the near owner state. In one embodiment, the rate at which new public keys are generated can also be reduced while the wireless accessory is in the near owner state.

[0061] The wireless accessory 201 can enter an alert state upon receiving a message from the mobile device 102 indicating that the wireless accessory 201 should enter an alert state. While in the alert state, the wireless accessory can initially enter an armed state in which the wireless accessory 201 can reduce or stop transmitting locator beacon signals, but other types of wireless signaling can continue to exist. The wireless accessory 201 can remain in the armed state until the mobile device 102 deactivates the state or an alarm is triggered. In one embodiment, the alarm can be triggered when movement is detected, for example, via an accelerometer within the wireless accessory 201. In one embodiment, the alarm can also be triggered when it is detected that the wireless accessory 201 has moved out of range of the mobile device 102 and is no longer in a near owner state. When the alarm is triggered, the rate of the beacon signal 301 can increase to increase the speed at which the wireless accessory 201 can be located.

[0062] The beacon signal 301 transmitted by the wireless accessory 201 can be detected by a set of detector devices 303 (the detector device can be the detector device 202) and / or the mobile device 102, which are other electronic devices that can receive the beacon signal transmitted by the wireless accessory and transmit the location and other data associated with the beacon signal 301 to the device locator server 203 via the wide area network 114. In one embodiment, the set of detector devices 303 includes a variation of the mobile device 102, or can be other types of electronic devices. For example, the set of detector devices 303 can perform operations (320) to associate the beacon signal 301 received from the wireless accessory 201 with the device location associated with the detector device 303. As described in detail in relation to FIG. Figure 2 The device location may be determined via a satellite positioning service or a ground positioning system using RF signals received from a wireless base station (e.g., a Wi-Fi access point or a cellular tower transmitter). In one embodiment, the group of detector devices 303 may also include fixed devices that can receive beacon signals 301, such as smart speaker devices, televisions, or television set-top boxes.

[0063] The set of detector devices 303 may encrypt the location data using the beacon identifier (e.g., a public key) received within the beacon signal 301 and send the location data (325) to the device locator server 203. The data sent by the set of detector devices 303 is sent anonymously, and identification information of the detector devices is not stored with the data sent by the detector devices.

[0064] The device locator server 203 may store the encrypted location data in a data repository 304, which in one embodiment may be a distributed database with multiple nodes. A hash of the beacon identifier / public key of the attachment may be sent together with the encrypted location data. The encrypted location data may be stored in a database node based on a hash of the beacon identifier. The encrypted location data may be indexed by the device locator server 203 using a hash of the beacon identifier. Sending a hash of the beacon identifier instead of a complete beacon identifier may prevent the complete beacon identifier from being stored in the server. Other information may also be sent and stored in an encrypted or unencrypted state with the location data. Other information may include a timestamp of when the beacon signal 301 was received, RSSI information of the received beacon, and / or ranging information such as determined via UWB ranging.

[0065] When a user or owner of the wireless accessory 201 wishes to locate the accessory, the user or owner may access the device locator UI 204 on the mobile device 102. The device locator UI 204 may be associated with a locator application 190 or feature of the mobile device 102. The device locator UI 204 may also have a web-based interface that may be accessed from the mobile device 102 or another type of electronic device, such as a laptop or desktop device. Upon loading the device locator UI 204, the mobile device 102 may send a request (330) for location data to the device locator server 203. The request 330 may include a set of public keys or public key hashes that may be used as beacon identifiers for beacon data. The mobile device 102 may generate this set of public keys based on secret information held by the mobile device 102 and the wireless accessory 201 and a timestamp at which the mobile device 102 wishes to receive location data. In one embodiment, this set of public keys is a sequence of public keys P generated based on an anti-tracking secret. i . Public key sequence P i Matching sequence d with the private key i The mobile device 102 may generate a public key sequence and a corresponding public key sequence d i , where i is a counter or timestamp. In one embodiment, the mobile device 102 may generate and send a public key for the previous 24 hours (or a hash of the public key for the previous 24 hours) in the request 330. If no data for a 24-hour public key is found, the mobile device 102 may send a key generated in an earlier time period, returning to a predetermined location data retention limit.

[0066] In one embodiment, encrypted location data is stored and indexed based on a hash of a public key rather than a public key to prevent providers of location service data from storing data that can be used to tie the encrypted location data to a specific device and, therefore, to a specific user or user account. The probe device may send a hash of a public key broadcast within a beacon signal 301 associated with the observed location. The owner of the device may query the device locator server 203 using the hash of the public key determined for the query period.

[0067] In some embodiments, if a location query is to be performed via a web-based interface from an electronic device, such as a laptop or desktop device, a key may need to be sent to the electronic device to enable decryption of the location data. In one embodiment, the decryption key for the location data may be sent to a server that provides a web-based interface to enable the server to decrypt the location data at least when viewing the location data through the web-based interface. Prior to displaying the location data via the web-based interface, a notification may be presented to inform the user that the location decryption key is being temporarily shared with the web-based interface server to enable decryption and presentation of the location data. In one embodiment, the sharing of the location decryption key may be performed via automatic and temporary authorization of location query permissions for a proxy account associated with the web-based interface.

[0068] In one embodiment, the wireless accessory 201 can be placed in a light lost mode. In the light lost mode, a set of future public keys can be generated for the wireless accessory and transmitted to the device locator server 203. The device locator server 203 can then notify the mobile device 102 if any location data corresponding to a key in the set of future public keys is received. In one embodiment, a detector device that sends the location of the wireless accessory in the light lost mode can be directed by the device locator server 203 to relay a message to the wireless accessory 201 that notifies the wireless accessory that it is in the light lost mode. A similar mechanism can also be used to relay a message to the wireless accessory 201 that places the accessory in an explicit lost mode. The user can enable the explicit lost mode via the device locator UI 204. In the explicit lost mode, the wireless accessory 201 cannot be paired with another device unless it is unlocked by the owner. Additional examples of paired devices using location services can be found in U.S. Patent Application No. 16 / 543,227, entitled “A System and Method for Locating Wireless Accessories,” filed on August 16, 2019, which is incorporated herein by reference in its entirety.

[0069] Figure 4is a flow chart illustrating a method for use with a device locator system according to embodiments described herein. Figure 4 A method 400 for pairing a mobile device with a wireless accessory is illustrated. Aspects of the method 400 are also described in Figure 2 and Figure 3 For example, the following description of operations involves mobile device 102, wireless accessory 201, and device locator server 203.

[0070] like Figure 4 As shown, method 400 includes an operation of performing an initial pairing with a wireless accessory (402). The initial pairing can be a Bluetooth pairing or another type of pairing performed using other radio technologies. During the initial pairing, the mobile device and the wireless accessory can exchange identifiers, keys, or other credentials that enable wireless data exchange between the mobile device or another electronic device and the wireless accessory. In one embodiment, the initial pairing with the wireless accessory can include the exchange of credentials associated with the wireless protocol for which the pairing is performed, thereby allowing all data exchanged wirelessly to have at least a first layer of encryption.

[0071] The mobile device can then generate a public / private key pair and one or more additional shared secrets (404). The device can then send the public key and the one or more additional shared secrets to the wireless accessory (406). A variety of key generation techniques can be used. In one embodiment, a variant of ECDH is used to generate a public key pair for encryption. In one embodiment, the one or more additional shared secrets can include an anti-tracking secret that enables the wireless accessory to derive a new public key based on an existing public key.

[0072] After generating the public / private key pair and one or more additional shared secrets, the mobile device may store the public / private key pair in a key store (408). In one embodiment, the key store is a cloud-based key store that can be synchronized with other devices associated with the same cloud service account or a series of cloud service accounts to which the mobile device and the wireless accessory are associated. The cloud-based key store allows the wireless accessory to be located by other synchronized devices. The mobile device may then register the wireless accessory with a device management server (410). Registering the wireless accessory with the device management server may form an association between the wireless accessory and the cloud service account to which the mobile device is associated. In some embodiments, the mobile device may register the wireless accessory with a device group 104. Information stored in a device group profile for the device group may also be synchronized between devices bound to a cloud service account (e.g., a user account). The device management server may communicate with other cloud-based servers such as Figure 2 and Figure 3 The other cloud-based server is associated with a device locator server 203) for facilitating cloud-based services accessible to mobile devices.

[0073] Figure 5 5 is a sequence diagram 500 illustrating a method for presenting a device locator user interface for use with the device locator system described herein, according to an embodiment. Mobile device 102 launches device locator UI 204 (502). Mobile device 102 sends a request to smart home device 107 to perform a beacon scan for advertisements from accessory device 201 (504). Advertisements may be sent before and / or after launching device locator UI 204. In one embodiment, BLE advertisements are received every two seconds. Signal strength measurements are determined by smart home device 107 for beacon signals from at least one accessory device 201 paired with mobile device 102 (506). Smart home device 107 sends location information for accessory device 201 to mobile device 102 (508), and mobile device 102 begins a search process for at least one accessory device (510). The location information may include, but is not limited to, the following data; signal strength information for received beacon signals, whether a beacon signal is detected, a defined spatial identifier for the location of smart home device 107, ranging information for wireless accessory device 201 and / or smart home device 107, battery life expectation for accessory device 201, device status for smart home device 107, and / or any other information that helps locate accessory device 201.

[0074] If the location information for accessory device 201 from smart-home device 107 indicates that smart-home device 107 can help locate accessory device 201 for mobile device 102, mobile device 102 can request that smart-home device 107 provide and / or help provide location services. In one embodiment, mobile device 102 can consider a variety of factors to determine which smart-home device 107 from a group of smart-home devices to use for location services, including, but not limited to, the following: location information from each smart-home device 107, services and / or hardware supported by smart-home device 107, proximity of smart-home device 107 to accessory device 201 and / or mobile device 102, and / or device state of smart-home device 107. For example, a smart-home device 107 from a group of smart-home devices that has one of a higher RSSI value and available processing power (e.g., not executing a higher priority process) can rely on the device locator service. In one embodiment, smart-home device 107 can be compared to a threshold RSSI value and must exceed the threshold to be selected as a smart-home device 107 to help with the finding process.

[0075] In some embodiments, mobile device 102 optionally sends a request to play media (e.g., sound, video, etc.) on behalf of at least one accessory device (512). In some embodiments, a radio connection can be formed between target accessory device 201 and smart home device 107 (512), thereby allowing smart home device 107 to play the sound.

[0076] Mobile device 102 can use the location information received from smart home device 107 to present a proximity view to a user interface for device locator UI 204 user interface, and can present the location information with a visual indicator representing the proximity of target accessory device 201 (516). The visual indicator can correspond to a signal strength measurement of a received advertisement from smart home device 107 and / or mobile device 102. Optionally, in some embodiments, a radio connection can be formed between target accessory device 201 and mobile device 102 (518), thereby allowing mobile device 102 to request target accessory device 201 to play a sound (e.g., content, etc.).

[0077] In some embodiments, a radio connection can be formed between the target accessory device 201 and the mobile device 102, and the mobile device can request to perform a ranging process. When the mobile device 102 is within a threshold range of the target accessory device (520), a ranging process can be performed to determine the distance to and direction to the target wireless accessory device 201. A ranging view for the device locator UI 204 can be presented (522). The ranging view can provide a direction and calculated distance relative to the wireless target to assist in finding the accessory device 201. The ranging view can optionally be combined with the proximity view. Figure 1 The device locator application 204 user interface may be presented together and / or independently within the device locator UI 204. The device locator application 204 user interface may be presented, asking the user whether the accessory device 201 is found (526).

[0078] Figure 6 and Figure 7A device locator system for enabling device location services for a target accessory device according to an embodiment is illustrated. One embodiment provides a system 600 for enhancing location finding using a device locator UI 204 when in a location environment 608 having at least one smart home device 107A proximate to a target accessory device 101 to be located and / or a smart home device 107B not proximate to a wireless accessory device 101. If a smart home device 107A can receive a wireless communication signal 605 (e.g., detect a beacon signal) from a wireless accessory device 101, the smart home device 107A can be proximate to help locate the wireless accessory device 101. The location environment 608 can include a mobile device 102 (e.g., a smart phone, a wearable device, a tablet, etc.). The mobile device 102 can wirelessly communicate with one or more smart home devices 107 (including 107A and 107B) and / or a wireless accessory device 101 including a radio 619.

[0079] Devices 101, 107, and 102 can communicate wirelessly via wireless communication signals 605 and by scanning wireless channels, transmitting and receiving beacons or beacon frames, establishing connections (e.g., by transmitting connection requests), and / or transmitting and receiving packets or frames (which may include requests and / or additional information such as data as payload). Wireless communication signals 605 can be carrier signals that conform to wireless communication technologies such as, but not limited to, Wi-Fi or Bluetooth. In addition to wireless communication, mobile device 102, smart home device 107, and / or wireless accessory device 101 perform wireless ranging operations using wireless ranging signals 606 (as shown between smart home device 107A and wireless accessory device 101). Wireless ranging signals can be, for example, ultra-wideband signals that can be used to determine the distance and / or angle between wireless accessory device 101 and smart home device 107 and / or wireless accessory device 101 and mobile device 102 using the techniques described herein. In one embodiment, the data provided by wireless ranging signals 606 can be correlated with other metrics such as the RSSI of wireless communication signals 605. In one embodiment, the communication processor of the mobile device 102 and / or smart home device 107 may fuse multiple types of ranging to provide a unified distance and / or angle estimate based on multiple types of radio data.

[0080] In some embodiments, mobile device 102 can provide a device locator UI 204 that presents a map and / or view of location environment 608. Mobile device 102 can send a request to perform a beacon scan (e.g., requests as shown by 109 to 107A and 11 to 107B) to one or more smart home devices 107A and 107B to detect wireless accessory device 101. In response to the request, smart home devices 107A and 107B can send location information including RSSI information and / or ranging information to mobile device 102. In some embodiments, smart home devices 107A and 107B can communicate and access a map or container of location information of wireless accessory devices 101 to determine which smart home device 107 should be prioritized for communicating location information to mobile device 102. Smart home device 107B does not detect a beacon signal from wireless accessory device 101 and can communicate that no signal was detected to help locate wireless accessory device 101 because the location in location environment 608 is eliminated. Alternatively, as shown, smart home device 107A detects a beacon signal (e.g., a wireless communication signal) and exchanges ranging signals 606 with wireless accessory device 101. Thus, mobile device 102 can use the location information from smart home device 107A to locate wireless accessory device 101. In this way, smart home devices 107 can act as a network of distributed search agents for locating wireless accessory devices 101.

[0081] In some embodiments, the mobile device 102 can display a map and / or view with a virtual representation of the wireless accessory device 101. The virtual representation can simplify the process of locating the wireless accessory device 101 by the user of the mobile device 102. For example, using a directional indicator pointing to the detected location of the wireless device 101, as provided by the location information from the smart home device 107, can simplify the process of finding the approximate location of the wireless accessory device 101. The directional indicator can be combined with the location environment 608. Figure 1 The map may include a virtual representation of the wireless accessory device 101. When the mobile device 102 is in proximity to the wireless accessory device 102 so that the beacon signal includes a wireless communication signal and / or a ranging signal, such as Figure 7 As shown, a proximity view and / or a ranging view of the location environment 608 can be presented, which can allow a user to identify the wireless accessory device 101 .

[0082] In some embodiments, the smart home device 107A can play a media file (e.g., an audio file) on behalf of the wireless accessory device 101 to assist the user of the mobile device 102 in finding the location of the wireless accessory device 101. In one scenario, the wireless accessory device 101 can be hidden by an item. The item can be, for example, a jacket or another article of clothing, and the wireless accessory device 101 can be a smart phone or tablet device located in a pocket of the item. The item and the wireless accessory device 101 can each be within a container such as a backpack, a locker, an item of luggage, or another item that reduces the effectiveness of audio-based positioning techniques that rely on the wireless accessory device 101 playing sounds that can be heard by the user.

[0083] In another scenario, the wireless accessory device 101 may be a beacon peripheral or locator tag attached to an item. To find the item, the user may use the device locator UI 204 on the mobile device 102 to find the wireless device 201, thereby enabling the user to locate the item to which the wireless device 201 is attached. The proximity enhancements described herein may be used to enhance the device locator UI 204, thereby allowing the user to quickly determine the location of an item via the wireless device 201 in scenarios where the item may be hidden by a container or another physical item.

[0084] In one embodiment, the mobile device 102 and / or smart home device 107 may establish a secure wireless communication connection (e.g., a Bluetooth connection as shown in 606) with the wireless device 101 and command the wireless device 101 to begin a wireless ranging process using, for example, a wireless ranging signal 606, which may be a UWB signal. Figure 7 A system 700 is illustrated for implementing distributed device location finding for a target accessory device when the mobile device 102 is close enough to exchange communication signals 710 and wireless ranging signals 706 according to an embodiment. Optionally, the mobile device 102 can utilize location information (e.g., RSSI and / or ranging information) from the smart home device 107 (as shown by the communication via 109 and 111).

[0085] Wireless ranging signal 606 enables mobile device 102 to receive range and angle measurements of wireless accessory device 101. In some embodiments, mobile device 102 can request that target wireless accessory device 101 increase the rate at which advertisements are sent (e.g., from every 2 seconds to every 30 ms) to aid in finding wireless accessory device 101.

[0086] In one embodiment, before establishing a secure wireless communication connection and / or under an established wireless connection, a signal strength measurement can be determined based on the received advertisement and can help guide the user to the wireless device 101. In addition, the location information from the smart home device 107A about the defined space within the location environment 608 can provide the user with sufficient directions to move within the range to establish a wireless communication signal and a ranging signal to locate the wireless accessory device 101.

[0087] In the embodiments described herein, wireless ranging can be performed using any standard or proprietary ranging technology, or any combination of standard and / or proprietary ranging technology. Wireless ranging operations can be performed to determine the distance between devices (e.g., between an initiator and a responder), the direction between devices, or both. For example, a flight time / arrival time (ToF / ToA) can be determined for one or more messages between devices, which can be used to establish a distance measurement. One or more messages can have any format and can be transmitted using any wireless protocol. In some embodiments, a two-way exchange of two or more messages can be used to determine ToF / ToA. In some embodiments, one or more messages used to perform ranging can be protected, for example, by encrypting or otherwise protecting at least a portion of the content. In addition, in some embodiments, the direction of the source of one or more wireless signals can be determined using a technique such as angle of arrival (AoA). For example, AoA estimation can be performed using multiple receiving elements (e.g., elements of an antenna array) to measure different times (TDOA) and / or different phases (PDOA) of signal arrival. Additionally or alternatively, in some embodiments, directivity can be determined by measuring Doppler shift to establish frequency difference of arrival (FDOA). Wireless ranging techniques can be applied alone or in combination to perform a single ranging operation. In addition, wireless ranging techniques can be applied alone or in combination to perform continuous ranging operations, such as continuous or discontinuous ranging, and the history of measurements can be captured and used in performing operations based on range and / or direction.

[0088] Incoming measurements may be processed to reduce measurement noise and stabilize the estimated location of the lost item. In one embodiment, the mobile device 102 may use sensor fusion techniques that rely on multiple measurement streams to obtain the best estimate of the device's location. For example, incoming range and / or angle measurements may be fused with the position and orientation measurements of the mobile device 102. Position and orientation measurements may be collected using visual inertial measurements (VIO), which use IMUs and computer vision using camera data to determine device position and orientation. VIO data enables software on the mobile device 102 to be aware of the movement and orientation of the device. Combining VIO data with range and / or angle measurements determined via wireless ranging signals 606 and / or 706 and / or enables the most likely location of the wireless accessory device 101 to be estimated and allows the user to point to the wireless accessory device 101. In addition, combining VIO data with signal strength measurements via wireless communication signals 605 and / or 710 may help guide the user to the wireless device 101 through a close view and / or in conjunction with a ranging view. Due to multipath signal propagation effects, VIO data may be particularly useful in scenarios where one or more of the range or angle measurements may be inaccurate.

[0089] The algorithm for determining the location of the lost item can be accessed via the device locator UI 204. When the user intends to locate the item, the user can select the item from the list of registered devices and select the "Find" button presented via the device locator UI 204. In various embodiments, multiple user interfaces can be presented. Some interfaces resemble a two-dimensional compass-like view, in which two-dimensional arrows guide the user to the target item or device to be found. In addition to presenting the trajectory that the user has taken in his search (where the proximity indicator is presented along the path taken), the proximity view and / or the combination of the proximity view and the ranging view of the device locator UI 204 can provide arrows to guide the user.

[0090] Figure 8 A flowchart 800 for a method for enabling location services for a target accessory device according to an embodiment is illustrated. A smart home device 107 receives a request for location information for a target accessory device 101 associated with an owner mobile device 102 (802). The request may include a data packet having at least one public key associated with the target accessory device 101. The data packet having at least one public key may enable the smart home device 107 to identify a wireless device within communication range as a target accessory device 101 associated with the owner mobile device 102 (e.g., as described in reference to FIG. 1 ). Figure 3In some embodiments, the request may include a data packet containing a set of rolling public keys that are expected from the target accessory device over time. The set of rolling public keys enables smart home device 107 to identify the target accessory device 101 for a time period corresponding to each rolling key from the set of rolling public keys.

[0091] Smart-home device 107 may receive the request from mobile device 102, device locator service 170, another smart-home device 107 in the location environment, and / or directly from the user via an interface (e.g., a voice user interface, a user interface with a verbal command, etc.). In one embodiment, smart-home device 107 is a voice-activated virtual assistant, and smart-home device 107 may receive a verbal command for location information of a target accessory device 101, the verbal command being interpreted as a request for location information including a data packet having at least one public key for target accessory device 101. Mobile device 102 and / or device locator service 170 may send the at least one public key directly to smart-home device 107 and / or provide access to the at least one public key via a user account. The at least one public key may be associated with a user account, and smart-home device 107 may access the at least one public key for the target accessory device 101 from the user account associated with owner mobile device 102. For example, if smart home device 107 has access to a user account for the owner of mobile device 102, smart home device 107 can use one or more rolling public keys (e.g., Figure 3 The beacon signal from the target accessory device 101 is identified using a rolling public key for the wireless accessory 201 in the wireless accessory 201.

[0092] Smart home device 107 receives a beacon signal from the wireless device and determines at least one proximity value based on the beacon signal (804). The proximity value can be a representation of an estimated distance, relative distance, and / or relative position to a target accessory device 101. The wireless device is a candidate for a target accessory device 101 within a location environment, and the proximity value can be calculated before, after, and / or in parallel with determining whether the wireless device is a target accessory device 101 in step 806. The proximity value can be used to prioritize the use of one or more smart home devices over another smart home device 107 to provide location information to assist mobile device 102 in locating a target accessory device 101.

[0093] In some embodiments, multiple factors can be used to determine the proximity value, including, but not limited to: the availability of processing power to process the request from mobile device 102, the hardware capabilities of smart home device 107 (e.g., UWB ranging capabilities, speakers, wireless communication capabilities, microphones, etc.), RSSI values, ranging data, sensor data, proximity to mobile device 102 (e.g., smart home device 107 within wireless communication range to mobile device 102), and / or any combination thereof. For example, smart home device 107 can determine at least one proximity value to a wireless device (e.g., candidate target accessory device 101) based on at least one signal strength value. In one embodiment, the signal strength value represents the proximity of smart home device 107 to target accessory device 101 by providing a value representing a relative distance to the candidate target accessory device, and can be used as the proximity value for smart home device 107. In this case, the higher the proximity value (e.g., signal strength value) compared to the proximity values ​​from other smart home devices 107, the greater the likelihood that smart home device 107 in a group of smart home devices is the smart home device that is closest to target accessory device 101. In this example, other factors may be used in the final calculation, such as the communication range and ranging capability to mobile device 102. For another example, if smart home device 107 does not receive a beacon signal or determines that the received beacon signal does not contain at least one public key, then smart home device 107 sends a proximity value to owner mobile device 102 with an indication that target accessory device 101 is not within the wireless communication range to smart home device 107.

[0094] In yet another example, smart home device 107 has a wireless controller that includes a ranging sensor, and smart home device 107 performs ranging operations via an ultra-wideband radio device. Smart home device 107 determines a range relative to and a direction to a target accessory device 101 via the ranging sensor of the wireless controller during the two-way ranging operation. Smart home device 107 determines a proximity value representing an estimate of a target position of target accessory device 101 relative to smart home device 107 based on the range relative to and the direction to the target accessory device 101. In some embodiments, for smart home devices that have ranging capabilities and are within range of target accessory device 101, the proximity value can be weighted higher to perform two-way communication and determine the range relative to and the direction to the target accessory device 101.

[0095] Smart-home device 107 determines that the received beacon signal for the wireless device contains at least one public key (806). A comparison is performed between the at least one public key and a key (if any) within the received beacon signal. If the received key matches at least one public key associated with target accessory device 101, then the wireless device is a target accessory device 101 associated with mobile device 102. In some embodiments, smart-home device 107 can have a set of public keys for target accessory devices 101 that are expected to be broadcast at various time periods.

[0096] In some embodiments, such as Figure 3 3 and 4. As shown in conjunction with wireless accessory 201, target accessory device 101 (e.g., a specific implementation of wireless accessory 201) periodically broadcasts a beacon signal 301 that includes device status information and a beacon identifier. The beacon identifier is a public key derived from a shared secret (e.g., an anti-tracking secret) established during a public key exchange between mobile device 102 and target accessory device 201, as shown in FIG. Figure 3 As shown. In addition, the target accessory device 101 can periodically perform public key derivation to generate a new public key and begin broadcasting the new public key as a beacon identifier. The public key is a K-byte key, where a new K-byte key is generated every M minutes. Therefore, the mobile device 102 can provide the set of public keys expected for multiple time periods (e.g., a time period of M minutes).

[0097] Smart home device 107 sends location information based on the comparison result (808). If the wireless device is determined to be the target accessory device by the key comparison (806), the location information for the target accessory device 101 is sent to mobile device 102. In some embodiments, the location information is sent to mobile device 101 when the proximity value exceeds a threshold for the proximity value. Smart home device 107 sends location information to mobile device 102, including, but not limited to, at least one of the following: a target location estimate for the target accessory device 101, the location of smart home device 107, at least one proximity value, and / or an indicator of the direction to the target accessory device 101. For example, the RSSI for the beacon signal can be transmitted along with the location data for the wireless accessory device 101 and / or the smart home device 107. In one embodiment, mobile device 102 can receive all location information from smart home devices 107 and prioritize the receipt of location information from smart home devices 107 with relatively higher proximity values. Alternatively, mobile device 102 can receive location information from a priority list of smart home devices 107 having relatively high proximity values ​​determined by smart home devices 107 and / or device locator server 170 that received proximity values ​​from smart home devices 107. The location information provided to mobile device 102 can include information about the location of smart home devices 107 within the location environment. For example, RSSI data can be used to determine the distance of wireless accessory 201 from smart home device 107 and facilitate triangulation on mobile device 102.

[0098] If smart home device 107 receives an indication from mobile device 102 that smart home device 107 has a higher proximity value (e.g., is prioritized for providing location information) than at least one other proximity value from a group of smart home devices, smart home device 107 establishes a radio connection with target accessory device 101 and can perform tasks on behalf of target accessory device 101.

[0099] In one embodiment, smart home device 107 can have a wireless controller that includes a ranging sensor. Smart home device 107 can perform ranging operations via an ultra-wideband radio. Smart home device 107 determines a range relative to target accessory device 101 and a direction to the target accessory device via the ranging sensor of the wireless controller during a two-way ranging operation. Smart home device 107 determines a target location estimate for target accessory device 101 relative to smart home device 107 based on the range relative to target accessory device 101 and the direction to the target accessory device. Smart home device 107 receives requests from target accessory device 101 that include at least one of a request to play audio, a request to send a target location of target accessory device 101 to mobile device owner 102, and / or a request to send location information for smart home device 107 to owner mobile device 102. Smart home device 107 can request that target accessory device 101 increase the rate at which it sends beacons to provide more data in order to locate target accessory device 101.

[0100] In one embodiment, presenting the signal strength proximity view and the ranging view includes presenting at least one indicator of a proximity value to the target accessory device 101 and an indicator for a direction to the target wireless accessory device 101 along a track in the device locator UI 204 .

[0101] Fig. 9 A flow chart 900 for a method for enabling location services for a target accessory device is illustrated according to an embodiment. A smart home device 107 receives a request to play content for a target accessory device 101 associated with an owner electronic device 102 (902). In one embodiment, one or more smart home devices 107 designated as being proximate to a target accessory device 101 can receive content for playback from a target accessory device 101, a mobile device 102, another smart home device 107, and / or a device locator server 170. In some embodiments, a smart home device 107 can access a user account having content designated for playback for the target accessory device 101.

[0102] The smart home device 107 determines an occupancy value for the defined space (904). The smart home device 107 may use sensors, microphones, wireless signals from mobile devices, known user routines, machine learning, and / or any other method for determining whether the defined space is occupied. In some embodiments, the received radar data and / or audio data may be used by the smart home device 107 to detect that a person is in a defined space within a location environment, enters a defined space, and / or leaves a defined space. Content playback settings may be specified for various conditions, including but not limited to the following: the number of occupants, identified occupants (e.g., establishing a connection with a user device), volume preferences, privacy issues, and / or preferences for a specific defined space. In one embodiment, when a user of the mobile device 102 is detected to enter a defined space, the playback of the content may be adjusted. The user of the mobile device 102 may be detected by any of the following: sensors (e.g., microphones, radar, audio data, etc.), being within range to exchange wireless communication signals or establish a wireless connection with one or more smart home devices 107, providing voice commands to the smart home device 107, voice recognition for the owner of the mobile device 102, or a mobile device 102 owner routine. Smart home devices 107 can communicate with other smart home devices 107 about the movement of occupants, including users of mobile devices 102. Smart home devices 107 can have a map of the locations of other smart home devices 107 in a defined space and know which smart home device 107 is adjacent to another smart home device 107.

[0103] The playback of the content is adjusted based on the occupancy value (906). The playback of the content can be adjusted by increasing or decreasing the volume, increasing or decreasing the playback speed, changing the brightness, switching the content, turning off the screen, and / or other adjustments to the playback of the content. In one embodiment, if someone other than the user of the mobile device 102 is located in the defined space, the content is not played. In some embodiments, all smart home devices 107 in proximity to the target accessory device 101 can play the content to assist the mobile device 101 in actively searching. For example, when a user enters the defined space and / or forms a wireless connection with the target accessory device 101 and / or smart home device 107, the playback can be stopped or adjusted to a lower level.

[0104] FIG. 10A to FIG. 10B A system 1000 for implementing distributed device location finding for a target accessory device according to an embodiment is illustrated. Fig. 10AAs shown, one embodiment provides a system 1000 for enhancing location finding using a device locator UI 204 when in a location environment 1008 having at least one smart home device (e.g., 107A, 107C, and 107D) proximate to a target accessory device 101 to be located and / or smart home devices (107B, 107E, and 107F) not proximate to the wireless accessory device 101. A mobile device 102 may request 109 (e.g., requests shown as 109A-109F) that each of the smart home devices 107 (e.g., 107A-107F) act as a search agent for a network to detect a beacon signal (e.g., illustrated with communication signals 111A, 111C, and 111D) from the accessory device 101 and provide location information in response. The request from the mobile device 102 may include a data packet with one or more public keys that the mobile device 102 expects the accessory device 101 to contain within the beacon signal. Each of smart-home devices 107 can attempt to detect and respond to the request using the RSSI value (if any) and location information for the defined space of smart-home device 107. The location information can include a defined space in the location environment, such as a room name in location environment 1008, as shown with "family room," "kitchen," "bedroom," "hallway," "garage," and / or "bathroom." In one embodiment, when smart-home devices 107 are registered during setup, a defined space within location environment 1008 can be assigned, and an association with the defined space for each smart-home device 107 can be stored in a database. If smart-home device 107 receives wireless communication signal 111 from wireless accessory device 101 (e.g., 111A, 111C, and 111D, a beacon signal is detected), smart-home devices 107A, 107C, and 107D can be proximate to wireless accessory device 101 to help locate wireless accessory device 101. As shown, wireless ranging signals 1006 can optionally be sent between smart home device 107A and accessory device 101, and ranging information can be provided along with location information sent to mobile device 102 to help locate accessory device 101. Location information responsive to the request can be sent to mobile device 102, device locator server 170, and / or communicated to other smart home devices 107 in location environment 1008. Fig. 10AIn the location environment 1008, smart home devices 107A, 107C, and 107D can provide location information with RSSI values, and comparisons between the received RSSI values ​​can be performed by mobile device 102, smart home devices 107, and / or device locator server 170 to determine which smart home device 107 can help mobile device 102 locate accessory device 101. In one embodiment, smart home devices 107 can communicate location information by synchronizing data across devices using the method provided in co-pending U.S. patent application Ser. No. 16 / 420,892, filed on May 23, 2019, entitled “Data Synchronization across Multiple Devices,” which is incorporated herein by reference in its entirety.

[0105] Location information from various smart home devices 107 can be used to update device locator UI 204 for locating accessory device 101. Location information can be used to update device locator UI 204 with identifiers for defined spaces corresponding to smart home devices 107, estimates of distance and direction of accessory device 101 determined by smart home devices 107 and / or by mobile device 102 using location information and / or ranging information from smart home devices 107. Location information can be used to update a proximity view and / or ranging view of device locator UI 204.

[0106] In some embodiments, smart home devices (e.g., 107A and / or 107C) with available processing power can play media files on behalf of accessory device 101 or in conjunction with the accessory device to supplement and / or amplify audio intended for accessory device 101. Optionally, smart home devices 107A and 107C can be paired with mobile device 102 as a group and communicate location information to mobile device 102, as shown at 109A. As mobile device 102 moves closer to accessory device 101, mobile device 102 can receive a beacon signal, shown along with wireless communication signal 120, from accessory device 101 and / or form a wireless connection (e.g., BLE, etc.) with accessory device 101. The wireless communication signal can be used to provide a proximity view on device locator UI 204, and ranging information 1006 exchanged between smart home device 107A and accessory device 101 can be used to create a ranging view on device locator UI 204 from the perspective of smart home device 107A and / or mobile device 102. As shown in FIG. Fig. 10B As shown, the mobile device 102 moves closer to the accessory device 101 and may be able to directly exchange wireless ranging signals 1006 with the accessory device 101 to provide a ranging view on the device locator UI 204 .

[0107] Fig.11 Flowchart 1100 is illustrated for a method for enabling location services for a target accessory device from the perspective of a target accessory device 101, according to an embodiment. Various signals and content (e.g., audio content) can be provided by a target wireless accessory 101 to a smart home device 107 to assist a mobile device 102 in locating the target wireless accessory 101. Optionally, mobile device 102 requests that the target accessory device 101 play audio. In some embodiments, the audio played by the target accessory device 101 can be encoded with information that allows the smart home device 107 to identify the target accessory device 101.

[0108] In some embodiments, the accessory device 101 sends a beacon signal having at least one public key (1104). Figure 3 As shown, accessory device 101 transmits a beacon signal that contains a key from a set of rolling public keys that are expected from the accessory device over time. The rolling public keys enable smart home device 107 to identify accessory device 101 for a time period corresponding to each rolling key from the set of rolling public keys. Additionally, target accessory device 101 can periodically perform public key derivation to generate a new public key and begin broadcasting the new public key as a beacon identifier. The public key is a K-byte key, where a new K-byte key is generated every M minutes. Thus, mobile device 102 can provide the set of public keys expected for multiple time periods (e.g., a time period of M minutes).

[0109] Optionally, accessory device 101 performs a two-way ranging operation with smart-home device 107 (1106). Smart-home device 107 is selected by owner mobile device 102, a group of smart-home devices 107, and / or device locator server 170 to communicate with accessory device 101. In one embodiment, smart-home device 107 is selected after comparing proximity values ​​of candidate smart-home devices 107. The proximity value may need to meet a threshold for proximity value and / or have a higher value than proximity values ​​of other smart-home devices 107. A ranging process can be performed between smart-home device 107 and target accessory device 101 to determine the distance from the target accessory device 101 and the direction to the target accessory device. When mobile device 102 is within wireless communication range to target accessory device 101, then smart-home device 102 can provide ranging information to mobile device 102, and mobile device 102 can perform a two-way ranging operation directly with target accessory device 101. The ranging information from smart-home device 107 and mobile device 102 can be used to present ranging views, proximity views, and / or combinations thereof as described herein.

[0110] Optionally, in one embodiment, when target accessory device 101 plays audio, smart home device 107 can use at least one microphone to estimate the location and direction of target accessory device 101. In one embodiment, smart home device 107 can have one or more directional microphones to estimate that the audio played by target accessory device 101 is coming from a specific location or direction.

[0111] In one embodiment, accessory device 101 sends media content (e.g., audio, video, etc.) to be played to smart-home device 107 on behalf of accessory device 101 (1108). Accessory device 101 can receive a request to send media content to smart-home device 107 to amplify sound that the accessory device can provide using a speaker.

[0112] Fig.12 A flowchart 1200 for a method for enabling location services for a target accessory device according to an embodiment is illustrated. A mobile device 102 (e.g., an owner electronic device) sends a request for location data for a target accessory device 101 associated with an owner electronic device (e.g., mobile device 102) to one or more smart home devices 107 (1202). Mobile device 102 may be an owner electronic device or a mobile device 102 that has been given the privilege to search for a time period on behalf of the owner electronic device. The request may be communicated to smart home device 107 wirelessly and / or via an interface (e.g., a voice user interface with a verbal command, a user interface, etc.). In one embodiment, smart home device 107 is a voice-activated virtual assistant, and smart home device 107 may receive a verbal command for location information of target accessory device 101, the verbal command being interpreted as a request for location information including a data packet having at least one public key for target accessory device 101. Mobile device 102 and / or device locator server 170 may send at least one public key directly to smart home device 107 and / or provide access to at least one public key via a user account. At least one public key can be associated with a user account of the owner electronic device and / or a user account of another mobile device that is authorized to access a set of keys to locate the target accessory device by the owner electronic device. Smart home device 107 can access at least one public key for target accessory device 101 from a user account associated with owner mobile device 102. For example, if smart home device 107 has access to a user account or a shared user account (e.g., a family account, a trusted friend account, etc.) for the owner of mobile device 102, smart home device 107 can use one or more rolling public keys (e.g., a family account, a trusted friend account, etc.) for the target accessory device 101 paired with mobile device 102. Figure 3 The beacon signal from the target accessory device 101 is identified using a rolling public key for the wireless accessory 201 in the wireless accessory 201.

[0113] Mobile device 102 receives one or more proximity values ​​from one or more smart home devices (1204). The proximity value can be a representation of an estimated distance, relative distance, and / or relative position to a target accessory device 101. The wireless device is a candidate for a target accessory device 101 within a location environment, and the proximity value can be calculated before, after, and / or in parallel with determining whether the wireless device is a target accessory device 101 in step 806. The proximity value can be used to prioritize the use of one or more smart home devices over another smart home device 107 to provide location information to assist mobile device 102 in locating a target accessory device 101.

[0114] The mobile device presents location information from at least one of the one or more smart-home devices within a user interface based on a comparison between the proximity value and the threshold value (1206). In one embodiment, the mobile device 102 may consider a variety of factors to determine which smart-home device 107 in a group of smart-home devices to use for location services, including, but not limited to, the following: location information from each smart-home device 107, services and / or hardware supported by the smart-home device 107, proximity of the smart-home device 107 to the accessory device 201 and / or the mobile device 102, and / or the device state of the smart-home device 107. For example, a smart-home device 107 in a group of smart-home devices that has one of a higher RSSI value and available processing power (e.g., not executing a higher priority process) may rely on the device locator service. In one embodiment, the smart-home device 107 may be compared to a threshold RSSI value and must exceed the threshold value to be selected as a smart-home device 107 to assist in the finding process.

[0115] A ranging view, a proximity view, or at least one of a ranging view and a proximity view is selectively presented within a user interface of the owner electronic device 102 (1208). Location information can be used to provide proximity information and / or ranging information. In some embodiments, the user interface 204 can indicate a defined space (e.g., a room) in which the smart home device 107 is located. In another embodiment, signal strength and / or ranging data can be used to assist the user in locating the target accessory device 101. For example, in one embodiment, signal strength measurements from signals received at the smart home device 107 and / or the mobile device 102 can be used to indicate the location of a nearby target accessory device 101. Optionally, within the user interface 204, a proximity indicator can be displayed to indicate when the mobile device 102 is in proximity to the target device as the user is moving through the location environment. The following description Fig.21 and Fig. 222 illustrates a device locator user interface 204 on a mobile device 2100 according to an embodiment described herein. Fig.21 The illustrated "proximity view" 2104 can present proximity information using visualization techniques to present proximity information about a target wireless accessory device. In one embodiment, the proximity view 2104 has visual indicators (such as user interface elements) positioned along a track 2118 presented within the user interface 204 to represent the path the user has taken in their search. In some embodiments, the mobile device can be moved within a threshold range of a target device, allowing a ranging process to determine the distance to and direction to the target device using communications between the mobile device and the target device. Fig. 22 As shown, the “ranging view” 2120 of the user interface 204 may provide a distance measurement 2116 that may be selectively displayed in addition to the direction to the target device or the defined space identifier 2108. In some embodiments, when the ranging data of the ranging view 2120 is not available because the target device is not within a threshold range of the mobile device, the target device transmitter is not in the field of view of the receiver at the mobile device, and / or the mobile device does not have a substantially unobstructed view to the target device, the proximity view 2104 for finding the target device may be used. In some embodiments, the proximity view of the user interface 204 may present the ranging information using the ranging measurement, and present an arrow 2128 indicating the direction of the target wireless accessory device 201 and the distance 2136 from the target wireless accessory device 201.

[0116] Fig.13 A flowchart 1300 is illustrated for a method for enabling location services for a target accessory device according to an embodiment. Fig.13As shown, method 1300 includes an operation of an electronic device launching a device locator UI (1301). In response to launching the device locator UI, an electronic device, which may be a mobile device as described herein, or another electronic device associated with the same cloud service account as the mobile electronic device, may perform operations to generate a set of public keys (1302) included in a beacon signal broadcast by the wireless accessory during a first time period. The first time period may be, for example, the previous 24 hours. The electronic device knows the frequency with which the wireless accessory generates new public keys, and using a shared secret generated with the wireless accessory, a set of public keys corresponding to the keys generated by the wireless accessory during the first time period may be generated. The electronic device may then send the set of public keys in a request to a device locator server to send location data corresponding to the set of public keys (1303). In one embodiment, the location data sent by the server in response to the request will be encrypted using a public key transmitted as a beacon identifier of the wireless accessory. The electronic device may decrypt the encrypted location data received by the server using a private key generated during the initial pairing with the wireless accessory (1304). The electronic device may then process the location data to determine the highest probability location of the wireless accessory (1305). In one embodiment, the location data may include data of accessory devices 201 in the device group.

[0117] Processing location data may include a variety of different operations. In one embodiment, the location data includes latitude and longitude information and a timestamp that determines the location. The electronic device may perform triangulation based on the timestamp and remove noise or abnormal locations. In one embodiment, the location data specifies the location of the detector device that detected the beacon. The location data may also include UWB ranging information and / or RSSI information of the beacon detected by the detector device. The electronic device may analyze the UWB ranging information and / or RSSI information in the context of the device location to obtain a more accurate location of the wireless accessory. Data that may be transmitted by the detector device and used for location processing is in Fig.14 and described below.

[0118] like Fig.14As shown, method 1400 includes operations that can be performed when a device locator server does not have location data to provide to an electronic device in response to a request. In the case of a device group, an electronic device (e.g., mobile device 102) can provide location data about devices in the device group. The electronic device can generate a first set of public keys (1401) included in a beacon signal broadcast by a wireless accessory during a first time period. The first time period can be, for example, 24 hours, but other initial search time periods can be used. The electronic device can perform subsequent operations to request the device locator server to send location data corresponding to the first set of public keys (1402). If the data is returned by the server (1403, "yes"), the electronic device can use a private key corresponding to the set of public keys to decrypt the location data received from the server (box 1409).

[0119] If the server does not return data (1403, "No"), the electronic device can generate a second set of public keys included in a beacon signal broadcast by the wireless accessory during a second time period (1404). The second time period can be 24, 48, or other number of hours before the first time period. The electronic device can then request the device locator server to send data corresponding to the second set of public keys (1405). If the server returns data in response to the request (1406, "Yes"), method 1400 can proceed to box 1409, where the electronic device decrypts the received data. If the server does not return data (1406, "No"), or the server sends a reply indicating that the data is not available, method 1400 includes: the electronic device can widen the search time (1407) by continuously requesting earlier time periods until the maximum time period is reached.

[0120] Fig.15 1 is a flow chart illustrating a method 1500 of broadcasting a signal beacon at a wireless accessory according to an embodiment. Aspects of the method 1500 are also described in Figure 2 and Figure 3. Method 1500 includes the wireless accessory deriving a public key (block 1502). The public key may be derived based on a shared secret and a timestamp determined based on a clock or timing device of the wireless accessory. Optionally, a determination is made as to whether the wireless accessory is part of a device group (1504). If the wireless accessory is part of a device group, status information and / or verifiable information for other accessory devices 201 in the device group is provided in the beacon signal (1506). The wireless accessory may indicate status information and / or verifiable information, such as whether any other wireless accessory in the device group is proximate, connected (physically or wirelessly), and / or any other information about other wireless accessories in the device group 105. In one embodiment, a set of bits included in the beacon signal may represent each accessory in the device group, and setting a Boolean value (e.g., true (1) or false (0)) may indicate whether the corresponding accessory is proximate and / or connected to the accessory device that sent the beacon signal. Alternatively, if the wireless accessory is not part of the device group, no information is provided on the device group (1504). The wireless accessory can then transmit a beacon signal at a first frequency, wherein the beacon signal includes the public key (1508). The first frequency can be variable, and in one embodiment is one beacon every two seconds.

[0121] After transmitting the beacon signal, the wireless accessory can listen for a response from the owner device (1510). If the wireless signal receives a response from the owner device (1510, "yes"), the wireless accessory can enter a near owner state (1512) and begin transmitting beacon signals at a second, lower frequency (1516). If the wireless accessory does not receive a response from the owner device (1510, "no"), the wireless accessory can continue to send beacons at the first frequency (1514).

[0122] Method 1500 also includes rotating the public key once every M minutes by the wireless device when sending beacons, where the value of M can be variable between implementations and / or based on device state. Based on a timer expiration, a counter, or another mechanism, the wireless accessory can determine whether the accessory has entered a new key time period (1518). Although the wireless accessory has not yet entered a new key time period (1518, "No"), the accessory can continue to send beacons using the current public key (1522). When the wireless accessory detects that it has entered a new key time period (1518, "Yes"), the accessory can derive a new public key using the current timestamp (1520). In one embodiment, the new public key can be derived using an existing public key, a timestamp, and an anti-tracking secret.

[0123] Figures 16 to 21 A device locator UI 204 is illustrated according to an embodiment. Fig.16A first graphical user interface of the device locator UI 204 according to an embodiment is shown showing notifications of various wireless accessories of the user. The device locator UI 204 can cause a detached notification 1602 to be presented on the home screen 1601 of the electronic device 1600. Fig.17 A second graphical user interface of the device locator UI 204 is shown that enables viewing of left-behind accessory devices on a map, adding a trusted location, or requesting notifications for an item be stopped, in accordance with an embodiment.

[0124] Fig.18 A third graphical user interface of a device locator UI 204 is shown that enables finding accessory devices 101 (including devices in a device group) on a map in accordance with an embodiment. Fig.19 A fourth graphical user interface of a device locator UI 204 according to one embodiment is shown, which enables a wireless accessory to be set to lost mode or to notify when found. The device locator UI 204 can be displayed on an electronic device, which can be a mobile device 102, or any other type of electronic device described herein. Fig. 20 A fifth graphical user interface of device locator UI 204 is shown that enables a wireless accessory to add a trusted location in accordance with an embodiment. Fig.21 A sixth graphical user interface of the device locator UI 203 is shown that uses signal strength measurements to implement a proximity view in accordance with an embodiment.

[0125] like Fig.17 As shown, the device locator UI 204 can present a unified graphical interface on the electronic device 1700 through which a variety of different types of devices and accessories can be located, including wireless devices with network or cellular access and wireless accessories without local network access. The device locator UI 204 may include a map 1704 with a marker 1705 that shows the current or last known location of the wireless accessory device 101. The marker 1705 can be an icon, image, graphic, or any other user interface element that identifies the accessory and communicates the location of the accessory. An optional element 1707 in the device locator UI 204 can present an option to request not to be notified of a particular item. An optional element 1706 in the device locator UI 204 can present a description or name of the wireless device or accessory, and can show an estimated distance between the wireless device or accessory and the current location of the electronic device 1800, such as Fig.18 shown.

[0126] like Fig.18As shown, the device locator UI 204 can present a user interface that enables the wireless accessory to view items 1803 and 1805 and the distance from the electronic device 1800. In one embodiment, the third user interface can be responsive to selecting Fig.18 The third user interface can present a user interface element 1802 representing and / or describing the wireless accessory under consideration, as well as a map 1801 and a marker 1802 showing the current or last known location of the wireless accessory.

[0127] like Fig.19 As shown, the device locator UI 204 may present a fourth graphical user interface that enables the wireless accessory to be set to lost mode. In one embodiment, when the wireless accessory cannot be located via the device locator UI 204, the map 1901 will not display a mark indicating the location of the accessory. The device locator UI 204 may present a user interface element 1904 representing and / or describing the wireless accessory under consideration and a set of optional user interface elements. An optional user interface element 1906 may present an option for notifying the user when the accessory is found. When notification when found is enabled, in one embodiment, the wireless accessory may be placed in a light lost mode. An electronic device associated with the device locator UI 204 may generate a set of public keys, and the wireless accessory will broadcast this set of public keys together with the beacon signal during a future time period (e.g., the next 24 hours, the next 48 hours, etc.). If a detector device or smart home device 107 detects a signal using one of the future keys, the device locator server may notify one or more electronic devices associated with the user.

[0128] Another optional user interface element 1907 can place the wireless accessory in an explicit lost mode. When explicitly placed in lost mode, the wireless accessory 101 will not be able to pair with other devices until the accessory 101 is unlocked by the user or owner who placed the device in lost mode. When a request is sent to place a wireless accessory in lost mode, the requesting user can be required to enter authentication information to ensure that the requesting user is authorized to request to initiate lost mode on the lost accessory. The authentication information can include a username or password associated with an account of the user, such as a cloud service account associated with the user, the electronic device, and the wireless accessory. The authentication information can also include biometric information, such as fingerprint or facial recognition data.

[0129] In one embodiment, the message and contact information provided by the requesting user can be displayed on the user device to prompt the person who finds the lost wireless accessory how to contact the requesting user. In one embodiment, the message and contact information can be displayed when another user attempts to pair another electronic device with the lost accessory.

[0130] like Fig. 20 As shown, the device locator UI 204 can present a fifth graphical user interface in the electronic device 2000, which enables the designation of a known location 2006 shown on the map with 2004 to become a trusted location with the selection of the selectable element 2003. In one embodiment, the fifth user interface can be responsive to the selection of Fig.18 1803. Device locator UI 204 can present user interface element 2005 representing and / or describing the wireless accessory in question.

[0131] like Fig.21 As shown, the device locator UI 204 can use the signal strength measurement to present a sixth graphical user interface with a proximity view in the electronic device 2100. The proximity view 2124 for finding "Tommy's AirPods" has indicators 2122, 2126, 2130, 2132, 2134, and 2128 at various locations along a track 2138 within the user interface 204. Each indicator can be a user interface element that indicates proximity to the target wireless accessory device 101 through the size, color, shape, color gradient, shading, pattern, and / or any other technique of the visual indicator within the user interface. As the user moves through the location environment, the indicators can be displayed along the track 2138. In the proximity view 2104, for example, the indicator 2106 is closest to the target wireless accessory device 101 along the track 2138 that the user has taken to find the target wireless accessory device 101, as indicated by a darker color and / or larger size than the other indicators. In some embodiments, the user interface 204 proximity view can present ranging information using ranging measurements and present an arrow 2128 indicating the direction of the target wireless accessory device 101 and the distance 2136 from the target wireless accessory device 101. In other embodiments, the track can be shown in a grid, such as a hexagonal grid, where the visual indicator consists of areas of the grid designated by color, gradient, shading, and / or any other markings along the track drawn on the grid to represent proximity values ​​for signal strength observed by the mobile device 102, smart home device 107, and / or a combination thereof in the corresponding area.

[0132] In one embodiment, signal strength measurements from signals received at mobile device 102 can be used to represent proximity to a target device within a user interface to indicate when mobile device 102 is close to the target device. In some embodiments, signal strength information for smart home devices 107 along the track can be used to present a proximity indicator. Fig.21The illustrated "proximity view" 2104 can present proximity information using visualization techniques to present proximity information about the target wireless accessory device 101. In one embodiment, the proximity view 2104 has visual indicators (such as user interface elements) positioned along the track 2118 presented within the user interface 204 to represent the path the user has taken in their search. In some embodiments, the visual indicator can be a user interface element displayed by gradient, color, color gradient, size, shape, and / or any other visualization technique that is used to represent the signal strength value and the corresponding defined proximity category (e.g., far, near, close, etc.) to the target device. The proximity view 2104 for finding "Tommy's AirPods" has indicators 2102, 2106, 2110, 2112, and 2114 at various locations along the track 2118 within the user interface 204. Each indicator in the proximity view 2104 can be a user interface element within the user interface 204 that represents proximity to the target wireless accessory device by size and color gradient. In the proximity view 2104, for example, the indicator 2106 is closest to the target wireless accessory device along the trajectory 2118 that the user has taken to find the target wireless accessory device, as indicated by a darker color and / or larger size compared to the other indicators (e.g., 2102, 2110, 2112, and 2114). An embodiment can use visual inertial odometry (VIO) measurements to determine the trajectory 2118 that the user has taken in his search within the user interface 204. VIO provides the ability to track the movement of the mobile device 102 in an arbitrary initial coordinate system. VIO techniques include analyzing a sequence of images collected by a mobile device to estimate camera motion over the sequence of images.

[0133] In some embodiments, the mobile device 102 may be moved within a threshold range of the target accessory device 101, thereby allowing the ranging process to use communications between the mobile device and the target device to determine the distance to the target device and the direction to the target device. Fig. 22 As shown, the “range view” 2120 of the user interface 204 may provide a distance measurement 2116 that may be selectively displayed in addition to the direction 2108 to the target device. In some embodiments, the proximity view 2104 for finding the target device may be used when range data for the range view 2120 is not available because the target device is not within a threshold range of the mobile device 102, the target device 101 transmitter is not in the field of view of the receiver at the mobile device, and / or the mobile device does not have a substantially unobstructed view to the target device 101. The target device 101 may be in the field of view of the mobile device 102 when the receiver at the mobile device 102 has a view of the target device transmitter.

[0134] In some embodiments, ranging using ultra-wideband (UWB) radio technology can provide relatively accurate location or distance data to a target device, but is a relatively short-range radio frequency (RF) technology wireless communication compared to Bluetooth technology. In some embodiments, it is desirable for the mobile device 102 UWB receiver to have a line of sight to the target device transmitter or a nearly unobstructed view of the target device to obtain optimal ranging location data. Proximity information in the form of signal strength information may be relatively less accurate than UWB, but may cover a wider area providing a longer range, and may be obtained from advertisements before a radio connection is established. In some embodiments, two-way communication cannot be established through a connection between the mobile device and the target device, but advertisements received at the mobile device may provide signal strength information before a connection is established to help guide the user to the target device. A combination of technologies may help a user locate a target wireless accessory device.

[0135] In various embodiments, description is made with reference to the accompanying drawings. However, certain embodiments may be practiced without one or more of these specific details or in combination with other known methods and structures. In the following description, many specific details such as specific configurations, sizes and processes are shown to provide a thorough understanding of the embodiments. In other cases, well-known semiconductor processes and manufacturing techniques are not described in particular detail to avoid unnecessarily obscuring the embodiments. "One embodiment" mentioned throughout the specification refers to the specific features, structures, constructions or characteristics described in conjunction with the embodiments being included in at least one embodiment. Therefore, the phrase "in one embodiment" that appears in multiple places throughout the specification does not necessarily refer to the same embodiment. In addition, specific features, structures, constructions or characteristics may be combined in one or more embodiments in any appropriate manner.

[0136] In the following discussion, a computing device including a touch-sensitive display is described. However, it should be understood that the computing device may include one or more other physical user interface devices. Various applications that may be executed on the device may use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the device may be adjusted and / or varied from one application to the next, and / or may be adjusted and / or varied within a corresponding application. In this way, the common physical architecture of the device (such as a touch-sensitive surface) may support a variety of applications with an intuitive and transparent user interface.

[0137] Some processes are described below with regard to some sequential operations. However, it should be understood that some operations in the described operations can be performed in different orders. In addition, some operations can also be performed in parallel rather than in order.

[0138] Fig.23is a block diagram illustrating an exemplary API architecture that can be used in some embodiments of the present invention. Fig.23 As shown, the API architecture 2300 includes an API implementation component 2310 (e.g., an operating system, library, device driver, API, application, software, or other module) that implements an API 2320. The API 2320 specifies one or more functions, methods, classes, objects, protocols, data structures, formats, and / or other features of the API implementation component that can be used by an API calling component 2330. The API 2320 may specify at least one calling convention that specifies how functions in the API implementation component receive parameters from the API calling component and how functions return results to the API calling component. The API calling component 2330 (e.g., an operating system, library, device driver, API, application, software, or other module) makes an API call through the API 2320 to access and use the features of the API implementation component 2310 specified by the API 2320. The API implementation component 2310 may return a value to the API calling component 2330 through the API 2320 in response to the API call.

[0139] It should be understood that the API implementation component 2310 may include additional functions, methods, classes, data structures, and / or other features that are not specified by the API 2320 and are not available to the API calling component 2330. It should be understood that the API calling component 2330 may be on the same system as the API implementation component 2310, or may be located remotely and access the API implementation component 2310 using the API 2320 over a network. Fig.23 A single API calling component 2330 is illustrated interacting with the API 2320 , but it should be understood that other API calling components written in a different language (or the same language) than the API calling component 2330 may use the API 2320 .

[0140] The API implementation component 2310, API 2320, and API calling component 2330 may be stored in a machine-readable medium, which includes any mechanism for storing information in a form readable by a machine (e.g., a computer or other data processing system). For example, a machine-readable medium includes a disk, an optical disk, a random access memory, a read-only memory, a flash memory device, etc.

[0141] Fig.2424 is a block diagram of a device architecture 2400 for a mobile or embedded device according to an embodiment. The device architecture 2400 includes a memory interface 2402, a processing system 2404 including one or more data processors, an image processor and / or a graphics processing unit, and a peripheral device interface 2406. The various components may be coupled via one or more communication buses or signal lines. The various components may be separate logic components or devices or may be integrated in one or more integrated circuits, such as a system-on-chip integrated circuit.

[0142] The memory interface 2402 can be coupled to the memory 2450, which may include high-speed random access memory such as static random access memory (SRAM) or dynamic random access memory (DRAM) and / or non-volatile memory such as but not limited to flash memory (e.g., NAND flash, NOR flash, etc.).

[0143] Sensors, devices, and subsystems may be coupled to the peripheral device interface 2406 to facilitate a variety of functionality. For example, a motion sensor 2410, a light sensor 2412, and a proximity sensor 2414 may be coupled to the peripheral device interface 2406 to facilitate mobile device functionality. There may also be one or more biometric sensors 2415, such as a fingerprint scanner for fingerprint recognition or an image sensor for facial recognition. Other sensors 2416 may also be connected to the peripheral device interface 2406, such as a positioning system (e.g., a GPS receiver), a temperature sensor, or other sensing device to facilitate related functionality. A camera subsystem 2420 and an optical sensor 2422 (e.g., a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) optical sensor) may be used to facilitate camera functions, such as recording photos and video clips.

[0144] Communication functions may be facilitated by one or more wireless communication subsystems 2424, which may include radio frequency receivers and transmitters and / or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the wireless communication subsystem 2424 may depend on the communication network over which the mobile device is intended to operate. For example, a mobile device including the illustrated device architecture 2400 may include a wireless communication subsystem 2424 designed to operate over a GSM network, a CDMA network, an LTE network, a Wi-Fi network, a Bluetooth network, or any other wireless network. Specifically, the wireless communication subsystem 2424 may provide a communication mechanism by which a media playback application can retrieve resources from a remote media server or retrieve scheduled events from a remote calendar or event server.

[0145] The audio subsystem 2426 may be coupled to a speaker 2428 and a microphone 2430 to facilitate voice-enabled functions such as voice recognition, voice replication, digital recording, and telephony. In the smart media devices described herein, the audio subsystem 2426 may be a high-quality audio system including support for virtual surround sound.

[0146] The I / O subsystem 2440 may include a touch screen controller 2442 and / or other input controllers 2445. For computing devices that include a display device, the touch screen controller 2442 may be coupled to a touch-sensitive display system 2446 (e.g., a touch screen). The touch-sensitive display system 2446 and the touch screen controller 2442 may detect contact and movement or pressure using, for example, any of a variety of touch and pressure sensing technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display system 2446. The display output of the touch-sensitive display system 2446 may be generated by the display controller 2443. In one embodiment, the display controller 2443 may provide frame data to the touch-sensitive display system 2446 at a variable frame rate.

[0147] In one embodiment, a sensor controller 2444 is included to monitor, control and / or process data received from one or more of the motion sensor 2410, light sensor 2412, proximity sensor 2414, or other sensor 2416. The sensor controller 2444 may include logic to interpret the sensor data to determine the occurrence of one of a plurality of motion events or activities by analyzing the sensor data from the sensors.

[0148] In one embodiment, the I / O subsystem 2440 includes other input controllers 2445 that may be coupled to other input / control devices 2448 such as one or more buttons, rocker switches, thumb wheels, infrared ports, USB ports, and / or pointer devices such as a stylus, or control devices such as up / down buttons for volume control of the speaker 2428 and / or microphone 2430.

[0149] In one embodiment, memory 2450 coupled to memory interface 2402 may store instructions for an operating system 2452, including portable operating system interface (POSIX) compatible and non-compatible operating systems or embedded operating systems. Operating system 2452 may include instructions for handling basic system services and for performing hardware-related tasks. In some specific implementations, operating system 2452 may be a kernel.

[0150] The memory 2450 may also store communication instructions 2454 to facilitate communication with one or more additional devices, one or more computers and / or one or more servers, such as retrieving web resources from a remote web server. The memory 2450 may also include user interface instructions 2456, including graphical user interface instructions to facilitate graphical user interface processing.

[0151] In addition, the memory 2450 may store sensor processing instructions 2458 that facilitate sensor-related processes and functions; phone instructions 2460 that facilitate phone-related processes and functions; instant messaging instructions 2462 that facilitate electronic instant messaging-related processes and functions; web browser instructions 2464 that facilitate web browsing-related processes and functions; media processing instructions 2466 that facilitate media processing-related processes and functions; location service instructions that facilitate location-based functionality, including GPS and / or navigation instructions 2468 and Wi-Fi-based location instructions; camera instructions 2470 that facilitate camera-related processes and functions; and / or other software instructions 2472 that facilitate other processes and functions (e.g., security processes and functions and system-related processes and functions). The memory 2450 may also store other software instructions, such as web video instructions that facilitate web video-related processes and functions; and / or web shopping instructions that facilitate web shopping-related processes and functions. In some specific implementations, the media processing instructions 2466 are divided into audio processing instructions and video processing instructions to facilitate processes and functions related to audio processing and processes and functions related to video processing, respectively. A mobile device identifier, such as an International Mobile Equipment Identity (IMEI) 2474 or similar hardware identifier may also be stored in memory 2450 .

[0152] Each of the instructions and applications identified above may correspond to an instruction set for performing one or more of the functions described above. These instructions need not be implemented as separate software programs, processes, or modules. Memory 2450 may include additional instructions or fewer instructions. In addition, various functions may be performed in hardware and / or software, including in one or more signal processing and / or application specific integrated circuits.

[0153] Fig.252500 is a block diagram of a computing system 2500 according to an embodiment. The computing system 2500 illustrated is intended to represent a series of computing systems (wired or wireless), including, for example, a desktop computer system, a laptop computer system, a tablet computer system, a mobile phone, a personal digital assistant (PDA) including a cellular PDA, a set-top box, an entertainment system or other consumer electronic device, an intelligent electrical appliance, or one or more specific implementations of an intelligent media playback device. Alternative computing systems may include more, less and / or different components. The computing system 2500 can be used to provide a computing device and / or a server device that a computing device may be connected to.

[0154] The computing system 2500 includes a bus 2535 or other communication device for transmitting information, and a processor 2510 coupled to the bus 2535 that can process information. Although the computing system 2500 is illustrated as having a single processor, the computing system 2500 may include multiple processors and / or coprocessors. The computing system 2500 may also include a memory 2520 in the form of a random access memory (RAM) or other dynamic storage device coupled to the bus 2535. The memory 2520 may store information and instructions that can be executed by the processor 2510. The memory 2520 may also be a main memory for storing temporary variables or other intermediate information during the execution of instructions by the processor 2510.

[0155] The computing system 2500 may also include a read-only memory (ROM) 2530 and / or another data storage device 2540 coupled to the bus 2535 that may store information and instructions for the processor 2510. The data storage device 2540 may be or include a variety of storage devices, such as a flash memory device, a magnetic disk, or an optical disk, and may be coupled to the computing system 2500 via the bus 2535 or via a remote peripheral interface.

[0156] The computing system 2500 may also be coupled to a display device 2550 via bus 2535 to display information to a user. The computing system 2500 may also include an alphanumeric input device 2560, which includes alphanumeric and other keys, which may be coupled to bus 2535 to communicate information and command options to the processor 2510. Another type of user input device includes a cursor control 2570 device, such as a touch pad, mouse, trackball, or cursor direction keys, for communicating direction information and command selections to the processor 2510 and controlling cursor movement on the display device 2550. The computing system 2500 may also receive user input from a communicatively coupled remote device via one or more network interfaces 2580.

[0157] The computing system 2500 may also include one or more network interfaces 2580 to provide access to a network such as a local area network. The network interface 2580 may include, for example, a wireless network interface having an antenna 2585, which may represent one or more antennas. The computing system 2500 may include multiple wireless network interfaces, such as Wi-Fi, A combination of near field communication (NFC) and / or a cellular phone interface. The network interface 2580 may also include, for example, a wired network interface to communicate with a remote device via a network cable 2587, which may be, for example, an Ethernet cable, a coaxial cable, a fiber optic cable, a serial cable, or a parallel cable.

[0158] In one embodiment, the network interface 2580 can provide access to a local area network, for example by complying with the IEEE 802.11 wireless standard, and / or the wireless network interface can provide access to a personal area network, for example by complying with the Bluetooth standard. Other wireless network interfaces and / or protocols may also be supported. In addition to or in lieu of communicating via a wireless LAN standard, the network interface 2580 can provide wireless communications using, for example, a time division multiple access (TDMA) protocol, a global system for mobile communications (GSM) protocol, a code division multiple access (CDMA) protocol, a long term evolution (LTE) protocol, and / or any other type of wireless communication protocol.

[0159] The computing system 2500 may also include one or more energy sources 2505 and one or more energy measurement systems 2545. The energy source 2505 may include an AC / DC adapter coupled to an external power source, one or more batteries, one or more charge storage devices, a USB charger, or other energy source. The energy measurement system includes at least one voltage or current measurement device that can measure the energy consumed by the computing system 2500 during a predetermined time period. In addition, one or more energy measurement systems may be included to measure, for example, the energy consumed by a display device, a cooling subsystem, a Wi-Fi subsystem, or other commonly used or high energy consumption subsystems.

[0160] Although the embodiments have been described in language specific to structural features and / or methodological acts, it should be understood that the appended claims are not necessarily limited to the specific features or acts described. Rather, the specific features and acts disclosed should be understood as illustrative embodiments of the claims.

Claims

1. An electronic device, comprising: Communications radio equipment; a memory storing instructions; and one or more processors configured to execute the instructions, wherein the instructions cause the one or more processors to: receiving a request for location information for a target accessory device associated with an owner electronic device, wherein the request includes a data packet having at least one public key; receiving a beacon signal from a wireless device and determining at least one proximity value to the wireless device based on the beacon signal; performing a comparison between a key for the wireless device from the received beacon signal and the at least one public key; as well as Location information is selectively transmitted to the owner electronic device based on the comparison result, wherein the location information includes the at least one proximity value and information about a defined space within a location environment.

2. The electronic device according to claim 1, wherein the one or more processors are configured to execute the instructions, wherein the instructions further cause the one or more processors to: determining that the received beacon signal does not include the at least one public key; and Location information indicating that the electronic device is not within a wireless communication range is sent to the target electronic device.

3. The electronic device according to claim 1, wherein the one or more processors are configured to execute the instructions, wherein the instructions further cause the one or more processors to: An indication is received from the owner electronic device that the at least one proximity value for the electronic device is a highest value compared to at least one other proximity value from a group of electronic devices.

4. The electronic device according to claim 3, wherein the one or more processors are configured to execute the instructions, wherein the instructions further cause the one or more processors to: A radio connection is established with the target accessory device.

5. The electronic device according to claim 3, further comprising: A wireless controller, wherein the wireless controller comprises a distance measuring sensor; the one or more processors, the one or more processors being configured to execute the instructions, the instructions further causing the one or more processors to: determining, via the ranging sensor of the wireless controller, a range relative to and a direction to the target accessory device during a two-way ranging operation; determining a target position estimate for the target accessory device relative to the electronic device based on the range relative to the target accessory device and the direction to the target wireless device; as well as At least one of the target position estimate for the target accessory device, a defined spatial identifier for the electronic device, or an indicator of the direction to the target accessory device is sent to the owner electronic device.

6. The electronic device according to claim 3, wherein the one or more processors are configured to execute the instructions, wherein the instructions further cause the one or more processors to: A request is received from the target accessory device, the request comprising at least one of a request to play audio, a request to send a target location of the target accessory device to the owner electronic device, or a request to send location information for the electronic device to the owner electronic device.

7. The electronic device according to claim 3, wherein the one or more processors are configured to execute the instructions, wherein the instructions further cause the one or more processors to: A request is sent to the target accessory device to increase a beaconing rate.

8. The electronic device of claim 5, wherein the one or more wireless ranging operations include ranging operations performed via an ultra wideband radio.

9. The electronic device according to claim 5, wherein the one or more processors are configured to execute the instructions, wherein the instructions further cause the one or more processors to: Presents signal strength proximity view and ranging view, including: At least one indicator of a proximity value from the electronic device to the target wireless device and an indicator for a direction to the target wireless device are presented along the trajectory in a user interface.

10. The electronic device of claim 1, wherein the electronic device is a voice-activated virtual assistant.

11. The electronic device of claim 1, wherein the proximity value is a signal strength measurement.

12. The electronic device of claim 1, wherein the one or more processors are configured to execute the instructions, wherein the instructions further cause the one or more processors to: The presence of an occupant in proximity to the defined space for the electronic device is detected, and playback of content requested by the owner electronic device is adjusted.

13. A method comprising: transmitting a beacon signal having at least one public key from a wireless device; as well as A two-way ranging operation is performed with a smart home device, wherein the smart home device is selected by an owner electronic device to communicate with the wireless device.

14. The method according to claim 13, further comprising: The audio content is sent to the smart home device for playback.

15. The method according to claim 13, further comprising: Stopping a two-way ranging operation with the smart home device; as well as A two-way ranging operation is performed on the owner electronic device.

16. A method comprising: sending a request to one or more smart home devices for location information regarding a target accessory device associated with an owner electronic device, wherein the request includes a data packet having at least one public key; receiving one or more proximity values ​​from the one or more smart home devices; as well as Position information from at least one smart home device of the one or more smart home devices is presented via a user interface, the at least one smart home device being selected based on a comparison between the proximity values.

17. The method according to claim 16, further comprising: selectively presenting at least one of a ranging view, a signal strength proximity view, or a ranging and signal strength proximity view within the user interface, wherein presenting the ranging view comprises: determining, via a ranging sensor of the wireless controller, a range relative to and a direction to a target accessory device during a two-way ranging operation; determining a target position estimate for the target accessory device relative to the owner electronic device based on the range relative to the target accessory device and the direction to the target wireless device; and Presenting the ranging view includes displaying at least one of the target position estimate for the target accessory device and an indicator for the direction to the target accessory device.

18. An electronic device, comprising: Communications radio equipment; speaker; a memory storing instructions; and one or more processors configured to execute the instructions, wherein the instructions cause the one or more processors to: receiving a request for location information for a target accessory device associated with an owner electronic device, wherein the request includes a data packet having at least one public key; receiving a beacon signal from a wireless device and determining at least one signal strength value based on the beacon signal; determining that a received beacon signal for the wireless device includes the at least one public key; estimating at least one proximity value to the target accessory device based on the at least one signal strength value; transmitting the at least one proximity value to the owner electronic device; receiving a request from the owner electronic device to play content for the target accessory device, wherein the owner device selects the electronic device to play content based on the proximity value; as well as Playback of the content is adjusted based on an occupancy value for a defined space, the electronic device being designated to be located proximate to the defined space.

19. The electronic device of claim 18, wherein adjusting playback comprises: A volume of the content is reduced based on the occupancy value.

20. The electronic device of claim 18, wherein adjusting playback comprises: receiving wireless communications from the owner electronic device and detecting an increase in an occupancy value for the defined space; as well as A volume of the playback for the content is increased.

21. The electronic device of claim 18, wherein adjusting playback comprises: detecting an increase in the occupancy value for the defined space based on at least one of the received audio or sensor data; as well as A volume for the playback of the content is reduced.

22. The electronic device of claim 18, wherein the instructions cause the one or more processors to: receiving an indication from another smart home device that the owner electronic device is proximate to the defined space for the electronic device; and A volume of the playback for the content is increased.

Images