Device Location Detection

Abstract

To locate devices using device locator services.SOLUTION: One or more inertial displacement measurement values may be received using an inertial sensor and received camera sensor data, a trajectory based on the one or more inertial displacement measurement values may be determined, a beacon signal from a target wireless device may be received, and at least one signal strength value may be determined from the beacon signal, at least one proximity value to the target wireless device may be estimated based on the at least one signal strength value corresponding to at least one position along the trajectory, and an indicator of the at least one proximity value to the target wireless device along the trajectory may be presented to a user interface.SELECTED DRAWING: Figure 1A

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Application No. 63 / 197,228, entitled "Device Location Finding," filed June 4, 2021, and U.S. Patent Application No. 17 / 543,421, entitled "Device Location Finding," filed December 6, 2021, each of which is incorporated herein by reference. [Technical Field]

[0002] The embodiments described herein relate to locating a device using a device locator service. [Background technology]

[0003] Previous device locator services do not provide meaningful assistance in locating devices when connectivity between the devices is not established. In particular, users often travel the same route as attempting to locate a device using previous approaches and may need more information to help guide the user to the missing device. Therefore, there is a need to provide an improved locator service. Summary of the Invention

[0004] Embodiments include electronic devices, non-transitory machine-readable media, and methods. In some embodiments, the electronic device includes a communication radio, an inertial sensor, a camera sensor, a display device, a memory for storing instructions, and one or more processors for executing the instructions. Embodiments may include instructions to cause the one or more processors to receive one or more inertial displacement measurements using the inertial sensor and received camera sensor data, determine a trajectory based on the one or more inertial displacement measurements, receive a beacon signal from a target wireless device, determine at least one signal strength value from the beacon signal, estimate at least one proximity value to the target wireless device based on the at least one signal strength value corresponding to the at least one position along the trajectory, and present, on a user interface, an indicator of the at least one proximity value to the target wireless device along the trajectory.

[0005] In one or more embodiments, the one or more processors execute instructions that further cause the one or more processors to determine a category from a plurality of categories for the signal strength value and present an indicator of the proximity value according to the determined category.

[0006] In one or more embodiments, the one or more processors execute instructions that further cause the one or more processors to present an indicator of the proximity value on a two-dimensional heat map along the trajectory in the user interface.

[0007] In one or more embodiments, the one or more processors execute instructions that further cause the one or more processors to present an indicator of the proximity value on the hexagonal grid.

[0008] In one or more embodiments, a wireless controller including a ranging sensor and one or more processors executing instructions, the instructions further causing the one or more processors to selectively present at least one of a ranging view, a signal strength proximity view, or a ranging and signal strength proximity view within a user interface, where presenting the ranging view includes determining a range and direction to a target wireless device via the ranging sensor of the wireless controller during a two-way ranging operation and determining a target position estimate of the target wireless device relative to the electronic device based on the range and direction to the target wireless device, and presenting the ranging view includes displaying at least one of a target position estimate for the target wireless device and an indicator of the direction to the target wireless device. In one or more embodiments, selecting to present the ranging view is performed in response to establishing a wireless connection with the target wireless device. In one or more embodiments, the one or more processors execute instructions, the instructions further causing the one or more processors to request the target wireless device to increase its beacon rate. In one or more embodiments, the one or more wireless ranging operations include ranging operations performed via an ultra-wideband radio. In one or more embodiments, presenting the signal strength proximity view and the ranging view includes presenting in the user interface at least one indicator of a proximity value to the target wireless device and an indicator of a direction to the target wireless device along the trajectory. [Brief explanation of the drawings]

[0009] [Figure 1A] 1 illustrates a device locator user interface according to embodiments described herein. [Figure 1B] 1 illustrates a device locator user interface according to embodiments described herein.

[0010] [Figure 1C]1 is a block diagram of a network operating environment 100 for a mobile device, according to one embodiment.

[0011] [Figure 2] 1 illustrates a system for locating a wireless accessory, according to one embodiment.

[0012] [Figure 3] 1 illustrates a system for pairing and locating wireless accessories according to embodiments described herein.

[0013] [Figure 4] FIG. 1 is a flow diagram illustrating a method for use with the device locator system described herein.

[0014] [Figure 5] FIG. 10 is a sequence diagram illustrating a method for presenting a device locator user interface for use with the device locator system described herein.

[0015] [Figure 6] 1 illustrates a system that enables range and attitude estimates to generate a target position estimate for a target device, according to one embodiment.

[0016] [Figure 7] 1 illustrates a system in which proximity enhancement can be enabled for a device locator application.

[0017] [Figure 8] 1 is a flowchart illustrating a method for a device locator user interface in one embodiment.

[0018] [Figure 9] SUMMARY OF THE INVENTION A method is presented that enables a proximity-enhanced user interface for device locator applications.

[0019] [Figure 10] 1 illustrates a method for determining the location of a wireless accessory via a device locator server.

[0020] [Figure 11] An additional method for determining the location of a wireless accessory via a device locator server is shown.

[0021] [Figure 12] 1 is a flow diagram illustrating a method for broadcasting a signal beacon in a wireless accessory, according to one embodiment.

[0022] [Figure 13] 10 illustrates method operations that may be performed by a finder device according to embodiments described herein. [Figure 14] 10 illustrates method operations that may be performed by a finder device according to embodiments described herein.

[0023] [Figure 15] 1 illustrates the collection of signal and ranging data by a finder device, according to one embodiment.

[0024] [Figure 16] 1 illustrates a device locator user interface in accordance with one or more embodiments. [Figure 17] 1 illustrates a device locator user interface in accordance with one or more embodiments. [Figure 18] 1 illustrates a device locator user interface in accordance with one or more embodiments. [Figure 19] 1 illustrates a device locator user interface in accordance with one or more embodiments. [Figure 20] 1 illustrates a device locator user interface in accordance with one or more embodiments. [Figure 21] 1 illustrates a device locator user interface in accordance with one or more embodiments.

[0025] [Figure 22] FIG. 1 is a block diagram illustrating an exemplary API architecture that may be used in some embodiments.

[0026] [Figure 23] FIG. 1 is a block diagram of a device architecture for a mobile or embedded device, according to one embodiment.

[0027] [Figure 24] FIG. 1 is a block diagram of a computing system, according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0028] The embodiments described herein generally provide techniques for presenting information on a mobile device to guide or assist a user actively involved in locating a target device. The mobile device can use various data sources and / or signals to present information to assist in locating the target device, including signals received from the target device itself. The accessibility of the types of signals received from the target device may change as the mobile device is moved around within the target device's location environment. Various user interface views may be presented within the mobile device's user interface to represent information when data sources / signals are accessible to the mobile device.

[0029] In one embodiment, signal strength measurements from signals received at a mobile device can be used to represent proximity to a target device in a user interface to indicate when the mobile device is in proximity to the target device. FIGS. 1A and 1B show a device locator user interface 204 on a mobile device 2100, according to embodiments described herein. A “proximity view” 2104, such as that shown in FIG. 1A, can present proximity information using visualization techniques to present proximity information related to a target wireless accessory device. In one embodiment, the proximity view 2104 has visual indicators, such as user interface elements positioned along a trajectory 2118 presented in the user interface 204, representing the path the user has taken in their search. In some embodiments, the visual indicators can be user interface elements displayed using gradients, colors, color gradients, sizes, shapes, and / or any other visualization techniques to represent signal strength values ​​and corresponding defined proximity categories (e.g., far, near, up close, etc.) to the target device. Proximity view 2104 for finding "Tommy's AirPods" has indicators 2102, 2106, 2110, 2112, and 2114 at various positions along a trajectory 2118 within user interface 204. Each indicator within proximity view 2104 may be a user interface element that represents proximity to a target wireless accessory device by its size and color gradient within user interface 204. In proximity view 2104, for example, indicator 2106 is closest to the target wireless accessory device along the trajectory 2118 taken by the user to find the target wireless accessory device, as represented by a darker color and / or larger size compared to the other indicators (e.g., 2102, 2110, 2112, and 2114). Embodiments may use visual inertial odometry (VIO) measurements to determine the trajectory 2118 taken by the user in their search within user interface 204. VIO provides the ability to track the movement of a mobile device in any initial coordinate system.The VIO technique involves the analysis of a sequence of images collected with a mobile device to estimate camera motion over the sequence of images.

[0030] In some embodiments, a mobile device can move within a threshold range of a target device, enabling a ranging process that uses communication between the mobile device and the target device to determine the range from and direction to the target device. As shown in FIG. 1B , a “ranging view” 2120 of the user interface 204 can provide distance measurements 2116 in addition to a direction 2108 to the target device, which can be selectively displayed. The proximity view 2104 for locating a target device can, in some embodiments, be used when ranging data for the ranging view 2120 is unavailable due to the target device not being within a threshold range of the mobile device, the target device transmitter not being within the field of view of a receiver at the mobile device, and / or the mobile device not having a substantially unobstructed view of the target device. A target device can be within the field of view of a mobile device when a receiver at the mobile device has a view of the target device transmitter.

[0031] In some embodiments, ranging using ultra-wideband (UWB) wireless technology is a radio frequency (RF) technology wireless communication that can provide relatively accurate location or distance data to a target device, but has a relatively short range compared to Bluetooth technology. In some embodiments, it may be desirable for the mobile device UWB receiver to have line of sight to the target device transmitter or a nearly unobstructed view of the target device to obtain optimal ranging location data. Proximity information in the form of signal strength information may be relatively less accurate compared to UWB, but may cover a wider area providing a longer range, and can be obtained from advertisements before a wireless connection is established. In some embodiments, two-way communication may not be established using a connection between the mobile device and the target device, but advertisements received at the mobile device may provide signal strength information to help guide the user to the target device before establishing a connection. A combination of technologies can assist a user in locating a target wireless accessory device.

[0032] Various embodiments are described with reference to figures. However, certain embodiments may be practiced without one or more of these specific details and in combination with other known methods and configurations. In the following description, numerous specific details are set forth, such as specific configurations, dimensions, and processes, in order to provide a thorough understanding of the embodiments. In other instances, well-known semiconductor processing and manufacturing techniques are not described in particular detail so as not to unnecessarily obscure the embodiments. Throughout this specification, references to "one embodiment" mean that a particular feature, structure, configuration, or characteristic described in connection with that embodiment is included in at least one embodiment. Thus, references to the phrase "in one embodiment" in various places throughout this specification are not necessarily referring to the same embodiment. Furthermore, particular features, structures, configurations, or characteristics may be combined in any suitable manner in one or more embodiments.

[0033] In the following description, a computing device including a touch-sensitive display is described. However, it should be understood that the computing device may include one or more other physical user interface devices. Various applications that may be executed on the device may use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device may be adjusted and / or changed from one application to the next and / or within each application. In this way, a common physical architecture of the device (such as a touch-sensitive surface) may support a variety of applications with intuitive and transparent user interfaces.

[0034] Some processes are described below with respect to some sequential operations. However, it should be understood that some of the described operations may be performed in a different order. Furthermore, some operations may be performed in parallel rather than sequentially.

[0035] FIG. 1C is a block diagram of a network operating environment 100 for mobile devices, according to one embodiment. The network operating environment 100 includes multiple mobile devices, such as an accessory device 101 and a mobile device 102. The accessory device 101 may be paired with the mobile device 102. In one embodiment, the mobile device 101 may be a pair of accessory devices that may be paired as a device group. Optionally, the device group with the accessory device 101 may be stored within the mobile device with a wired connection, such as a case that holds the accessory device 101. The case may also be an accessory device that may be paired with the mobile device 102 in some embodiments. By way of example, the accessory devices 101 may collectively be devices such as Apple AirPods® or EarPods®. In some embodiments, the accessory devices 101 may not be able to communicate over a wide area network. In other embodiments, the mobile devices 101 and 102 may each be any electronic device capable of communicating with a wireless network and wireless accessory devices. Some exemplary mobile devices include, but are not limited to, smartphones, tablet computers, notebook computers, wearable computers (e.g., smart watches or other wearable computing accessories), mobile media players, personal digital assistants, AirPods®, EarPods®, locator tags, headphones, head-mounted displays, health equipment, and other similar devices. Mobile device 101 and mobile device 102 may each optionally include a user interface, such as user interface 104 of mobile device 102. In other embodiments, mobile device 101, as an accessory device, may not have a user interface. Mobile devices 101 and 102 may be third-party devices that utilize an application programming interface to access a device locator service.The third-party devices may be provided by a different device manufacturer or may be part of a different ecosystem (e.g., operating system) than the mobile devices 101 and 102. The mobile devices 101 and 102 may communicate over one or more wired and / or wireless networks 110 to perform data communications. For example, the wireless network 112 (e.g., cellular network, Wi-Fi network) may communicate with a wide area network 114, such as the Internet, by using a gateway 116. Similarly, an access device 118, such as a mobile hotspot wireless access device, may provide communication access to the wide area network 114. The gateway 116 and the access device 118 may then communicate with the wide area network 114 via a combination of wired and / or wireless networks.

[0036] In some implementations, both voice and data communications may be established via wireless network 112 and / or access device 118. For example, mobile device 102 may make and receive telephone calls (e.g., using a VoIP protocol), send and receive email messages (e.g., using a POP3 protocol), and retrieve electronic documents and / or streams, such as web pages, photos, and videos, via wireless network 112, gateway 116, and wide area network 114 (e.g., using a TCP / IP or UDP protocol). In some implementations, mobile device 102 may make and receive telephone calls, send and receive email messages, and retrieve electronic documents via access device 118 and wide area network 114. In some implementations, mobile device 101 and / or mobile device 102 may be physically connected to access device 118 using one or more cables, for example, if access device 118 is a personal computer. In this configuration, mobile device 101 or mobile device 102 may be referred to as a “tethered” device. In one embodiment, mobile device 101 can communicate with mobile device 102 via a wireless peer-to-peer connection 120. Wireless peer-to-peer connection 120 can be used to synchronize data between the devices.

[0037] Mobile device 101 or mobile device 102 can communicate with one or more services, such as telephony service 130, messaging service 140, media service 150, storage service 160, and device locator service 170, over one or more wired and / or wireless networks 110. For example, telephony service 130 can enable telephone communications between mobile devices or between mobile devices and wired telephone devices. Telephony service 130 can route Voice over IP (VoIP) calls over wide area network 114 or can access a cellular voice network (e.g., wireless network 112). Messaging service 140 can provide, for example, email and / or other messaging services. Media service 150 can provide access to media files, such as, for example, song files, audiobooks, movie files, video clips, and other media data. Storage service 160 can provide network storage capabilities for mobile device 101 and mobile device 102 to store documents and media files. Device locator service 170 may enable a user to locate a lost or misplaced device that was, at least at some point, connected to one or more wired and / or wireless networks 110. Other services may also be provided, including a software update service for updating operating system or client software on the mobile device. In one embodiment, messaging service 140, media service 150, storage service 160, and device locator service 170 may each be associated with a cloud service provider, and the various services are facilitated via cloud service accounts associated with mobile devices 101 and 102.

[0038] In some embodiments, the accessory device 101 and the mobile device 102, and / or a device group may be registered with the certificate authority 106. In some embodiments, the certificate authority 106 is an entity that issues digital certificates, and the service may be implemented using a set of servers managed by the device manufacturer, a service provider, or a registration service. The certificate provided by the certificate authority 106 may attest to the validity of received verifiable information about the device, such as the device's specific manufacturer, serial number, a device group identifier or other identifier, an indicator that the device is part of a device group, and / or any other verifiable information. In some embodiments, a device manufacturer may establish a device group by grouping the serial numbers of accessory devices within the device group. In further embodiments, the certificate may be encrypted by the devices 101 and 102 before being sent to a third party and may be decrypted at an authentication service (e.g., a certificate authority or another authentication service) when the third party requests verification of information provided by the accessory device 101, the mobile device 102, and / or a device in the device group. In some embodiments, the secure token may be provided in a pairing request by the accessory device 101. Further examples of paired devices using location services can be found in U.S. Patent Application No. 17 / 219,595, entitled "Secure Pairing and Pairing Lock for Accessory Devices," filed March 21, 2021, the entire contents of which are incorporated herein by reference.

[0039] Mobile devices 101 and 102 may have locally accessible applications, services, and features on the device, including location services 180. In particular, mobile devices 101 and / or 102 may have a device locator application (e.g., a "Find my" application) 190 to utilize device locator service 170 and location services 180. Locally accessible data may be stored in known locations 182 and secure or trusted locations 184. In some cases, machine learning algorithms 186 may be used to identify known locations 182 and / or trusted locations 184. Cluster analysis is provided as an example of a machine learning algorithm that may be used, but those skilled in the art will recognize that other algorithms may be used to identify potential known or trusted locations. By way of example, cluster data analysis may be used to identify, classify, and provide semantic labels for locations, such as locations frequently visited by a user. Secure or trusted locations 184 may be explicitly designated or confirmed as such by the user of device 102A-B after data analysis. In other examples, known locations 182 or trusted locations 184 may be classified offline and provided by the device locator service 170 or a third party (eg, a database with map information).

[0040] On-device heuristics and / or machine learning models may be used to infer relationships between users and locations based on analysis of locally stored data on frequently visited locations, including locations frequently visited by the user, known locations, and / or any other locations. For example, frequently visited locations such as home, vehicle, workplace, any locations frequently visited by a user with a mobile device (e.g., accessory device 101 and mobile device 102), and / or any other locations designated by the user as trusted locations 184. Known locations 182 may be business locations, public spaces, parks, museums, and / or any other locations that the user may frequently visit. Boundary information for each stored location may be stored along with a classification type for the location and any semantic labels assigned to the location. The stored information may include a defined set of boundaries or a radius distance around a point location to enable creation of a geofence for the location. A geofence is a virtual boundary of a real-world geographic area. A global positioning system (GPS) can be used to create a virtual fence around a location and track the physical location of mobile devices 101 and 102 within the geofence boundary as well as their entry and exit into the bounded area.

[0041] The machine learning algorithm 186 may include on-device heuristics, machine learning algorithms, or a combination thereof, to analyze and assign a label related to the device's movement or motion, such that it is designated as being in an “in motion” or “stable” state at a particular location for a period of time. The analysis may be performed using various signals from data sources available to the mobile device 102, including, but not limited to, sensor data, positioning data, calendar data, transit card usage data, application data, historical data related to travel patterns / routines, and / or any other data accessible to the mobile device 102. In some embodiments, the mobile device 102 may be classified with a “stable” semantic label after remaining within the geographic boundaries defining a location (e.g., trusted location 184) for a defined period of time. In the simplest case, positioning data for the mobile device 102 may remain within the boundaries of a geofence for a particular location for a certain duration (e.g., five minutes). Sensor data, such as accelerometer data, may indicate that the mobile device 102 is stationary, supporting the inference that it is stable. Application data can support an inference that the mobile device 102 is in a steady state, such as the mobile device being located at a calendar appointment location. Application data indicating the type of application in use may also provide an inference that the device is in a steady state, such as using a media application. A user's historical data regarding on-the-go routines or patterns may be used to determine whether the mobile device 102 is in a steady state, such as a bedtime routine at home or a hotel location. A mobile device 102 may be classified as having an "on-the-go" label based on previous behavior, patterns, or routines for the user and analyzed on the mobile device 102. For example, a user may have a routine of going to work at the same time every day, and if data on the device supports that the pattern is repeating, an "on-the-go" state may be assigned.In the simplest case, the speed at which a mobile device is moving or entering or leaving a known geographic area (e.g., using a geofence) may allow for the inference that the mobile device 102 is in motion. If the mobile device 102 is detected accelerating in a known transit area (e.g., a road, highway, railroad line, etc.), the mobile device 102 may be given an "in motion" status. Similarly, if a transit application / card is used / in use, the mobile device 102 may be designated as "in motion."

[0042] 2 illustrates a system 200 for locating a wireless accessory 201, according to one embodiment. In one embodiment, the wireless accessory 201 is another embodiment of the accessory device 101 (and optionally the case) that may be paired as part of a device group. In other embodiments, the accessory devices 201 are separate accessory devices that are not part of a device group, each paired separately with the mobile device 102. Each accessory device 201 includes one or more wireless transceivers and can communicate with a companion device (e.g., the mobile device 102) either directly or indirectly (e.g., through another device or computer) over a wireless network or a peer-to-peer communication link. The accessory device 201 can provide a beacon signal to the case and any accessories within the case. The accessory device 201 is separate from the case and can be found independently and separately by providing a beacon signal. Some examples of wireless accessory devices 201 include, but are not limited to, wireless earbuds, EarPods, AirPods, input devices, charging devices, accessory cases, headphones, headsets, fitness equipment, health equipment, display devices, external hard drives, other wearable device (e.g., smart watches, fitness bands, optical head-mounted displays), adapters, speakers, and / or other devices. A paired group of accessories may be devices of the same type (e.g., speakers, AirPods, fitness weights, etc.) or devices of different types (e.g., smartphones and credit card readers, etc.). Wireless accessories 201 may also include other wireless devices, such as input devices, including, but not limited to, credit card reading devices, stylus devices, mice, keyboards, game controllers, or remote controls.In one embodiment, the wireless accessory 201 also includes a smartphone, tablet computer, laptop computer, smart speaker device, television, or television set-top box that is at least temporarily unable to access a wide area network such as the Internet (e.g., wide area network 114 of FIG. 1 ). The wireless accessory 201 may also be any other wireless device, including a beacon or locator tag that can be attached to other devices to enable tracking or locating the other devices. In one embodiment, the wireless accessory 201 may be from a device group of accessory devices that are paired with the mobile device 102 using a wireless technology standard such as, but not limited to, Bluetooth®. The wireless accessory 201 may also communicate with the mobile device 102 via wireless technology, including implementations of any wireless standards and protocols, such as Wi-Fi Direct, Zigbee®, or AirPlay. The companion device with which the wireless accessory 201 is paired is generally referred to as the mobile device 102, although companion devices are not limited to mobile devices. Companion devices may also include, in some embodiments, laptop or desktop devices, as well as some wearable accessories, such as, but not limited to, smartwatch devices or wearable displays.

[0043] In one embodiment, the wireless accessory 201 may periodically transmit a wireless beacon signal. The wireless accessory 201 may transmit the beacon signal using one of the various wireless technologies described herein (e.g., Bluetooth, Wi-Fi, etc.), and in one embodiment may also transmit the beacon using ultra-wideband (UWB) wireless technology. The beacon signal may be transmitted using a single wireless technology, one of multiple selectable wireless technologies, or multiple simultaneous wireless technologies. The beacon signal may transmit a beacon identifier that includes information to specifically identify an individual wireless accessory 201 and / or a device group. In one embodiment, the beacon identifier is a public encryption key associated with the device.

[0044] The beacon signal may also convey information about the wireless accessory 201, such as device status information and / or verifiable information. The device status information in the beacon signal may include, but is not limited to, a beacon type, a device classification, a battery level, any predefined device status, a device state, a lost status, an alarm status, a status away from owner, a status near owner, a status in proximity to one or more accessory devices in a device group status, a wired or wireless connection status, a status physically connected to one or more accessory devices in a device group status, a pairing status indicating whether the accessory device is paired or unpaired, a pairing pending status, a battery life status, a charging status, and / or any other status information. The lost status or “away from owner” status may indicate that the wireless accessory 201 has determined itself lost or has been placed in a lost state by the device owner. The alarm status may indicate that the wireless accessory 201 has been placed in a status that should trigger an alarm if the device moves from its current location. The near owner status may indicate that the wireless accessory 201 has detected the nearby presence of a mobile device 102 associated with the accessory owner.

[0045] In some embodiments, the verifiable information may include any information that may be needed to establish trust or authority that the pairing and / or discovery process may proceed with the device presenting the verifiable information. By way of example, the verifiable information may include information established by a device manufacturer, such as a serial number or set of serial numbers within a device group. In some embodiments, the verifiable information may include status or state information of the device. The verifiable information may include, but is not limited to, the device type, membership in a device group, the serial number, the device group, the serial numbers of other devices in the device group, state or status information, software version, and / or any other verifiable information. The verifiable information may be transmitted to a certificate authority 106 or other attestation service to verify received information presented by the device to another device. The verifiable information may be encrypted and / or transmitted with a token to enable further verification of the device.

[0046] In some embodiments, the beacon signals can be detected by a finder device 202 in local proximity to the wireless accessory 201 to use crowdsourcing to locate the lost wireless accessory 201. The finder device 202 can be a device similar to the mobile device 102 and can transmit and receive data over the wide area network 114 and can transmit and receive using similar wireless technologies (e.g., Bluetooth, etc.) as the wireless accessory 201. In particular, the finder device 202 can receive data using the wireless protocol over which the beacon signals are transmitted. The finder device 202 can determine its location using one or more location and / or positioning services, including, but not limited to, satellite positioning services 206 or terrestrial positioning systems using RF signals received from wireless base stations 205, such as Wi-Fi access points or cell tower transmitters of a cellular telephone network. In one embodiment, the finder device 202 periodically stores its location determined based on one or more location and / or positioning services. The stored location can be associated with a timestamp at which the location was determined. When the finder device 202 receives a beacon signal from the wireless accessory 201, the finder device 202 may transmit the location of the finder device 202 to the device locator server 203 over the wide area network 114. A timestamp of the determined location of the finder device 202 may be correlated with the timestamp at which the beacon signal was received to associate a geographic location with the received beacon signal.

[0047] If the wireless accessory 201 provides a public key in the beacon signal, the finder device 202 can encrypt the determined location data and transmit the encrypted location data to the device locator server 203 over the wide area network 114. In one embodiment, additional data can be encrypted and transmitted with the location data, or transmitted unencrypted to the device locator server 203. For example, a received signal strength indicator (RSSI) of the beacon signal can be transmitted with the location data. The RSSI data can then be used to determine the distance of the wireless accessory 201 from the finder device 202 and assist in triangulation on the owner device. If the RSSI data is transmitted unencrypted, in one embodiment, the server can use the RSSI information to reduce noise by discarding very weak signals in the presence of other stronger signals. In one embodiment, UWB ranging data can also be provided, where such data is available.

[0048] In one embodiment, when the finder device 202 receives a beacon signal from the wireless accessory 201, it can behave differently depending on the device status communicated by the wireless accessory 201. For a standard beacon signal, the finder device 202 can queue encrypted location data and transmit the location data to the device locator server 203 during a periodic transmission window. However, if the wireless accessory 201 indicates an alarm condition, the finder device 202 can immediately transmit the location data to the device locator server 203. Additionally, if the beacon signal of the wireless accessory 201 indicates that the accessory is near the accessory's owner, the finder device 202 may not transmit the location data to the device locator server 203. Alternatively, the finder device 202 may delay transmitting the encrypted location data.

[0049] When the owner of the wireless accessory 201 wishes to locate the location of the wireless accessory, the owner can access a device locator user interface 204 on the mobile device 102. The device locator user interface 204 may be associated with a device locator application used to locate electronic devices and accessories registered to a user's online account, such as a cloud service account or another type of online account. The device owner can use the device locator UI 204 to query the device locator server 203 for location data that may have been sent to the device locator server by the finder device 202 of the wireless accessory 201. In one embodiment, the mobile device 102 can send a public encryption key associated with the wireless accessory 201 to the device locator server 203. The device locator server 203 can then return any stored location data that corresponds to the public encryption key. The location data returned to the mobile device 102 can be encrypted data encrypted by the finder device 202 using the public encryption key. The mobile device 102 can decrypt the encrypted location data using the associated private key. The decoded location data may then be processed by the mobile device 102 to determine the most likely location of the wireless accessory 201. In various embodiments, the most likely location of the wireless accessory 201 may be determined by triangulation from multiple received locations and using other data, such as the beacon signal RSSI associated with each location and timestamps or UWB ranging data included within the location data.

[0050] 3 illustrates a system 300 for pairing and locating a wireless accessory according to embodiments described herein. In one embodiment, a mobile device 102 (e.g., an example of the device 101) of a user of a wireless accessory 201 can present an accessory pairing UI 302 that allows the user to pair the mobile device 102 with the wireless accessory 201. During the initial pairing (305) between the mobile device 102 and the wireless accessory 201, a public key exchange (310) can be performed between the mobile device and the wireless accessory 201. In one embodiment, during the public key exchange (310), the mobile device 102 and the accessory 201 exchange public keys of a public key pair generated by the device and the wireless accessory 201. In one embodiment, the public key exchange (310) is a one-way transfer in which the mobile device 102 sends the public key of a public / private key pair to the wireless accessory 201. Alternatively or additionally, the public key exchange (310) can be a Diffie-Hellman key exchange in which the device and the accessory establish a shared secret between the two parties. In one embodiment, the public key exchange (310) further establishes a shared secret using elliptic curve cryptography. For example, elliptic curve Diffie-Hellman (ECDH) may be used to enable the establishment of a public key pair and one or more shared secrets. In one embodiment, the one or more shared secrets include an anti-tracking secret from which the wireless accessory 201 can periodically derive additional public keys.

[0051] After the wireless accessory 201 is paired with the mobile device 102, the wireless accessory 201 can periodically broadcast a beacon signal 301 that includes device status information and a beacon identifier. In one embodiment, the beacon identifier is a public key derived from a shared secret established during a public key exchange (310). Additionally, the wireless accessory 201 can periodically perform a public key derivation (315) to generate a new public key and begin broadcasting the new public key as a beacon identifier. The public key is a K-byte key, and a new K-byte key is generated every M minutes. The values ​​K and M can vary between embodiments. In one embodiment, a K value of 28 bytes is used. In one embodiment, a K value of 27 bytes is used. The value K can be determined based at least in part on a beacon length associated with a wireless protocol used to transmit the beacon signal 301. In one embodiment, the beacon signal can transmit a variant of a beacon advertisement packet associated with a low energy wireless protocol, such as Bluetooth® Low Energy.

[0052] In one embodiment, the value M is 15 minutes, and a new K-byte key is generated every 15 minutes. The public key can be derived deterministically based on the timestamp and the anti-tracking secret generated during public key exchange 310. The public key derivation (315) process allows the wireless accessory 201 to use different keys over time, preventing long-term association of a particular key with a particular device. The key can be derived based on the anti-tracking secret known only to the mobile device 102 and the wireless accessory 201, allowing only the mobile device 102 and the mobile device to determine which public key is broadcast by the wireless accessory 201 at any given timestamp. The anti-tracking secret can be generated along with the ECDH public key and transferred to the wireless accessory 201. The anti-tracking secret can then be used to enable the wireless accessory 201 to generate a sequence of public keys P. In one embodiment, the sequence of public keys P=λ·P defines a group operation between a scalar or exponent value λ and a group element, such as an elliptic curve point P. A scalar or exponential value λ=KDF(AT,i), where KDF is a key derivation function, AT is an anti-tracking secret, and i is a counter or timestamp.

[0053] In one embodiment, backtracking resistance can be enabled to protect the anti-tracking secret if the wireless accessory 201 is compromised. When backtracking resistance is enabled, the anti-tracking secret is transferred to the wireless accessory 201 but is not retained by the wireless accessory. Instead, the accessory calculates the value λi+1 = H(λi||time), where λo = A T and H is a cryptographic hash function. The wireless accessory 201 then stores λi for a given period of time i. If the wireless accessory 201 is compromised, only the current and future values ​​of i, λi, are exposed, not the anti-tracking secret A T. In one embodiment, backtracking resistance is performed by periodically writing λi to the non-volatile memory of the wireless accessory 201.

[0054] In one embodiment, the wireless accessory 201 may transmit a beacon signal 301 every two seconds, although other beacon rates may be used, and the beacon rate may change under certain circumstances. For example, the wireless accessory 201 may decrease its beacon rate when in close proximity to its owner. The beacon rate may also change based on an event triggered by an accelerometer. For example, the wireless accessory 201 may increase its beacon rate when in an alarm state, which may be triggered by an accelerometer on the wireless accessory 201.

[0055] The wireless accessory 201 may enter the near-owner state if, after transmitting the beacon signal 301, the wireless accessory 201 receives a response from a mobile device 102 associated with the accessory's user indicating that the mobile device 102 is within range of the wireless accessory. Additionally, while the wireless accessory is in the near-owner state, the amount of data transmitted by the beacon signal 301 may be reduced. In one embodiment, the rate at which new public keys are generated may also be reduced while the wireless accessory is in the near-owner state.

[0056] The wireless accessory 201 can enter an alarm state upon receiving a message from the mobile device 102 indicating that the wireless accessory 201 should enter an alarm state. When in the alarm state, the wireless accessory can first enter an armed state in which the wireless accessory 201 can reduce or stop transmitting locator beacon signals, although other types of wireless signaling can continue. The wireless accessory 201 can remain in the armed state until the state is deactivated by the mobile device 102 or an alarm is triggered. In one embodiment, the alarm can be triggered when movement is detected, for example, via an accelerometer within the wireless accessory 201. In one embodiment, the alarm can also be triggered when the wireless accessory detects that it has moved out of range of the mobile device and is no longer in proximity to its owner. When an alarm is triggered, the rate of the beacon signal 301 can be increased to increase the speed at which the wireless accessory 201 can be located.

[0057] The beacon signal 301 transmitted by the wireless accessory 201 can be detected by a set of finder devices 303 (the finder devices can be finder devices 202) and / or mobile devices 102, which are other electronic devices that can receive the beacon signal transmitted by the wireless accessory and transmit location and other data associated with the beacon signal 301 to the device locator server 203 over the wide area network 114. In one embodiment, the set of finder devices 303 can include variations of mobile devices 102 or can be other types of electronic devices. For example, the set of finder devices 303 can perform an operation (320) to correlate the beacon signal 301 received from the wireless accessory 201 with a device location associated with the finder devices 303. As described with respect to FIG. 2 , device location can be determined via a satellite positioning service or a terrestrial positioning system using RF signals received from wireless base stations (e.g., Wi-Fi access points or cell tower transmitters). In one embodiment, the set of finder devices 303 may also include fixed devices such as smart speaker devices, televisions, or television set-top boxes that are capable of receiving the beacon signals 301 .

[0058] The set of finder devices 303 may encrypt the location data using the beacon identifier (e.g., public key) received in the beacon signal 301 and transmit the location data (325) to the device locator server 203. The data transmitted by the set of finder devices 303 is transmitted anonymously, and the identification information of the finder devices is not stored with the data transmitted by the finder devices.

[0059] The device locator server 203 can store the encrypted location data in a data store 304, which in one embodiment can be a distributed database having multiple nodes. A hash of the accessory's beacon identifier / public key can be transmitted along with the encrypted location data. The encrypted location data can be stored in a database node based on the hash of the beacon identifier. The encrypted location data can be indexed by the device locator server 203 using the hash of the beacon identifier. Transmitting the hash of the beacon identifier instead of the full beacon identifier prevents storage of the full beacon identifier in the server. Other information, either encrypted or unencrypted, can also be transmitted and stored along with the location data. The other information can include a timestamp of when the beacon signal 301 was received, RSSI information of the received beacon, and / or ranging information determined, for example, via UWB ranging.

[0060] When a user or owner of the wireless accessory 201 wishes to locate the accessory, the user or owner can access a device locator UI 204 on the mobile device 102. The device locator UI 204 may be associated with the locator application 190 or a feature of the mobile device 102. The device locator UI 204 may also have a web-based interface that can be accessed from the mobile device 102 or another type of electronic device, such as a laptop or desktop device. Once the mobile device 102 loads the device locator UI 204, the mobile device 102 can send a request (330) for location data to the device locator server 203. The request 330 can include a set of public keys or public key hashes that can serve as beacon identifiers for the beacon data. The mobile device 102 can generate a set of public keys based on secret information maintained by the mobile device 102 and the wireless accessory 201 and a timestamp at which the mobile device 102 wishes to receive location data. In one embodiment, the set of public keys is a sequence Pi of public keys generated based on a tracking prevention secret. The sequence of public keys Pi corresponds to a matching sequence of private keys dj. The mobile device 102 can generate a sequence of public keys and a corresponding sequence of public keys dj, where i is a counter or timestamp. In one embodiment, the mobile device 102 can generate and send in the request 330 public keys (or hashes of the 24-hour public keys) for the previous 24 hours. If no data is found for the 24-hour public keys, the mobile device 102 can send generated keys for an earlier period to revert to a predetermined location data retention limit.

[0061] In one embodiment, the encrypted location data is stored and indexed based on a hash of the public key instead of the public key to prevent location service data providers from storing data that can be used to tie the encrypted location data to a particular device and therefore to a particular user or user account. A finder device can transmit a hash of the public key broadcast in a beacon signal 301 associated with the observed location. The device owner can query the device locator server 203 using the hash of the public key determined for the query period.

[0062] In some embodiments, when a location query is performed via a web-based interface from an electronic device such as a laptop or desktop device, a key to enable decryption of the location data may be required to be sent to the electronic device. In one embodiment, a decryption key for the location data may be sent to a server providing the web-based interface to enable the server to decrypt the location data, at least while the location data is being viewed via the web-based interface. Before the location data is displayed via the web-based interface, a notice may be presented to inform the user that a location decryption key has been temporarily shared with the web-based interface server to enable the location data to be decrypted and presented. In one embodiment, sharing of the location decryption key may be performed via automatic and temporary delegation of location query rights by a proxy account associated with the web-based interface.

[0063] In one embodiment, the wireless accessory 201 can be placed in simple lost mode. In simple lost mode, a set of future public keys can be generated for the wireless accessory and sent to the device locator server 203. The device locator server 203 can then notify the mobile device 102 whether any location data corresponding to a key in the set of future public keys has been received. In one embodiment, a finder device transmitting the location of a wireless accessory in simple lost mode can instruct the device locator server 203 to relay a message to the wireless accessory 201 informing the wireless accessory that it is in simple lost mode. A similar mechanism can be used to relay a message to the wireless accessory 201 that puts the accessory into explicit lost mode. Explicit lost mode can be enabled by the user via the device locator UI 204. In explicit lost mode, the wireless accessory 201 cannot be paired with another device unless it is unlocked by the owner. Further examples of paired devices using location services can be found in U.S. Patent Application No. 16 / 543,227, entitled "A System and Method for Locating Wireless Accessories," filed August 16, 2019, which is incorporated herein by reference in its entirety.

[0064]

[0023] Figure 4 is a flow diagram illustrating a method for use with the device locator system described herein. Figure 4 illustrates a method 400 for pairing a mobile device with a wireless accessory. Aspects of method 400 are also illustrated in Figures 2 and 3, as discussed above. For example, the following operational description refers to a mobile device 102, a wireless accessory 201, and a device locator server 203.

[0065] As shown in FIG. 4 , method 400 includes an act of performing (402) initial pairing with a wireless accessory. The initial pairing can be Bluetooth® pairing or another type of pairing using other wireless technologies. During initial pairing, the mobile device and wireless accessory can exchange identifiers, passkeys, or other authentication information that enables wireless data exchange to occur between the mobile or other electronic device and the wireless accessory. In one embodiment, the initial pairing with the wireless accessory can include an exchange of authentication information associated with the wireless protocol over which pairing is being performed, allowing all data exchanged wirelessly to have at least a first layer of encryption.

[0066] The mobile device may then generate a public / private key pair and one or more additional shared secrets (404). The device may then send the public key and one or more additional shared secrets to the wireless accessory (406). Various key generation techniques may be used. In one embodiment, a variant of ECDH is used to generate the public key pair for encryption. In one embodiment, the one or more additional shared secrets may include an anti-tracking secret that allows the wireless accessory to derive a new public key based on an existing public key.

[0067] After generating the public / private key pair and one or more additional shared secrets, the mobile device may store the public / private key pair in a key store (408). In one embodiment, the key store is a cloud-based key store that can be synchronized with other devices associated with the same cloud service account or family of cloud service accounts with which the mobile device and wireless accessory are associated. The cloud-based key store allows the wireless accessory to be located by other synchronized devices. The mobile device may then register the wireless accessory with a device management server (410). Registering the wireless accessory with the device management server may form an association between the wireless accessory and the cloud service account with which the mobile device is associated. In some embodiments, the mobile device may register the wireless accessory and a device group 104. Information stored in a device group profile for the device group may also be synchronized among devices bound to the cloud service account (e.g., a user account). The device management server may be associated with other cloud-based servers used to facilitate cloud-based services accessible to mobile devices, such as the device locator server 203 of FIGS. 2 and 3 .

[0068] 5 is a sequence diagram 500 illustrating a method for presenting a device locator user interface for use with the device locator system described herein. The mobile device 102 may launch the device locator application 204 (506). The mobile device 102 may receive a beacon signal from the accessory device 201 (508) that includes an advertisement. The advertisement may be transmitted before launching the application (506) and / or after launching the device locator application 204. In some embodiments, a BLE advertisement may be transmitted before establishing a wireless connection to the mobile device 102. In one embodiment, the BLE advertisement may be received every two seconds. Signal strength measurements may be determined from the beacon signal, and a process may begin to locate at least one accessory device 201 (510) paired to the mobile device 102 (510) using a proximity view for the user interface 204. A proximity view for the user interface of the device locator application 204 may be presented with a visual indicator representing the proximity of the target accessory device 201. The visual indicator may correspond to a signal strength measurement (512) from an advertisement received at the mobile device 102. Optionally, in some embodiments, a wireless connection can be formed (514) between the target accessory device 201 and the mobile device 102, allowing the mobile device 102 to request that the target accessory device 201 play a sound. In some embodiments, a wireless connection may be formed between the target accessory device 201 and the mobile device 102, and the mobile device may request that a ranging process be performed. When the mobile device 102 is within a threshold range of the target accessory device (515), a ranging process can be performed to determine the distance from and direction to the target wireless accessory device 201. A ranging view of the user interface of the device locator application 204 may be presented (516).The ranging view may provide (518) a direction and calculated distance to a wireless target to aid in locating the accessory device 201. The ranging view may be presented selectively with the proximity view and / or independently within the user interface 204. A user interface of the device locator application 204 may be presented (520) querying the user as to whether the accessory device 201 has been found.

[0069] FIG. 6 illustrates a system 600 that enables range and attitude estimates to generate a target position estimate for a target device, according to one embodiment. In one embodiment, the system 600 includes a wireless controller 630, an IMU 612, and a set of frameworks executing on an application processor of the mobile device 102 as described herein. The set of frameworks may include an augmented reality (AR) framework 610 and a sensor fusion framework 611. The wireless controller 630 can send range measurements 601 to interpolator logic 604 within the sensor framework 611. The AR framework 610 can receive sensor measurements 603 from the IMU 612 and generate an attitude estimate 602 for the mobile device 102. The attitude estimate 602 can be sent to the interpolator logic 604 within the sensor framework 611. The interpolator logic 604 can process the range measurements 601 and attitude estimate 602 to generate measurements at an aligned attitude 605. The interpolator logic 604 can generate measurements at an aligned attitude 605 by interpolating and aligning the range measurements 601 and attitude estimates 602 based on the times at which the measurements and estimates were determined. The measurements at the aligned attitude 605 can be provided to the estimator logic 606. The estimator logic 606 can then generate a three-dimensional target position estimate 607 for use by the mobile device 102. This target position estimate can be used by the mobile device 102 to generate a view of the location environment.

[0070] The interpolator logic 604 and estimator logic 606 can include various algorithmic techniques to overcome anomalous data and sensor interference to improve the accuracy of the three-dimensional target position estimate 607. In one embodiment, the system 600 can fuse inertial data collected from the IMU 612 with images captured via one or more cameras to perform visual inertial odometry (VIO). VIO techniques involve analyzing a sequence of images to estimate camera motion over the sequence of images. Image-based camera motion estimation and inertial measurements captured over the same period can be used to estimate the motion of the mobile device through a coordinate system, giving the software of the system 600 the ability to estimate both the position of the mobile device 102 (or any viewfinder device) and a target device located within a three-dimensional coordinate space.

[0071] In one embodiment, the target location estimate may take the form of a nonlinear least-squares problem that can be solved using a nonlinear least-squares (NLLS) batch filter. For example, given r as the range measurement at time t and simultaneously p(t) as the pose estimate relative to the center of the VIO coordinate system, the target location x=[xyz]T that minimizes the cost function can be determined.

number

[0072] In one embodiment, once an initial target position for the wireless device is determined in three-dimensional coordinate space, the relative target position of the wireless device may be updated using AR data even in the absence of updated range measurements between the wireless device 102 and the accessory device 201. In such a scenario, the frequency of wireless ranging operations may be throttled to reduce power consumption by the target device 201 and the mobile device 102.

[0073] 7 illustrates a system 700 in which proximity enhancements may be enabled for a device locator application 204. One embodiment provides a system 700 that enhances the functionality of the device locator application 204 when in a location environment 708 in proximity to a located item or target device 201. The location environment 708 may include a mobile device 102 (e.g., a smartphone, wearable, tablet, etc.). The mobile device 102 may wirelessly communicate with a wireless device 101 that includes a radio 719. The mobile device 102 may also include a variation of the radio 719 within the wireless device 102. The wireless device 201 may also be a mobile device or may be a wireless accessory 201 as described herein.

[0074] The devices may detect each other by scanning wireless channels, send and receive beacons or beacon frames on the wireless channels, establish connections (e.g., by sending connection requests), and / or communicate wirelessly via wireless communication signals 705 by sending and receiving packets or frames (which may include requests and / or additional information, such as data, as payloads). The wireless communication signals 705 may be carrier signals that comply with wireless communication technologies such as, but not limited to, Wi-Fi or Bluetooth. In addition to wireless communication, the mobile device 102 and the wireless device 201 perform wireless ranging operations using wireless ranging signals 706. The wireless ranging signals may be, for example, ultra-wideband signals that may be used to determine the distance and / or angle between the wireless device 201 and the mobile device 102 using the techniques described herein. In one embodiment, the data provided by the wireless ranging signals 706 may be correlated with other metrics, such as the RSSI of the wireless communication signals 705. In one embodiment, the communications processors of the mobile device 102 and the wireless device 201 can fuse multiple types of ranging to provide a unified distance and / or angle estimate based on multiple types of wireless data.

[0075] The mobile device 102 may provide a device locator UI 204 that presents a map and / or view of the location environment 708. The map and / or view may present a virtual representation of the wireless device 201. The virtual representation may streamline the process of locating the wireless device 201 by a user of the mobile device 102. For example, using a directional indicator to point to the detected location of the wireless device 201 may simplify the process of finding the approximate location of the wireless device 201. The directional indicator may be paired with a map of the location environment 708, and the map may include a virtual representation of the wireless device 201. When the mobile device 102 is in proximity to the wireless device 201, such that the beacon signal includes a wireless communication signal 705 and / or a ranging signal 706, a proximity view and / or a ranging view of the location environment 708 may be presented, allowing the user to identify the wireless device 201 when it is obscured from view.

[0076] In one scenario, the wireless device 201 may be hidden by an item. The item may be, for example, a jacket or another article of clothing, and the wireless device 201 may be a smartphone or tablet device in a pocket of the item. The item and the wireless device 201 may each be in a container such as a backpack, bin, luggage, or another item that reduces the effectiveness of audio-based location techniques that rely on the wireless device 201 to play sounds that the user can hear.

[0077] In another scenario, the wireless device 201 may be a beacon peripheral or a locator tag attached to an item. To find the item, a user may use a device locator UI 204 on the mobile device 102 to locate the wireless device 201, allowing the user to find the item to which the wireless device 201 is attached. The proximity enhancements described herein may be used to enhance the device locator UI 204, allowing the user to quickly determine the location of an item via the wireless device 201 in scenarios where the item may be obscured by a container or another physical item.

[0078] In one embodiment, the mobile device 102 may establish a secure wireless communication connection (e.g., a Bluetooth connection) with the wireless device 201 (e.g., via wireless communication signal 705) and instruct the wireless device 201 to begin a wireless ranging process using a wireless ranging signal 706, which may be, for example, a UWB signal. The wireless ranging signal 706 allows the mobile device 102 to receive range and angle measurements to the wireless device 201. In some embodiments, the mobile device 102 may request that the target wireless device 201 increase the rate at which it sends advertisements (e.g., from every 2 seconds to every 30 ms) to help locate the wireless device 201.

[0079] In one embodiment, prior to establishing a secure wireless communication connection and / or with an established wireless connection, signal strength measurements may be determined from the received advertisement to help direct the user to the wireless device 201.

[0080] In embodiments described herein, wireless ranging may be performed using any standard or proprietary ranging technique, or any combination of standard and / or proprietary ranging techniques. Wireless ranging operations may be performed to determine the distance between devices (e.g., between an initiator and a responder), the direction between the devices, or both. For example, the time of flight / time of arrival (ToF / ToA) may be determined for one or more messages between the devices, and these values ​​may be used to establish a distance measurement. The one or more messages may have any format and may be transmitted using any wireless protocol. In some embodiments, the ToF / ToA may be determined using a two-way exchange of two or more messages. In some embodiments, one or more messages used to perform ranging may be secured, for example, by encrypting or otherwise protecting at least a portion of their contents. Additionally, in some embodiments, the direction of a source of one or more wireless signals may be determined using techniques such as angle of arrival (AoA). For example, AoA estimation may be performed using multiple receiving elements (e.g., elements of an antenna array) to measure different times of arrival (TDOA) and / or different phases (PDOA) of the signal. Additionally or alternatively, in some embodiments, directivity may be determined by measuring Doppler shift to establish frequency difference of arrival (FDOA). Wireless ranging techniques may be applied individually or in combination to perform a single ranging operation. Furthermore, wireless ranging techniques may be applied individually or in combination to perform ongoing ranging operations, such as continuous or intermittent ranging, and a history of measurements may be captured and used in performing range and / or direction-based operations.

[0081] Incoming measurements can be processed to reduce measurement noise and stabilize the estimated location of the lost item. In one embodiment, the mobile device 102 can use sensor fusion techniques that rely on multiple streams of measurements to arrive at a best estimate of the device location. For example, incoming range and / or angle measurements can be fused with position and orientation measurements of the mobile device 102. Position and orientation measurements can be collected using visual inertial odometry (VIO), which uses an IMU to determine the device's location and orientation and camera data to determine computer vision. The VIO data allows software on the mobile device 102 to recognize the device's movement and orientation. Combining the VIO data with range and / or angle measurements determined via the wireless ranging signals 706 enables an estimate of the wireless device's most likely position and allows the user to be directed to the wireless device 201. Furthermore, combining the VIO data with signal strength measurements via the wireless communication signals 705 can assist in directing the user to the wireless device 201 using a proximity view and / or in combination with a ranging view. VIO data can be particularly useful in scenarios where one or more of the range or angle measurements may be inaccurate due to multipath signal propagation effects.

[0082] Algorithms for determining the position of a lost item can be accessed via the device locator UI 204. When a user attempts to locate an item, the user can select the item from a list of registered devices and select a "find" button presented via the device locator UI 204. In various embodiments, multiple user interfaces can be presented. Some interfaces resemble a two-dimensional compass-like view with a two-dimensional arrow that guides the user to the target item or device to be found. The proximity view and / or a combination of the proximity view and ranging view of the device locator UI 204 can provide arrows to guide the user in addition to presenting the trajectory the user has taken in their search with proximity indicators presented along the path taken.

[0083] FIG. 8 is a flowchart 800 illustrating a method for the device locator user interface 204 in one embodiment. One or more inertial displacement measurements may be determined (802) using received inertial sensor data and received camera sensor data. The one or more inertial displacement measurements are determined from an origin in any 2D or 3D coordinate system using VIO. In one embodiment, the origin (e.g., ([0,0,0] x, y, and z axes)) is established when a user selects to begin a process to locate a target wireless device 201. The target wireless device may be the wireless accessory device 201 and / or the wireless accessory device 101 shown in FIGS. 1-3. A trajectory may be determined based on the collected one or more inertial displacement measurements (804). The displacement measurements from the origin and the displacement measurements collected as the user moves within the location environment may be used to plot a trajectory within the user interface 204 to represent the path taken by the user within the location environment. The trajectory may be presented in a user interface to alert the user to the path they have already taken, so they can use the information to try other paths to find the device or to stay on course.

[0084] A beacon signal may be received from the target wireless device 201 (806). The mobile device 102 may receive the beacon signal from the target wireless device 201 along with one or more advertisements before establishing a wireless connection and / or while the target wireless device 201 is establishing a wireless connection to the mobile device 102. Although two-way communication may not be established between the mobile device 102 and the target wireless device 201, the advertisements received by the mobile device 102 may be used to determine proximity to the target wireless device 201. A signal strength value may be determined from the received beacon signal (806). The signal strength value may be an average of signal strength measurements determined from advertisements in beacon signals received while the user is located within an area on the trajectory. In some embodiments, an average of signal strength values ​​taken over an area (e.g., as opposed to one taken over time) can reduce the effects of fading. Fading occurs when beacon signals received at a wireless receiver on the mobile device 102 simultaneously cause a drop in signal strength values. An alternative embodiment may use an average of signal strength values ​​taken over time. A proximity value to the target wireless device 201 may be estimated based on signal strength values ​​corresponding to positions along the trajectory (808) within the area from which the signal strength values ​​were received.

[0085] An indicator of the proximity value to the target wireless device may be displayed along the trajectory (810). Defined categories of signal strength values ​​may be determined for each set of one or more signal strength values ​​averaged for an area along the trajectory, and an indicator of the proximity value may be presented according to the defined categories. For example, a range of values ​​may be defined for each of a set of buckets to designate up close, near, and far categories, and / or any other category that provides information to the user about the proximity to the wireless accessory device 201. The visual indicator may be any type of user interface element, including, but not limited to, a marker, a particular color or gradient, a shape, or any other visual indication displayed on the user interface.

[0086] A ranging view, a signal strength proximity view, and / or a combination of ranging and signal strength proximity views (as shown in FIG. 21 ) may be selectively presented within user interface 812. By way of example, the ranging view may be shown only when a wireless connection is established. Distance View

[0087] The performance and accuracy of distance, bearing, and range calculations can be improved using various algorithmic techniques. In one scenario, as a device follows a trajectory through space toward a target, the device may encounter multipath signal propagation effects when ranging to the target. Multipath propagation is a phenomenon in which a radio signal reaches a receiving antenna by more than one path. Multipath propagation can occur when the line of sight to the target is blocked by obstacles in the environment, when the antenna pattern significantly attenuates the line-of-sight signal, or when the user blocks the signal. As a mobile device tracks a trajectory through space toward a target, the measured range to the target can evolve over time along with the distance traveled over the trajectory. During this evolution, anomalies and interference can occur.

[0088] Sometimes, the measurement range determined from sensor data may appear to jump more than the distance traveled in space. A sudden increase in the measurement range that differs from the distance traveled over a period of time can be used to determine when multipath conditions have changed. If the change in range is positive and greater than the distance traveled, there may be an increase in measurement bias due to multipath. Similarly, if the change in range is negative and greater in magnitude than the distance traveled, multipath components were likely present previously and are now reducing in effect. When a decrease in multipath propagation is detected, the uncertainty of previous measurements can be increased to account for potential multipath interference. In one embodiment, previous measurements considered multipath measurements can be removed from the measurement history by setting the weights associated with those measurements to zero. When an increase in multipath interference is detected, measurements received after the detected increase can similarly be weighted less or the measurements can be rejected.

[0089] In one embodiment, range measurements may be deweighted due to the passage of time. As range measurements are collected over the device's orbit, the target position solution becomes over-determined. Range measurements may be deweighted as the measurements become older and / or as VIO errors accumulate. For moving targets, analysis of range measurements as a function of time may be particularly important because a time series of range measurements may be used to determine angle measurements for the device without relying on angle measurements associated with any one ranging operation. Thus, weights associated with older measurements may be decayed based on a function of time.

[0090] In one embodiment, range measurements may be deweighted due to the measured and / or estimated distance to the target. As measurements indicate more range, the likelihood of multipath interference increases, making the measurements less useful. Once a solution is formed, an estimated distance to each target for which a measurement was made can be calculated. Based on that distance, the measurements can be reweighted. Furthermore, the measurements themselves are ranges, which can be used to estimate the uncertainty of the measurements.

[0091] In one embodiment, angle-of-arrival measurements can be used to further refine the target position when a range-only solution indicates that the target is within the device's field of view. A device position solution can be formed using only range measurements (e.g., without angle measurements). Given a position solution, the probability can be calculated that each angle measurement was made from the mobile device when the target was within the mobile device's field of view. If the probability meets a threshold, the angle measurements can be used to form a new position solution. The use of angle measurements can resolve ambiguities that may exist in the range solution due to the device trajectory being primarily in a single plane (e.g., the x-y plane), particularly along the z-axis.

[0092] After the measurements are used to form a position solution, the measurement error for each contributing measurement can be calculated. In an over-determined solution, the measurement error will be non-zero, giving some indication of the quality of the position determination. If large errors are observed, the confidence in the position determination may be reduced. A threshold on the confidence may be used to determine whether the position should be shown to the user. If all measurements are observed to disagree by only a small amount, the error can be used to increase the uncertainty of the position.

[0093] FIG. 9 illustrates a method 900 for enabling a proximity-enhanced user interface for a device locator application. Method 900 includes operations that can fuse range measurements determined by a wireless controller with pose estimates determined based on sensor data from an IMU to generate pose alignment measurements and three-dimensional or two-dimensional target position estimates. In some embodiments, the three-dimensional target position estimates can be used to enable an AR view of a location environment including a target device or item to be located. Range measurements determined by the wireless controller and estimates determined based on sensor data from an IMU can also be used to determine a two-dimensional target position estimate. Method 900 can be performed by a mobile device (e.g., mobile device 102) described herein to locate a target wireless accessory device (e.g., wireless device 101) or a target wireless accessory device (e.g., wireless accessory device 201) or an item associated with the wireless accessory device 201, such as an item tagged with a beacon or locator tag.

[0094] In one embodiment, the method 900 includes an act of optionally receiving 901, at the mobile device 102, a geographic location of the target wireless accessory device. The geographic location of the target wireless accessory device 201 can be a location determined by the mobile device 102 that has access to a user account for the mobile device 102, or a location received from a device locator server 203.

[0095] The mobile device 102 may then present (902) an indicator of the target wireless accessory device 201 on an interface of a device locator application 204 executed by the mobile device 102. The indicator may be a map indicator that shows the location of the target wireless accessory device on a map of the nearby environment. For example, a map with a map indicator may be presented as shown in FIGS. 17-20. Distance and range measurements to the target wireless accessory device 201 may be presented on the map along with the map indicator that shows the location of the target wireless accessory device. The user of the mobile device 102 may then navigate toward the location of the target wireless accessory device 201.

[0096] If the mobile device 102 is within a threshold range of the target wireless accessory device 201, the wireless accessory device 201 may determine 903 the range and / or direction to the target wireless accessory device 201 using one or more wireless ranging operations. The one or more ranging operations may include continuous or periodic ultra-wideband ranging operations. The ultra-wideband ranging operations may be performed in conjunction with other range determination techniques, such as RSSI-based range determination. In one embodiment, the ultra-wideband ranging operations are secure ranging operations in which the exchanged ranging packets are encrypted. In one embodiment, both range and direction determinations may be made based on one or more ranging operations. In one embodiment, a range-only determination may be made, and the direction may be determined based on an analysis of multiple range measurements. Where both range and angle measurements may be determined based on instantaneous wireless ranging signal analysis, a history of range measurements may be used to improve the accuracy of the determined angle measurements.

[0097] The mobile device 102 may then determine 904 the attitude of the mobile device 102 via sensor data received from the inertial measurement unit within the mobile device 102. The inertial measurement unit data may be processed by the system to determine the attitude of the mobile device 102. The attitude data may be used to further improve direction determination made for the target wireless accessory device 201 based on the wireless ranging sensors.

[0098] The mobile device 102 can then determine 905 a position estimate of the target wireless accessory device 201 based on the range, orientation, and attitude. In one embodiment, the particular range, orientation, and attitude determined for the target wireless accessory device 201 and the mobile device 102 are determined based on a fusion of sensor data from the wireless ranging system and the mobile device 102's system. The position estimate can be a position relative to the mobile device 102 or an absolute position in 3D and / or 2D coordinate space. The mobile device 102 can then generate and display 906 a device locator UI 204 that includes the position estimate of the wireless accessory device 201. The device locator UI can be, for example, a ranging view with distance 2116 and direction information 2108 to the wireless accessory device 201, as shown in FIG. 21 . A proximity view, also shown in FIG. 21 , provides a trajectory in 2D coordinate space with a visual indicator for proximity to the target wireless accessory device 201 as determined by signal strength measurements.

[0099] FIG. 10 illustrates a method 1000 for determining the location of a wireless accessory via a device locator server. FIG. 11 illustrates an additional method 1100 for determining the location of a wireless accessory via a device locator server. As shown in FIG. 10 , method 1000 includes an operation in which an electronic device launches a device locator UI (1001). In response to launching the device locator UI, the electronic device, which may be a mobile device described herein or another electronic device associated with the same cloud service account as the mobile electronic device, may perform an operation to generate a set of public keys that were included in beacon signals broadcast by the wireless accessory during a first period of time (1002). The first period of time may be, for example, the past 24 hours. The electronic device, knowing the frequency at which the wireless accessory generates new public keys, may use the shared secret key generated by the wireless accessory to generate a set of public keys corresponding to the keys generated by the wireless accessory over the first period of time. The electronic device may then transmit the set of public keys in a request for the device locator server to transmit location data corresponding to the set of public keys (1003). In one embodiment, the location data transmitted by the server in response to the request is encrypted using a public key transmitted as the beacon identifier of the wireless accessory. The electronic device can decrypt the encrypted location data received by the server using a private key generated during initial pairing with the wireless accessory (1004). The electronic device can then process the location data to determine the most probable location of the wireless accessory (1005). In one embodiment, the location data can include data about accessory devices 201 in a device group.

[0100] Processing the location data can include a variety of different operations. In one embodiment, the location data includes latitude and longitude information along with a timestamp at which the position was determined. The electronic device can triangulate based on the timestamp to remove noise or outlier locations. In one embodiment, the location data specifies the location of the finder device that detected the beacon. The location data can further include UWB ranging information and / or RSSI information for the beacon detected by the finder device. The electronic device can analyze the UWB ranging information and / or RSSI information in conjunction with the device location to reveal a more precise location of the wireless accessory. Data transmitted by the finder device and that can be used for location processing is shown in FIG. 12 and described below.

[0101] As shown in FIG. 11, method 11 includes operations that can be performed when the device locator server does not have location data to provide to the electronic device in response to a request. In the case of a device group, the electronic device (e.g., mobile device 102) can provide location data for devices in the device group. The electronic device can generate a first set of public keys that were included in a beacon signal broadcast by the wireless accessory during a first period of time (1101). The first period can be, for example, 24 hours, although other initial search periods can also be used. The electronic device can perform a subsequent operation of requesting the device locator server to send location data corresponding to the first set of public keys (1102). If data is returned by the server (1103, “Yes”), the electronic device can decrypt the location data received from the server using a private key corresponding to the set of public keys (block 1109).

[0102] If no data is returned by the server (1103, “No”), the electronic device may generate a second set of public keys that were included in beacon signals broadcast by the wireless accessory during a second period of time (1104). The second period of time may be 24 hours, 48 ​​hours, or another time before the first period of time. The electronic device may then request the device locator server to send data corresponding to the second set of public keys (1105). If data is returned by the server in response to the request (1106, “Yes”), method 1100 may proceed to block 1109, where the electronic device decrypts the received data. If data is not returned by the server (1106, “No”) or the server sends a response indicating that the data is not available, method 1100 includes the electronic device extending the search time by requesting successively older periods until a maximum period is reached (1107).

[0103] FIG. 12 is a flow diagram illustrating a method 1200 of broadcasting a signal beacon in a wireless accessory, according to one embodiment. Aspects of method 1200 are also shown in FIGS. 2 and 3. Method 1200 includes the wireless accessory deriving a public key (block 1202). The public key may be derived based on a shared secret and a timestamp determined based on a clock or timing device of the wireless accessory. Optionally, a determination is made as to whether the wireless accessory is part of a device group (1204). If the wireless accessory is part of the device group, status and / or verifiable information about other accessory devices 201 in the device group is provided in the beacon signal (1206). The wireless accessory may indicate status and / or verifiable information, such as whether any other wireless accessories in the device group are in proximity or connected (physically or wirelessly), and / or any other information about other wireless accessories in the device group 105. In one embodiment, a set of bits included in the beacon signal can represent each accessory in the device group, and setting a Boolean value (e.g., true (1) or false (0)) can indicate whether the individual accessory is in proximity to and / or connected to the accessory device transmitting the beacon signal. Alternatively, if the wireless accessory is not part of a device group, no information about the device group is provided (1204). The wireless accessory can then transmit a beacon signal at a first frequency, where the beacon signal includes the public key (1208). The first frequency can vary, and in one embodiment is one beacon every two seconds.

[0104] After transmitting the beacon signal, the wireless accessory may listen for a response from the owner device (1210). If the wireless signal receives a response from the owner device (1210, "yes"), the wireless accessory may enter a near-owner state (1212) and begin transmitting a beacon signal at a second, lower frequency (1216). If the wireless accessory does not receive a response from the owner device (1210, "no"), the wireless accessory may continue transmitting beacons at the first frequency (1214).

[0105] Method 1200 additionally includes, for the wireless device, rotating the public key every M minutes during beacon transmission, where the value of M may vary across embodiments and / or based on device state. Based on a timer expiration, a counter, or another mechanism, the wireless accessory may determine whether the accessory has entered a new key period (1218). While the wireless accessory has not entered a new key period (1218, “No”), the accessory may continue beacon transmission using the current public key (1222). Upon detecting that the wireless accessory has entered a new key period (1218, “Yes”), the accessory may derive a new public key using the current timestamp (1220). In one embodiment, the new public key may be derived using the existing public key, the timestamp, and the anti-tracking secret.

[0106] 13-14 illustrate operations of a method 1300 that may be performed by a finder device according to embodiments described herein. Aspects of the method 1300 are also illustrated in FIGS.

[0107] 13, method 1300 includes a finder device performing periodic beacon scans using a wireless baseband processor while an application processor of the finder device is in a low power mode (1301). While beacon scans may also be performed when the application processor is active, beacon scans may be performed by the wireless processor and wireless receiver as a low power operation while the finder device is idle, inactive, or otherwise in a low power state. The finder device may store a timestamp and a beacon identifier in a beacon scan buffer for any beacon data received by the finder device (1302). The beacon identifier in one embodiment is a public key generated by the wireless device based on the timestamp and a shared secret generated at the owner's mobile device.

[0108] Method 1300 further includes the finder device performing periodic Wi-Fi scans using the radio processor while the application processor is in a low power mode (1303). The Wi-Fi scans may also be performed when the application processor is active, but the Wi-Fi scans may be performed by the radio processor and radio receiver as a low power operation while the finder device is idle, inactive, or otherwise in a low power state. The device may then store the Wi-Fi service set identifier (SSID) and scan timestamp in a Wi-Fi scan buffer on the finder device (1304).

[0109] In one embodiment, the Wi-Fi scan buffer is a rolling buffer that stores the most recently detected SSIDs while overwriting older detected SSIDs. In one embodiment, the beacon scan buffer may be a fixed-size buffer with space for a predetermined number of entries. The finder device may wake up the application processor when the beacon scan buffer is full (1305) and correlate those beacon scans with the most recently detected SSIDs in the Wi-Fi scan buffer. If the beacon indicates that the beacon signal was received from a device group (1306), a set of device locations corresponding to the received beacon based on the Wi-Fi scan buffer data may be performed for the beacon signal from the device group (1310). For example, if a beacon signal is received from a first accessory device in the device group 105 and includes information about a set of nearby devices physically or wirelessly connected to the first accessory device, the last known location of the first accessory device may be attributed / stored to the first accessory device and each of the nearby devices in the device group 105. Alternatively, the correlation may enable the finder device to determine a set of device locations corresponding to the received beacons based on the Wi-Fi scan buffer data (1308).

[0110] Method 1300 continues with FIG. 14 and includes the finder device correlating 1407 the device location from the Wi-Fi scan buffer data with other location data, if other location data is available, to generate a refined device location. If a refined device location is generated, the finder device may optionally combine 1408 beacon data with the refined device location. The finder device may also add 1409 signal strength (RSSI) and / or ranging data to the location data. Signal strength and ranging data (e.g., UWB ranging data) may be collected as beacon signals are received by the finder device. The finder device may then encrypt 1410 the location data using one or more public keys received in the beacon data. The signal and ranging data may be encrypted along with the location data or may be transmitted unencrypted along with the encrypted location data. The finder device may enqueue 1411 the encrypted location data for transmission to a device locator server. The device locator server may be one of multiple cloud services servers, to which communication is typically performed in a batch and throttled manner. Batches of encrypted data may be collected and placed in a transmission queue until a transmission interval arrives, during which the finder device may transmit the data to the cloud services server (1412).

[0111] FIG. 15 illustrates the collection of signal and ranging data by a finder device, according to one embodiment. In one embodiment, the finder device 202 can collect signal strength information (e.g., RSSI 1504A-1504N) of beacon signals 301 received from the wireless accessory 201 across multiple locations 1502A-1502N. The finder device 202 can also represent multiple finder devices, such as the set of finder devices 303 of FIG. 3, with each finder device detecting a beacon signal at a different location. Each finder device 202 can transmit a different location and signal strength, and the location and signal strength data received from the multiple finder devices is aggregated by a device locator server. In one embodiment, the finder device and wireless device each include a UWB radio, UWB ranging 1506 can be performed when the finder device and wireless device are within range of a UWB transmission. The UWB ranging and signal strength data can be transmitted to the device locator server along with the finder device's location data.

[0112] The owner device can retrieve RSSI and / or UWB information from the device locator server along with location data, which in one embodiment is provided in the form of latitude and longitude information along with a timestamp at which the location was determined. The owner device can then use the location data, timestamp, and signal information to triangulate the most likely location of the wireless accessory 201.

[0113] 16-21 illustrate a device locator UI 204, according to one embodiment. FIG. 16 illustrates a first graphical user interface of the device locator UI 204, according to one embodiment, showing notifications for a user's various wireless accessories. The device locator UI 204 can cause a separation notification 1602 to be presented on a home screen 1601 of an electronic device 1600. FIG. 17 illustrates a second graphical user interface of the device locator UI 204, according to one embodiment, which allows a user to request that a left-behind accessory device be seen on a map, add a trusted location, or stop notifications for the item.

[0114] FIG. 18 shows a third graphical user interface of the device locator UI 204, according to one embodiment, that allows for locating accessory devices 201, including devices in a device group, in a map. FIG. 19 shows a fourth graphical user interface of the device locator UI 204, according to one embodiment, that allows for notification when a wireless accessory is set to lost mode or is found. The device locator UI 204 can be displayed on an electronic device, which may be a mobile device 102 or any other type of electronic device described herein. FIG. 20 shows a fifth graphical user interface of the device locator UI 204, according to one embodiment, that allows for a wireless accessory to add a trusted location. FIG. 21 shows a sixth graphical user interface of the device locator UI 203, according to one embodiment, that allows for a proximity view using signal strength measurements.

[0115] As shown in FIG. 17 , the device locator UI 204 can present a unified graphical interface on the electronic device 1700 through which multiple different types of devices and accessories can be located, including wireless devices with network or cellular access and wireless accessories without native network access. The device locator UI 204 can include a map 1704 with a marker 1705 indicating the current or last known location of the wireless accessory device 201. The marker 1705 can be an icon, image, graphic, or any other user interface element that identifies the accessory and conveys the accessory's location. Selectable elements 1707 in the device locator UI 204 can present an option to request not to be notified about certain items. Selectable elements 1706 in the device locator UI 204 can present a description or name of the wireless device or accessory and, as shown in FIG. 18 , can indicate the estimated distance between the wireless device or accessory and the current location of the electronic device 1800.

[0116] 18, the device locator UI 204 may present a user interface that allows the wireless accessory to view items 1803 and 1805 and their distance from the electronic device 1800. In one embodiment, in response to selection of the selectable element 1706 shown in FIG. 17, a third user interface may be displayed. The third user interface may present user interface elements 1802 that represent and / or describe the wireless accessory in question, as well as a map 1801 and a marker 1802 that indicate the current or last known location of the wireless accessory.

[0117] As shown in FIG. 19 , the device locator UI 204 may present a fourth graphical user interface that allows the wireless accessory to be placed in lost mode. In one embodiment, if the wireless accessory cannot be located via the device locator UI 204, the map 1901 does not display a marker indicating the accessory's location. The device locator UI 204 may present user interface elements 1904 that represent and / or describe the wireless accessory in question and a set of selectable user interface elements. One selectable user interface element 1906 may present an option to notify the user when the accessory is found. When found notification is enabled, in one embodiment, the wireless accessory may be placed in simplified lost mode. An electronic device associated with the device locator UI 204 may generate a set of public keys that the wireless accessory will broadcast with its beacon signal during a future period (e.g., the next 24 hours, the next 48 hours, etc.). If a signal is detected by a finder device using one of the future keys, the device locator server may notify one or more electronic devices associated with the user.

[0118] Another selectable user interface element 1907 can place the wireless accessory in explicit lost mode. When explicitly placed in lost mode, the wireless accessory cannot be paired with other devices until the accessory is unlocked by the user or owner who places the device in lost mode. When sending a request to place the wireless accessory in lost mode, the requesting user may be required to enter authentication information to ensure that the requesting user is authorized to request that lost mode be initiated on the lost accessory. The authentication information may include a username or password associated with the user, the electronic device, and the user's account, such as a cloud service account with which the wireless accessory is associated. The authentication information may also include biometric information, such as fingerprint or facial recognition data.

[0119] In one embodiment, a message and contact information provided by the requesting user may be displayed on the user device to alert the finder of the lost wireless accessory on how to contact the requesting user. In one embodiment, the message and contact information may be displayed when another user attempts to pair another electronic device with the lost accessory.

[0120] As shown in Figure 20, the device locator UI 204 may present a fifth graphical user interface within the electronic device 2000, allowing for the designation of a known location 2006 shown on the map with 2004 to become a trusted location upon selection of a selectable element 2003. In one embodiment, the fifth user interface may be displayed in response to selection of the selectable element 1703 shown in Figure 17. The device locator UI 204 may present a user interface element 2005 that represents and / or describes the wireless accessory in question.

[0121] As shown in FIG. 21 , the device locator UI 204 can use signal strength measurements to present a sixth graphical user interface with a proximity view within the electronic device 2100. The proximity view 2124 for locating “Tommy’s AirPods” has indicators 2122, 2126, 2130, 2132, 2134, and 2128 at various positions along a trajectory 2138 within the user interface 204. Each indicator may be a user interface element that represents proximity to the target wireless accessory device 201 by size, color, shape, color gradient, shading, pattern, and / or any other technique for visual indicators within a user interface. The indicators may be displayed along the trajectory 2138 as the user moves through the location environment. In the proximity view 2104, for example, indicator 2106 is closest to the target wireless accessory device 201 along the trajectory 2138 taken by the user to locate the target wireless accessory device 201, as represented by a darker color and / or larger size compared to the other indicators. In some embodiments, the user interface 204 proximity view may present ranging information using ranging measurements and present an arrow 2128 indicating the direction of the target wireless accessory device 201 and the distance 2136 to the target wireless accessory device 201. In other embodiments, the ranging information may be presented in a separate user interface without the proximity indicators 2122, 2126, 2130, 2132, and 2134, as shown in FIG. 1B . In other embodiments, the trajectory may be shown in a grid, such as a hexagonal grid, with a visual indicator consisting of areas of the grid designated by color, gradient, shading, and / or any other marking along the trajectory plotted on the grid to represent the proximity value of the signal strength observed within each area.

[0122] FIG. 22 is a block diagram illustrating an exemplary API architecture that may be used in some embodiments of the present invention. As shown in FIG. 22, the API architecture 2200 includes an API implementation component 2210 (e.g., an operating system, library, device driver, API, application program, software, or other module) that implements an API 2220. The API 2220 specifies one or more functions, methods, classes, objects, protocols, data structures, formats, and / or other features of the API implementation component that may be used by an API calling component 2230. The API 2220 may specify at least one calling convention that specifies how functions of the API implementation component receive parameters from the API calling component and how the functions return results to the API calling component. The API calling component 2230 (e.g., an operating system, library, device driver, API, application program, software, or other module) makes API calls through the API 2220 to access and use the functionality of the API implementation component 2210 as specified by the API 2220. The API implementation component 2210 may return a value to the API calling component 2230 through the API 2220 in response to the API call.

[0123] It will be understood that API implementation component 2210 may include additional functions, methods, classes, data structures, and / or other functionality not specified through API 2220 and not available to API invocation component 2230. It should be understood that API invocation component 2230 may be on the same system as API implementation component 2210 or may be located remotely and may access API implementation component 2210 using API 2220 over a network. While Figure 22 shows a single API invocation component 2230 interacting with API 2220, it should be understood that other API invocation components, which may be written in a different language (or the same language) as API invocation component 2230, may use API 2220.

[0124] The API implementation component 2210, the API 2220, and the API calling component 2230 can be stored on a machine-readable medium, which includes any mechanism for storing information in a form readable by a machine (e.g., a computer or other data processing system). For example, machine-readable media include magnetic disks, optical disks, random access memory, read-only memory, flash memory devices, and the like.

[0125] 23 is a block diagram of a device architecture 2000 for a mobile or embedded device, according to one embodiment. The device architecture 2300 includes a memory interface 2302, a processing system 2304 including one or more data processors, image processors, and / or graphics processing units, and a peripherals interface 2306. The various components may be coupled by one or more communication buses or signal lines. The various components may be separate logic components or devices, or may be integrated into one or more integrated circuits, such as a system on a chip integrated circuit.

[0126] The memory interface 2302 can be coupled to memory 2350, which can include high-speed random access memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM), and / or non-volatile memory, such as, but not limited to, flash memory (e.g., NAND flash, NOR flash, etc.).

[0127] Sensors, devices, and subsystems can be coupled to the peripherals interface 2306 to facilitate multiple functions. For example, a motion sensor 2310, a light sensor 2312, and a proximity sensor 2314 can be coupled to the peripherals interface 2306 to facilitate mobile device functions. One or more biometric sensor(s) 2315, such as a fingerprint scanner for fingerprint authentication or an image sensor for facial recognition, may also be present. Other sensors 2316 can also be connected to the peripherals interface 2306, such as a positioning system (e.g., a GPS receiver), a temperature sensor, or other detection devices, to facilitate related functions. A camera subsystem 2320 and an optical sensor 2322 (e.g., a charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) optical sensor) can be utilized to facilitate camera functions such as recording pictures and video clips.

[0128] Communication functions may be facilitated via one or more wireless communication subsystems 2324, which may include radio frequency receivers and transmitters and / or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the wireless communication subsystem 2324 may depend on the communication network(s) over which the mobile device is intended to operate. For example, a mobile device including the illustrated device architecture 2300 may include a wireless communication subsystem 2324 designed to operate over a GSM network, a CDMA network, an LTE network, a Wi-Fi network, a Bluetooth network, or any other wireless network. In particular, the wireless communication subsystem 2324 may provide a communication mechanism by which a media playback application can retrieve resources from a remote media server or scheduled events from a remote calendar or event server.

[0129] The audio subsystem 2326 can be coupled to a speaker 2328 and a microphone 2330 to facilitate voice-enabled functions such as voice recognition, voice duplication, digital recording, and telephony functions. In the smart media devices described herein, the audio subsystem 2326 can be a high-quality audio system that includes support for virtual surround sound.

[0130] The I / O subsystem 2340 may include a touchscreen controller 2342 and / or other input controller(s) 2345. In the case of a computing device that includes a display device, the touchscreen controller 2342 may be coupled to a touch-sensitive display system 2346 (e.g., a touchscreen). The touch-sensitive display system 2346 and touchscreen controller 2342 may detect contact and movement and / or pressure using any of a number of touch and pressure sensing technologies, including, for example, but not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display system 2346. Display output for the touch-sensitive display system 2346 may be generated by a display controller 2343. In one embodiment, the display controller 2343 may provide frame data to the touch-sensitive display system 2346 at a variable frame rate.

[0131] In one embodiment, a sensor controller 2344 is included to monitor, control, and / or process data received from one or more of the motion sensor 2310, light sensor 2312, proximity sensor 2314, or other sensors 2316. The sensor controller 2344 can include logic to interpret the sensor data and determine the occurrence of one of more motion events or activities through analysis of the sensor data from the sensors.

[0132] In one embodiment, I / O subsystem 2340 includes other input controller(s) 2345 that can be coupled to other input / control devices 2348 such as one or more buttons, rocker switches, thumbwheels, infrared ports, USB ports, and / or pointer devices such as a stylus, or control devices such as up / down buttons for volume control of speaker 2328 and / or microphone 2330.

[0133] In one embodiment, memory 2350 coupled to memory interface 2302 can store instructions for operating system 2352, including Portable Operating System Interface (POSIX) compliant and non-compliant operating systems or embedded operating systems. Operating system 2352 can include instructions for handling basic system services and performing hardware-dependent tasks. In some implementations, operating system 2352 can be a kernel.

[0134] Memory 2350 may also store communications instructions 2354 to facilitate communication with one or more additional devices, one or more computers, and / or one or more servers, for example, to retrieve web resources from a remote web server. Memory 2350 may also include user interface instructions 2356, including graphical user interface instructions that facilitate processing of a graphical user interface.

[0135] Additionally, memory 2350 may store sensor processing instructions 2358 to facilitate sensor-related processes and functions, telephone instructions 2360 to facilitate telephone-related processes and functions, messaging instructions 2362 to facilitate electronic messaging-related processes and functions, web browser instructions 2364 to facilitate web browsing-related processes and functions, media processing instructions 2366 to facilitate media processing-related processes and functions, location services instructions including GPS and / or navigation instructions 2368 and Wi-Fi-based location instructions to facilitate location-based functions, camera instructions 2370 to facilitate camera-related processes and functions, and / or other software instructions 2372 to facilitate other processes and functions, such as security processes and functions, and system-related processes and functions. Memory 2350 may also store other software instructions, such as web video instructions to facilitate web video-related processes and functions, and / or web shopping instructions to facilitate web shopping-related processes and functions. In some implementations, the media processing instructions 2366 are divided into audio processing instructions that facilitate audio processing related processes and functions and video processing instructions that facilitate video processing related processes and functions. A mobile device identifier, such as an International Mobile Equipment Identity (IMEI) 2374 or similar hardware identifier, can also be stored in memory 2350.

[0136] Each of the above-identified instructions and applications may correspond to a set of instructions that perform one or more of the functions described above. These instructions need not be implemented as separate software programs, procedures, or modules. Memory 2350 may include additional instructions or fewer instructions. Furthermore, various functions may be implemented in hardware and / or software, including one or more signal processing and / or application specific integrated circuits.

[0137] 24 is a block diagram of a computing system 2400, according to one embodiment. The computing system 2400 shown in the figure is intended to represent a variety of computing systems (wired or wireless), including, for example, one or more implementations of a desktop computer system, a laptop computer system, a tablet computer system, a cellular telephone, a personal digital assistant (PDA) including a cellular-enabled PDA, a set-top box, an entertainment system or other consumer electronic device, a smart home appliance device, or a smart media playback device. Alternative computing systems may include more, fewer, and / or different components. The computing system 2400 may be used to host a computing device and / or a server device to which a computing device may connect.

[0138] Computing system 2400 includes a bus 2435 or other communication device for communicating information, and processor(s) 2410 coupled to bus 2435 that may process information. While computing system 2400 is illustrated with a single processor, computing system 2400 may include multiple processors and / or coprocessors. Computing system 2400 may further include memory 2420, such as random access memory (RAM) or other dynamic storage device, coupled to bus 2435. Memory 2420 may store information and instructions that may be executed by processor(s) 2410. Memory 2420 may also be used for storing temporary variables or other intermediate information during execution of instructions by processor(s) 2410.

[0139] Computing system 2400 may also include a read-only memory (ROM) 2430 and / or another data storage device 2440 coupled to bus 2435 that may store information and instructions for the processor(s) 2410. Data storage device 2440 may be or include a variety of storage devices such as a flash memory device, a magnetic disk, or an optical disk, and may be coupled to computing system 2400 via bus 2435 or via a remote peripheral interface.

[0140] Computing system 2400 may also be coupled to a display device 2450 via bus 2435 to display information to a user. Computing system 2400 may also include an alphanumeric input device 2460, including alphanumeric and other keys, that may be coupled to bus 2435 to communicate information and command selections to processor(s) 2410. Another type of user input device includes a cursor control device 2470, such as a touchpad, mouse, trackball, or cursor direction keys, that communicates directional information and command selections to processor(s) 2410 and controls cursor movement on display device 2450. Computing system 2400 may also receive user input from remote devices communicatively coupled via one or more network interface(s) 2480.

[0141] Computing system 2400 may further include one or more network interface(s) 2480 to provide access to a network, such as a local area network. Network interface(s) 2480 may include, for example, a wireless network interface having antenna(s) 2485, which may represent one or more antennas (e). Computing system 2400 may include multiple wireless network interfaces, such as a combination of Wi-Fi, Bluetooth, near field communication (NFC), and / or cellular telephone interfaces. For example, network interface(s) 2480 may also include a wired network interface for communicating with remote devices via network cable 2487, which may be, for example, an Ethernet cable, a coaxial cable, a fiber optic cable, a serial cable, or a parallel cable.

[0142] In one embodiment, the network interface(s) 2480 may provide access to a local area network, for example, by conforming to the IEEE 802.11 wireless standard, and / or the wireless network interface may provide access to a personal area network, for example, by conforming to the Bluetooth standard. Other wireless network interfaces and / or protocols may also be supported. In addition to or instead of communicating via a wireless LAN standard, the network interface(s) 2480 may provide wireless communication using, for example, a time division multiple access (TDMA) protocol, a global system for mobile communications (GSM) protocol, a code division multiple access (CDMA) protocol, a long term evolution (LTE) protocol, and / or any other type of wireless communication protocol.

[0143] The computing system 2400 may further include one or more energy sources 2405 and one or more energy measurement systems 2445. The energy source 2405 may include an AC / DC adapter coupled to an external power source, one or more batteries, one or more charge storage devices, a USB charger, or other energy source. The energy measurement system includes at least one voltage or amperage measurement device capable of measuring energy consumed by the computing system 2400 over a predetermined period of time. The computing system 2400 may further include one or more energy measurement systems that measure energy consumed by, for example, a display device, a cooling subsystem, a Wi-Fi subsystem, or other frequently used or high energy consuming subsystems.

[0144] Although the embodiments have been described in language specific to structural features and / or methodological acts, it should be understood that the appended claims are not necessarily limited to the specific features or acts described above. Rather, the specific features and acts disclosed should be understood as illustrative embodiments of the claims.

Claims

1. 1. An electronic device comprising: A communication radio; a wireless controller including a distance measurement sensor; A display device; a memory for storing instructions; one or more processors of an electronic device that executes the instructions; The instructions cause the one or more processors to: determining a range and a direction to a target wireless device via the ranging sensor of the wireless controller during a two-way ranging operation; determining a target position estimate of the target wireless device relative to the electronic device based on the range and the direction to the target wireless device; causing the target wireless device to present an indicator of a proximity value to the target position estimate along a trajectory, the trajectory representing a path taken by the electronic device; Electronic devices.

2. The one or more processors execute the instructions, and the instructions are transmitted to the one or more processors, further comprising: causing at least one of the target position estimate of the target wireless device along the trajectory and an indicator of the direction to the target wireless device to be presented; The electronic device of claim 1 .

3. The one or more processors execute the instructions, and the instructions are transmitted to the one or more processors, further comprising: receiving a beacon signal from a target wireless device and determining at least one signal strength value from the beacon signal; estimating at least one proximity value to the target wireless device based on the at least one signal strength value; The electronic device of claim 1 .

4. The one or more processors execute the instructions, and the instructions are transmitted to the one or more processors, further comprising: determining a category of the at least one signal strength value from a plurality of categories of signal strength values; causing the indicator of proximity value to be presented according to the determined category. The electronic device according to claim 3 .

5. The one or more processors execute the instructions, and the instructions are transmitted to the one or more processors, further comprising: causing a distance to the target wireless device to be displayed; The electronic device of claim 1 .

6. The one or more processors execute the instructions, and the instructions are transmitted to the one or more processors, further comprising: establishing a wireless communication connection with the target wireless device; The electronic device of claim 1 .

7. The one or more processors execute the instructions, and the instructions are transmitted to the one or more processors, further comprising: causing the target wireless device to request an increase in beacon rate; The electronic device of claim 1 .

8. The one or more processors execute the instructions, and the instructions are transmitted to the one or more processors, further comprising: The electronic device of claim 1 , configured to perform one or more wireless ranging operations, including ranging operations performed via an ultra-wideband radio.

9. The electronic device of claim 1 , wherein presenting a signal strength proximity view and a ranging view includes presenting at least one indicator of a proximity value to the target wireless device.

10. 1. A computer program comprising instructions for causing one or more processors of an electronic device to perform operations, said operations comprising: determining a range and direction to a target wireless device via a ranging sensor of the wireless controller during a two-way ranging operation; determining a target position estimate for the target wireless device relative to the electronic device based on the range and the direction to the target wireless device; presenting an indicator of a proximity value of the target wireless device to the target position estimate along a trajectory, the trajectory representing a path taken by the electronic device. Computer program.

11. The operation may further include: receiving a beacon signal from a target wireless device and determining at least one signal strength value from the beacon signal; and estimating at least one proximity value to the target wireless device based on the at least one signal strength value.

11. A computer program according to claim 10.

12. The operation further comprises: determining a category of the at least one signal strength value from a plurality of categories of signal strength values; and presenting the indicator of proximity value according to the determined category.

12. A computer program according to claim 11.

13. The operation further comprises: establishing a wireless communication connection with the target wireless device; 11. A computer program according to claim 10.

14. The operation further comprises: requesting the target wireless device to increase its beacon rate; 11. A computer program according to claim 10.

15. The operation further comprises: selecting to present a ranging view is performed in response to establishing a wireless communication connection with the target wireless device.

11. A computer program according to claim 10.

16. The operation further comprises: performing one or more wireless ranging operations, including ranging operations performed via an ultra-wideband radio; 11. A computer program according to claim 10.

17. The operation further comprises: presenting a signal strength proximity view and a ranging view includes presenting at least one indicator of the proximity value to the target wireless device.

11. A computer program according to claim 10.

18. determining a range and direction to a target wireless device via a ranging sensor of the wireless controller during a two-way ranging operation; determining a target position estimate of the target wireless device relative to the electronic device based on the range and the direction to the target wireless device; presenting an indicator of a proximity value of the target wireless device to the target position estimate along a trajectory, the trajectory representing a path taken by the electronic device. method.

19. The method further comprises: receiving a beacon signal from a target wireless device and determining at least one signal strength value from the beacon signal; and estimating at least one proximity value to the target wireless device based on the at least one signal strength value.

20. The method of claim 18.

Images