Device location seeking

Abstract

The invention relates to device location seeking. Embodiments of a device locator application are described. In one embodiment, one or more inertial displacement measurements may be received using the inertial sensor and received camera sensor data, a trajectory may be determined based on the one or more inertial displacement measurements, a beacon signal is received from a target wireless device, and at least one signal strength value is determined from the beacon signal, at least one proximity value to the target wireless device may be estimated based on the at least one signal strength value corresponding to the at least one location along the trajectory, and an indicator of the at least one proximity value to the target wireless device may be presented along the trajectory in a user interface.

Description

[0001] This application is a divisional application of invention patent application 202210606447.8 entitled "Device Location Finding", filed on May 31, 2022. Technical Field

[0002] The implementation scheme described in this article involves using a device locator service to locate devices. Background Technology

[0003] When a connection between devices has not yet been established, previous device locator services do not provide meaningful assistance in locating devices. Specifically, users often repeatedly travel along the same paths, attempt to locate devices using existing methods, and require more information to help guide them to lost devices. Therefore, an improved locator service is needed. Summary of the Invention

[0004] Implementations include electronic devices, non-transitory machine-readable media, and methods. In some implementations, the electronic devices include communication radio equipment, inertial sensors, camera sensors, display devices, memory for storing instructions, and one or more processors for executing the instructions. Implementations may have instructions that cause one or more processors to: use the inertial sensor and received camera sensor data to receive one or more inertial displacement measurements; determine a trajectory based on the one or more inertial displacement measurements; receive a beacon signal from a target wireless device and determine at least one signal strength value based on the beacon signal; estimate at least one proximity value to the target wireless device based on the at least one signal strength value corresponding to at least one position along the trajectory; and present an indicator of the at least one proximity value to the target wireless device along the trajectory in a user interface.

[0005] In one or more embodiments, the one or more processors execute the instructions, wherein the instructions further cause the one or more processors to: determine a category of the signal strength value from a plurality of categories, and to present an indicator of the proximity value according to the determined category.

[0006] In one or more embodiments, the one or more processors are used to execute the instruction, wherein the instruction further causes the one or more processors to display the indicator of the proximity value on a two-dimensional heatmap along the trajectory within the user interface.

[0007] In one or more embodiments, the one or more processors are used to execute the instruction, wherein the instruction further causes the one or more processors to display the indicator of the proximity value on a hexagonal grid.

[0008] In one or more embodiments, the wireless controller includes a ranging sensor; the one or more processors execute the instruction, which further causes the one or more processors to: selectively present a ranging view, a signal strength proximity view, or at least one of a ranging and signal strength proximity view within the user interface, wherein presenting the ranging view includes: determining, during two-way ranging operations, a range relative to the target wireless device and a direction to the target wireless device via the ranging sensor of the wireless controller; determining a target position estimate of the target wireless device relative to the electronic device based on the range relative to the target wireless device and the direction to the target wireless device; and presenting the ranging view includes displaying at least one of the target position estimate of the target wireless device and an indicator of the direction to the target wireless device. In one or more embodiments, the selection to present the ranging view is performed in response to establishing a radio connection with the target wireless device. In one or more embodiments, the one or more processors execute the instruction, which further causes the one or more processors to request an increase in the beacon rate from the target wireless device. In one or more embodiments, the one or more wireless ranging operations include ranging operations performed via ultra-wideband radio. In one or more embodiments, presenting the signal strength proximity view and the ranging view includes presenting at least one indicator of the proximity value to the target wireless device along the trajectory in the user interface, as well as the indicator of the direction to the target wireless device. Attached Figure Description

[0009] Figure 1A and Figure 1B A device locator user interface according to the embodiment described herein is shown.

[0010] Figure 1C This is a block diagram of a network operating environment 100 for mobile devices according to the implementation plan.

[0011] Figure 2 A system for locating wireless accessories according to an implementation scheme is shown.

[0012] Figure 3 A system for pairing and locating wireless accessories according to an embodiment described herein is shown.

[0013] Figure 4 This is a flowchart illustrating a method for use with the device locator system described herein.

[0014] Figure 5 This is a sequence diagram illustrating a method for presenting a device locator user interface used in conjunction with the device locator system described herein.

[0015] Figure 6A system for performing ranging and attitude estimation to generate a target position estimate of a target device, according to an implementation scheme, is shown.

[0016] Figure 7 A system that enables proximity enhancement for device locator applications is shown.

[0017] Figure 8 This is a flowchart illustrating a method for a device locator user interface in an implementation scheme.

[0018] Figure 9 A method for implementing a proximity-enhanced user interface for a device locator application is shown.

[0019] Figure 10 A method for determining the location of a wireless accessory via a device locator server is shown.

[0020] Figure 11 An additional method for determining the location of a wireless accessory via a device locator server is shown.

[0021] Figure 12 This is a flowchart illustrating a method for broadcasting a signal beacon at a wireless accessory according to an implementation scheme.

[0022] Figures 13 to 14 The operation of a method that can be performed by a detector device according to the embodiment described herein is illustrated.

[0023] Figure 15 The acquisition of signal and ranging data performed by the detector device according to the implementation scheme is shown.

[0024] Figures 16 to 21 A device locator user interface according to one or more implementation schemes is shown.

[0025] Figure 22 This is a block diagram illustrating an exemplary API architecture that can be used in some implementation schemes.

[0026] Figure 23 It is a block diagram of a device architecture for mobile or embedded devices according to the implementation plan.

[0027] Figure 24 It is a block diagram of the computing system according to the implementation plan. Detailed Implementation

[0028] The embodiments described herein typically provide techniques for presenting information on mobile devices to guide or assist users in actively locating a target device. Mobile devices may use various data sources and / or signals to present information to aid in locating the target device, including signals received from the target device itself. The accessibility of the various types of signals received from the target device may vary as the mobile device moves within the target device's location environment. When the mobile device has access to the data source / signal, various user interface views can be presented within the mobile device's user interface to represent the information.

[0029] In one implementation, a signal strength measurement of a signal received from a mobile device can be used within a user interface to indicate proximity to a target device, thus indicating when the mobile device is approaching the target device. Figure 1A and Figure 1B A device locator user interface 204 on a mobile device 2100 according to an embodiment described herein is shown. Figure 1A The "proximity view" 2104 shown may use visualization techniques to present proximity information, thereby presenting proximity information about the target wireless accessory device. In one embodiment, the proximity view 2104 has visual indicators (such as user interface elements) positioned along a trajectory 2118 presented within the user interface 204 to indicate the path the user has taken in their search. In some embodiments, the visual indicators may be user interface elements displayed through gradients, colors, color gradients, sizes, shapes, and / or any other visualization techniques used to represent signal strength values ​​and a defined proximity category (e.g., far, nearby, close, etc.) corresponding to the target device. The proximity view 2104 for finding "Tommy's AirPods" has indicators 2102, 2106, 2110, 2112, and 2114 located at various positions along the trajectory 2118 within the user interface 204. Each indicator in the proximity view 2104 may be a user interface element within the user interface 204 that uses size and color gradients to indicate proximity to the target wireless accessory device. In proximity view 2104, for example, indicator 2106 is the closest to the target wireless accessory device along trajectory 2118 that the user has taken to find the target wireless accessory device, as indicated by a darker color and / or a larger size compared to other indicators (e.g., 2102, 2110, 2112, and 2114). Implementations can use visual inertial ranging (VIO) measurements to determine the trajectory 2118 that the user has taken in their search within user interface 204. VIO provides the ability to track the movement of the mobile device in an arbitrary initial coordinate system. VIO techniques include analyzing image sequences collected by the mobile device to estimate camera motion on the image sequences.

[0030] In some implementations, the mobile device can move within a threshold range of the target device, allowing the ranging process to use communication between the mobile device and the target device to determine the distance to the target device and the direction to the target device. For example... Figure 1B As shown, the "ranging view" 2120 of the user interface 204 can selectively display distance measurements 2116 in addition to the direction 2108 to the target device. In some embodiments, a proximity view 2104 for locating the target device can be used when ranging data from the ranging view 2120 is unavailable because the target device is not within a threshold range of the mobile device, the target device transmitter is not in the field of view of the receiver at the mobile device, and / or the mobile device does not have a nearly unobstructed view leading to the target device. When the receiver at the mobile device has a view of the target device transmitter, the target device may be in the field of view of the mobile device.

[0031] In some implementations, ranging using ultra-wideband (UWB) radio technology can provide relatively accurate location or distance data to a target device, but is a relatively short-range radio frequency (RF) wireless communication technology compared to Bluetooth. In some implementations, it may be desirable for the mobile device's UWB receiver to have a line-of-sight to the target device's transmitter or a nearly unobstructed view of the target device to obtain optimal ranging location data. Proximity information in the form of signal strength information may be relatively less precise than UWB, but can cover a wider area with a longer range and can be obtained from advertising before a radio connection is established. In some implementations, two-way communication cannot be established via a connection between the mobile device and the target device, but advertising received at the mobile device can provide signal strength information before a connection is established to help guide the user to the target device. A combination of technologies can help users locate target wireless accessory devices.

[0032] Various embodiments are described with reference to the accompanying drawings. However, certain embodiments may be practiced without one or more of these specific details or in combination with other known methods and constructions. In the following description, numerous specific details such as particular configurations, dimensions, and processes are shown to provide a thorough understanding of the embodiments. In other instances, well-known semiconductor processes and manufacturing techniques are not described in particular detail to avoid unnecessarily obscuring the embodiments. The phrase "an embodiment" as used throughout the specification means that a particular feature, structure, construction, or characteristic described in connection with an embodiment is included in at least one embodiment. Therefore, the repeated use of the phrase "in an embodiment" throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, construction, or characteristic may be combined in any suitable manner in one or more embodiments.

[0033] In the following discussion, a computing device including a touch-sensitive display is described. However, it should be understood that the computing device may include one or more other physical user interface devices. Various applications that can run on the device may use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface, as well as the corresponding information displayed on the device, may be adjusted and / or changed from one application to the next, and / or within the respective application. In this way, the shared physical architecture of the device (such as the touch-sensitive surface) can support various applications using an intuitive and transparent user interface.

[0034] The following describes some procedures for sequential operations. However, it should be understood that some of these operations can be performed in a different order. Furthermore, some operations can be performed in parallel rather than sequentially.

[0035] Figure 1C This is a block diagram of a network operating environment 100 for mobile devices according to an embodiment. The network operating environment 100 includes multiple mobile devices, such as accessory device 101 and mobile device 102. Accessory device 101 can be paired with mobile device 102. In one embodiment, mobile device 101 can be a pair of accessory devices that can be paired as a device group. Optionally, the device group having accessory device 101 can be stored in the mobile device via a wired connection, such as a case for holding accessory device 101. In some embodiments, the case can also be an accessory device that can be paired with mobile device 102. For example, accessory device 101 can collectively be for devices such as Apple AirPods. ® Or EarPods ® The devices are described. In some embodiments, accessory device 101 may not be able to communicate over a wide area network. In other embodiments, mobile devices 101 and 102 may each be any electronic device capable of communicating with wireless networks and wireless accessory devices. Some exemplary mobile devices include, but are not limited to, smartphones, tablets, laptops, wearable computers (e.g., smartwatches or other wearable computing accessories), mobile media players, personal digital assistants, and AirPods. ® EarPods ®Locator tags, headphones, head-mounted displays, health equipment, and other similar devices. Each of mobile devices 101 and 102 may optionally include a user interface, such as user interface 104 for mobile device 102. In other embodiments, mobile device 101, as an accessory device, may not have a user interface. Mobile devices 101 and 102 may be third-party devices that utilize application programming interfaces to access device locator services. Third-party devices may be provided by different device manufacturers or be part of different ecosystems (e.g., operating systems) of mobile devices 101 and 102. Mobile devices 101 and 102 may communicate via one or more wired and / or wireless networks 110 to perform data communication. For example, wireless network 112 (e.g., cellular network, Wi-Fi network) may communicate with wide area network 114 (such as the Internet) using gateway 116. Similarly, access device 118, such as a mobile hotspot wireless access device, may provide communication access to wide area network 114. Gateway 116 and access device 118 may then communicate with wide area network 114 via a combination of wired and / or wireless networks.

[0036] In some implementations, both voice and data communications can be established via wireless network 112 and / or access device 118. For example, mobile device 102 can make and receive telephone calls (e.g., using VoIP), send and receive email messages (e.g., using POP3), and retrieve electronic documents and / or streams, such as web pages, photos, and videos, via wireless network 112, gateway 116, and wide area network 114 (e.g., using TCP / IP or UDP protocols). In some implementations, mobile device 102 can make and receive telephone calls, send and receive email messages, and retrieve electronic documents via access device 118 and wide area network 114. In some implementations, mobile device 101 and / or mobile device 102 can be physically connected to access device 118 using one or more cables, for example, where access device 118 is a personal computer. In this configuration, mobile device 101 or mobile device 102 may be referred to as a “tethered” device. In one implementation, mobile device 101 can communicate with mobile device 102 via wireless peering connection 120. Wireless peering connection 120 can be used to synchronize data between devices.

[0037] Mobile device 101 or mobile device 102 can communicate with one or more services, such as telephone service 130, instant messaging service 140, media service 150, storage service 160, and device locator service 170, via one or more wired and / or wireless networks 110. For example, telephone service 130 enables telephone communication between mobile devices or between a mobile device and a wired telephone device. Telephone service 130 can route IP-based voice (VoIP) calls via wide area network 114 or can access a cellular voice network (e.g., wireless network 112). Instant messaging service 140 can, for example, provide email and / or other instant messaging services. Media service 150 can, for example, provide access to media files, such as song files, audiobooks, movie files, video clips, and other media data. Storage service 160 can provide network storage capabilities to mobile devices 101 and 102 to store documents and media files. Device locator service 170 enables users to locate lost or misplaced devices that are connected to one or more wired and / or wireless networks 110 at at least some point. Other services may also be provided, including software update services for updating operating system software or client software on mobile devices. In one embodiment, instant messaging service 140, media service 150, storage service 160, and device locator service 170 may each be associated with a cloud service provider, wherein the various services are facilitated via cloud service accounts associated with mobile devices 101 and 102.

[0038] In some implementations, accessory device 101 and mobile device 102 and / or device group may register with Certificate Authority 106. In some implementations, Certificate Authority 106 is the entity that issues digital certificates, and the service may be implemented using a set of servers managed by the device manufacturer, service provider, or registration service. Certificates provided by Certificate Authority 106 can verify the validity of received verifiable information about the device, such as the device's specific manufacturer, serial number, device group identifier or other identifier, indication that the device is part of the device group, and / or any other verifiable information. In some implementations, the device manufacturer may create a device group by grouping the serial numbers of accessory devices within the device group. In other implementations, the certificate may be encrypted by devices 101 and 102 before being sent to a third party, and may be decrypted at a verification service (e.g., a Certificate Authority or another verification service) when the third party requests verification of information provided by accessory device 101, mobile device 102, and / or devices within the device group. In some implementations, a security token may be provided by accessory device 101 in a pairing request. Additional examples of paired devices using location services can be found in U.S. Patent Application No. 17 / 219,595, filed March 21, 2021, entitled "Secure Pairing and Pairing Lock for Accessory Devices," the entire contents of which are incorporated herein by reference.

[0039] Mobile devices 101 and 102 may have locally accessible applications, services, and functions on the device, including location service 180. Specifically, mobile devices 101 and / or 102 may have a device locator application (e.g., a "Find My" application) 190 to utilize device locator service 170 and location service 180. Locally accessible data may be stored on known locations 182 and secure or trusted locations 184. In some cases, machine learning algorithm 186 may be used to identify known locations 182 and / or trusted locations 184. While cluster analysis is provided as an example of a machine learning algorithm that can be used, those skilled in the art will recognize that other algorithms may be used to identify potential known or trusted locations. For example, cluster data analysis may be used to identify and classify locations and provide semantic labels for locations, such as locations frequently visited by users. Secure or trusted locations 184 may be explicitly specified or confirmed by users of devices 102A-B after data analysis. In other cases, known locations 182 or trusted locations 184 may be classified offline and provided by device locator service 170 or a third party (e.g., a database with map information).

[0040] On-device heuristics and / or machine learning models can be used to infer relationships between users and locations based on analysis of locally stored data on frequently visited locations, including locations frequently visited by the user, known locations, and / or any other locations. For example, frequently visited locations include homes, vehicles, workplaces, any location frequently visited by users with mobile devices (e.g., accessory devices 101 and mobile device 102), and / or any other location designated by the user as a trusted location 184. Known location 182 can be a commercial location, a public space, a park, a museum, and / or any other location the user might frequently visit. Boundary information for the corresponding stored location can be stored along with the location's classification type and any semantic labels assigned to the location. The stored information may include a defined set of boundaries or radii around the point location to allow the creation of a geofence for that location. A geofence is a virtual perimeter of a real-world geographic area. The Global Positioning System (GPS) can be used to create virtual fences around the location and track the physical location of mobile devices 101 and 102 within the geofence boundaries, as well as entry and exit from the bounded area.

[0041] Machine learning algorithm 186 may include on-device heuristics, machine learning algorithms, or combinations thereof to analyze and assign labels regarding the movement or travel of the device, which is designated as being in a "on-the-go" or "stable" state at a specific location for a certain period of time. The analysis can be performed using various signals from data sources available to mobile device 102, including but not limited to: sensor data, location data, calendar data, transit card usage data, application data, historical data about travel patterns / routines, and / or any other data accessible to mobile device 102. In some implementations, mobile device 102 can be classified with a "stable" semantic label after remaining within the geographic boundaries of a defined location (e.g., trusted location 184) for a defined period of time. In the simplest case, location data of mobile device 102 may remain within the boundaries of a geofence at a specific location for a certain period of time (e.g., 5 minutes). Sensor data (such as accelerometer data) may indicate that mobile device 102 is stationary to support the inference of stabilization. Application data may support the inference of stabilization of mobile device 102, such as the mobile device being located at a calendar-approved location. Application data indicating the type of application in use can also provide inferences about the device being stabilized, such as the use of a media application. Historical data about a user's routines or patterns of movement can be used to determine whether mobile device 102 is being stabilized, such as a bedtime routine at home or a hotel location. Mobile device 102 can be categorized with a "on the way" label based on the user's previous behavior, patterns, or routines, and analyzed on mobile device 102. For example, a user may have a routine of working around the same time every day, and if data on the device supports repeating this pattern, the "on the way" status can be assigned. In the simplest case, the speed at which the mobile device moves or enters and exits a known geographic area (e.g., using a geofence) can allow inferences that mobile device 102 is on the way. If mobile device 102 is detected accelerating in a known transportation area (e.g., on a road, highway, train route, etc.), mobile device 102 can be given the "on the way" status. Similarly, if a transportation application / card is being used / used, mobile device 102 can be designated as "on the way".

[0042] Figure 2A system 200 for locating a wireless accessory 201 according to an embodiment is shown. In one embodiment, the wireless accessory 201 is another embodiment of an accessory device 101 (and optionally a box) that can be paired as part of a device group. In other embodiments, the accessory device 201 is a separate accessory device that is not part of a device group and is each paired separately with a mobile device 102. Each wireless accessory 201 includes one or more wireless transceivers and can communicate directly or indirectly (e.g., via another device or computer) with an accompanying device (e.g., mobile device 102) via a wireless network or peer-to-peer communication link. The accessory device 201 can provide beacon signals for the box and any accessories within the box. The accessory device 201 is separate from the box and can be discovered independently and separately by providing beacon signals. Examples of wireless accessory devices 201 include, but are not limited to, wireless earbuds, EarPods, AirPods, input devices, charging devices, accessory cases, headphones, headband headphones, fitness equipment, health equipment, display devices, external hard drives, other wearable device adapters (e.g., smartwatches, fitness trackers, optical head-mounted displays), speakers, and / or other devices. Paired accessory groups can be of the same type of device (e.g., speakers, AirPods, fitness weights, etc.) or different types of devices (e.g., smartphones and credit card readers, etc.). Wireless accessory 201 may also include other wireless devices, such as input devices, including but not limited to credit card readers, stylus devices, mice, keyboards, game controllers, or remote controls. In one embodiment, wireless accessory 201 also includes smartphones, tablets, laptops, smart speaker devices, televisions, or set-top boxes that cannot at least temporarily access a wide area network such as the Internet (e.g., WAN 114 as shown in Figure 1). Wireless accessory 201 can also be any other wireless device, including beacons or locator tags that can be attached to other devices to allow tracking or locating those devices. In one embodiment, the wireless accessory 201 may be from a group of accessory devices that pair with the mobile device 102 using wireless technology standards such as, but not limited to, Bluetooth. The wireless accessory 201 may also communicate with the mobile device 102 via wireless technologies, including specific implementations of any wireless standards and protocols such as Wi-Fi Direct, Zigbee, or AirPlay. Although the accessory device to which the wireless accessory 201 is paired is generally referred to as the mobile device 102, the accessory device is not limited to mobile devices. In some embodiments, the accessory device may also include laptops or desktop devices, and may also include wearable accessories such as, but not limited to, smartwatches or wearable displays.

[0043] In one embodiment, the wireless accessory 201 may periodically transmit a wireless beacon signal. The wireless accessory 201 may use one of the various wireless technologies described herein (e.g., Bluetooth, Wi-Fi, etc.) to transmit the beacon signal, and in one embodiment, ultra-wideband (UWB) radio technology may also be used to transmit the beacon. The beacon signal may be transmitted using a single wireless technology, one of several alternative wireless technologies, or several concurrent wireless technologies. The beacon signal may transmit a beacon identifier, which includes information specifically identifying the individual wireless accessory 201 and / or group of devices. In one embodiment, the beacon identifier is a public encryption key associated with the device.

[0044] The beacon signal may also transmit information about the wireless accessory 201, such as device status information and / or verifiable information. Device status information in the beacon signal may include, but is not limited to, the following: beacon type, device classification, battery level, any predefined device status, device status, lost status, alarm status, separated from owner status, near owner status, near-owner status, near-owner status, near-owner status, wired or wireless connection status, physically connected to one or more accessory devices in the device group, pairing status indicating whether the accessory device is paired, pending pairing status, battery life status, charging status, and / or any other status information. A lost or "separated from owner" status may indicate that the wireless accessory 201 has determined that it is lost or has been placed in a lost state by the device's owner. An alarm status may indicate that the wireless accessory 201 is in a state where the device should trigger an alarm if it moves from its current location. A near-owner status may indicate that the wireless accessory 201 has detected the presence of a mobile device 102 associated with the accessory's owner in the vicinity.

[0045] In some implementations, verifiable information may include any information that may be needed to establish trust or authorization, i.e., the pairing and / or search process can proceed, where the device presents verifiable information. For example, verifiable information may include information established by the device manufacturer, such as serial numbers or sets of serial numbers in a device group. In some implementations, verifiable information may include device status or condition information. Verifiable information may include, but is not limited to, the following: device type, member of the device group, serial number, device group, serial numbers of other devices within the device group, status or condition information, software version, and / or any other verifiable information. Verifiable information may be sent to a Certificate Authority 106 or other verification service to verify the information presented by one device to another. Verifiable information may be encrypted and / or sent with a token to allow further verification of the device.

[0046] In some implementations, the beacon signal can be detected by a detector device 202 locally proximate to the wireless accessory 201 to enable crowdsourcing for locating the lost wireless accessory 201. The detector device 202 may be a device similar to mobile device 102 and can receive and transmit data over a wide area network 114, using wireless technologies similar to those used with the wireless accessory 201 (e.g., Bluetooth). Specifically, the detector device 202 may use a wireless protocol to receive data, through which the beacon signal is transmitted. The detector device 202 may use one or more location and / or positioning services to determine its location, including but not limited to satellite positioning service 206 or a terrestrial positioning system using RF signals received from a cellular tower transmitter such as a Wi-Fi access point or a cellular phone network from a wireless base station 205. In one implementation, the detector device 202 periodically stores its location determined based on one or more location and / or positioning services. The stored location may be associated with a timestamp indicating that location was determined. When detector device 202 receives a beacon signal from wireless accessory 201, detector device 202 can transmit its location to device locator server 203 via wide area network 114. The timestamp used to determine the location of detector device 202 can be associated with the timestamp of receiving the beacon signal to associate the geographic location with the received beacon signal.

[0047] With the wireless accessory 201 providing a public key within the beacon signal, the detector device 202 can encrypt the determined location data and transmit the encrypted location data to the device locator server 203 via the wide area network 114. In one embodiment, additional data can be encrypted and transmitted along with the location data, or transmitted to the device locator server 203 unencrypted. For example, the received signal strength indication (RSSI) of the beacon signal can be transmitted along with the location data. The RSSI data can then be used to determine the distance between the wireless accessory 201 and the detector device 202 and can aid in triangulation on the owner device. In the case where the RSSI data is transmitted unencrypted, in one embodiment, the server can use the RSSI information to reduce noise by discarding very weak signals if other stronger signals are present. In one embodiment, UWB ranging data may also be provided, where such data is available.

[0048] In one implementation, the detector device 202 may behave differently when receiving a beacon signal from the wireless accessory 201, depending on the device status conveyed by the wireless accessory 201. For a standard beacon signal, the detector device 202 may queue encrypted location data and transmit the location data to the device locator server 203 during a periodic transmission window. However, if the wireless accessory 201 is indicating an alarm status, the detector device 202 may immediately transmit the location data to the device locator server 203. Alternatively, if the beacon signal from the wireless accessory 201 indicates that the accessory is near its owner, the detector device 202 may not transmit the location data to the device locator server 203. Alternatively, the detector device 202 may delay the transmission of encrypted location data.

[0049] If the owner of wireless accessory 201 wishes to locate the wireless accessory, the owner can access device locator user interface 204 on mobile device 102. Device locator user interface 204 may be associated with a device locator application used to locate electronic devices and accessories registered with a user's online account (such as a cloud service account or another type of online account). The device owner can use device locator UI 204 to query device locator server 203 for location data that may have been transmitted to the device locator server by the detector device 202 of wireless accessory 201. In one embodiment, mobile device 102 may transmit a public encryption key associated with wireless accessory 201 to device locator server 203. Device locator server 203 may then return any stored location data corresponding to the public encryption key. The location data returned to mobile device 102 may be encrypted data encrypted by detector device 202 using the public encryption key. Mobile device 102 may use an associated private key to decrypt the encrypted location data. The decrypted location data is then processed by mobile device 102 to determine the most probable location of wireless accessory 201. In various implementations, the most probable location of the wireless accessory 201 can be determined by triangulation from multiple receiving locations and by using other data, such as beacon signal RSSI associated with each location and timestamp or UWB ranging data included in the location data.

[0050] Figure 3A system 300 for pairing and locating a wireless accessory according to an embodiment described herein is illustrated. In one embodiment, a user's mobile device 102 (e.g., an example of device 101) of the wireless accessory 201 may present an accessory pairing UI 302 through which the user pairs the mobile device 102 with the wireless accessory 201. During the initial pairing (305) between the mobile device 102 and the wireless accessory 201, a public key exchange (310) may be performed between the mobile device and the wireless accessory 201. In one embodiment, during the public key exchange (310), the mobile device 102 and the wireless accessory 201 exchange the public key in the public key pair generated by the device and the accessory 201. In one embodiment, the public key exchange (310) is a one-way transmission, wherein the mobile device 102 transmits the public key in the public / private key pair to the wireless accessory 201. Alternatively or otherwise, the public key exchange (310) may be a Diffie-Hellman key exchange, wherein the device and the accessory establish a shared secret between the two parties. In one implementation, the public key exchange (310) further utilizes elliptic curve cryptography to establish a shared secret. For example, elliptic curve Diffie-Hellman (ECDH) can be used to establish public key pairs and one or more shared secrets. In one implementation, the one or more shared secrets include an anti-tracking secret, which can be used by the wireless accessory 201 to periodically derive additional public keys.

[0051] After the wireless accessory 201 has been paired with the mobile device 102, the wireless accessory 201 may periodically broadcast a beacon signal 301 including device status information and a beacon identifier. In one embodiment, the beacon identifier is a public key derived from a shared secret established during public key exchange (310). Additionally, the wireless accessory 201 may periodically perform public key derivation (315) to generate a new public key and begin broadcasting the new public key as the beacon identifier. The public key is a K-byte key, wherein a new K-byte key is generated every M minutes. The values ​​K and M may vary between embodiments. In one embodiment, a 28-byte K value is used. In another embodiment, a 27-byte K value is used. The value K may be determined at least in part based on the beacon length associated with the wireless protocol used to transmit the beacon signal 301. In one embodiment, the beacon signal may transmit a variant of a beacon advertising packet associated with a low-power radio protocol such as Bluetooth Low Energy.

[0052] In one implementation, the value M is 15 minutes, such that a new K-byte key is generated every 15 minutes. The public key can be derived qualitatively based on a timestamp and an anti-tracking secret generated during public key exchange 310. The public key derivation (315) process enables the wireless accessory 201 to use different keys over time, thereby preventing a specific key from being associated with a specific device for an extended period. The key can be derived based on an anti-tracking secret known only to mobile device 102 and wireless accessory 201, thereby allowing mobile device 102 and only mobile device to determine which public key wireless accessory 201 will broadcast at any given timestamp. The anti-tracking secret can be generated along with the ECDH public key and transmitted to wireless accessory 201. The anti-tracking secret can then be used to enable wireless accessory 201 to generate a public key sequence. In one implementation, a public key sequence Its scalar or exponential value is limited. With group elements (e.g., elliptic curve points) Group operations between scalar or exponential values. , where KDF is the key derivation function, AT is the anti-tracking secret, and i is a counter or timestamp.

[0053] In one implementation, reverse tracking resistance can be enabled to protect the anti-tracking secret in the event that the wireless accessory 201 is compromised. When reverse tracking resistance is enabled, the anti-tracking secret is transmitted to the wireless accessory 201, but the wireless accessory does not retain the anti-tracking secret. Instead, the accessory calculates the value... ( ||Time), among which ,and For the cryptographic hash function. Wireless accessory 201 then stores. The duration is specified for a given time period i. If wireless accessory 201 is compromised, only the values ​​i used for the current and future periods are exposed. And without exposing the anti-tracking secret AT. In one implementation, reverse tracking resistance is achieved by periodically... The process is executed by writing to the non-volatile memory of the wireless accessory 201.

[0054] In one implementation, the wireless accessory 201 may transmit the beacon signal 301 every two seconds, but other beacon rates may be used, and these beacon rates may vary under certain conditions. For example, when in a near-owner state, the wireless accessory 201 may reduce the beacon rate. The beacon rate may also vary based on an event triggered by an accelerometer. For example, when in an alarm state, the wireless accessory 201 may increase the beacon rate, which may be triggered by an accelerometer on the wireless accessory 201.

[0055] If, after transmitting beacon signal 301, wireless accessory 201 receives a response from mobile device 102 associated with the user of the accessory, indicating that mobile device 102 is within range of the wireless accessory, then wireless accessory 201 may enter a near-owner state. Additionally, when the wireless accessory is in a near-owner state, the amount of data transmitted by beacon signal 301 may be reduced. In one embodiment, the rate at which a new public key is generated may also be reduced when the wireless accessory is in a near-owner state.

[0056] Wireless accessory 201 may enter an alarm state upon receiving a message from mobile device 102 instructing it to do so. While in an alarm state, the wireless accessory may initially enter a standby state in which it reduces or stops transmitting locator beacon signals, but other types of wireless signaling may persist. Wireless accessory 201 may remain in this standby state until mobile device 102 deactivates the state or an alarm is triggered. In one embodiment, the alarm may be triggered, for example, when movement is detected via an accelerometer within wireless accessory 201. In another embodiment, the alarm may also be triggered when it is detected that the wireless accessory has moved out of range of the mobile device and is no longer in a near-owner state. When an alarm is triggered, the rate of beacon signal 301 may be increased to increase the speed at which the locator wireless accessory 201 can be located.

[0057] The beacon signal 301 transmitted by the wireless accessory 201 can be detected by a set of detector devices 303 (which may be detector device 202) and / or mobile devices 102. These detector devices and / or mobile devices are other electronic devices capable of receiving the beacon signal transmitted by the wireless accessory and transmitting location and other data associated with the beacon signal 301 to the device locator server 203 via the wide area network 114. In one embodiment, the set of detector devices 303 includes a variant of the mobile device 102, or may be other types of electronic devices. For example, the set of detector devices 303 may perform an operation (320) to associate the beacon signal 301 received from the wireless accessory 201 with the device location associated with the detector device 303. (See also...) Figure 2 The device location can be determined via satellite positioning services or a terrestrial positioning system using RF signals received from wireless base stations (e.g., Wi-Fi access points or cell tower transmitters). In one embodiment, the group of detector devices 303 may also include fixed devices capable of receiving beacon signals 301, such as smart speaker devices, televisions, or television set-top boxes.

[0058] The group of detector devices 303 can encrypt location data using a beacon identifier (e.g., a public key) received in the beacon signal 301 and send the location data (325) to the device locator server 203. The data sent by this group of detector devices 303 is sent anonymously, and the identification information of the detector devices is not stored along with the data sent by the detector devices.

[0059] The device locator server 203 can store encrypted location data in a data repository 304, which in one embodiment may be a distributed database with multiple nodes. A hash of the attached beacon identifier / public key may be sent along with the encrypted location data. The encrypted location data may be stored in the database nodes based on the hash of the beacon identifier. The encrypted location data may be indexed by the device locator server 203 using the hash of the beacon identifier. Sending the hash of the beacon identifier instead of the complete beacon identifier prevents the complete beacon identifier from being stored on the server. Other information may also be sent and stored along with the location data in an encrypted or unencrypted state. This other information may include a timestamp of when the beacon signal 301 was received, the RSSI information of the received beacon, and / or ranging information, such as that determined via UWB ranging.

[0060] When the user or owner of wireless accessory 201 wishes to locate the accessory, the user or owner can access device locator UI 204 on mobile device 102. Device locator UI 204 may be associated with locator application 190 or features of mobile device 102. Device locator UI 204 may also have a web-based interface accessible from mobile device 102 or another type of electronic device, such as a laptop or desktop device. When device locator UI 204 is loaded, mobile device 102 may send a request (330) for location data to device locator server 203. Request 330 may include a set of public keys or public key hashes that can be used as beacon identifiers for beacon data. Mobile device 102 may generate this set of public keys based on secret information held by mobile device 102 and wireless accessory 201 and a timestamp indicating when mobile device 102 wishes to receive location data. In one embodiment, this set of public keys is a sequence of public keys generated based on an anti-tracking secret. Public key sequence Matching private key sequence Correspondingly, mobile device 102 can generate a public key sequence and a corresponding public key sequence. , where i is a counter or timestamp. In one implementation, mobile device 102 may generate and send the public key (or a hash of the public key for the previous 24 hours) within request 330. If no data for the 24-hour public key is found, mobile device 102 may send the generated key in an earlier time period, returning to the predetermined location data retention limit.

[0061] In one implementation, encrypted location data is stored and indexed based on a hash of the public key, rather than the public key itself, to prevent location service data providers from storing data that could be used to bind encrypted location data to a specific device and thus to a specific user or user account. The detector device may send a hash of the public key broadcast within a beacon signal 301 associated with the observed location. The device owner may use the hash of the public key, determined for a specific query period, to query the device locator server 203.

[0062] In some implementations, if a location query is to be performed via a web-based interface from an electronic device, such as a laptop or desktop device, it may be necessary to send a key to the electronic device to enable decryption of the location data. In one implementation, the decryption key for the location data may be sent to a server that provides a web-based interface so that the server can decrypt the location data, at least when the location data is viewed through the web-based interface. Before displaying the location data via the web-based interface, a notification may be presented to inform the user that a location decryption key is being temporarily shared with the web-based interface server to enable decryption and presentation of the location data. In one implementation, the sharing of the location decryption key may be performed via automatic and temporary authorization of location query permissions for an agent account associated with the web-based interface.

[0063] In one implementation, the wireless accessory 201 can be placed in a light-loss mode. In light-loss mode, a set of future public keys can be generated for the wireless accessory and transmitted to the device locator server 203. Then, if any location data corresponding to a key in this set of future public keys is received, the device locator server 203 can notify the mobile device 102. In one implementation, a detector device sending the location of the wireless accessory in light-loss mode can be guided by the device locator server 203 to relay a message to the wireless accessory 201 notifying it that it is in light-loss mode. A similar mechanism can also be used to relay messages to the wireless accessory 201 that puts it in explicit loss mode. The user can enable explicit loss mode via the device locator UI 204. In explicit loss mode, the wireless accessory 201 cannot pair with another device unless it is unlocked by its owner. Additional examples of paired devices using location services can be found in U.S. Patent Application No. 16 / 543,227, filed August 16, 2019, entitled “A System and Method for Locating Wireless Accessories,” the entire contents of which are incorporated herein by reference.

[0064] Figure 4This is a flowchart illustrating a method for use with the device locator system described herein. Figure 4 A method 400 for pairing a mobile device with a wireless accessory is shown. Aspects of method 400 are also described in... Figure 2 and Figure 3 As shown above, the following description of operations relates to mobile device 102, wireless accessory 201, and device locator server 203.

[0065] like Figure 4 As shown, method 400 includes performing an initial pairing operation with the wireless accessory (402). The initial pairing can be Bluetooth pairing or another type of pairing utilizing other radio technologies. During the initial pairing, the mobile device and the wireless accessory may exchange identifiers, keys, or other credentials that enable wireless data exchange between the mobile device or another electronic device and the wireless accessory. In one embodiment, the initial pairing with the wireless accessory may include the exchange of credentials associated with the wireless protocol for which pairing is performed, thereby allowing all data exchanged wirelessly to have at least a first encryption layer.

[0066] The mobile device can then generate a public / private key pair and one or more additional shared secrets (404). The device can then send the public key and one or more additional shared secrets to the wireless accessory (406). Various key generation techniques can be used. In one embodiment, a variant of ECDH is used to generate a public key pair for encryption. In one embodiment, the one or more additional shared secrets may include a trace-proof secret that enables the wireless accessory to derive a new public key based on the existing public key.

[0067] After generating a public / private key pair and one or more additional shared secrets, the mobile device can store the public / private key pair in a keystore (408). In one embodiment, the keystore is a cloud-based keystore that can be synchronized with other devices associated with the same cloud service account or a series of cloud service accounts to which the mobile device and wireless accessory are associated. The cloud-based keystore allows the wireless accessory to be located by other synchronized devices. The mobile device can then register the wireless accessory with a device management server (410). Registering the wireless accessory with the device management server establishes an association between the wireless accessory and the cloud service account to which the mobile device is associated. In some embodiments, the mobile device can register the wireless accessory and device group 104. Information stored in the device group profile of the device group can also be synchronized between devices bound to a cloud service account (e.g., a user account). The device management server can be synchronized with other cloud-based servers (such as... Figure 2 and Figure 3 The device locator server (203) is associated with this other cloud-based server, which is used to facilitate cloud-based services accessible to mobile devices.

[0068] Figure 5 This is a sequence diagram illustrating a method 500 for presenting a device locator user interface used with the device locator system described herein. A mobile device 102 may launch a device locator application 204 (506). The mobile device 102 may receive a beacon signal (508) including an advertisement from an accessory device 201. The advertisement may be sent before and / or after launching the application (506). In some embodiments, BLE advertisements may be sent before establishing a radio connection with the mobile device 102. In one embodiment, BLE advertisements may be received every two seconds. Signal strength measurements may be determined based on the beacon signal, and the process may begin by searching for at least one accessory device 201 (510) paired with the mobile device 102 (510) through a proximity view of the user interface 204. The proximity view of the user interface of the device locator application 204 may be presented with a visual indicator indicating the proximity of the target accessory device 201. The visual indicator may correspond to a signal strength measurement (512) of the received advertisement from the mobile device 102. Optionally, in some embodiments, a radio connection (514) may be established between the target accessory device 201 and the mobile device 102, thereby allowing the mobile device 102 to request the target accessory device 201 to play a sound. In some embodiments, a radio connection may be established between the target accessory device 201 and the mobile device 102, and the mobile device may request the execution of a ranging process. When the mobile device 102 is within a threshold range (515) of the target accessory device, the ranging process may be performed to determine the distance from the target wireless accessory device 201 and the direction to the target wireless accessory device. A ranging view (516) of the user interface of the device locator application 204 may be presented. The ranging view provides direction and calculated distance relative to the wireless target to aid in locating the accessory device 201 (518). The ranging view may be selectively presented together with a proximity view and / or independently within the user interface 204. The user interface of the device locator application 204 may be presented to ask the user whether the accessory device 201 has been found (520).

[0069] Figure 6A system 600 for performing ranging and attitude estimation to generate a target position estimate for a target device, according to an embodiment, is illustrated. In one embodiment, system 600 includes a wireless controller 630, an IMU 612, and a set of frameworks executing on an application processor of a mobile device 102 as described herein. This set of frameworks may include an augmented reality (AR) framework 610 and a sensor fusion framework 611. The wireless controller 630 may transmit range measurements 601 to interpolator logic 604 within the sensor framework 611. The AR framework 610 may receive sensor measurements 603 from the IMU 612 and generate an attitude estimate 602 for the mobile device 102. The attitude estimate 602 may be transmitted to the interpolator logic 604 within the sensor framework 611. The interpolator logic 604 may process the range measurements 601 and the attitude estimate 602 to generate a measurement with an aligned attitude 605. The interpolator logic 604 may interpolate and align the range measurements 601 and the attitude estimate 602 based on the time determined by the measurements and estimates to generate a measurement with an aligned attitude 605. Measurements with alignment attitude 605 can be provided to estimator logic 606. Estimator logic 606 can then generate a three-dimensional target position estimate 607 for use by the mobile device. The mobile device 102 can use this target position estimate to generate a view of the location environment.

[0070] Interpolator logic 604 and estimator logic 606 may include various algorithmic techniques to overcome anomalous data and sensor interference, thereby enhancing the accuracy of the 3D target position estimation 607. In one embodiment, system 600 may perform visual inertial distance (VIO) by fusing inertial data collected from IMU 612 with images captured via one or more cameras. VIO techniques include analyzing the image sequence to estimate camera motion over the image sequence. Using image-based camera motion estimation and inertial measurements captured within the same time period, the motion of the mobile device in the coordinate system can be estimated, thereby providing the software of system 600 with the ability to determine the positions of both the mobile device 102 (or any detector device) and the target device to be located in the 3D coordinate space.

[0071] In one implementation, target location estimation can take the form of a nonlinear least squares problem that can be solved using nonlinear least squares (NLLS) batch filters. For example, given As in time The range measurement on, and at the same time, given As an attitude estimate relative to the center of the VIO coordinate system, the target position can be determined. x = [xyz] T The target location minimizes the cost function:

[0072]

[0073] The specific attitude estimation used can be based on or relative to the position of the wireless ranging radio antenna on the mobile device. Alternatively, methods based on Kalman filters or space occupancy grids can be applied.

[0074] In one implementation, once the initial target position of the wireless device is determined in three-dimensional coordinate space, the relative target position of the wireless device can be updated using AR data, even if there is no updated range measurement between the wireless device 102 and the accessory device 201. In this scenario, the frequency of the wireless ranging operation can be throttled to reduce the power consumption of the target device 201 and the mobile device 102.

[0075] Figure 7 A system 700 is shown in which proximity enhancement can be implemented for a device locator application 204. One embodiment provides a system 700 for enhancing the functionality of a device locator application 204 when it is in a location environment 708 near an item or target device 201 to be located. The location environment 708 may include a mobile device 102 (e.g., a smartphone, wearable device, tablet, etc.). The mobile device 102 may wirelessly communicate with a wireless device 101 including a radio device 719. The mobile device 102 may also include variations of the radio device 719 in the wireless device 102. The wireless device 201 may also be a mobile device, or it may be a wireless accessory 201 as described herein.

[0076] These devices can communicate via wireless communication signal 705 in the following ways: detecting each other by scanning a wireless channel, transmitting and receiving beacons or beacon frames on a wireless channel, establishing a connection (e.g., by transmitting a connection request), and / or transmitting and receiving packets or frames (which may include requests and / or additional information such as data as a payload). Wireless communication signal 705 can be a carrier signal conforming to wireless communication technologies (such as, but not limited to, Wi-Fi or Bluetooth). In addition to wireless communication, mobile device 102 and wireless device 201 use wireless ranging signal 706 to perform wireless ranging operations. The wireless ranging signal can be, for example, an ultra-wideband signal, which can be used to determine the distance and / or angle between wireless device 201 and mobile device 102 using the techniques described herein. In one embodiment, the data provided by wireless ranging signal 706 can be correlated with other metrics such as the RSSI of wireless communication signal 705. In one embodiment, the communication processors of mobile device 102 and wireless device 201 can fuse multiple types of ranging to provide a unified distance and / or angle estimate based on multiple types of radio data.

[0077] Mobile device 102 may provide a device locator UI 204 that presents a map and / or view of the location environment 708. The map and / or view may present a virtual representation of wireless device 201. The virtual representation simplifies the process of locating wireless device 201 by a user of mobile device 102. For example, using a directional indicator pointing to the detected location of wireless device 201 simplifies the process of finding the approximate location of wireless device 201. The directional indicator may be paired with a map of location environment 708, wherein the map may include a virtual representation of wireless device 201. When mobile device 102 approaches wireless device 201 such that a beacon signal includes wireless communication signal 705 and / or ranging signal 706, a proximity view and / or ranging view of location environment 708 may be presented, which allows the user to identify wireless device 201 when it is hidden from view.

[0078] In one scenario, the wireless device 201 can be hidden by an object. The object can be, for example, a jacket or other clothing item, and the wireless device 201 can be a smartphone or tablet device located in a pocket of the object. The object and the wireless device 201 can each be inside a container such as a backpack, locker, luggage, or another object that reduces the efficiency of audio-based positioning technology that relies on the wireless device 201 to play sounds that can be heard by the user.

[0079] In another scenario, the wireless device 201 can be a beacon peripheral or locator tag attached to an item. To locate the item, a user can use the device locator UI 204 on the mobile device 102 to locate the wireless device 201, thereby enabling the user to pinpoint the item to which the wireless device 201 is attached. The proximity enhancement described herein can be used to enhance the device locator UI 204, allowing the user to quickly determine the location of the item via the wireless device 201 in scenarios where the item may be hidden by a container or another physical object.

[0080] In one implementation, mobile device 102 may establish a secure wireless communication connection (e.g., via wireless communication signal 705) with wireless device 201, and command wireless device 201 to begin a wireless ranging process, for example, using a wireless ranging signal 706, which may be a UWB signal. The wireless ranging signal 706 enables mobile device 102 to receive measurements of the range and angle of wireless device 201. In some implementations, mobile device 102 may request the target wireless device 201 to increase the rate at which it sends advertisements (e.g., from every 2 seconds to every 30 ms) to aid in locating wireless device 201.

[0081] In one implementation, signal strength measurements can be determined based on received advertisements before and / or while a secure wireless communication connection is established, and can help direct users to wireless device 201.

[0082] In the embodiments described herein, wireless ranging can be performed using any standard or proprietary ranging technology, or any combination of standard and / or proprietary ranging technologies. Wireless ranging operations can be performed to determine the distance between devices (e.g., between an initiator and a responder), the direction between devices, or both. For example, time of flight / time of arrival (ToF / ToA) can be determined for one or more messages between devices, which can be used to establish distance measurements. One or more messages can have any format and can be transmitted using any wireless protocol. In some embodiments, the ToF / ToA can be determined using the bidirectional exchange of two or more messages. In some embodiments, one or more messages used to perform ranging can be protected, for example, by encrypting or otherwise protecting at least a portion of their content. Furthermore, in some embodiments, the direction of origin of one or more wireless signals can be determined using techniques such as angle of arrival (AoA). For example, AoA estimation can be performed using multiple receiving elements (e.g., elements of an antenna array) to measure the different times of arrival (TDOA) and / or different phases (PDOA) of the signal. Additionally or alternatively, in some embodiments, directionality can be determined by measuring Doppler drift to establish the frequency difference of arrival (FDOA). Wireless ranging techniques can be applied alone or in combination to perform a single ranging operation. Furthermore, wireless ranging techniques can be applied alone or in combination to perform continuous ranging operations, such as continuous or intermittent ranging, and the measurement history can be captured and used in operations based on range and / or direction.

[0083] Incoming measurements can be processed to reduce measurement noise and stabilize the estimated location of lost items. In one embodiment, mobile device 102 may use sensor fusion techniques that rely on multiple measurement streams to obtain the best estimate of the device's location. For example, incoming range and / or angle measurements may be fused with position and orientation measurements of mobile device 102. Visual inertial measurement (VIO) may be used to collect position and orientation measurements, which uses an IMU and computer vision using camera data to determine the device's position and orientation. VIO data enables software on mobile device 102 to know the device's movement and orientation. Combining VIO data with range and / or angle measurements determined via wireless ranging signal 706 makes it possible to estimate the most probable location of the wireless device and allow the user to point to wireless device 201. Additionally, combining VIO data with signal strength measurements via wireless communication signal 705 can help guide the user to wireless device 201 through proximity view and / or combined ranging view. VIO data may be particularly useful in scenarios where one or more of the range or angle measurements may be inaccurate due to multipath signal propagation effects.

[0084] The algorithm used to determine the location of a lost item is accessible via the device locator UI 204. When a user intends to locate an item, the user can select the item from the list of registered devices and choose the "Find" button presented via the device locator UI 204. In various implementations, multiple user interfaces may be presented. Some interfaces resemble a two-dimensional compass-like view, where two-dimensional arrows guide the user to the target item or device to be found. In addition to presenting the trajectory the user has already taken in their search (where proximity indicators are presented along the path taken), the device locator UI 204's proximity view and / or a combination of proximity and distance views can provide arrows to guide the user.

[0085] Figure 8 This is a flowchart illustrating a method 800 for a device locator user interface 204 in one embodiment. One or more inertial displacement measurements (802) can be determined using received inertial sensor data and received camera sensor data. One or more inertial displacement measurements are determined from the origin in any 2D or 3D coordinate system using VIO. In one embodiment, the origin (e.g., ([0,0,0] x, y, and z axes)) is established when the user selects to initiate the process of finding the target wireless device 201. The target wireless device can be as shown in Figure 1 to... Figure 3 The wireless accessory device 201 and / or wireless accessory device 101 shown. A trajectory (804) can be determined based on one or more collected inertial displacement measurements. Displacement measurements from the origin are collected as the user moves within the location environment, and these displacement measurements can be used to plot a trajectory in the user interface 204 to represent the path taken by the user within the location environment. The trajectory can be presented within the user interface to alert the user to the path already taken, so the user can use this information to try alternative routes or maintain the route used for positioning devices.

[0086] A beacon signal (806) can be received from the target wireless device 201. The mobile device 102 may receive the beacon signal from the target wireless device 201 via one or more advertisements before a radio connection is established and / or when the target wireless device 201 has already established a radio connection with the mobile device 102. Two-way communication may not be established between the mobile device 102 and the target wireless device 201, but proximity to the target wireless device 201 may be determined using advertisements received by the mobile device 102. A signal strength value (806) can be determined based on the received beacon signal. The signal strength value may be an average of signal strength measurements determined based on advertisements in the beacon signal received while the user is positioned in an area on the trajectory. In some embodiments, an average signal strength value obtained over an area (e.g., the opposite of that obtained over time) may reduce the effects of attenuation. Attenuation occurs when beacon signals received at the radio receiver on the mobile device 102 simultaneously cause a decrease in signal strength value. Alternatively, an average signal strength value obtained over time may be used. The proximity value to the target wireless device 201 can be estimated (808) based on the signal strength value of the location along the trajectory in the area in which the signal strength value has been received.

[0087] An indicator (810) can be displayed along a trajectory to show the proximity value to the target wireless device. A defined category of signal strength values ​​can be determined for each set of averaged signal strength values ​​along the trajectory region, and an indicator of the proximity value can be presented according to the defined category. For example, a range of values ​​can be defined for each of a set of buckets to specify near, nearby, and far categories and / or any other category to provide the user with information about proximity to the wireless accessory device 201. The visual indicator can be any type of user interface element, including but not limited to: markers, specific colors or gradients, shapes, or any other visual indication that can be displayed on the user interface.

[0088] The user interface can optionally display a ranging view, a signal strength proximity view, and / or a combination of ranging and signal strength proximity views (e.g., ...). Figure 21 (as shown in Figure 812). For example, the ranging view can only be shown if a radio connection has been established.

[0089] Distance view

[0090] Various algorithmic methods can be used to enhance the performance and accuracy of distance, orientation, and range calculations. In one scenario, when a device traces a trajectory through space toward a target, it may encounter multipath propagation effects when ranging relative to the target. Multipath propagation is the phenomenon that causes radio signals to reach the receiving antenna via two or more paths. Multipath propagation can occur if the line of sight to the target is obstructed by obstacles in the environment, the antenna pattern severely attenuates the line-of-sight signal, or the user themselves block the signal. As the mobile device follows a trajectory through space toward the target, the measured range relative to the target, along with the distance traveled along the trajectory, can evolve over time. During this evolution, anomalies and interference may occur.

[0091] Sometimes, the measurement range determined from sensor data can appear to skip distances greater than the distance traveled in space. Sudden increases in measurement ranges that differ from the traveled distance over a period of time can be used to determine when multipath conditions have changed. When the change in range is positive and greater than the traveled distance, the measurement bias due to multipathing may have increased. Similarly, when the change in range is negative and greater in magnitude than the traveled distance, the multipath component is likely present previously and has actually decreased. When a decrease in multipath propagation is detected, the uncertainty of previous measurements can be increased to account for potential multipath interference. In one implementation, previous measurements considered to be multipath measurements can be removed from the measurement history by setting the weights associated with these measurements to zero. When an increase in multipath interference is detected, measurements received after the detected increase can be similarly deweighted, or those measurements can be rejected.

[0092] In one implementation, range measurements may be deweighted over time. When range measurements are collected along the device's trajectory, the solution for the target location becomes overdetermined. Range measurements may be deweighted with measurement age and / or with the accumulation of VIO errors. In the case of moving targets, the analysis of range measurements as a function of time may be particularly important because the time series of range measurements can be used to determine angle measurements of the device, independent of angle measurements associated with any given ranging operation. Therefore, the weights associated with older measurements can be attenuated based on a time function.

[0093] In one implementation, the range measurements can be deweighted based on the measured and / or estimated distances to the target. As the measurements indicate further range, the likelihood of multipath interference increases, and the measurements become less useful. When a solution is formed, an estimated distance to each location that formed the measurement can be calculated. Based on this distance, the measurements can be reweighted. Furthermore, the measurement itself constitutes the range and can be used to infer measurement uncertainties.

[0094] In one implementation, when only the range solution indicates that the target is in the device's field of view, the angle of arrival measurement can be used to further refine the target's location. The location solution for the device can be formed using only range measurements (e.g., without angle measurements). Given a location solution, the probability of each angle measurement taken from the mobile device when the target is in the device's field of view can be calculated. If the probability meets a threshold, the angle measurement can be used to form a new location solution. Using angle measurements can resolve ambiguities that may exist in the range solution, especially along the z-axis, because the device trajectory is mostly in a single plane (e.g., the xy plane).

[0095] After using measurements to arrive at a location solution, the measurement errors of each contributing measurement can be calculated. In an overdetermined solution, the measurement errors will be non-zero and provide some indication of the quality of the positioning. If large errors are found, the confidence level of the positioning can be reduced. A confidence threshold can be used to determine whether the location should be displayed to the user. If slight differences are found among all measurements, errors can be used to increase the uncertainty of the location.

[0096] Figure 9 A method 900 for implementing a proximity-enhanced user interface for a device locator application is illustrated. Method 900 includes operations such as fusing range measurements determined by a wireless controller with an attitude estimate determined based on sensor data from an IMU to generate a measurement of attitude alignment and a three-dimensional or two-dimensional target position estimate. In some embodiments, the three-dimensional target position estimate can be used to implement an AR view including the location environment of the target device or item to be located. The range measurements determined by the wireless controller and the estimate determined based on sensor data from the IMU can also be used to determine the two-dimensional target position estimate. Method 900 can be performed by a mobile device (e.g., mobile device 102) as described herein to locate a target wireless accessory device (e.g., wireless device 101) or a target wireless accessory device (e.g., wireless accessory device 201), or an item associated with wireless accessory device 201, such as an item tagged with a beacon or locator tag.

[0097] In one implementation, method 900 includes (901) optionally receiving the geographic location of a target wireless accessory device at mobile device 102. The geographic location of the target wireless accessory device 201 may be a location determined by mobile device 102 that has access to a user account of mobile device 102, or a location received from device locator server 203.

[0098] Then, mobile device 102 can present an indicator (902) of the target wireless accessory device 201 on the interface of device locator application 204 executed by mobile device 102. The indicator may be a map indicator showing the location of the target wireless accessory device on a map of the nearby environment. For example, a map with map indicators may be presented, such as... Figures 17 to 20 As shown. The distance measurement to the target wireless accessory device 201 and the range measurement relative to the target wireless device can be presented on a map along with a map indicator showing the location of the target wireless accessory device. The user of the mobile device 102 can then move toward the location of the target wireless accessory device 201.

[0099] When mobile device 102 is within a threshold range of target wireless accessory device 201, wireless accessory device 201 may use one or more wireless ranging operations to determine range and / or orientation relative to target wireless accessory device 201 (903). The one or more ranging operations may include continuous or periodic ultra-wideband ranging operations. Ultra-wideband ranging operations may be performed in conjunction with other range determination techniques, such as RSSI-based distance determination. In one embodiment, the ultra-wideband ranging operation is a secure ranging operation in which ranging packet switching is encrypted. In one embodiment, both range and orientation determination may be based on one or more ranging operations. In one embodiment, range-only determination may be performed, and orientation may be determined based on analysis of multiple range measurements. Where both range and angle measurements can be determined based on analysis of instantaneous wireless ranging signals, range measurement history may be used to improve the accuracy of the determined angle measurements.

[0100] The mobile device 102 can then determine its attitude (904) via sensor data received from an inertial measurement unit within the mobile device 102. The inertial measurement unit data can be processed by the system to determine the attitude of the mobile device 102. The attitude data can be used to further enhance orientation determination for the target wireless accessory device 201 based on a wireless ranging sensor.

[0101] Then, mobile device 102 can determine the location estimate of the target wireless accessory device 201 based on range, orientation, and attitude (905). In one embodiment, the specific range, orientation, and attitude determined for the target wireless accessory device 201 and mobile device 102 are determined based on the fusion of wireless ranging system data from mobile device 102 and system sensor data. The location estimate can be the position relative to mobile device 102 or the absolute position in 3D and / or 2D coordinate space. Then, mobile device 102 can generate and display a device locator UI 204 (906) that includes the location estimate of wireless accessory device 201. The device locator UI can be, for example, Figure 21The ranging view shown has distance 2116 and direction information 2108 relative to the wireless accessory device 201. For example... Figure 21 The proximity view shown also provides a trajectory in 2D coordinate space with visual indicators of proximity to the target wireless accessory device 201 (e.g., determined by signal strength measurements).

[0102] Figure 10 A method 1000 for determining the location of a wireless accessory via a device locator server is shown. Figure 11 An additional method 1100 for determining the location of a wireless accessory via a device locator server is shown. (e.g.) Figure 10 As shown, method 1000 includes an electronic device activating a device locator UI (1001). In response to activating the device locator UI, an electronic device, such as the mobile device described herein, or another electronic device associated with the same cloud service account as the mobile electronic device, may perform an operation to generate a set of public keys included in a beacon signal broadcast by the wireless accessory during a first time period (1002). The first time period may be, for example, a previous 24 hours. The electronic device knows the frequency at which the wireless accessory generates new public keys and, using a shared secret generated by the wireless accessory, can generate a set of public keys corresponding to the keys generated by the wireless accessory during the first time period. The electronic device may then send the set of public keys to send location data corresponding to the set of public keys within a request to the device locator server (1003). In one embodiment, the location data sent by the server in response to the request is encrypted using the public key transmitted as a beacon identifier of the wireless accessory. The electronic device may use a private key generated during the initial pairing with the wireless accessory to decrypt the encrypted location data received by the server (1004). The electronic device may then process the location data to determine the highest probability location of the wireless accessory (1005). In one implementation, location data may include data from accessory device 201 in the device group.

[0103] Processing location data can include a variety of different operations. In one embodiment, the location data includes latitude and longitude information and a timestamp determining the location. Electronic devices can perform triangulation based on the timestamp and remove noise or anomalous locations. In one embodiment, the location data specifies the location of a detector device that detected a beacon. The location data may also include UWB ranging information and / or RSSI information of the beacon detected by the detector device. Electronic devices can analyze the UWB ranging information and / or RSSI information in the context of the device's location to obtain a more accurate location of the wireless accessory. Data that can be transmitted by the detector device and used for location processing is... Figure 12 It is shown in the figure and described below.

[0104] like Figure 11As shown, method 11 includes operations that can be performed if the device locator server does not have location data to provide to the electronic device in response to a request. In the case of a group of devices, the electronic device (e.g., mobile device 102) can provide location data about the devices in the group. The electronic device can generate a first set of public keys (1101) included in a beacon signal broadcast by a wireless accessory during a first time period. The first time period can be, for example, 24 hours, but other initial search time periods can be used. The electronic device can perform subsequent operations to request the device locator server to send location data corresponding to the first set of public keys (1102). If the data is returned by the server (1103, "Yes"), the electronic device can use the private key corresponding to this set of public keys to decrypt the location data received from the server (box 1109).

[0105] If the server does not return data (1103, "No"), the electronic device may generate a second public key included in the beacon signal broadcast by the wireless accessory during a second time period (1104). The second time period may be 24, 48, or other hours prior to the first time period. The electronic device may then request the device locator server to send data corresponding to the second public key (1105). If the server returns data in response to the request (1106, "Yes"), method 1100 may proceed to box 1109, whereby the electronic device decrypts the received data. If the server does not return data (1106, "No"), or the server sends a reply indicating that the data is unavailable, method 1100 includes: the electronic device may extend the search time by continuously requesting earlier time periods until a maximum time period is reached (1107).

[0106] Figure 12 This is a flowchart illustrating method 1200 for broadcasting a signal beacon at a wireless location according to an embodiment. Aspects of method 1200 are also... Figure 2 and Figure 3As shown in the diagram. Method 1200 includes deriving a public key from a wireless accessory (box 1202). The public key can be derived based on a shared secret and a timestamp determined by a clock or timing device of the wireless accessory. Optionally, it is determined whether the wireless accessory is part of a device group (1204). If the wireless accessory is part of a device group, status information and / or verifiable information of other accessory devices 201 in the device group are provided in the beacon signal (1206). The wireless accessory may indicate status information and / or verifiable information, such as whether any other wireless accessory in the device group is near, connected (physically or wirelessly), and / or any other information about other wireless accessories in the device group 105. In one embodiment, a set of bits included in the beacon signal may represent each accessory in the device group, and setting a Boolean value (e.g., true (1) or false (0)) may indicate whether the corresponding accessory is near and / or connected to the accessory device that sent the beacon signal. Alternatively, if the wireless accessory is not part of a device group (1204), no information is provided on the device group. The wireless accessory can then transmit beacon signals at a first frequency, wherein the beacon signals include the public key (1208). The first frequency can be varied, and in one implementation, it is one beacon every two seconds.

[0107] After transmitting the beacon signal, the wireless accessory can listen for a response from the owner device (1210). If the wireless accessory receives a response from the owner device (1210, "Yes"), it can enter a near-owner state (1212) and begin transmitting the beacon signal at a second lower frequency (1216). If the wireless accessory does not receive a response from the owner device (1210, "No"), it can continue transmitting the beacon at a first frequency (1214).

[0108] Method 1200 further includes rotating the public key once every M minutes when transmitting beacons, where the value of M may vary between implementations and / or based on device state. Based on timer expiration, a counter, or other mechanism, the wireless attachment can determine whether it has entered a new key period (1218). Even if the wireless attachment has not yet entered a new key period (1218, "No"), it can continue transmitting beacons using the current public key (1222). When the wireless attachment detects that it has entered a new key period (1218, "Yes"), it can derive a new public key using the current timestamp (1220). In one implementation, the new public key can be derived using an existing public key, a timestamp, and an anti-tracking secret.

[0109] Figures 13 to 14 Operation of method 1300, which can be performed by a detector device according to the embodiment described herein, is illustrated. Aspects of method 1300 are also... Figure 2 and Figure 3 As shown in the image.

[0110] like Figure 13 As shown, method 1300 includes the detector device performing periodic beacon scanning using a wireless baseband processor when the application processor of the detector device is in a low-power mode (1301). While beacon scanning can also be performed when the application processor is active, it can be performed by the wireless processor and radio receiver as low-power operations when the detector device is idle, inactive, or otherwise in a low-power state. The detector device can store a timestamp and a beacon identifier in a beacon scanning buffer for use with any beacon data received by the detector device (1302). In one embodiment, the beacon identifier is a public key generated by the wireless device based on the timestamp and a shared secret generated using the owner's mobile device.

[0111] Method 1300 also includes the detector device performing periodic Wi-Fi scans using the wireless processor when the application processor is in a low-power mode (1303). While Wi-Fi scanning can also be performed when the application processor is active, when the detector device is idle, inactive, or otherwise in a low-power state, the Wi-Fi scan can be performed by the wireless processor and radio receiver as a low-power operation. The detector device can then store the Wi-Fi Service Set Identifier (SSID) and scan timestamp in a Wi-Fi scan buffer on the detector device (1304).

[0112] In one implementation, the Wi-Fi scan buffer is a rolling buffer that stores the most recently detected SSID while overwriting earlier detected SSIDs. In another implementation, the beacon scan buffer may be a fixed-size buffer with space for a predetermined number of entries. When the beacon scan buffer is full, the detector device may wake up the application processor (1305) and associate those beacon scans with the most recently detected SSIDs in the Wi-Fi scan buffer. If a beacon indicates that a beacon signal has been received from the device group (1306), a set of device locations corresponding to the received beacon may be performed based on the Wi-Fi scan buffer data for the beacon signal from the device group (1310). For example, if a beacon signal is received from a first accessory device from device group 105, and the beacon signal includes information about a set of proximity devices physically or wirelessly connected to the first accessory device, the last known location of the first accessory device may be attributed to / stored in device group 105 for each of the first accessory device and proximity devices. Alternatively, this association may enable the detector device to determine a set of device locations corresponding to the received beacon based on the Wi-Fi scan buffer data (1308).

[0113] Method 1300 in Figure 14 The process continues, and includes: if other location data is available, the detector device correlates the device location from the Wi-Fi scan buffer data with other location data (1407) to generate a refined device location. If a refined device location is generated, the detector device may optionally combine beacon data with the refined device location (1408). The detector device may also add signal strength (RSSI) and / or ranging data to the location data (1409). When the detector device receives a beacon signal, it may collect signal strength and ranging data (e.g., UWB ranging data). The detector device may then encrypt the location data using one or more public keys received within the beacon data (1410). The signal and ranging data may be encrypted along with the location data, or may be transmitted unencrypted along with the encrypted location data. The detector device may enqueue the encrypted location data for transmission to a device locator server (1411). The device locator server may be one of multiple cloud service servers, typically communicating in batches and throttling. A batch of encrypted data can be collected and placed in a transmission queue until the transmission interval is reached, during which the detector device can transmit the data to the cloud service server (1412).

[0114] Figure 15 The acquisition of signal and ranging data performed by a detector device according to an embodiment is illustrated. In one embodiment, detector device 202 can collect signal strength information (e.g., RSSI 1504A-1504N) for beacon signals 301 received from wireless accessory 201 across multiple locations 1502A-1502N. Detector device 202 may also represent multiple detector devices, such as... Figure 3 A group of detector devices 303 are configured, each detecting beacon signals at a different location. Each detector device 202 can transmit different locations and signal strengths, and the location and signal strength data received from multiple detector devices are aggregated by a device locator server. In one embodiment, where both the detector devices and the wireless device include UWB radio equipment, UWB ranging 1506 can be performed if the detector devices and the wireless device are within UWB transmission range. The UWB ranging and signal strength data can be transmitted to the device locator server along with the location data of the detector devices.

[0115] The owner device can retrieve RSSI and / or UWB information, as well as location data, from the device locator server. In one embodiment, the location data is provided in the form of latitude and longitude information and a timestamp determining the location. The owner device can then use the location data, timestamp, and signal information to triangulate the most probable location of the wireless accessory 201.

[0116] Figures 16 to 21 The device locator UI 204 according to the implementation scheme is shown. Figure 16 A first graphical user interface of device locator UI 204 according to an embodiment is shown, which displays notifications for various wireless accessories of the user. Device locator UI 204 enables separate notifications 1602 to be displayed on the main screen 1601 of electronic device 1600. Figure 17 A second graphical user interface for the device locator UI 204 according to the embodiment is shown, which enables viewing of left-behind devices on a map, adding trusted locations, or requesting to stop notifications for items.

[0117] Figure 18 A third graphical user interface of the device locator UI 204 according to the embodiment is shown, which enables the location of the accessory device 201 (including devices in the device group) on a map. Figure 19 A fourth graphical user interface (GUI) of device locator UI 204 according to one embodiment is shown, which enables a wireless accessory to be set to lost mode or to notify when it is found. Device locator UI 204 can be displayed on an electronic device, which can be mobile device 102, or any other type of electronic device described herein. Figure 20 A fifth graphical user interface of the device locator UI 204 according to the embodiment is shown, which enables wireless accessories to add trusted locations. Figure 21 A sixth graphical user interface of the device locator UI 203 according to the embodiment is shown, which uses signal strength measurement to achieve proximity view.

[0118] like Figure 17 As shown, the device locator UI 204 can present a unified graphical interface on the electronic device 1700, through which various types of devices and accessories can be located, including wireless devices with network or cellular access and wireless accessories without local network access. The device locator UI 204 may include a map 1704 with markers 1705 indicating the current or last known location of the wireless accessory device 201. Marker 1705 can be an icon, image, graphic, or any other user interface element that identifies the accessory and conveys its location. Optional element 1707 in the device locator UI 204 can present an option to request that a specific item not be notified. Optional element 1706 in the device locator UI 204 can present a description or name of the wireless device or accessory and can show the estimated distance between the wireless device or accessory and the current location of the electronic device 1800, such as... Figure 18 As shown.

[0119] like Figure 18As shown, the device locator UI 204 can present a user interface that allows the wireless accessory to view items 1803 and 1805 and their distance from the electronic device 1800. In one embodiment, a third user interface can respond to selection. Figure 17 The optional element 1706 shown is displayed. The third user interface may present user interface elements 1802 that represent and / or describe the wireless accessory under consideration, as well as a map 1801 and markers 1802 that display the current or last known location of the wireless accessory.

[0120] like Figure 19 As shown, the device locator UI 204 may present a fourth graphical user interface that allows the wireless accessory to be set to a lost mode. In one embodiment, when the wireless accessory cannot be located via the device locator UI 204, the map 1901 will not display a marker indicating the location of the accessory. The device locator UI 204 may present a user interface element 1904 representing and / or describing the wireless accessory under consideration and a set of optional user interface elements. An optional user interface element 1906 may present options for notifying the user when the accessory is discovered. When notification upon discovery is enabled, in one embodiment, the wireless accessory may be placed in a light lost mode. The electronic device associated with the device locator UI 204 may generate a set of public keys, which the wireless accessory will broadcast along with a beacon signal during a future time period (e.g., the next 24 hours, the next 48 hours, etc.). If a detector device detects a signal using one of the future keys, the device locator server may notify one or more electronic devices associated with that user.

[0121] Another optional user interface element 1907 allows the wireless accessory to be placed in explicit lost mode. When explicitly placed in lost mode, the wireless accessory cannot be paired with other devices until it is unlocked by the user or owner who placed the device in lost mode. When a request to place the wireless accessory in lost mode is sent, the requesting user may be prompted to enter authentication information to ensure that the requesting user is authorized to request to initiate lost mode on the lost accessory. Authentication information may include a username or password associated with the user's account, such as the user's, the electronic device's, and the cloud service account associated with the wireless accessory. Authentication information may also include biometric information, such as fingerprint or facial recognition data.

[0122] In one implementation, a message and contact information provided by the requesting user can be displayed on the user's device to indicate how the person who found the lost wireless accessory can contact the requesting user. In another implementation, the message and contact information can be displayed when another user attempts to pair another electronic device with the lost accessory.

[0123] like Figure 20As shown, the device locator UI 204 can present a fifth graphical user interface in the electronic device 2000, which enables the designation of a known location 2006 shown on a map having 2004 to become a reliable location upon selection of an optional element 2003. In one embodiment, the fifth user interface can respond to selection Figure 17 The optional element 1703 shown is displayed. The device locator UI 204 may present user interface elements 2005 that represent and / or describe the wireless accessory under consideration.

[0124] like Figure 21 As shown, the device locator UI 204 can use signal strength measurements to present a sixth graphical user interface with a proximity view in the electronic device 2100. The proximity view 2124 for finding “Tommy’s AirPods” has indicators 2122, 2126, 2130, 2132, 2134, and 2128 at various locations along a trajectory 2138 within the user interface 204. Each indicator can be a user interface element that represents proximity to the target wireless accessory device 201 through the size, color, shape, color gradient, shadow, pattern, and / or any other technology of a visual indicator within the user interface. Indicators can be displayed along the trajectory 2138 as the user moves through the location environment. In proximity view 2104, for example, indicator 2106 is the closest to the target wireless accessory device 201 along the trajectory 2138 that the user has taken to find the target wireless accessory device 201, as indicated by a darker color and / or a larger size compared to other indicators. In some embodiments, the proximity view of user interface 204 may use ranging measurements to present ranging information and display arrows 2128 indicating the direction of the target wireless accessory device 201 and the distance 2136 from the target wireless accessory device 201. In other embodiments, the ranging information may be presented in a separate user interface without proximity indicators 2122, 2126, 2130, 2132, and 2134, such as Figure 1B As shown. In other embodiments, the trajectory may be represented by a grid, such as a hexagonal grid, wherein the visual indicator consists of regions of the grid specified by color, gradient, shading and / or any other markings along the trajectory drawn on the grid to represent approximate values ​​for the signal strength observed in the respective regions.

[0125] Figure 22 This is a block diagram illustrating an exemplary API architecture that can be used in some embodiments of the present invention. For example... Figure 22As shown, API architecture 2200 includes API implementation component 2210 (e.g., operating system, library, device driver, API, application, software, or other module) that implements API 2220. API 2220 specifies one or more functions, methods, classes, objects, protocols, data structures, formats, and / or other characteristics of the API implementation component that can be used by API calling component 2230. API 2220 may specify at least one calling convention that specifies how functions in the API implementation component receive parameters from the API calling component and how functions return results to the API calling component. API calling component 2230 (e.g., operating system, library, device driver, API, application, software, or other module) makes API calls through API 2220 to access and use the characteristics of API implementation component 2210 specified by API 2220. API implementation component 2210 may return values ​​to API calling component 2230 through API 2220 in response to API calls.

[0126] It should be understood that API implementation component 2210 may include additional functions, methods, classes, data structures, and / or other features not specified through API 2220 and not available to API invocation component 2230. It should be understood that API invocation component 2230 may be on the same system as API implementation component 2210, or may be remotely located and accessed via a network using API 2220. Although Figure 22 The diagram shows a single API call component 2230 interacting with API 2220, but it should be understood that other API call components written in a different language (or the same language) than API call component 2230 can use API 2220.

[0127] API implementation component 2210, API 2220, and API calling component 2230 may be stored in machine-readable media, including any mechanism for storing information in a machine-readable form (e.g., a computer or other data processing system). For example, machine-readable media include disks, optical disks, random access memory, read-only memory, flash memory devices, etc.

[0128] Figure 23 This is a block diagram of a device architecture 2000 for a mobile or embedded device according to an implementation scheme. Device architecture 2300 includes a memory interface 2302, a processing system 2304 including one or more data processors, an image processor and / or graphics processing unit, and a peripheral device interface 2306. Various components can be coupled via one or more communication buses or signal lines. These components can be individual logic components or devices or can be integrated into one or more integrated circuits, such as system-on-a-chip (SoC) integrated circuits.

[0129] The memory interface 2302 can be coupled to the memory 2350, which may include high-speed random access memory such as static random access memory (SRAM or dynamic random access memory (DRAM)) and / or non-volatile memory such as, but not limited to, flash memory (e.g., NAND flash, NOR flash, etc.).

[0130] Sensors, devices, and subsystems can be coupled to peripheral interface 2306 to facilitate a variety of functions. For example, motion sensor 2310, light sensor 2312, and proximity sensor 2314 can be coupled to peripheral interface 2306 to facilitate mobile device functionality. One or more biometric sensors 2315 may also be present, such as a fingerprint scanner for fingerprint recognition or an image sensor for facial recognition. Other sensors 2316 may also be connected to peripheral interface 2306, such as positioning systems (e.g., GPS receivers), temperature sensors, or other sensing devices to facilitate related functions. Camera subsystem 2320 and optical sensor 2322 (e.g., charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) optical sensor) can be used to facilitate camera functions such as recording photos and video clips.

[0131] Communication functionality can be facilitated by one or more wireless communication subsystems 2324, which may include radio frequency receivers and transmitters and / or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the wireless communication subsystem 2324 may depend on the communication network through which the mobile device intends to operate. For example, a mobile device including the illustrated device architecture 2300 may include a wireless communication subsystem 2324 designed to operate over a GSM network, CDMA network, LTE network, Wi-Fi network, Bluetooth network, or any other wireless network. Specifically, the wireless communication subsystem 2324 may provide a communication mechanism in which a media playback application can retrieve resources from a remote media server or retrieve scheduled events from a remote calendar or event server.

[0132] The audio subsystem 2326 can be coupled to the speaker 2328 and the microphone 2330 to facilitate voice-enabled functions such as voice recognition, voice copying, digital recording, and telephone functionality. In the smart media device described herein, the audio subsystem 2326 can be a high-quality audio system that includes support for virtual surround sound.

[0133] I / O subsystem 2340 may include touchscreen controller 2342 and / or other input controller 2345. For computing devices including display devices, touchscreen controller 2342 may be coupled to touch-sensitive display system 2346 (e.g., a touchscreen). Touch-sensitive display system 2346 and touchscreen controller 2342 may detect contact and motion or pressure using, for example, any of a variety of touch and pressure sensing technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with touch-sensitive display system 2346. Display output of touch-sensitive display system 2346 may be generated by display controller 2343. In one embodiment, display controller 2343 may provide frame data to touch-sensitive display system 2346 at a variable frame rate.

[0134] In one embodiment, a sensor controller 2344 is included to monitor, control, and / or process data received from one or more motion sensors 2310, light sensors 2312, proximity sensors 2314, or other sensors 2316. The sensor controller 2344 may include logic to interpret the sensor data to determine the occurrence of one of a plurality of motion events or activities by analyzing the sensor data from the sensors.

[0135] In one implementation, the I / O subsystem 2340 includes other input controllers 2345 that can be coupled to other input / control devices 2348, such as one or more buttons, rocker switches, thumbwheels, infrared ports, USB ports, and / or pointer devices such as styluses, or up / down buttons for volume controls of control devices such as speakers 2328 and / or microphones 2330.

[0136] In one implementation, memory 2350 coupled to memory interface 2302 may store instructions for operating system 2352, including POSIX-compliant and incompatible operating systems or embedded operating systems. Operating system 2352 may include instructions for handling basic system services and for performing hardware-related tasks. In some specific implementations, operating system 2352 may be a kernel.

[0137] The memory 2350 may also store communication instructions 2354 to facilitate communication with one or more additional devices, one or more computers, and / or one or more servers, such as retrieving web resources from a remote web server. The memory 2350 may also include user interface instructions 2356, including graphical user interface instructions to facilitate graphical user interface processing.

[0138] In addition, memory 2350 may store sensor processing instructions 2358 to facilitate sensor-related processing and functions; telephone instructions 2360 to facilitate telephone-related processes and functions; instant messaging instructions 2362 to facilitate electronic messaging-related processes and functions; web browser instructions 2364 to facilitate web browsing-related processes and functions; media processing instructions 2366 to facilitate media processing-related processes and functions; location service instructions including GPS and / or navigation instructions 2368 and Wi-Fi-based location instructions to facilitate location-based functionality; camera instructions 2370 to facilitate camera-related processes and functions; and / or other software instructions 2372 to facilitate other processes and functions, such as security processes and functions, and system-related processes and functions. Memory 2350 may also store other software instructions, such as web video instructions to facilitate web video-related processes and functions; and / or web shopping instructions to facilitate online shopping-related processes and functions. In some embodiments, media processing instructions 2366 are divided into audio processing instructions and video processing instructions, respectively used to facilitate audio processing-related processes and functions and video processing-related processes and functions. Mobile device identifiers, such as International Mobile Equipment Identity (IMEI) 2374 or similar hardware identifiers, may also be stored in memory 2350.

[0139] Each of the instructions and applications identified above may correspond to a set of instructions for performing one or more of the functions described above. These instructions do not need to be implemented as a separate software program, process, or module. Memory 2350 may include additional instructions or fewer instructions. Furthermore, various functions may be performed in hardware and / or software, including in one or more signal processing and / or application-specific integrated circuits.

[0140] Figure 24 This is a block diagram of a computing system 2400 according to an implementation scheme. The computer system 2400 shown is intended to represent one or more specific implementations of a series of computing systems (wired or wireless), including, for example, desktop computer systems, laptop computer systems, tablet computer systems, cellular phones, personal digital assistants (PDAs) including cellular-enabled PDAs, set-top boxes, entertainment systems or other consumer electronic devices, smart electrical devices, or smart media playback devices. Alternative computing systems may include more, fewer, and / or different components. The computing system 2400 can be used to provide computing devices and / or server devices that may be connected to the computing devices.

[0141] Computer system 2400 includes a bus 2435 or other communication device for transmitting information, and a processor 2410 coupled to the bus 2435 for processing information. Although computing system 2400 is illustrated as having a single processor, computing system 2400 may include multiple processors and / or coprocessors. Computing system 2400 may also include memory 2420 in the form of random access memory (RAM) or other dynamic storage device coupled to bus 2435. Memory 2420 may store information and instructions executable by processor 2410. During the execution of instructions by processor 2410, memory 2420 may also be main memory for storing temporary variables or other intermediate information.

[0142] The computing system 2400 may also include a read-only memory (ROM) 2430 and / or other data storage device 2440 coupled to the bus 2435 for storing information and instructions for the processor 2410. The data storage device 2440 may be or include various storage devices, such as flash memory devices, disks, or optical disks, and may be coupled to the computing system 2400 via the bus 2435 or via a remote peripheral device interface.

[0143] The computing system 2400 can also be coupled to a display device 2450 via a bus 2435 to display information to a user. The computing system 2400 may also include a numeric-alphanumeric input device 2460, which includes numeric keys and other keys, and can be coupled to the bus 2435 to send information and command options to the processor 2410. Another user input device includes a cursor control device 2470, such as a touchpad, mouse, trackball, or cursor arrow keys, for transmitting directional information and command selections to the processor 2410 and controlling cursor movement on the display device 2450. The computing system 2400 can also receive user input from a communicatively coupled remote device via one or more network interfaces 2480.

[0144] The computing system 2400 may also include one or more network interfaces 2480 to provide access to a network such as a local area network (LAN). The network interface 2480 may include, for example, a wireless network interface with an antenna 2485, which may represent one or more antennas. The computing system 2400 may include multiple wireless network interfaces, such as Wi-Fi and Bluetooth. ® A combination of near field communication (NFC) and / or cellular telephone interfaces. Network interface 2480 may also include, for example, a wired network interface for communicating with remote devices via network cable 2487, which may be, for example, an Ethernet cable, coaxial cable, fiber optic cable, serial cable, or parallel cable.

[0145] In one implementation, network interface 2480 may provide access to a local area network (LAN) for example, via a wireless standard compliant with the IEEE 802.11 standard, and / or wireless network interface may provide access to a personal area network (PAN) for example, via a Bluetooth standard compliant with the standard. Other wireless network interfaces and / or protocols may also be supported. In addition to or instead of communicating via wireless LAN standards, network interface 2480 may provide wireless communication using, for example, Time Division Multiple Access (TDMA), Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Long Term Evolution (LTE), and / or any other type of wireless communication protocol.

[0146] The computing system 2400 may also include one or more energy sources 2405 and one or more energy measurement systems 2445. The energy source 2405 may include an AC / DC adapter coupled to an external power source, one or more batteries, one or more charge storage devices, a USB charger, or other energy sources. The energy measurement system includes at least one voltage or current measuring device capable of measuring the energy consumed by the computing system 2400 over a predetermined time period. Furthermore, one or more energy measurement systems may be included to measure, for example, the energy consumed by a display device, cooling subsystem, Wi-Fi subsystem, or other commonly used or high-energy-consuming subsystems.

[0147] Although the embodiments are described in language specific to structural features and / or methodological behavior, it should be understood that the appended claims are not necessarily limited to the specific features or behaviors described. Rather, the specific features and behaviors disclosed should be understood as embodiments of the illustrative claims.

Claims

1. An electronic device, comprising: Communication wireless equipment; Wireless controller, including ranging sensors; Display devices; The memory stores instructions; as well as One or more processors of the electronic device are configured to execute the instructions, wherein the instructions cause the one or more processors to: During bidirectional ranging operations, the range and direction to the target wireless device are determined by the ranging sensor of the wireless controller; The target location estimate of the target wireless device relative to the electronic device is determined based on the range and direction to the target wireless device; as well as At least one indicator along the trajectory represents at least one proximity value to the target location estimate of the target wireless device, wherein the trajectory represents the path taken by the electronic device.

2. The electronic device of claim 1, wherein the one or more processors are configured to execute the instructions, wherein the instructions further cause the one or more processors to: Presents at least one of a target location estimate of the target wireless device along the trajectory or an indicator of the direction to the target wireless device.

3. The electronic device of claim 1, wherein the one or more processors are configured to execute the instructions, the instructions further causing the one or more processors to: Receive beacon signals from the target wireless device and determine at least one signal strength value from the beacon signals; and The at least one proximity value to the target wireless device is estimated based on the at least one signal strength value.

4. The electronic device of claim 3, wherein the one or more processors are configured to execute the instructions, the instructions further causing the one or more processors to: Determine the category of the at least one signal strength value from multiple categories of signal strength values; and Based on the category of the at least one signal strength value, at least one indicator of the at least one proximity value is presented.

5. The electronic device of claim 1, wherein the one or more processors are configured to execute the instructions, the instructions further causing the one or more processors to: The distance to the target wireless device is presented.

6. The electronic device of claim 1, wherein the one or more processors are configured to execute the instructions, the instructions further causing the one or more processors to: Establish a radio connection with the target wireless device.

7. The electronic device of claim 1, wherein the one or more processors are configured to execute the instructions, the instructions further causing the one or more processors to: Request the target wireless device to increase the beacon rate.

8. The electronic device of claim 1, wherein the one or more processors are configured to execute the instructions, the instructions further causing the one or more processors to: Perform one or more wireless ranging operations, including ranging operations performed via ultra-wideband radio.

9. The electronic device according to claim 1, wherein presenting the signal strength proximity view and the ranging view includes the at least one indicator presenting at least one proximity value to the target wireless device.

10. A non-transitory machine-readable medium storing instructions that cause one or more processors of an electronic device to perform operations, the operations comprising: During two-way ranging operations, the range and direction to the target wireless device are determined by the ranging sensor of the wireless controller; The target location estimate of the target wireless device relative to the electronic device is determined based on the range and direction to the target wireless device; as well as At least one indicator along the trajectory represents at least one proximity value to the target location estimate of the target wireless device, wherein the trajectory represents the path taken by the electronic device.

11. The non-transitory machine-readable medium of claim 10, further comprising: Receive beacon signals from the target wireless device and determine at least one signal strength value from the beacon signals; as well as The at least one proximity value to the target wireless device is estimated based on the at least one signal strength value.

12. The non-transitory machine-readable medium of claim 11, further comprising: Determine the category of the at least one signal strength value from multiple categories of signal strength values; as well as Based on the category of the at least one signal strength value, present the at least one indicator of the at least one proximity value.

13. The non-transitory machine-readable medium of claim 10, further comprising: Establish a radio connection with the target wireless device.

14. The non-transitory machine-readable medium of claim 10, further comprising: Request the target wireless device to increase the beacon rate.

15. The non-transitory machine-readable medium of claim 10, further comprising: In response to establishing a radio connection with the target wireless device, a selection of the ranging view is performed.

16. The non-transitory machine-readable medium of claim 10, further comprising: Request the target wireless device to increase the beacon rate.

17. The non-transitory machine-readable medium of claim 10, further comprising: Perform one or more wireless ranging operations, including ranging operations performed via ultra-wideband radio.

18. The non-transitory machine-readable medium of claim 10, further comprising: Presenting a signal strength proximity view and a ranging view includes at least one indicator that presents at least one proximity value to the target wireless device.

19. A method comprising: During two-way ranging operations, the range and direction from the electronic device to the target wireless device are determined by the ranging sensor of the wireless controller of the electronic device. Based on the range and direction to the target wireless device, a target position estimate of the target wireless device relative to the electronic device is determined; and An indicator along the trajectory shows the proximity value to the target location estimate of the target wireless device, wherein the trajectory represents the path taken by the electronic device.

20. The method of claim 19, further comprising: Receive beacon signals from the target wireless device and determine at least one signal strength value from the beacon signals; as well as Based on the at least one signal strength value, estimate at least one proximity value to the target wireless device.

Images